2019-05-07

Serialization Benchmark

It's interesting to see the result of jeromefoe's metser and smallnest's gosercomp serialization benchmark, the combined best results are:
There's also interesting result (that optimized for deserializing speed, since it's only indexing/pointer to value, but in expense of bandwidth) such as FlatBuffers (but perform badly on metser's benchmark), or ZeroFormatter which not included in both benchmark, but on the original C# implementation has best result (also explanation how it works). But if you have to use JSON anyway for browser compatibility, please use jsoniter instead of Golang's default. If you really need to communicate between services, it's preferably to use binary format (gRPC instead of REST, or near-dead spec like SOAP). For best practice on using gRPC, see videos below:



How FlatBuffers works:

self-note tl;dr
  • use FlatBuffers if you don't care about bandwidth, want real fast deserialization 
  • use ProtoBuf if you use care about bandwidth, communicating between services using gRPC (since it's already been implemented on most languages' library)
  • use JSON if you need browser compatibility, eg. using REST
  • use Colfer or Gencode if you care about bandwidth, real fast in both case (serialization and deserialization), also both client and server written in Go
  • use ZeroFormatter if both client and server written in C#
Haven't research about bound check tho (when network package forged/tampered), not sure which of those binary formats are secure against those kind of attack.

Also checkout these interesting new database (still experimental so it doesn't support important feature such as replication, but you can use this for any embedded database use-case) that tries to reduce/minimize serialization-deserialization process from disk to memory/network (which most databases do convert from row/column/document to struct then serialize before sending to client, not sure how it will affect the query performance tho):
EDIT 2019-06-05: dammit, there's FastBinaryEncoding, that even more faster than them all -_-, should write a new benchmark for this..