Hacker News Reader

Comment by the_other_guy | Hacker News Reader

Yet protobuf is probably the most compact, efficient and performant serialization method especially when saving bandwidth is important. I experimented with protofbuf, flatbuffers and messagepack and always found protobuf messages the most compact by a noticeable margin

View on HN

That's the core of the author's argument. Protobuffers optimize for something besides usability and maintainability, because Google cares more about incremental performance than developer-friendliness. Which is a fine thing to care about at Google's scale, but maybe others' calculations should be different.

note that you could succinctly put in one sentence something that the author took a whole page. I feel there is something jib about compression to be said, but I'll leave that as an exercise to the reader.

That doesn't seem like the author's main argument - they say: "Protobuffers were obviously built by amateurs because they offer bad solutions to widely-known and already-solved problems."

That's a fine argument. But then the author decides to call the people who wrote protobuf idiots and amateurs.

That really depends on the nature of your data. The space saving in protobufs really comes from its variable-length zig-zag encoding of numeric fields (including lengths for arrays and strings).

Tbh, i'm surprised msgpack didn't produce smaller outputs for you, because it really should (at the cost of being slower)

No way that msgpack can produce more compact messages than protobuf, it is schemaless and stores the element names inside the message. Here is a simple benchmark I have of an array of objects where each {id: int,x: int,y: int,rot: float}. An array of 20 elements with random values of such object would take in JSON 804 bytes, in mpack 538 bytes and in protobuf3 only 304 bytes, of course this changes if you put other values but as you see for this case protobuf is almost 60% of msgpack message size and it's around the same ration even if you make the array length 200 instead of 20.

Yep, you're right. I forgot about the keys.

Protobuf is not size efficient. The half self-describing nature and built-in backwards/forwards compatibility features sacrifice a lot of bits in efficiency. I'm not saying that this is a bad thing, just that choosing protobuf if bandwidth efficiency is your main goal is probably not a good idea.

See my other reply below under this comment tree, I also tried flatbuffers and it was extremely inefficient compared to protobuf but I don't have numbers currently for it

"msgpack and flatbuffers are worse" is faint praise.

It definitely is not.

ASN.1 PER encodes data on a bit-level (that is, it represents integers using minimal number of bits they can fit in) and doesn't waste space on tags if not necessary. It is used in all kinds of telecom protocol, particularly mobile protocols such as LTE. Obviously, it needs to be both space-efficient and efficient in processing.

You can find more about ASN.1 uses here: https://www.marben-products.com/

Reply on news.ycombinator.com