Faster Protocol Buffers (2019) | Hacker News Reader

Faster Protocol Buffers (2019)

blog.najaryan.net

59 pointsngaut16 comments

My go-to for needing to deserialize structured data in a fast way these days is flatbuffers[1]. It compacts nicely and more importantly is zero copy/allocation(within the constraints of your language where possible) in deserialize. Which lets you do neat things like mmap it from disk.

We used to store 20-30mb of animation data with it and we'd just mmap the whole file and let the kernel handle paging it in/out, worked great.

I don't know how up to date their benchmarks[2] are but my experience has been that it beats almost every other off-the-shelf solution(other than maybe capn-proto which has some similar properties).

[1] https://google.github.io/flatbuffers/

[2] https://google.github.io/flatbuffers/flatbuffers_benchmarks....

Those old benchmarks (that have since been deleted) are highly misleading. The protobuf column is essentially what you would get if you just ignored all the performance best practices of protobuf. There's no reason why in C++ you'd need to allocate and deallocate a message struct for every parse. You just reuse it. And you can have allocation-free parses if you want, and you can alias network or file buffers if you want.

But the main reason people use protobuf is it is compact, which this table does accurately show. Flatbuffers are always larger.

There is a tradeoff in space / performance, I've just never found for my use cases it was large enough vs the significant(~10x) perf boost I got.

From what I recall in the message format[1] protocol buffers require you to walk quite a few parts of the data structures where in flatbuffers you are just reading offset and traversal is significantly faster.

Flatbuffers also let's you control data layout into the packed buffer so you can guarantee cache locality which makes a huge difference when it comes to real-world traversal performance. Data oriented design and FlatBuffers go together really well(which isn't a surprise given some of the background of where FlatBuffers comes from).

[1] https://developers.google.com/protocol-buffers/docs/encoding...

OK, but I just want readers to be aware that the whole idea that it could take five minutes to parse a million protobufs is completely preposterous. I reimplemented their benchmark just now and it runs at roughly 8 million protos per second, orders of magnitude faster than they state, and I didn't even do anything to optimize it.

https://github.com/jwbee/protobuf-flatbuffer-benchmark

Article author here, good to see it on HN, someone else has submitted it (thanks :-)).

If you are interested in the topic you may be also interested in a research library I wrote recently: https://github.com/splunk/exp-lazyproto, which among other things exploits the partial (de)serialization technique. This is just a prototype for now, one day I may actually do a production quality implementation.

How relevant is this (excellent) post in 2022? Has the tech changed at all on this front?

One option not mentioned in this article is use of unknown fields [1] (e.g. C++ documentation is at [2]). Rather than changing the types of the field from a message to bytes, simply remove them altogether:

    message Metric {
      // MetricDescriptor metric_descriptor = 1;
      // Resource resource = 2;
      repeated Int64TimeSeries int64_timeseries = 3;
    }

When then message is deserialised, those will be placed in a separate binary buffer, which is used directly when reserialising. It even works for other types of field. Originally this functionality was lost in the transition from proto2 to proto3 but was added back a long time ago now.

This feels like a cleaner solution to me as it's what this feature is really intended for.

[1] https://developers.google.com/protocol-buffers/docs/proto3#u...

[2] https://developers.google.com/protocol-buffers/docs/referenc...

Did the OP consider sending less data? For example, registering the metric and using an int, rather than sending all of the static data every time. I guess the major downside there is it would make the metric server stateful and I think OpenTelemetry uses UDP.

Otel uses grpc, tcp.

The issue is the agent/collector is stateless and agnostic about where the data eventually goes. It's not the final destination, just an API router / transformer.

This. The statelessness of the OTLP is by design. I did consider stateful designs with e.g. shared state dictionary compression but eventually chose not to, so that the intermediaries can remain stateless.

An extension to OTLP that uses shared state (and columnar encoding) to achieve more compact representation and is suitable for the last network leg in the data delivery path has been proposed and may become a reality in the future: https://github.com/open-telemetry/oteps/pull/171

Windows has something like 15,000 performance counters and error metrics that can be collected. There isn’t a system on earth that can even approach this. At scale, I have to pick and choose maybe 20-100 counters for fear of overloading a cluster(!) of servers collecting the data… once a minute.

That’s because the protocol overheads cause “write multiplication” of a hundred-to-one or worse. Every byte of metric ends up nearly a kilobyte on the wire.

Meanwhile I did some experiments that showed that even with a tiny bit of crude data-oriented design and delta compression a single box could collect 10K metrics across 10K endpoints every second without breaking a sweat.

The modern REST / RPC approach is fine for business apps but is an unmitigated disaster for collecting tiny metrics.

Set your goals higher than collecting a selected subset of 1% of the available metrics 60x less frequently than admins would like…

The partial decoding is also interesting for another case, I think: polymorphism. The OneOf implementation in go protobuf is ugly, but this trick could make it bareable. I'll have to explore further :)

Here is a OneOf Go implementation I wrote that hopefully is less ugly and is significantly faster: https://github.com/splunk/exp-lazyproto#oneof-fields

In this case it is also possible to encode the message up to and including fields 1 and 2, then repeatedly encoding field 3 and concatenating the resulting buffers. Protobuf is designed to do this.

Oooh, I never even thought about trying that!

that's less useful if you're trying to do it over multiple rpc calls, which is how this is used

Reply on news.ycombinator.com