Protobufs are very important for Google. A significant percentage of all compute cycles is used on parsing protobufs. I am surprised that the parsing is not doing using handwritten assembly if it's possible to improve performance so much.
As a lingua franca between interpreted languages it's about par for the course but you'd think the fast language should be the fast path (ie: zero parsing/marshalling overhead in Rust/C/C++, no allocations) as you're usually not writing in these languages for fun but because you need the thing to be fast.
It's also the kind of choice that comes back to bite you years into a project if you started with something like Python and then need to rewrite a component in a systems language to make it faster. Now you not only have to rewrite your component but change the serialization format too.
Unfortunately Protobuf gets a ton of mindshare because nobody ever got fired for using a Google library. IMO it's just not that good and you're inheriting a good chunk of Google's technical debt when adopting it.
With the techniques described in this article, protobuf parsing speed is reasonably competitive, though if your yardstick is zero-parse, it will never match up.
You only need to pay for decompression once to load the message into ram rather than being forced to either make a copy or pay for decoding all throughout the program whenever fields are accessed. And if the link is bandwidth constrained then the added latency of decompression is probably negligible.
The separation of concerns between compression format and encoding also allows specifically tuned compression algorithms to be used, for example like when switching zstd's many compression levels. Separating the compression from encoding also lets you compress/decompress on another processor core for higher throughput.
Meanwhile you can also do a one shot decompression or skip compression of a stream for replay/analysis; when talking over a low latency high bandwidth link/IPC; or when serializing to/from an already compressed filesystem like btrfs+zstd/lzo.
It's just more flexible this way with negligible drawbacks.
One nice design choice it has is to make default values zero on the wire by xor'ing all integral fields with the field default value.
This composes well with another nice feature it has, which is an optional run-length style packed encooding that compresses these zero bytes down. Overall, not quite msgpack efficiency but still very good.
One even more awesome feature is you can unpack the packed encoding without access to the original schema.
Overall I think it's a well designed and balanced feature set.
- No need to build messages 'inside out' / leaves first to get zero copy behaviour. This is huge.
- Fully functional RPC implementation with really interesting latency hiding behaviour via asynchronous promises. It's got most of what I want from gRPC without the horrid googlisms and dependencies.
- CapnProto has a canonical encoding, which makes it suitable for hashing and comparison.
- It has a packed encoding, whereas with Flatbuffers you're going to have to compress with a more hardcore codec like zlib.
- CapnProto supports runtime reflection via a dynamic API. This is useful in cases where you really need it, like when creating generic bindings for dynamically typed languages or use cases. Like protobufs it has a "lite" mode to opt out of this.
- CapnProto JSON support in the library is better, probably due to the above.
Major cons:
- Ugly schema format.
- Not header only (I've not tested whethwr or not there is a positive here with respect to build times)
- The "kj" support library is a bit of a pain in the ass, refusing to use common C++ vocabulary types like string_view or support common STL data structures like vectors or strings. Writing a message in to a std::vector for instance will require a specialization of an interface.
The only thing I’ve seen that’s competitive to raw struct is writing a C++ template visitor style pattern to walk the struct fields using inline methods. (This can be achieved in some other compiled languages, of course.)
It lets you compress things as you walk the struct, and breaks the dependency on struct layout (which can matter for cross-platform stuff, and for versioned protocols). It’s not quite as fast as a memcpy off the network buffer, but it can be branch and virtual dispatch free (not counting branches in the compression logic).
Also, it can validate as it copies the data into the struct, so now you can auto-generate application-specific validation logic. This is a big deal, since validating the output of a deserializer is almost as hard as hand deserialization!
I really liked the article however. Forcing tail optimization in the c++ template serializer sounds like it will substantially improve the (worst case) generated assembly.
https://developers.google.com/protocol-buffers/docs/encoding
- wire_type=1 64 bit: fixed64, sfixed64, double
- wire_type=5 32 bit: fixed32, sfixed32, float
Pregenerating protobuf messages you want to send and then modifying the bytes in-place before sending is going to give you a nice performance boost over "normal" protobuf serialization. It can be useful if you need to be protobuf compatible, but it's obviously better to use something like SBE - https://github.com/real-logic/simple-binary-encoding
But yes, once you want real high performance, protobuf will disappoint you when you benchmark and find it responsible for all the CPU use. What are the options to reduce parsing overhead? flatbuffers? xdr?
if you're figuring out serialization from scratch nowadays, for most apps, I'd def start by evaluating arrow. A lot of the benefits of protobuf, and then some
Think pandas in python, but language agnostic.
When messages look more like data, even better, as columnar representations are often better for compression & compute. So it's pretty common now for data frameworks, and increasingly, DB connectors. Ex: Our client https://github.com/graphistry/pygraphistry takes a bunch of formats, but we convert to Arrow before sending to our server, we zero-copy it between server-side GPU processes, and stream it to WebGL buffers in the client, all without touching it (beyond our own data cleaning).
The main reason I'd stick to protobuf is legacy, especially awkward data schemas / highly sparse data, and I'm guessing, Google probably has internal protobuf tooling that's faster+easier+better tooled than the OSS stuff.
For server->client, arrow's record batch representation is useful for streaming binary blobs without having to touch them, but for the surrounding protocol, we care way more about http/browser APIs and how the protocol works, vs the envs + problems flight was built for. So we built some tuned layers for pushing arrow record batches through GPU -> CPU -> network -> browser. The NodeJS ecosystem is surprisingly good for streaming buffers, so it's a few libs from there, careful protocol use to ensure zero-copy on the browser side, and tricks around compression. Flight, at least at the time, was more in the way than a help for all that, and all arrow sponsor money goes to ursa labs, so we never ended up having the resources to OSS that part, just the core Arrow JS libs. Our commercial side is starting to launch certain cloud layers that may make it more aligned to revisit & OSS those pieces, but for now, still low on the priority list for what our users care about. Our protocol here is about to get upgraded again, but not fundamentally wrt the network/client parts.
Interestingly, for the server case... despite our using dask etc. increasingly on the server, we don't explicitly use arrow-flight, and I don't believe our next moves here will either. I recorded a talk for Nvidia GTC about our early experiments on the push to doing 90+ GB/s per server, and while parquet/arrow is good here, flight still seems more of a distraction: https://pavilion.io/nvidia (with more resources, I think it can be aligned)
I've been playing with pushing arrow batches using (db -> server-side turbodbc -> network -> browser -> webworker using transferable support), leveraging a custom msgpack protocol over websockets, since most databases don't support connectivity from a browser. Not dealing with big data by any means (max a million records or so), and so far it's been fairly performant.
for saturating network, afaict, it gets more into tuning around browser support for parallel streams, native compression, etc. we've actually been backing off browser opts b/c of unreliability across users, and thus focus more on what we can do on the server..
Plus it's C so it's infinitely more maintainable and way more portable.
Portability is typically not a big issue, because you can have a fallback C++ implementation.
A proof that they don't care enough about protobuf parsing performance.
https://devblogs.microsoft.com/aspnet/grpc-performance-impro...
> Support for Protobuf buffer serialization was a multi-year effort between Microsoft and Google engineers. Changes were spread across multiple repositories.
The implementation being discussed there is the main C# implementation from https://github.com/protocolbuffers/protobuf/tree/master/csha...
Changing very widely used APIs is not easy, these things take time.
Something that by good CI/CD practices shouldn't even be part of checked in code.