Hacker News Reader

Comment by drej | Hacker News Reader

This might be a contrarian view, maybe I'm misunderstanding it. To me, much of message passing is not performance critical, so it would be well served by JSON/YAML/XML for easy implementation/testing/debugging. If one needs performance, he can just send bytes over, which can be (de)serialised in a (couple) dozen SLOC. Sure, when you're talking about very complex structures, RPC, dynamic schemas etc., then you might opt for something like this, but let's be honest - that's quite a minority of current users, isn't it?

I never minded these frameworks, but then I wanted to write a few parsers for some file formats and they used Thrift/Flatbuffers to encode a few ints, which seemed like a major overkill. There was no RPC, no packet loss, no nothing.

View on HN

Your view isn't contrarian at all -- in fact it's widely-held by smart people. But I'll present the counter-argument.

True, in many use cases, performance is not critical. However, when you're doing big data processing or serving at scale, serialization performance does in fact start to matter a lot. If Google Search passed all backend messages as JSON rather than Protobuf, it would probably require 10x the hardware (we may be talking about millions of machines here) and wouldn't be able to respond as quickly. Similarly, Cloudflare couldn't handle its logging load without Cap'n Proto.

Many times, companies who started small and thought performance didn't matter later found themselves needing to do massive, expensive rewrites in order to keep up with scale (e.g., famously, Twitter).

Hand-written binary serializers may be easy to write initially, but they are very hard to maintain. Over time, you'll need to add and remove fields, while maintaining compatibility with old binaries already running in the wild, or compatibility across languages. It's very easy to screw this up with a hand-written encoder, but very easy to get right with Protobuf or Cap'n Proto.

Finally, even if performance isn't an issue, you may find that type safety is useful. If you are writing code in a type-safe language, correctly dealing with JSON actually tends to be a huge pain involving lots of branching. By defining a schema upfront and generating code, you can get a much nicer interface. And if you're doing that anyway, then you might as well take advantage of the more-efficient encoding while you're at it -- it's easy to dump human-readable text for debugging when needed. (As of this release, Cap'n Proto ships with a library for converting to/from JSON, and there are lots of third-party libraries to do the same with Protobuf. Both libraries also have one-liner "dump a debug string" functions.)

Or, in slightly fewer words: there's a reason ethernet, IP, and UDP packets aren't encoded in JSON. It's that when you have to encode a lot of a thing, and the encoding is relatively expensive compared to the rest of the processing that needs to be done on that thing, having inexpensive encoding is important.

A more approachable example: logging to a logging process in compute-heavy and/or time sensitive application (simulators, games, rasterizers, ...). In such situations the transport layer is usually bandwidth bound, so the use of a well-designed serialization layer (compression) is highly desireable. That being said, due to Zipf's law and the fact that whitespace delimiting is a pseudo-RLE compressor, I've found most binary serialization formats to be a wash compared to text.

You don't need a schema for type safety. Just an encoding that preserves type information like all the schemaless binary serializations (BSON, CBOR, etc.) do.

You're talking about a different kind of "type safety".

I mean that when I write code to consume a message, I would like to detect (at compile time) if I typo a field name, and I also don't want to worry about what happens if a (possibly-malicious) sender sending me a different type than I expected.

I like to use the term "binding" to differentiate between the compile-time type checks and static typing of messages in a programming language vs. the runtime type equivalence checks performed between sender and receiver of a message.

Please don't use the term "binding". It has many uses and it mostly tends to confuse people.

What kinds of uses of the term "binding" causes confusion in the context of message data types?

I think binding is precisely what is going on when a compiler generates code for marshaling and unmarshaling messages based on a schema.

Binding, as a term, is used for anything from assigning values to variables to what is essentially dependency injection of implementations of more or less abstract interfaces.

I'm merely pointing out that the term is vague and can cause confusion. That you make a strong association with one use of the term doesn't really govern what others associate with it.

> there are lots of third-party libraries to do the same with Protobuf

Recent releases have it built in already, just FYI.

(Also, hi Kenton! I spent lots of time looking at your code when hacking on protobuf :))

It isn't just the serialization, but the fact that being careless about these things tends to co-occur with other kinds of sloppiness.

Here's one example that has repeated itself a few times in a few companies. I kinda like having logging that works without strangling what you are monitoring and/or the server doing the monitoring. I also like having extensible log servers that you can write plugins for (although I'm not that fond of the idea anymore as people invariably will write _slow_ plugins, and then you are screwed. Do put in the APIs, but don't tell anyone it is easy to add processing chains). I like it to the point where people made fun of me for this "obsession" even 15 years ago.

So it goes like this: I use some fast serialization that can be parsed using next to no CPU per message -- or in later years, using Protobuf or similar. Because it is more convenient and I can throw in binary payloads in exceptional cases. I benchmark and I usually get high ridiculously high throughput without even making much of an effort. Great, this saves me the trouble of having lots of log server horsepower and I can get on with life. (Stuff will need to be sharded eventually, but since we keep everything dead simple and we actually have a plan for how to do this when needed I don't worry).

Then at some point someone who has never had to write fast code comes along. It used to be "let's use XML". Now it is invariably "let's use JSON". Because "it is fast enough". This is where it starts. And they'll use Ruby, or something else that is godawfully useless for anything that is supposed to be high throughput, because hey, it is what they understand.

And for a while, it is fast enough. Sure, you now have only a fraction of the original throughput, but the systems are lightly loaded and you are not bumping into any CPU limits. "See!? I told you it would work!".

Then people start to do serious logging and the stuff can't keep up anymore. So they start coming up with lots of schemes for dealing with the load. For every scheme implemented, stuff gets a bit more brittle and complex because now the code has all of these assumptions and mechanisms to preserve. And before you know it, you have a slow, complex logging system, that everyone depends on and nobody wants to touch. Well, sometimes someone will reimplement it, but they'll use whatever is hip at the moment. Like JavaScript and then ooh'ing and aah'ing when they double the throughput -- even though they kind of originally lost 2-3 orders of magnitude of performance.

And it isn't like making a performant version is a big task. With Cap'n Proto, Protobuffers and whatnot, someone has done the heavy lifting for you. Someone a lot smarter than you. And it is probably going to be faster to implement, more efficient, and easier to maintain in the long run.

Performance matters where it matters. And it _is_ stupid to forego it when it costs you little or no effort. And yes, being an old fart means I don't give a shit about people's egos anymore and I'll call them stupid to their faces when they are being stupid.

If you log using JSON (or XML, yech!) as your envelope format, you _are_ stupid.

IMHO performance is only one reason for using a system like Cap'n'Proto/grpc/Thrift vs some handwritten json/yaml/xml/binary protocol.

Others reasons are:

- Starting from an IDL you get a nice service contract. Which means both sides of the service (client & server implementors) get a strong description on how the service should behave and what they need to implement. The more roles you have (architects, client-implementors, service-implementors, testers, mock-service implementors), the more you profit from a good specification.

- If you generate all the serialization/deserialization code you avoid the tedious handwriting of this code and the possibility of introducing errors in this. E.g. some misspelling of field names or functions or copy&paste errors can always occur and some of these errors are only detected late in the development process.

These advantages might not be visible if you have a small service and development team - you might even experience an overhead compared to just "changing the client and server code", which are both under your control. But once you are working on a system with thousands of APIs you are quite happy if you don't have to care about handwriting serialization and RPC code and instead can focus on business logic. I personally was responsible for deploying a similar solution for an in-vehicle infotainment system with hundreds of services and around 4000 functions, which worked out really well.

Thanks for making the point about large teams -- I was coming here to make it myself. Even if you are using JSON, having a contract between client and server is super helpful.

Out of curiosity, what format did your infotainment system for serialization?

it's a proprietary binary serialization. However nothing revolutionary, just comparable to thrift and protobuf.

Another advantage: using an IDL defines your contract in a way that is both language agnostic and portable, which is great for client support and for long-term maintenance.

I once worked on a rather convoluted bunch of microservices where 40% of CPU time was spend inside cJSON library. Cap'n Proto does exactly what you propose - sends bytes over, except in a documented, organized, forward- and backward-compatible manner, instead of a haphazard implementation burdened by technological debt.

When you start having polyglot microservices this all will make sense. For example protobuf gives you versioning and compiles to a dozen languages...just because it is not useful to you don't dismiss the idea. And since I'm part of the minority I'm happy to have libs like this around. Just a sidenote: this behavior of neglecting performance alltogether leads to systems which take ages to build and collapse under pressure.

> When you start having polyglot microservices this all will make sense.

I'll be surprised if you use a language that has a good protobuf implementation that doesn't also have a good json/yaml/xml implementation.

Incidentally, as far as I can tell, protobuf also neglects performance. For high performance you want as much of your message to be predictably positioned in the bytestream as possible, but protobuf uses a key-value style system which means you always have to look at each key-value pair to determine what it is and how big it is. This makes good sense for compatibility, but it's a definite trade off that limits how fast deserialization can be.

If you use a type safe language, Protobuf gives you type safety. This helps a lot as a system grows complex. You don't get that with JSON/XML/etc.

Huh? All (?) of the XML parsers/unmarshallers I've ever used have been type safe - certainly the built in ones in C#, Java, and Go.

I think there's two distinct notions of type safety being discussed here: most deserializers in typed languages provide type safety once successfully deserialized, but the IDL/shared-schema approach helps guarantee that you'll be able to successfully deserialize the data into a type-safe representation in the first place.

I can't speak to XML, but can say that even with "type safe" json serialization, I've seen several bugs due to some json libraries treating all numbers as doubles internally, meaning bad things happen when you're actually serializing integers larger than 52 bits (say, a nano timestamp). Sure, maybe it's a bug in the json library because the json spec doesn't establish any max precision for numbers, but it's not hard to run into it when different components/languages with independently implemented json libraries try to talk to each other.

Shared schema approaches like proto also make it really hard to accidentally typo the name of the field, or accidentally try to read a field as a string rather than a list of strings.

Just like I wouldn't consider code to be type safe if it passed data around as raw Objects and each usage cast it to the expected type, I wouldn't consider services that communicate with each other using independently (manually) implemented parsers/extractors as being type safe, even if the "meat" of the code is dealing with strongly typed data on either end.

Another numeric oddity; the spec says that ints should have from 1 to 9 digits (in theory limiting it to 35 bit numbers or so)! Few libraries actually honour this, but it's really pot luck what they'll do with anything greater than a 32 bit integer.

Is that actually the case? The reference I see for 1-9 in the spec is not talking about length of the number, but is referring to digits 1,2,3...,9 as a single token in the grammar. digit1-9 is a token that's used to disallow leading zeros: a number must start (ignoring the possibility of "-" prefix) with a `digit1-9` which may then be followed by any number of `digit` tokens (inclusive of the digit "0" this time), or else it must start with a "0" which may only be followed by "." etc.

I don't see any reference to range/precision in the spec, other than the suggestion that numbers that can be represented as an IEEE double are likely to be interoperable, and integers within 53 bits of range will generally be interoperable due to exact agreement on their value.

In Golang you can have type safety with simple JSON marshaling between structures.

It's been a while since I looked at it but back when I was trying to map a JSON wire format onto a Go structure, this was a PITA, as it typically is in strongly-typed languages. You say it's "simple". What's the best way to go about it?

I've found it very easy just using the Marshal and Unmarshal packages for golang:

https://golang.org/pkg/encoding/json/#Unmarshal

They have some reasonable examples there.

The idea is that if you're inventing a new data format, you can write the Go types first, and the JSON schema is based on that. This works ok if you're starting in Go.

Mapping arbitrary JSON to Go types may or may not be simple, depending on the data format.

The other commenter posted a link to the official docs which explain it pretty well. I'm new to Go so I don't know what was there before but basically all I have to do is define a Go struct with the types that I want and then using struct field tags I can name the fields (for example if I want to use snakecase or something for JSON). Then when I encode to a string it automatically takes care of formatting it as JSON and I can decode a string and get back the typed struct (it will throw an error if it can't be done).

json/yaml/xml have much less support for a schema-first workflow. XML offers it but only in a terrible schema language; JSON/YAML don't offer it at all as far as I know, and have difficulty expressing sum types, never mind anything more complicated.

The serialization format is an implementation detail. That's why there are similar schema/definition formats that ARE natively represented in JSON, like JSON-Schema.

Personally I think it's easier to maintain a format defined in an IDL like Protobuf or Thrift than a JSON format. Everyone is always on about how JSON is a "human-readable" format, which is nice and probably 90% true. What we want is a human-writable format, which neither JSON nor YAML really are. It's easier to define a structure in Thrift than in JSON.

I still gotta review this release of Cap'n Proto, but after struggling with some buggy language implementations from the upstream Thrift project and evaluating Protobuf 3.0 + grpc, I think that Protobuf 3.0 is going to be the way forward for now.

I work on such a system (multiple "microservices" written in a variety of languages) and we use JSON on top of zeromq. It's super easy to debug without any fancy tool. And as long as I'm consistent with my API design it's trivial to extend and keep everybody up to date.

If performance ever becomes an issue (unlikely in my case) I could probably switch to some binary serialization protocol eventually.

Clearly there are needs for very efficient message parsing. That's why these serialization frameworks exist. I've got servers that spend too many cycles in serialization, so I pay careful attention to it. I put a lot of effort into using JSON despite the performance problems, and have pretty low-level code to read write it without too many allocations etc. Nothing thats easy on the eyes.

But I'd hazard a guess that you're spot on regards the normal needs of normal programs. The vast majority of programs utilizing message-passing are not serialization-bound. For these programs I'd recommend JSON because it is easy for humans to inspect and debug, and there are plenty of libraries to choose from if you are not performance-sensitive.

(YAML is hopeless regards security and XML is just horrid to look at when you're debugging).

Being able to inspect messages is down to the proper tooling.

But I can certainly understand where you are coming from. I've designed several text based protocols in my time for that exact reason. And I still do for stuff that I positively know will never require any sort of performance (mostly on embedded systems, somewhat ironically since those are sometimes constrained down to just a few hundred bytes of memory :-))

The "you can just send bytes over" bit is kind of the whole point. While I use Google's flatbuffers instead of Capn'Proto due to easier use w/ in-memory buffers they more or less have the same design idea: the underlying data is little-endian and stored in machine native types.

For my use case in electronic trading, a format like this is great b/c you don't waste many CPU cycles in decoding/encoding while at the same time you can take complex message structures and easily generate code stubs to read/write them in various languages. You change something? Just regenerate the code stub. You can even guarantee backwards compatibility if needed.

I'd be curious what you mean by "easier use w/ in-memory buffers".

One advantage of things like Protobufs/Thrift/Avro/etc, beyond the performance (and the performance difference can be _huge_), is that they have a (generally upgradeable) schema; this is particularly handy when using strongly-typed languages. For JSON and YAML you, at best, can use a bolt-on schema checker thing.

I'd also disagree that performance on message passing doesn't matter; depends how many messages you're passing :) I've seen CPU-bound systems which spend most of their CPU time decoding JSON.

There's MessagePack for substential improvement of parsing speed and size of JSON while still being straight-forward to parse and debug. These guys (Capt'n'Proto) are for cases when that's not enough either. As I understood, Capt'n'Proto removes the need of parsing itself, so you don't have to parse a data structure into your language's structure if you want to access a field, you just keep your data as a binary blob in memory without copying or parsing the whole thing.

I've found myself in all three situations: where performance didn't matter much and something like JSON or XML was fine, where performance mattered and a binary format was an obvious win, and one situation where serialized data needed to be as compact as possible and I could only achieve that by hand coding.

To me, formats like JSON or XML have one clear advantage: debugging is easier. I can fire up a decoder for a binary format, but that slows down the investigation.

That said, most binary formats have a schema file due to necessity of not serializing field names. That schema file makes it easy to have tools that spit out language bindings. And that reduces the tedium of actually writing code that uses (and validates) the data, so I generally prefer those formats for that reason.

In addition to the problem of premature binarification, I have the sneaking suspicion that a lot of the remaining use cases could well be handled by ASN.1.

Experientially, nothing is handled well by ASN.1. Google for "asn.1 cve" and see how many security vulnerabilities are due to it being approximately impossible to parse correctly.

Reply on news.ycombinator.com