Hacker News Reader

Comment by onion2k | Hacker News Reader

Sure, it's simple and familiar, but it has serious drawbacks such as the time and CPU resources it takes for parsing and the lack of schema.

Before looking for something 'better' than JSON, build a basic prototype of your app and see if your application actually needs to use something else. The data in most applications is so small and will parse so easily that actually measuring the CPU utilisation will be quite hard. 99.99% of applications aren't moving data around at the sort of speed where the format really matters.

Protocol buffers are very clever and really cool but using them where you don't need them is a huge waste of effort.

As for the lack of a schema that's not true. There is a schema. What's in the file is defined somewhere. For applications that consume JSON that's in your code rather than the file format.

View on HN

JSON will easily dominate CPU usage in any modern application that uses it for its API (and is not itself something intrinsically CPU-hungry like crypto or machine learning).

We benchmarked one of our apps, written in Go, which uses JSON extensively both for the API and for internal storage, and we found that around 90% of the CPU time is spent serializing and deserializing JSON.

The JSON code itself is a tiny portion of any app, so it's one of those rare optimization cases where a small change can yield significant speed-ups for the entire app.

Was it a JSON echo server?

Our Scala based servers handling thousands of RPS only deal with JSON via Jackson and JSON encode/decode doesn’t even register in profiles.

Depends on the way you handle JSON. You don't need to deserialize the entire received JSON string and then walk the node tree, you can walk the string as is. With this, you can process 500MB/s or more of JSON on a single core.

On the other side, you can produce JSON string directly, saving the need to produce some intermediate representation in memory.

You should be able to saturate 10gbit line on a multicore server easilly with this approach.

Agreed; in data-centric apps the JSON parsing time is a major time suck. But sometimes it can be heavily optimised with a good library, like fastjson for Java.

There are formats that fall between unrestricted JSON and strict predefined schemas like Protobufs or Thrift. I have used MessagePack for internal APIs and message queues where JSON was too slow but we still wanted a self-describing format. You can decode msgpack for debugging without needing the original definition (there is none), unlike a fixed schema format.

No affiliation, just satisfied user.

http://msgpack.org/index.html

> Before looking for something 'better' than JSON, build a basic prototype of your app and see if your application actually needs to use something else.

That implies that JSON is somehow the gold standard of serialization formats. I don't see it that way. The fact that JSON is ubiquitous in some areas doesn't mean that it's the gold standard for all other uses.

In my experience, protobufs are easy to work with, concise, safe, and efficient. There's never any ambiguity to them, and both creating and parsing messages is fast and easy. That doesn't mean that there aren't cases in which JSONs aren't a better choice, for instance when working with browsers, or needing a text based format for debugging or inspection purposes.

> Protocol buffers are very clever and really cool but using them where you don't need them is a huge waste of effort.

In my experience protobufs require very little effort, with most of the effort being put into defining the schema. It's pretty frictionless afterwards with one notable exception - when you want to manually inspect a message. That's where JSON wins hands down.

Protobuf seems more complicated to debug manually.

That implies that JSON is somehow the gold standard of serialization formats.

I'm not suggesting JSON is the best format, because that depends on the project you're working on. I'm saying it's good enough for most tasks, and has ubituous support, and it's easy to work with. There are better formats for particular use cases but if your application has requirements that can work with JSON it probably isn't worth finding something else.

Protobufs do require more effort in the toolchain. Pretty much the only way to parse a .proto file is with protoc and you might need to build it from source. So you need to run the C++ compiler and depending in the build system you're using, adding a dependency on protoc at build time can be a pain.

Contrast with JSON where every decent language ecosystem has a parser readily available. That's what makes it the "gold standard" for me.

Protobuffers have one major advantage over JSON: the schemas are versioned. When your requirements for the payload inevitably change, Protobuffers can handle that with a lot less muss and fuss than JSON can.

Plus, the byte count over the wire is much smaller - something that matters even in today's world of gigabit ethernet.

You have to compile the protocol buffers so there is overhead. Also the messages aren't human readable eg through kafka so you need to write something to look through the messages. JSON is faster if your dev team need to get from A to B in a hurry, but protocol buffers are a generally a better long term choice if you can put in the effort up front. Both have pros and cons.

> Protobuffers can handle that with a lot less muss and fuss than JSON can.

Are you speaking from personal experience? FWIW it doesn't match mine.

Not sure what you mean by "schemas are versioned". Protobufs have backward compatibility rules so you can add fields without changing versions. I don't think there's anything about explicit versioning in the spec?

Reply on news.ycombinator.com