Hacker News Reader

Comment by Klathmon | Hacker News Reader

They say in the linked article that this should only be used for objects about 10kb and larger.

I'd argue that if you have 10kb or larger object literals in your codebase, you are already missing the mark on readability and maintainability in some ways.

View on HN

Where you'll usually find this:

- exporting data from server to client for initialization

- localization data

- environment variables (feature maps, configuration etc)

- preloading datasets for graphs/tables

Also cache initialization scenarios, larger datasets used for common dropdown/select lists like countries w/ ISO codes etc.

If it's exclusively going to be used for heavyweight operations like these, it's probably better to benchmark against protobuf decoding. I guess using JSON has a "works out of the box" appeal, and doesn't require defining any protobuf schema. But personally I don't see defining proto files as too prohibitive in terms of development cost.

> benchmark against protobuf decoding

Protobuf isn't built into the browser, so it can't bypass the JS parse & execute time. Instead you'd be parsing protobuf's JS, executing it, parsing proto, and producing objects. It'd be worth doing, sure, but it'd almost certainly be the slowest option by far since it's doing way more stuff in JS than either of the other two options and the JS syntax parse is the slow part.

These benchmarks indicate better protobuf performance [1]. Compute time these days is often dominated by memory transfer rates. The "slowness" of javascript seems to be offset by there being less data to begin with. Collapsing a 100KB resource down to, 50 or 25KB is usually worth it even if you have to do more operations in javascript. Not to mention end to end load time (which is probably what people are usually trying to optimize for) can be lower by reducing how much data needs to travel over the wire or radio.

At the end of the day, who knows if the use case hits edge cases or stresses parts of the implementation that is not optimized for JSON decode or protobuf. Getting meaningful performance data ultimately needs to be experimental, and resists categorical answers about whether X is faster than Y.

1. https://www.npmjs.com/package/protobufjs#performance

This article goes into a bit more detail: https://auth0.com/blog/beating-json-performance-with-protobu...

> These benchmarks indicate better protobuf performance [1].

We're exclusively talking about cold start performance here. Single, one-time object creation. Hence why JS syntax parse is the dominate factor and not execution performance. Those benchmarks are not that, they are hot performance. That's a completely different thing.

> Not to mention end to end load time (which is probably what people are usually trying to optimize for) can be lower by reducing how much data needs to travel over the wire or radio.

Wire transfer size would need to be looked at differently. The JS code & JSON string are both also going to be compressed unless you're not using a compressed Content-Type for some reason.

What is the "completely different thing" you're referring to here. Between:

1. Having a static JSON string, and decoding that string.

and

2. having a static blob, and using protobufs to decode that blob.

these two things accomplish the same thing. I'm not sure why you seem to think one is a "cold start" and the other is "hot" - they're both "single, one-time object creation". The former is going to be parsing ints and floats as ascii, and reading in "true" and "false". Regardless of compression, the memory-inefficient JSON encoding is going to be used (whether it's over the wire, or just as an intermediate representation during parsing). I've used protobuf decoding for things like localizations and configurations before - the "cold start" use case you're talking about - and it does in many circumstances result in faster loading. My napkin paper reasoning is that this will be much more heavily weighted to booleans and integers that are much more efficiently encoded in protobufs than JSON, so maybe if you had a use case that almost entirely decoded strings your performance differences may not be the same.

Are you including the cost of loading protobuf itself? You seem to be basing your argument on an assumed already present & loaded protobuf library.

You need to benchmark starting from nothing at all. Your link that you seem to be basing this off of has a loaded and fully JIT'd protobuf. That's not the start state.

You can measure the impact on loading time, and the size of the protobuf implementation you're using probably has an impact on the threshold at which it becomes more efficient. I don't doubt that parsing a 500 character long JSON string is probably faster than loading a protobuf to do it instead. In fact, apparently this JSON parsing trick is only effective beyond 10K or so. But past a certain threshold memory bandwidth is more crucial than loading code. If your data consists mostly of booleans and integers then JSON can often be an order of magnitude larger in size than protobufs. If it's compressed, then decompressing it takes clock cycles and the parsing code is still parsing the larger uncompressed JSON text. A protobuf library can often skip compression altogether by virtue of using normal ints and bits for numbers and booleans. So while the protobuf library does have some additional overhead it's often higher throughput for many types of data.

You’re repeatedly missing the point. This is about optimizing startup time.

The comparison should be:

cost of downloading payload + runtime cost of parsing JSON

cost to download protobuf lib + parse and execute JS protobuf lib + download payload + runtime cost of parsing

Specifically, the article talks about how parsing JS is more costly than JSON - this cost will apply to the protobuf library which certainly far exceeds 10KB. There is no way the math will work on your favor until you get to MBs of data.

> Specifically, the article talks about how parsing JS is more costly than JSON - this cost will apply to the protobuf library which certainly far exceeds 10KB.

I would suggest reading the links I posted. The minimal protobuf library, which is suitable for working with static decoding, is 6.5KB [1]. Again, you're right that the size of the protobuf library will be an important factor in dictating the scale at which it's more effective than JSON parsing but your sense of the factors is off - a light protobuf library doesn't reach 10kB let alone "far exceeds 10KB".

Furthermore, if your pages use the protobuf library already for other uses like decoding and encoding RPC messages then loading and parsing the protobuf library is basically free - you're going to be doing this anyway.

1. https://www.npmjs.com/package/protobufjs#installation

I currently work on a project using protobufjs. Our generated static classes are ~500KB and ~1.5MB, or around 140KB gzipped. The schemas are not that large, and this does not even include any network code (not part of protobufjs).

None of these need readability though (or are particularly readable to begin with, regardless of if object literals or parsed).

Either way it's really more data than object at that point so it's appropriate to store it as JSON. Normally I'd place such data in a different file, but I can imagine that that might not be best for webpages.

> fata

you created a noun to describe "fat data" from a typo.

But is it pronounced FAY-ta or FAT-a?

FAY-ta is a type of cheese. Not unlike this comment thread.

Sadly we pronounce it FEH-ta in the states

“Fata” is the word for a bucket in icelandic. Quite apt indeed.

Now I feel bad for fixing the typo...

Like a lot of people who have given interviews, I have my own set of very odd stories.

I interviewed a developer and asked him to explain how the system he was currently working on worked on the whiteboard. As he talked he drew two boxes. He drew a line between those boxes. Then as he talked he kept drawing over the line between the boxes. (Now, he was jr to mid-career so I didn't expect a magnum Opus but we value people who can explain themselves because at least if they're wrong we find out before the mess gets too big. But I digress.)

Your analysis reminded me of that interaction. What kind of information architecture do you have if you're building objects that big?

I mean, as others have said, if this is the main payload being transferred from client to server, it's probably going to arrive as JSON and you're going to turn it into Objects.

If it's not that data (they're talking about cold loads) how many other categories do you have that can approach 10k?

Configuration? We have libraries for that and they often read a JSON file.

Lookup tables for fixed relationships of data in the system? Maybe, but that complicates your testing situation.

How many of those categories get loaded more than once per session? Are these really such large startup bottlenecks that we tackle this instead of other problems? GP implied incompetence but I get more of a whiff of desperation here.

Smells like a God Object pattern to me.

Apt term: https://en.m.wikipedia.org/wiki/God_object

Reply on news.ycombinator.com