Hacker News Reader

Comment by umvi | Hacker News Reader

> In the last year or two I have worked on and off on making PyPy's JSON faster, particularly when parsing large JSON files

I think this is the key takeaway. If you are parsing huge JSON documents, this will make a difference.

Otherwise, it's super hard to beat Python3's native implementation. I've tried them all - hyperjson, orjson, ujson... none of them provide any significant gains on parse speed of smallish JSON documents, even if you are parsing thousands of small(ish) JSON documents in a row.

View on HN

Something else to note here is that the Python interpreter is slow compared to native code, so if you're parsing a lot of small JSON files, whatever difference in speed the JSON parser makes is masked somewhat by the interpreter overhead (assuming you're doing something with that JSON you just parsed). When you're using a JIT like PyPy, the overhead of the rest of your code becomes less, and so then the bottleneck could become something like JSON parsing, I/O, etc.

This is the main bottleneck in pysimdjson[1]. 95-99%[2] of the time is spent creating Python objects rather than parsing the documents.

There was no trick I could find (although treating all strings as bytes instead of using PyUnicode_DecodeUTF8/unicode_decode_utf8 was a fairly large gain on string-heavy documents) to reduce the cost of bringing the document into Python land, so I added a quick hack to push the filtering down into C, since most of my use cases only need a small part of the full document.

[1]: https://github.com/TkTech/pysimdjson [2]: https://github.com/TkTech/pysimdjson/issues/22

(one of the original authors of simdjson here)

I thought a bit about this - I think you could do a lot faster if you had a 'bulk' method to create lots of Python objects in one hit. I don't know it that's feasible, as I don't know much about the innards of Python, but there's no reason one couldn't keep goofing around with SIMD code and branch-free stuff all the way until you've created objects.

This may be ferociously unmaintainable (i.e. depend on things that Python implementations don't advertise as a part of the API) but I don't know. I doubt you're the only one to have this problem.

Like you said, it would unfortunately be unmaintainable, requiring a fork of CPython. Aggressive optimizations like this could go into pypy, but IMO CPython is still trying to be a "reasonably readable" reference implementation.

I think the time is currently best spent trying to push more of the work back into native code, extending the simple filtering that's in there now to also handle mutation (similar to jq), reducing the amount of back-and-forth with the interpreter.

This whole discussion makes me thing that perhaps there would be a point in parsing JSON into a "JSON database" object instead of a tree of individual hashmaps. In theory it would allow for a more memory efficient representation and more faster search or transformation operations.

> so if you're parsing a lot of small JSON files, whatever difference in speed the JSON parser makes is masked somewhat by the interpreter overhead

Does this vary based on the size/quantity of the input JSON files? I would expect this to hold regardless of size/quantity of the input files? Further, isn't CPython's implementation already native code? If so, I would expect the bottleneck to be allocating the output objects or pressure on the GC or something?

Of course, the smaller the JSON file, the shorter the amount of time spent in native code, and proportionally more time will be spent in the interpreter around whatever looping construct is used to feed data into json.loads (and presumably process that data).

    # interpreter time spent here looping
    for file_name in stuff:
        # interpreter: do stuff with file name, load file into string, etc.
        str_data = ...
        # interpreter: dispatch call to native function
        # native: actually perform json.loads
        data = json.loads(str_data)
        # interpreter: time spent processing the data
        process_data(data)

    # what happens as native time goes to zero (json parse time decreases)?
    # answer: interpreter time dominates

So "interpreter time" is defined as "time not spent parsing JSON" and thus "the less time you spend parsing JSON the less time you spend parsing JSON"? Of course that's correct, but it's so obvious as to be unsubstantial. Surely the OP is making a more interesting point...

No, that's it - for most json documents, the time the interpreter spends in all the stuff surrounding (and executing) json.loads dominates the time spent in native code parsing. That's why replacing json.loads with a faster native implementation doesn't result in noticeable improvements - it's speeding up the fastest bit, and if it's already 0.1% of the wall clock time, then you can't get much more than about 0.1% faster.

PyPy is a JITed language though, so the interpreted bits are much faster. OP is making the point that Python 3/CPython is so slow that speeding up json.loads is a premature (and unnecessary) optimization.

I would think that's because the overhead tends to dominate when calling on small JSON, whether it's function call overhead or python's interpreter, etc.

Really? I just tried with a sample of one of my files, using python 3.6. With text in memory, 1M smallish strings decoding is 12s with ujson and 25s with the built in json. orjson and rapidjson seemed about the same for this test.

Hmm, I'm starting to think my definition of "small" is different from everybody else's. I consider "smallish" JSON to be at most 100KB (and on average 1-10 KB). 1 MB I consider to be "large"... basically anything that doesn't parse near instantaneously I consider "large".

I think GP might've been saying 1M small JSON documents, rather than a single document containing 1M strings.

If you use a language like Rust (which is really easy for these kind of jobs), then 1mb will parse almost instantly.

I've been working with a 90gb JSON file recently. I'd probably start counting a file as large at around 100mb.

I have to ask - why? I understand if you're dealing with an external resource that you can't change, but I'd still like to ask them the same question. JSON's strength, it seems to me, is that it's great for human consumption. It's a very readable data structure when serialized! But who is ever going to read 90GB of JSON?

Not out of choice! It was a backup of a firebase database (also not my choice). I totally agree that JSON is not a good format for files this large, but it was the only format the data was available in so what can you do.

Not OP, but I deal with a variety of proprietary binary/encoded files in the 10s of MB to the 10s of GB range at work, and we have started decoding to JSON as the first step in our normalization, indexing and analytics pipeline (To be honest we essentially just throw everything in to MongoDB). Daily we're in the terabyte range.

It's the perfect solution because you can use the same decoder + something like jq for debugging and manipulation on the command line.

A question - do know of any good resources for jq? I haven't quite been "angry enough" to use jq much beyond using it for formatting json - but that's also partly because I can't seem to quite grasp the syntax.

It isn't quite "awk-ish", not quite cut/join/sort/uniq. But also not quite graphql? Which I suppose would be the "natural" language for a json-query tool?

I feel like I'm missing a piece - and missing out...

The manual is pretty dang good, tbh.

https://stedolan.github.io/jq/manual/

I agree that it is clear, but I still struggle to figure out (by myself) how to do (in an awk sense) simple things like:

Filter out empty or null values:

https://stackoverflow.com/questions/39500608/remove-all-null...

I seem to either end up with unwieldy, long/verbose solutions - or shorter ones that does not work :)

Oh wow, that's terrible!

I tend to get 90% of the way with jq and then awk/sed/grep my way to the desired result (jq -r).

It's not necessarily that it's great for human consumption, more that it's super interchangeable and ridiculously easy to get up and running, and for the most part 90Gb of space is so cheap that it's not worth the overhead of using a binary format.

I'm guessing that OP wasn't parsing a single 90Gb object, but a stream of many newline delimited JSON objects.

Plus the human element helps.

> I'm guessing that OP wasn't parsing a single 90Gb object, but a stream of many newline delimited JSON objects.

I wish that were true. Alas it really was a 90gb object. Actually, the first thing I did was split it by top-level key before processing further.

Especially considering that JSON compresses well.

Isn't that tautological? Clearly, no JSON parser provides a significant gain on parse speed of small JSON documents, when you define large as those where parse time is significant.

>Isn't that tautological?

No, it's no.

>Clearly, no JSON parser provides a significant gain on parse speed of small JSON documents, when you define large as those where parse time is significant.

A rather bizarre deduction.

E.g. "define large as those where parse time is significant".

So let's say a large file is one whose parse time is > 10 seconds.

So, while looking at small, ie. < 10 second parse time files, if a JSON parser takes files that others load in 500ms and load them in 20ms, then that's "a significant gain on parse speed of small JSON documents"...

We didn't have to change any of the definitions...

Well, Python2 -> Python3 introduced a dramatic performance increase, even for small JSON documents.

These were small though, I just made a small sample of a million of them and still saw a 2x improvement.

Oh I think you misunderstood, they are all small, there's just a million of them. I think they were all about 1k.

Note the qualifiers about document size — anything measured in millions probably isn't “smallish”, and unless your app does nothing other than decode those strings the total runtime change is unlikely to be anywhere near 2:1.

Each document was small, there was just a lot of them.

They can be huge changes. With slightly larger docs we had a setup that had to return batches of results back. The bottleneck was absolutely the single thread part of the API encoding and decoding JSON.

Even if the docs were just the size of my example that's 25s for 1m docs. With a few threads I can easily run 10k+/s of the useful part so that'd be 100s + 25s + encode time. With a few more threads hitting 50k/s that's 20s useful time + 25s + more.

With larger docs it becomes far more of a problem. More importantly the fix is a few seconds to implement.

Again, note the qualifiers: nobody is arguing that faster JSON parsing isn't a good thing, only that anything measured in the millions is probably not considered “small”. A significant amount of code is never profiled since it's fast enough and if someone does find the stdlib json library to be using enough time to care about it is certainly easily replaced.

The small qualifier seemed to be on the size of the documents, and on small documents I see a 2x or often higher improvement. At a scale of 10k docs that's still 100ms difference, and in my case can be a huge proportion of the total time spent (more with larger docs, and more again if you include encoding time).

Plenty of things measured in millions are small. A million bytes is a small amount of memory. There are plenty of domains where a working definition of "small" is roughly "can be slurped into memory on a single node without jumping through hoops for provisioning".

A million iterations that total to order of ten seconds seems quite reasonable for a quick performance comparison. It's just underspecified for the rest of us to take away much beyond that.

I have had pretty large speedups in the past when switching JSON implementations. About 5x. It doesn't matter much when the times are 0.2 Vs 1s, but when you are working with a couple of MB of JSON the difference will definitely be noticeable.

Python's STDlib is nice, but rarely spectacular. An old naive CSV parser I wrote for Chicken scheme (using a simple state machine) beat python's C-based one.

> Python's STDlib is nice, but rarely spectacular. An old naive CSV parser I wrote for Chicken scheme (using a simple state machine) beat python's C-based one.

The stdlib is also conservative: I've run into quite a few cases where someone had reported huge gains until they realized they weren't properly handling escaping, Unicode, etc. and once updated the gap narrowed considerably.

You're basically agreeing with the above, and I have experienced the same. The only place where this makes a difference is when the JSON file is in MBs. For smaller file, it's not really worth the extra library.

Depends on how many of the small docs you deal with, I see useful differences on small documents because I process reasonable amounts of them. A 10s change on a million docs for just decoding for a single library is easily worth it for me.

Reply on news.ycombinator.com