Hacker News Reader

The advantage over standard json is that this is trivial to parse incrementally, i.e. for a dataset that is too big to fit in memory.

I also wouldn't recommend you to reimplement a functionality that is already part of the standard library, especially because the standard library function is going to be much faster because it's implemented in C.

View on HN

Actually, the approach with commas and a wrapped brackets are equally trivial to parse incrementally.

Just start by erasing the open "[", and then read one row at a time, just like you would do without the "[" and comma.

I would say any tool that outputs lists of things in JSON and there is reasonable suspicion that the output may be large, should do this. I.e. do one-thing-per-line with opening and closing brackets on separate lines. It is quite annoying to find oneself with huge dataset in JSON which was produced by a naive generator that doesn't do that - when you need to extract something like "first N elements that satisfy condition X" it is annoyingly more work that it should be. By doing this people that don't care still get valid JSON, people that do care save some time.

Thanks for that info. I didn't know the module was written in C.

Only a small subset of the python standard library is actually written in python.

Reply on news.ycombinator.com