Show HN: A fast JSON library for Go | Hacker News Reader

Show HN: A fast JSON library for Go

github.com

119 pointsgoccy31 comments

I develop a super fast JSON library ( https://github.com/goccy/go-json ) . It is drop-in replacement of encoding/json.

This library is available by simply replacing the import statement. It's more compatible and much faster than json-iterator/go.

Please give me your feedback ! Thanks !

Hi mate,

very interesting library. Keep it up.

Improvement suggestion: While reading the README, the JSON library comparison [1] it's confusing, at least to me. It uses symbols like: ○, and △ which are not obvious. I would rather use "yes/no" or the classic check/x (unicode chars, HN does not support them here) if you want to use symbols. I don't know what "△" even means, so I cannot suggest an alternative.

Just my 2 cents.

[1]: https://github.com/goccy/go-json#json-library-comparison

Edit: This 2021 I have the objective to contribute more to OS projects. Is it OKay if I submit a PR with the change? Thanks!

Another suggestion in the chart: it isn't obvious which line is about your project because it doesnt list the `goccy/` part. I would add that and move it to the top, out it and the standard in bold.

Yes, Of course !! Welcome !!

As a Non-Gopher, is the stdlib so slow?

I think it depends on the application. In many cases, encoding/json is fast enough and stable, but I think you can try to use a faster library if JSON encoding / decoding is performance critical on your application.

Or you could try something like Apache Arrow or Flexbuffers?

If you write off the "JSON", sure, there's a bajillion options. But that's not always possible or desirable. None of those bajillion options have the combination of support and mindshare that JSON has obtained, for better or worse.

We're talking differences of a thousandth of a millisecond (0.001ms) for most structs, but yes.

encoding/json from stdlib makes a lot of allocations (~30) due its use of reflection, which is notoriously slow.

How does it handle de-serializing to structs with interface members? With encoding/json and ffjson, I had to write some code to handle that myself and add a type-tag member to my struct, which wasn't a deal-breaker, but if I could avoid that, I would like that very much.

That looks great, especially some of the unique optimisations. I gave it a nonscientific test run with a set of 1k different 2-4kb JSON encoded messages that we saw in our day to day traffic using the default Go benchmark library. Compared to easyjson (generated parser) goccy/go-json is unfortunately 20-25% slower and allocated four times the number of bytes.

Any chance that you will implement code generation like easyjson?

Are you unmarshaling to interface{}? Have you tried commenting out the easyjson json.Un/Marshaler implementations? By default they wrap the easyjson code and add reflection overhead.

I had the same results when testing with a project that is already using easyjson, but after commenting out easyjson's json.Un/Marshaler impls I am seeing much improved performance using goccy/go-json.

Indeed. Thanks for the hint! While the allocations are still 2.5x, not by accident calling into easyjson makes goccy/go-json perform faster than easyjson with generated code by ~10%. That is some impressive result.

Thank you for your report. It's an interesting result. If you have code that can be reproduced, I will try to optimize it to be faster than easyjson.

I find the circles, crosses and triangles on https://github.com/goccy/go-json#json-library-comparison hard to comprehend. I guess that circle means "yes", cross is "no" and triangle is "partial" or something. However that is really just based on context. (I guessed the symbols from what I thought the data should be)

I would suggest using words like "yes", "no" and "partial". They are short and clear. If you really want to do symbols I recommend using for yes, and still spelling out whatever triangle is supposed to mean.

The author looks to be Japanese, so that's probably why. In Japan, grade school teachers often use the symbols like so:

○ = correct × = incorrect △ = partially incorrect

I'd prefer the yes, no, partial though as well.

Ah, thanks for explaining that.

However yes, for an English README it probably makes sense to stick to words to clear any confusion.

I fixed it ! Thanks !

My first thought was "yes", "no" and "ish" since the last one is a similar length to the first two.

My second thought was that that was probably just going to move the confusion to non-native english speakers.

Given a math background, ~~ for 'ish' is also tempting ... but while writing this out I've circled back around to "I can't think of anything better than partial".

Great job!

>json-iterator/go ... it hasn't been supported for a long time.

I wouldn't say so. The latest release was last summer and the latest commit was last november.

The README is a treasure trove of information on go internals, thank you for writing all of that out. Fascinating to read what looks like your your thought process as you built this library, I always sort of thought README's should sometimes be a "diary of design decisions"

That being said, I'm not really at a point where json decoding is a bottleneck in any of the code I write, but will keep this in mind. Great job!

Thank you for your nice feedback !

Could you please reference it on awesome go and publish a reference to it on go-nuts ? I would be interested to see feedback from go experts.

How is the performance compared with simdjson (https://github.com/simdjson/simdjson)?

I tried to compare it with simdjson-go, but the interface of this library is very different from other libraries and it was difficult to compare. However, I think simdjson-go has the best decoder performance.

Not according to this comparison https://github.com/ohler55/compare-go-json.

Very interesting read in the README. It's a bit scary, so I'm not sure I'd use this in a real program, but I learned some cool stuff reading the description!

Thank you for your feedback! In fact, it's understandable to be scared because it uses a lot of uncommon techniques. By growing this project, I would like to be able to use advanced techniques with peace of mind.

Have you compare it to OjG (https://github.com/ohler55/ojg)?

You could also add your parser to https://github.com/ohler55/compare-go-json.

Great job. I starred this repository because of the interesting optimization techniques, could come in handy in my own projects.

Thank you so much !

Reply on news.ycombinator.com