Hacker News Reader

Comment by nly | Hacker News Reader

Not OP, but I deal with a variety of proprietary binary/encoded files in the 10s of MB to the 10s of GB range at work, and we have started decoding to JSON as the first step in our normalization, indexing and analytics pipeline (To be honest we essentially just throw everything in to MongoDB). Daily we're in the terabyte range.

It's the perfect solution because you can use the same decoder + something like jq for debugging and manipulation on the command line.

View on HN

A question - do know of any good resources for jq? I haven't quite been "angry enough" to use jq much beyond using it for formatting json - but that's also partly because I can't seem to quite grasp the syntax.

It isn't quite "awk-ish", not quite cut/join/sort/uniq. But also not quite graphql? Which I suppose would be the "natural" language for a json-query tool?

I feel like I'm missing a piece - and missing out...

The manual is pretty dang good, tbh.

https://stedolan.github.io/jq/manual/

I agree that it is clear, but I still struggle to figure out (by myself) how to do (in an awk sense) simple things like:

Filter out empty or null values:

https://stackoverflow.com/questions/39500608/remove-all-null...

I seem to either end up with unwieldy, long/verbose solutions - or shorter ones that does not work :)

Oh wow, that's terrible!

I tend to get 90% of the way with jq and then awk/sed/grep my way to the desired result (jq -r).

Reply on news.ycombinator.com