jq 1.7 | Hacker News Reader

jq 1.7

github.com

68 pointsjdnier26 comments

jq 1.7 - https://news.ycombinator.com/item?id=37412698 - Sept 2023 (205 comments)

Have only recently started using JQ, and it's absolutely brilliant. I'm on wsl2/Ubuntu - just pipe any Json content to it, and you get an easy to read colour version rather than your regular monotone output (which seems incompatible with my brain).

And you can then create queries to select certain keys - highly useful if you're dealing with monster documents, and customise the output of that (eg to reduce into just the info you need). Plus probably much more, but those are the features I've used so far. Big thanks to the Devs!

jq is great! Just so you know, there are similar programs for many other languages, like yq [0]. Also, there are interactive `jq` prompts like `ijq` [1].

[0]: https://github.com/kislyuk/yq

[1]: https://sr.ht/~gpanders/ijq/

And there is also a rust clone of `jq`, `jaq` [1], which aims for correctness, performance and simplicity.

I use it for the performance because there is a big startup time for `jq`, with `jaq` I can use it in scripts that are called every seconds in my status bar without delay.

[1]: https://github.com/01mf02/jaq

Yes, with curl and jq you can do a lot. But every time I have to relearn the syntax.

This is really great news. jq deserves an active team maintaining it. The new features (in particular pick()) look great.

Just one tiny personal regret. I get GPT-4 to write almost all of my jq scripts for me, and it's going to stay stuck on that 2021 version for an unknown time going forward.

I guess I could try and come up with some custom instructions that teach it about the new syntax!

Yeah no kidding re: pick(). It's always been a major pain point in jq when trying to write concise jq commands. Super glad they added it.

For their pick() example:

  jq -n '{"a": 1, "b": {"c": 2, "d": 3}, "e": 4} | pick(.a, .b.c, .x)

wouldn't a more jq-y syntax be:

  | {a, b.c, x}

(which fails on the 'b.c' term)

yeap that could be an extension of the shorthand syntax, but note that pick/1 also works with any "path expression" ex: pick(if .a then .b.c else .x end)

You do still get the number formatting advantage if you use the new version with GPT queries. It means for example jq . now pretty prints without converting all numbers to floats, for example.

Recently I needed to query something from a json document, but I just couldn't figure out how to express this in jq. In my spare time I'm working on a toy configuration language that is a superset of json, and expressing the query there was pretty straightforward, at least to me. So I added a mode to do that, and it comes in pretty handy once in a while now as a jq alternative.

https://fosstodon.org/@ruuda/111120049523534027

    % jq -c '[path(.tags[]|select(contains(["tag:primary"])))[-1]]' hosts.json
    ["ams01","zrh02"]

2m28s from time I downloaded hosts.json to producing this solution, though I must admit I had to take a moment to look up contains to realized it had to take an array—I could've used index/1 and had a better time.

    % jq -c '[.hosts[]|select(.tags|index("tag:primary")).name]' hosts_advanced.json
    ["ams01","zrh02"]

This one was much quicker, just 1m5s, but of course after I'd already done the previous one.

It's a language. You can learn it, and then it can be easier to write programs in it. Or you can blame the tool, sure.

edit: and for the curious, this is the path I took to produce these programs:

    wget https://gist.github.com/ruuda/31cffffaa595492a335a4637503f4b7a/raw/7ae5bef0ac331eb2fb7d67cac6cc867e13a675f3/hosts.json
    jq . hosts.json
    jq .tags hosts.json
    jq '.tags|path(.[]|select(contains("tag:primary")))' hosts.json
    jq '.tags|path(.[]|select(contains(["tag:primary"])))' hosts.json
    jq '.tags|path(.[]|select(contains(["tag:primary"])))[-1]' hosts.json
    jq 'path(.tags[]|select(contains(["tag:primary"])))[-1]' hosts.json
    jq '[path(.tags[]|select(contains(["tag:primary"])))[-1]]' hosts.json
    jq -c '[path(.tags[]|select(contains(["tag:primary"])))[-1]]' hosts.json
    jq -c '[path(.tags[]|select(index("tag:primary")|debug))[-1]]' hosts.json
    jq -c '[path(.tags[]|select(index("tag:primary")))[-1]]' hosts.json
    wget https://gist.github.com/ruuda/31cffffaa595492a335a4637503f4b7a/raw/7ae5bef0ac331eb2fb7d67cac6cc867e13a675f3/hosts_advanced.json
    jq . hosts_advanced.json
    jq '.hosts' hosts_advanced.json
    jq '.hosts[]' hosts_advanced.json
    jq '.hosts[]|select(.tags|index("tag:primary"))' hosts_advanced.json
    jq '[.hosts[]|select(.tags|index("tag:primary")).name]' hosts_advanced.json
    jq -c '[.hosts[]|select(.tags|index("tag:primary")).name]' hosts_advanced.json

I didn't time myself, but took 25-50% more queries.

I'll echo the value of ijq, https://sr.ht/~gpanders/ijq/ The live updating makes for less to remember and visualize. Though there is some value in having intermediate history recorded, for the second and later questions you ask of the data. (And supports commenting out for error diagnosis without deleting half the expression, `# ...`, hash and space required.)

I hadn't known of `debug`, quite convenient. I've often used multiple outputs at the tail, for example `... | keys, .` to remind myself of the structure as I explore down. Also `gron` and `fzf --multi`.

Even GPT-3.5 can solve that on the first try. I didn't measure time but it should be within a minute. https://chat.openai.com/share/7f6a97de-9aef-49db-946b-15bfd9...

I tried GPT-3.5, it got it wrong but it did get the overall setup right, just forgot about a detail. But I don't want to reach for ChatGPT every time I need to query something, I want something that I can write without having to solve a puzzle first. (True, maybe I should spend some time learning jq better.)

Isn’t it:

  .tags | to_entries | map(select(.value[] == "tag:primary") | .key))

This took me a little while as I wasn’t at a computer so I couldn’t build the query interactively.

I think the big advantage jq has is that ChatGPT can generate filters from English descriptions.

I'm a bit out of the loop, is there an article explaining the death and rebirth of jq development, alluded to in the release notes?

* First release of jq in 5 years

114 points by drewda 62 days ago | past | 27 comments

https://news.ycombinator.com/item?id=36951830

* jq 1.7

434 points by wwader 25 days ago | past | 206 comments

https://news.ycombinator.com/item?id=37412698

I’m not sure this answers the question. I was expecting some post about someone like the main contributor quitting or something.

To the best of my recollection no one in the former HN comment threads expanded on the long hiatus .. suggesting an absence of any real drama.

Which leaves us with the RC release candidate Announcement:

    After a five year hiatus we're back with a GitHub organization, with new admins and new maintainers who have brought a great deal of energy to make a long-awaited and long-needed new release.

https://github.com/jqlang/jq/issues/2802

and the 1.7 release Announcement:

    It's been almost five years since the release of jq 1.6. Recently we have had a great influx of contributor and maintainer energy. We now have 12 maintainers (up from 3), with about 4 or 5 active maintainers (up from 0), we have 8 admins with several active admins, and several owners at least two of whom are active. As a result the project will not again have such a long hiatus.

https://github.com/jqlang/jq/issues/2876

Either any dirty laundry is well buried or (more likely) this was simply a case of the original implementers hitting some plateau of achievement and fading away leaving JQ to eventually be picked up by others.

Others would have to chime in, I have nothing I can add.

As far as I'm aware, there was no real drama, just Maintainers Having Lives. There was actually some progress being made in the intervening time on the timescale of months, it just never made it into a release—some distros actually took to shipping HEAD iirc.

For those pining for a similar yaml query tool for working through acres of config: https://github.com/mikefarah/yq

jq is awesome and thanks to the new team for their recent efforts and energy, it massively appreciated.

Alternatively, https://fx.wtf/ is also a great interactive viewer with JavaScript filtering that I recently discovered through HN.

Visidata is also a good tool for exploring large JSON files interactively.

https://www.visidata.org/

jq probably saved my life doing region builds at AWS with LPT. jq is such a good tool to know.

Nice to see this. I use jq a lot.

Reply on news.ycombinator.com