Hacker News Reader

Hacker News

Comment by password4321 | Hacker News Reader

Full thread

password4321

Discussion of a similar tool last month:

yq: command-line YAML, JSON, XML, CSV and properties processor

https://news.ycombinator.com/item?id=34656022

Also mentions gojq, Benthos, xsv, Damsel, a 2nd yq, htmlq, cfn-flip, csvq, zq, and zsv.

View on HN

I have made a repository cataloguing tools like this: https://github.com/dbohdan/structured-text-tools.

Thanks for this, good job. I always mean to do something similar but just end up bookmarking HN threads I then never look at.

Guilty as charged... I have so many bookmarks I forget to look at...

APPlet Idea: an app that picks a random bookmark from your saves and sends you a reminder to click it and read it... and schedule when you want to see it - like every 7AM send me a link from my bookmarks. Set an alarm for 15 minutes to get me back on schedule.

So chatgpt and I made this: https://github.com/erasei/randombookmark Load in developer mode and when you click the icon it will open a new tab with a random bookmark.

There's also GoAWK, which supports CSV natively ("-i csv"): https://github.com/benhoyt/goawk

Big shouts to visidata -- very underrated tool imo.

You should include qsv (https://github.com/jqnatividad/qsv) and goawk (https://github.com/benhoyt/goawk) into the CSV section! Goawk got a csv/tsv mode and qsv is loaded with features being developed.

Thank you! That list is perfect since I don't use these tools often enough to remember them and some are hard to find again without knowing exactly what to search for.

I also discovered pawk on it which looks interesting, I made a somewhat related tool [1] which is probably out of scope for your list, but can be used in some of the same ways [2] and may of of interest nonetheless.

[1] https://github.com/elesiuta/pyxargs

[2] cat /etc/hosts | pyxargs -d \n --im json --pre "d={}" --post "print(dumps(d))" --py "d['{}'.split()[0]] = '{}'.split()[1]"

Benthos is a really cool tool when you want to take this idea to the next level. It can do the usual text/csv/json/yaml mangling you'd do with awk or jq (since it includes a version of those in addition to its own processing langauge) but it also has decoders and encoders for a bunch of different binary formats like protobuf or Avro. And in addition to stdin/out and files it can talk to Kafka, MQTT, HTTP, ES and a bunch of other stuff. I was able to put together a log processor that consumes from Kafka, does a bunch of field mangling and then ingests into Elasticsearch in a couple dozen lines of yaml.

Anyone else find jq really hard to use? I found the syntax/interface bewildering at times. Although not as bad as awk.

I'm pretty sure that's one big reason for all these other tools!

There's also https://github.com/dkogan/vnlog/ which is a wrapper around the existing coreutils, so all the options work, and there's nothing to learn

Seems like you meant https://github.com/TomWright/dasel (instead of damsel)

Correct; thanks!

the obvious missing reference is jq https://github.com/stedolan/jq

I've been getting a lot of mileage out of https://github.com/itchyny/gojq#readme recently due to two things: its vastly superior error messages and the (regrettably verbose) `--yaml-input` option

I also have https://github.com/01mf02/jaq#readme installed but just haven't needed it

-y and -Y unused so I patch those in as short flags for --yaml-input/output in my copy.

True, for me that one's a given!

Another similar discussion last year where Miller was mentioned:

New(ish) command line tools

https://news.ycombinator.com/item?id=31009313

Reply on news.ycombinator.com