Angle-grinder: Slice and dice logs on the command line

I maintain a list of tools like these as part of the docs for my own tool klp (https://github.com/dloss/klp), which I think has a few useful features that are not in angle-grinder, but is orders of magnitude slower, because it's implemented in pure Python instead of Rust.

See also: https://github.com/Textualize/toolong (terminal application to view, tail, merge, and search log files)

Specifically, the "killer app" feature for agrind, is that it's a live reporting tool. You can just throw a CSV file through agrind to get a nice one-shot aggregation out. But if you stream a live data source into it — think something like "your k8s ingress controller's access logs" — then agrind will grab your TTY; draw a table to fills it; and begin live-updating that table with the report, with the rows re-ordering to stay sorted as more data is received.

Want to see e.g. the top 10 highest-duration HTTP requests by path given some particular HTTP status code? You can just start streaming logs into it, and the report will gradually come into focus as it finds matching events. Hit ^C when the data has "stabilized" (i.e. when datapoint ranks are no longer jumping around), and you'll be left with the last-refreshed state of the table in your terminal scrollback, to use in further investigations.

agrind has jq-like manipulation tools, but honestly, I mostly still use jq for that part. I have no idea how to unwrap JSON list items in agrind (or if that's even possible), but if I want to do that, I'll just:

    http localhost:8001/v1/debug/inflight-requests | jq -c '.[].jsonPayload' | agrind '* | json | sum(dur) by path'

Bonus recommendation, as it's something I use together with jq and agrind: stern (https://github.com/rancher/stern) — for live tail(1)ing of the O(N) pods of a k8s Deployment, right from the cluster, without any external log multiplexer.

    stern ingress-nginx-controller -i some_path_frag -o raw 2>/dev/null | agrind '* | json | where status == 403 | count by client_addr' 2>/dev/null

The first one is to remove the "connected to pod XYZ" meta info from stern(1) output, that would muck up the agrind table.

The second one, though, is more interesting: it's there because unlike jq, agrind doesn't die when it hits an unparseable line! Instead, it just spits a parse error and continues. So you can have agrind consume a "dirty" data stream — e.g. a combined "structured JSON access log + plaintext error log" stream — and then just discard stderr, to silence the resulting errors printed from the failed attempts at parsing the error-log lines of the stream.

I think log processing is something that is easy enough to do (stream through STDIN lines) and has specific enough requirements to individual teams that it's worth building ones own little pipe-in binary.

Can start by highlighting common log fields, but later add subcommands to aggregate, or analyze known problems, etc.

Makes me wonder if the 'cargo install' method requires MUSL libc. This is not stated but that also makes it unclear since it comes right after the former variant.