`pv` (pipe view) is rather simple but I use it for viewing a lot and sometimes rate-limiting transfers. Usage is simple: `curl huge-file | pv | some-operation` will show you the progress (okay, curl already does... but pv works for any pipe input, and pv can work in line mode with -l so you can see how many lines (records, presumably) it processed).
`progress` is even simpler! Just type progress in a random terminal while, in another, you are running an operation like cp, gzip, etc.
$ progress
[10123] bzip2 /tmp/test
32.5% (127.0 MiB / 390.6 MiB)
Use -m to monitor instead of doing a one-shot. This will also allow it to calculate the remaining time (for anything running at more than a few kilobytes per second).I also love progress just because it's such a hack. It hooks into the process and monitors file operations. Simple but effective. Somehow it works super well because it manages to ignore everything irrelevant and only shows you the file handle you care about (you can give it hints if necessary, I've used that maybe once in ten years).
`jq` takes JSON data and either pretty-prints (default) or operates on it:
$ echo '{"whoami": "root"}' | jq
{
"whoami": "root"
}
# imagine colored output above
$ echo '{"whoami": "root"}' | jq .whoami
"root"
$ echo '{"users": [{"name": "abc", ...}, ...]}' | jq '.users[].name'
# prints the name of all users in this array
$ curl localhost/api/v1/users | jq '.users[] | "\(.name) \(.age)"' | sort -n -k 2 | tail -1 && echo "is our oldest user!"I’m definitely going to take a note of this and add to my TIL Notes so I can always refer back to this
I just wanted to thank you for teaching me something totally new today
Also, jq is amazing and is basically sed for json data.
Also xidel is a tool everyone should know.
Just out of curiosity are you into DevOps
$ seq 1 10 | pyp "sum(map(int, lines))"
55or along with `jq` and `pup` (html parser)
$ curl https://public-api.wordpress.com/rest/v1.1/sites/username.wordpress.com/posts | jq -r ".posts.content" | pyp "html.unescape(x)" | pup a attr{href}