Hacker News Reader

Comment by chrisshroba | Hacker News Reader

I have this shell function defined:

csv_to_json () { python -c 'import csv, json, sys; print(json.dumps([dict(r) for r in csv.DictReader(sys.stdin)]))' | jq . }

It converts a csv to a json list of objects, mapping column names to values. I find it way easier to then operate on json by filtering with jq or gron, or just pasting it into other tools for post-processing. The jq at the end isn't necessary but makes for nice formatting!

View on HN

I'm two weeks late, but you could do something like

   python -c 'import csv, json, sys; [print(json.dumps(r)) for r in csv.DictReader(sys.stdin)]' | jq -s .

to stream the csv rows instead of reading the whole file into ram.

That won’t scale well to files approaching or exceeding system RAM, though.

Also, unless it’s the coloring you’re concerned with, you could replace the pipe to jq with pprint, a native Python library.

pprint won't work here because the output is a json string, not a python object, so pprint'ing it will just print the string as is. And plus, pprint doesn't output valid json (it uses single quotes, for example), so the output isn't usable for piping into other filters and programs.

And it's true that it won't scale well but it still has worked every time I've wanted it to :)

Reply on news.ycombinator.com