Hacker News Reader

Comment by michaelcampbell | Hacker News Reader

The builtin CSV/TSV parsing seems the killer feature here, at least for me.

I've been using SQL on CSV and JSON lately as it's mostly easiest for me, but I can think back a few times where I've used an *awk for a specific need, AND had to deal with CSV parsing at the same time.

View on HN

Both GNU awk and "one true awk" added --csv last year, so as the README note mentions you probably don't need frawk for this (any more).

If you need just csv/tsv parsing, you can also take a look at https://github.com/eBay/tsv-utils

interestingly, i had read somewhere earlier that those utilities were written in D.

confirmed it by looking at the site just now.

I only found out recently that my favourite csv-munging tool, csvtool, is written in OCaml.

Yes, and with heavy use of one of the biggest D strengths -- CTFE (compile time eval). Great lib!

What's the common tool people use these days for manipulating json in an awk-like manner?

Probably: https://jqlang.github.io/jq/

I would say that lots of people use jq. But it’s not really awk-like.

You could probably use gron with awk pretty easily, since gron stands for “greppable json”.

But I also wonder how an awk with native json support would look like.

Food for thought: have a look at this paper[0] about structural regular expressions. The author (Rob Pike) sketches in the last section an awk-support. I remember using such regexps a while ago to tweak indented JSON and JSON-like data (the indentation allowed to loop on hashes easily).

An awk with json support would for the most part need to be able to loop on hashes and arrays, and provide ways to travel in-depth. So far regular awk can travel through arrays (line-separator), and "in-depth" (e.g. nested "arrays") via regular loops & cie. Probably easier to think about it with a few concrete examples though.

[0]: http://doc.cat-v.org/bell_labs/structural_regexps/se.pdf

gron (https://github.com/tomnomnom/gron) to transform it and query and then invert the transformation?

How do you use SQL on JSON?

You mean only for strictly tabular JSON, i.e. no nesting beyond rows and columns?

Spark and Drill have options, but DuckDB makes it almost trivial: https://duckdb.org/docs/data/json/overview

Thanks, will check that out.

duckdb, as noted by someone else as well.

Reply on news.ycombinator.com