Hacker News Reader

jq is convenient, but I don't see the draw in building data processing pipelines on it. It's like writing complex software in shell.

Recently, I found myself wanting to do a join by filename on two sets of about 300,000 files. Tried bashing my head against jq with INDEX and various tricks and couldn't get the runtime below minutes.

Then I just gave up, fired up Python, loaded the dataset into Pandas, and did a join. Completed too fast to notice.

View on HN

Created this tiny JavaScript project and published on npm for when you just can't be arsed remembering/learning JQ: https://www.npmjs.com/package/jsling .

Not shilling it so much as saying.. IMHO, for a lot of use cases, there are far better approaches than dealing with the occasional JQ struggle bus ride.

> bash

Did the `join` command help?

No. join in jq is just syntactic sugar over object property lookup anyway. Turns out index() was the expensive step.

I meant the coreutils join command.

Reply on news.ycombinator.com