Hacker News Reader

Comment by A4ET8a8uTh0 | Hacker News Reader

In their defense, if it was anything but CSV files, they would be accused of making it too complicated/locking into proprietary formats and so on. I can't say CSV would be my first choice, but I don't really want to think what the alternative would be.

View on HN

NDJSON? Sqlite?

csv is many times over again a simpler format than sqlite and easier to understand by anyone across the world.

Never heard of ndjson, can’t see one publishing this data in a format that isn’t nearly as common as something like csv (or regular json which some of the data is published in).

CSV only seems simple. Lots of parsing edge cases. Sqlite isn't readable by hand but it's basically bulletproof. NDJSON is literally just newline seperated JSON, it's just easier to process as a stream without a special parser.

Sure, could be simpler, but there's a spec and multiple implementations: https://www.rfc-editor.org/rfc/rfc4180.html And the spec is ~5 pages.

Not sure why NDJSON is considered simpler, as json objects can be arbitrarily nested. Breaking into records is easier, but parsing is harder.

(ND)JSON is simpler because people actually follow the spec, unlike with CSV.

People violate the JSON spec all the time.

And following the CSV spec is much easier than following the JSON spec. And there's only like three edge cases.

> And there's only like three edge cases.

Unless you involve MS Excel or, worse, MS Excel on macOS. OTOH, pitfalls are: UTF-8 BOM, comma vs semicolon, single vs double quote, multiline cell content and escaping, escaping in general...

Then you’d have to get your hands on a parser. Who wants to write up the business case and spend months going through change control (they’d probably just say no).

JSON and NDJSON can also be much larger than CSV files if the wrong structure is used.

All this banter arguing over CSV, JSON, sqlite seems unnecessary when you can just push format X through a pipe and get whichever format Y you want back out: https://github.com/liquidaty/zsv/blob/main/docs/csv_json_sql...

(disclaimer: I'm one of the zsv authors)

Millions of .xlsx files

Reply on news.ycombinator.com