Hacker News Reader

Comment by AdmiralAsshat | Hacker News Reader

CSV was a thing long before I was born, so I'm not privy to how it came about. But at least in day-to-day work, the single biggest drawback of CSV in my experience is the fact that the comma and most of the other common delimiters occur regularly in real data, forcing all of the cumbersome escape sequences. To say nothing of someone misplacing a quote somewhere and throwing off the cell count.

So, question to the greybeards: when this format was coming about, why didn't we use one of the dedicated Data Structure separator control codes (e.g. File/Group/Record/Unit separator) that were part of the ASCII standard?

https://en.wikipedia.org/wiki/C0_and_C1_control_codes#Basic_...

It seems like it would've saved us all several decades of headache.

View on HN

Some people did.

But commas can be seen, edited, and typed with ease in any text editor.

As per comment above: your CEO, SWE, or secretary can all use or contribute to a csv file. And using an easily recognizable and typeable separator has proven to be worth the downsides.

Agree that the commma delimiter is a pain, as it is a frequently used character in the content. Then adding escape-type chars or quotes around fields just makes things more and more messy.

Consider also that IBM EBDCIC and other non-ASCII character sets were (and are) in common use, so C0/C1 may not have made sense 'back in the day'

> why didn't we use one of the dedicated Data Structure separator control codes (e.g. File/Group/Record/Unit separator) that were part of the ASCII standard?

Because you can't see them. CSV is, at its core, a text format. Using FS/GS/RS/US would effectively make it a binary format.

Also, what happens if one of those bytes appears in data?

The simple fact is you will never be able to come up with a text-based format that can handle all possible values without escape sequences, quoting, or length-encoding. And really, that's not that big a deal. It just means you have to write a simple state machine instead of using regex or your language's equivalent of split().

Csv is only one form of delimited files. Tsv or Tab Separated Values is yet another. If you have either field delimiters or row delimiters (new line) in your data, change your delimiters and process appropriately.

Reply on news.ycombinator.com