Hacker News Reader

Comment by cpeterso | Hacker News Reader

ASCII also includes control characters for delimiting text records. If people used these purposed-designed characters instead of comma or tab characters as delimiters, we could avoid many headaches quoting and escaping CSV data.

https://en.wikipedia.org/wiki/C0_and_C1_control_codes#Field_...

  ASCII 28 for File Separator
  ASCII 29 for Group Separator
  ASCII 30 for Record Separator
  ASCII 31 for Unit Separator

View on HN

This is a terrible idea! You're not avoiding anything, you would still have to escape these characters in CSV because it's entirely possible (in fact, almost certain) that someone will want to use the character itself in a CSV file.

Besides, it's arguably a GOOD thing that the delimiter in CSV (the comma) is such a common character, because it forces all parsers to properly support escaping and quoting. If it used some sort of unique character that was almost never used except in CSV, then 99.9% it would work without correct escaping, but the 0.01% when someone entered "you should use the character X to separate fields" as a column it would fail, and those cases are much more likely to slip through the cracks.

Also: the whole point of CSV is to be human readable. There's no obvious way to render "Record separator" on screen, and the format would essentially become a binary format.

> it's entirely possible (in fact, almost certain) that someone will want to use the character itself in a CSV file.

Our business has parsed and emitted significant volumes of data have never encountered one of these code points in the wild.

My understanding of why not to use these code points is that they don't have printable glyphs and thus don't easily lend themselves to ad hoc data exploration tools.

Unfortunately the comma is the decimal separator in some locales but the thousands separator in others. As the US doesn’t really use commas in floating point numbers escaping these is often an afterthought causing issues that only surface abroad. Another reason why CSV is a terrible format.

Quoting/escaping would still be necessary, but the need would be less obvious. It would be deferred from a common annoyance at development time to a common cause of nasty surprises in production.

And replace them with a whole new set of headaches, like difficult typing and copy-pasting.

Typing should be easy. Just type ctrl-], ctrl-/, ctrl-[ or ctrl-^. In the bash shell, you may need to type the quote character (ctrl-v) first, depending on your try settings. Copy and paste worked fine for me in lxterminal an xterm.

Typing those in anywhere that’s not a shell is still a pain.

Isn't that mainly because these weren't norm? For the vast majority of text editing software you do not type Ctrl+J for a new line after all.

Reply on news.ycombinator.com