Hacker News Reader

Comment by tbrownaw | Hacker News Reader

The biggest issue I have with csv files, is when the people sending them don't get the $%^&@ quotes right. The second biggest issue, is when they say "sorry, we're not able to do that" after I tell them exactly how to fix what they're sending.

View on HN

Substringed JSON arrays work great for this as well. Basically you just convert each row to an array, convert the array to JSON, and the strip the leading and trailing brackets ("[" and "]"). Newlines will be properly represented as \n as well as other funky characters.

I prefer DSV files for this reason http://en.wikipedia.org/wiki/Delimiter-separated_values

I have a simple scheme where comma is the delimiter and , in content is escaped as \,. There are no quotes around values.

What is the proper way to encode the quotes? I'm playing around with writing a Livejournal data dump reader and will have to do the quote dance pretty soon. In LJ files they are encoded like so:

col1,col2,col3,"Longer data and somethin ""with"" quotes",col5

Yes, that's how they're supposed to be. Any field with quotes, or commas, or newlines in the data needs to have quotes around it. Any quotes in the data need to be escaped by preceding them with another quote.

From what I understand, you're basically right, but you're basically right only in a sense that that's what most people do. There's no "CSV Data Format" spec. It's all just what most people agree on, most of the time. Unless someone has another idea.

text/csv is defined in RFC 4180: https://tools.ietf.org/html/rfc4180

Strictly, the definition in RFC 4180 mandates ASCII which makes it unusable for many purposes. I guess there's nothing practical stopping you from using another encoding though.

  >      Common usage of CSV is US-ASCII, but other character sets defined
  >      by IANA for the "text" tree may be used in conjunction with the
  >      "charset" parameter.

http://www.rfc-editor.org/rfc/rfc6657.txt

https://www.iana.org/assignments/character-sets/character-se...

Reply on news.ycombinator.com