Hacker News Reader

The reason I don't like CSV is that CSV files can use any text encoding: ASCII, ISO-8859, UTF-8.. It's not nice to try to guess the encoding when reading files.

RFC 4180 also fails to define the default text encoding. It seems that text encoding can be given only when MIME type is given for the file.

JSON is so much easier.

View on HN

Default encoding is now UTF-8, due to the way the RFC process works it is not in the original document: https://www.iana.org/assignments/media-types/text/csv

Excel generates RFC-compliant CSV files if you are using the en-us/en-uk locales, but not for many other locales. Of course, this is a system-wide setting, and Excel has no option to generate a standard CSV independent of locale.

I used Excel recently and it added a BOM, which is pretty awful. Is there any way to stop that nonsense?

AFAIK: Nope. Microsoft loves that BOM, for reasons I cannot comprehend.

Almost certainly for backwards compatibility. Excel in particular uses the BOM as a single to open a file in a Unicode encoding, rather than cp1252 etc.

I use a script to strip it from the file. Definitely not ideal.

> the en-us/en-uk locales

Though be careful if you communicate with people using both locals. Things interpreted as dates will get corrupted in fun ways.

Reply on news.ycombinator.com