Hacker News Reader

Comment by ORioN63 | Hacker News Reader

That's not a bad thought. Maybe RFC4180 shouldn't have defined csv, but another similar alternative.

It doesn't so much as it says here are some things that people have done.

> While there are various specifications and implementations for the CSV format (for ex. [4], [5], [6] and [7]), there is no formal specification in existence, which allows for a wide variety of interpretations of CSV files. This section documents the format that seems to be followed by most implementations:

RFC4180 misses some rather famous practical problems, such as until very recently Excel was limited to some 16 bit number of rows, 32767 or 65536, and the bigger and richer the corporation the less likely their software is updated, so you send them a 500K line CSV dump of engineering data, they use Excel as a viewer, and they insta-file a bug report that they are paying good money for 500K lines of data but they checked and we only sent them the first 65536 lines or whatever. Bonus fun if they save the truncated file on itself permanently deleting the excess data. Or if their corporate standard DBMS is Excel, as it often is at larger companies.

This has been fixed, but only in very recent versions of Excel, where I work this was a near daily issue until last year or two.

> This has been fixed, but only in very recent versions of Excel, where I work this was a near daily issue until last year or two.

This has been fixed since Excel 2007. I don't know whether 9 years counts as very recently.

I also agree with the fact that it has been an issue. Even at customers who have upgraded past Office 2007, a lot of corporate forms and standardized data exports are based on old work that was completed before the new Office file formats. Updates to these documents and exports continue to use the old file formats because people make changes to existing artifacts, which are handled in compatibility mode in Office, and no one changes the settings to export .xlsx files.

2007 is still in the future for some enterprise companies.

Absolutely. I am not denying enterprise inertia. But the parent of my original comment specifically said "recent versions of Excel". To me, this sounds like "a version of Excel released recently", not "a version of Excel that certain enterprise clients have only recently started migrating to".

CSV is inferior to TSV, there's really very little reason to keep using CSV other than taking advantage of its ubiquity.

CSV allows commas and tabs in fields (actually, all characters, if you choose to). TSV does not allow tabs.

What's more common, tabs in text content or commas in text content?

I'd suggest you'll see commas far more frequently, such as in addresses and freetext fields.

Can fields with commas be quoted to prevent them from being interpreted as delimiters? Yes they can, but that requires extra vigilance, and in my experience it's frequently overlooked.

As for TSV not allowing tabs... the same type of post-processing that goes into allowing for commas to be quoted can go into replacing tabs with multiple spaces.

> As for TSV not allowing tabs... the same type of post-processing that goes into allowing for commas to be quoted can go into replacing tabs with multiple spaces.

No, replacing tabs with spaces means you cannot distinguish between spaces and tabs.

CSV escaping is completely reversible.

I agree tabs aren't common, but it sure is nice to have your format allow arbitrary data, rather than just the data a particular person prefers.

It's a shame we don't use the ASCII delimiter characters. It would have solved a lot of the quoting issues.

https://en.m.wikipedia.org/wiki/C0_and_C1_control_codes#Fiel...

Agreed, that would be the best delimiter choice for plain text data files.

Reply on news.ycombinator.com