Hacker News Reader

Comment by atrus | Hacker News Reader

Don't use broken tools. The key phrase is "in control of producing data". If you're forced to use Excel, then it's not really you in control, is it?

View on HN

>> […] Excel […]

> Don't use broken tools.

Tell that to your accounting and finance department and let us know how the message is received.

> If you're forced to use Excel, then it's not really you in control, is it?

In which case the up-thread's advice to "just produce strict RFC 4180-compliant CSV data" is worthless. "Just."

We're stuck with whatever CSVs we get, so 'just' doing X is not an option.

From a pragmatic viewpoint, the CSVs that I get from finance (usually saved as .xlsx) have the same issues for parsing the data as a CSV. But since the issues are consistent, I can automate conversion from .xlsx to CSV, then process the CSV using awk to eliminate errors in further parsing the CSV (for import, analysis, etc.). Sure, I'm essentially parsing the CSV twice but, because the parsing issues are consistent, I can automate to make the process efficient.

Obviously that wouldn't work for CSVs with different structures, but can be effective in the workplace in certain scenarios.

As long as a human didn't generate the file, all things can be automated.

However, if you ever have the misfortune of dealing with human generated files (particularly Excels) then you will suffer much pain and loss.

I once had to deal with a "CSV" which had not one, not two but 6(!) distinct date formats in the same file. Life as a data scientist kinda sucks sometimes :shrug:.

Before 2010 and UTF-8 everywhere , I regularly had the misfortune of dealing with multi encoding CSVs. Someone got CSVs from multiple sources and catted them together. One source uses ISO 8859-1, another -15, another UTF-8, sometimes a greek or russian or even ebcdic was in there. Fun trying to guess where one stopped and the other begun . Of course, none of them were consistent crlf or escape wise.

The up-thread’s comment (emphasis mine):

> And *if you are in control of producing data*, just produce strict RFC 4180-compliant CSV data

The point of the comment was that you likely aren’t in control of producing the data, so the article’s recommendation of using an entirely different format is likely also invalid. I’m not sure what you are arguing against as you seem to actually agree with them.

This is some next level response where Excel is called a “broken tool”. You may not agree with their choices or design or anything else. But calling the entirety of the product isn’t making a strong case towards the prior point.

“Just be in control” is unfortunately bad advice when you’re … not in control.

I don't think the advice is "Just be in control" as much as "Acknowledge you're not in control most of the time."

Reply on news.ycombinator.com