Hacker News Reader

Comment by dacryn | Hacker News Reader

it would be so easy to just implement an opt-out function, but Microsoft doesn't want to.

There doesn't have to be a tradeoff if you make it optional.

An opt-out doesn't solve the problem, because one person in the chain of custody if an excel file who hasn't opted out would be enough to destroy the data.

Only if the option was implemented as a user preference which isn’t stored in the file. The right way to do this would be to add the flag as a file header, and give people a UI control to set it by default. That way you’d always see spreadsheets as the person who gave it to you did, and you would create new ones with your preferred behavior.

Channeling my inner QA person, this breaks in many subtle and not so subtle ways:

- people love to export as CSV. And I kinda understand them, csv looks simple enough, what could go wrong ?

- other spreadsheet softs need to recognize the same field and act the same way. Google Sheets, LibreOffice, KingSoft's one etc. Not counting old office versions that aren't updated anymore, because the owner doesn't want to move into the cloud.

- the flag needs to stay when moving sheets around, copy to another set, etc.

That could still be better than nothing. But could still lead to catastrophic loses as people get used to it working most of the time, there's no perfect solution.

It might be easier to think about as the difference between perfection and practical improvements. Right now 100% of Excel users are at risk of silent data corruption. The proposed change would allow people to reliably prevent this category of corruption for all of the documents they create and over time that’d become a majority in their field.

This would also offer the path for changing the default over time to safe by default. There will be more spreadsheets created in the future than exist now.

> Right now 100% of Excel users are at risk of silent data corruption.

They already added an option in the settings last year to deal with this specific issue. Their solution isn't perfect either, they made the tradeoff to optimize for up to date excel users.

Still the same problems. Unless you exclude people with older excel versions they can still ruin the file. And if the first person in the chain didn't make the change, who would even notice it? So we're back at requiring extreme vigilance without any assurance.

Also keep in mind that the first person doesn't necessarily use excel at all. The file might start and end as a text file or TSV.

I don't understand what's wrong with the existing solution of:

Right Click > Format Cells > Text.

Set the data type of the column and Excel will not fiddle with it. Let it stay implicit and Excel will to guess. This change is saved in the file. The article only slightly touches on this: converting to CSV and losing all the Excel formatting and then opening it in Excel again.

The problem with that is that you have to remember to do it before Excel loses data.

Excel doesn't support CSV so expecting that to work is like expecting your car to fly.

Just because they have something they call CSV import or export doesn't mean the feature actually works.

As usual, I think it's a lot more nuanced than that.

An option was added last year [0] the way you propose it, but until then users already had the option to mark the columns as text. The same way we have to deal with any data where auto detection might be problematic: explicitely set a type, and potentially a format.

But of course that's bothersome, sometime people forget to do it, sometimes they don't even know how to properly do it etc. And some of these people will still probably forget to set the option, and there are still edge cases where it can fail (macros are cited)

[0] https://www.theverge.com/2023/10/21/23926585/microsoft-excel...

> but until then users already had the option to mark the columns as text

This only works on Excel files though, which save a type for columns/cells. With CSV files, Excel would just auto-format them. I guess you could save everything as xls/xlsx, but I shouldn't have to use a vendors file format because they're threatening to corrupt my data if I don't.

CSV doesn't do what you want and excel doesn't support CSV files. Why insist on a completely wrong tool and file format combination?

If the excel developers cared about CSV support they would have added more settings to the CSV import tool that let you specify everything. Every single data base tool that lets you import CSV files has worked better than Excel. It simply means Excel wasn't built to import CSV files. Tough luck.

.xlsx file extension is well supported by many open source libraries. If you insist on working with Excel then you should use the appropriate file format, which is .xlsx. I do that all the time and it just works.

That's just CSV being unusable shit. There's always a schema for the types of the fields, but there's no way to communicate that schema, so CSV is inherently ambiguous. If your data includes fancy things like numbers you should avoid CSV.

It's still a shame that Excel's JSON support is so good, but no business user trusts it because "you can't double click the files" and CSV remains the worst best thing you can double click.

I don't want Excel to take over the file association for .json but I do sometimes wish Excel would just invent a file extension like .xljson that loads JSON so that at least you could trivially rename a JSON file to .xljson and get working double-click for your business users but a file format (with optional schema support even) that's a more modern and saner de facto standard than CSV.

I think that one dumb trick would improve everyone's productivity a great deal.

You're talking about users of an operating system that hides file extensions by default.

No, I'm talking about developing for those users. Right now if you want to write the simplest, dependency lightest export code that those users can double-click and open in Excel you are writing yet another CSV export. It would be nice if instead you could take an existing API that returns JSON and wrap it in a mimetype like application/json+excel, and add a Content-Disposition: attachment; filename=thisapi.xljson and get Excel double-click support without another flaky CSV file.

Reply on news.ycombinator.com