Hacker News Reader

Comment by samwillis | Hacker News Reader

This reminds me of this one from a couple of years ago: https://www.theverge.com/2020/8/6/21355674/human-genes-renam...

“Scientists rename human genes to stop Microsoft Excel from misreading them as dates”

> …over the past year or so, some 27 human genes have been renamed, all because Microsoft Excel kept misreading their symbols as dates.

> The problem isn’t as unexpected as it first sounds. Excel is a behemoth in the spreadsheet world and is regularly used by scientists to track their work and even conduct clinical trials. But its default settings were designed with more mundane applications in mind, so when a user inputs a gene’s alphanumeric symbol into a spreadsheet, like MARCH1 — short for “Membrane Associated Ring-CH-Type Finger 1” — Excel converts that into a date: 1-Mar.

Original paper leading to the changes: https://genomebiology.biomedcentral.com/articles/10.1186/s13...

View on HN

And I still can’t believe that after multiple decades it removes the leading “0” for things like zip codes. It does this even though the default data type for cells is “general” and not explicitly numeric. To my thinking that should cause excel to defer to the source data characters.

When I worked at a lower level I had to deal with this constantly when folks doing mail merges had bad zip formats coming through, and as the “data guy” it all got referred to me to fix.

Still, I think excel is great when used within certain confines. I use it for ad hoc purposes all of the time for simpler bits of work that are one-offs where it would take a fair bit longer to explore the data a bit and make some minimally presentable visuals & tables and some inline notes on analysis and interpretation.

> the default data type for cells is “general” and not explicitly numeric. To my thinking that should cause excel to defer to the source data characters

you think zip codes are a more common use case than numbers?

I think if you explicitly type in a leading zero with five digits it seems reasonable that it should infer it’s a zip code.

Maybe, given the appropriate locale in the system settings.

Oh no, Jesus, please no. Nothing better then Excell file what behaves completely differently when moved to another computer, purely because of something like locale in system settings.

First, you should not conflate the locale of the user or program with the locale of the (span in the) file.

Second, how would you handle international addresses? For many businesses, a program that can't do that would be useless.

programmers who don't work with time be like

It needs Clippy-ing: "recognise column as zip codes? always, now, not now, never, turn off format recognition".

If the column is going to be called something stupid like A, B or C clippy is going to have a hard time....

I would expect it to leave my data alone unless told so. It’s one of the little things that drive me crazy with Excel the few times I use it, and every time I need to search how to make it behave because I don’t use it nearly enough to remember.

I would expect it to store 01234 as a number unless explicitly told to store it as a string. This is exactly how it should work. I’m all for hating on excel but being upset because the software can’t read your mind and your eccentric edge case isn’t automatically picked up is a bit entitled

I think I want to do the opposite of reading my mind to be honest. The default cell type is generic, so just assume everything is a string and don’t do any parsing. Maybe show a tooltip to quickly change the cell type if you detect something that might be a number or a date.

Probably it’s just different usage - when I use excel is mostly to store a table of raw data where I want to store the actual data, not what excel thing the data is. I’m using it as the equivalent of a database, but simple to update and share with other people (even not technical folks) or to open a csv and quickly hide/filter some data.

> The default cell type is generic, so just assume everything is a string and don’t do any parsing.

So, if you have `1234` in A1 and `5678` in B1, `=A1+B1` should by default produce `VALUE ERROR` rather than `6912`?

That's... not why spreadsheets were invented.

> when I use excel is mostly to store a table of raw data where I want to store the actual data, not what excel thing the data is. I’m using it as the equivalent of a database,

Then use a database program, not a spreadsheet!

However, if there are no database programs that fit your needs, and you end up using a spreadsheet instead, don't complain that the spreadsheet acts like a spreadsheet instead of a database. It's not the tool's fault that you're the one using the wrong tool for the job.

> So, if you have `1234` in A1 and `5678` in B1, `=A1+B1` should by default produce `VALUE ERROR` rather than `6912`?

If you didn't change anything else, just made A1 and B1 text cells, then adding them would still return 6912.

> That's... not why spreadsheets were invented.

It's decades later. Why they were invented is not the priority.

> Then use a database program, not a spreadsheet!

Databases only have a fraction of the capabilities of a spreadsheet. That's not a reasonable response.

Umm, if you are treating them as text cells, then adding them would product 12345678 since you are concatenating strings. I just don't think there is any good solution for this, since it's effectively impossible to automatically distinguish numbers that the user wants to treat as strings (like a zip code) and numbers that should be treated as floats or ints.

+ is not concatenation in any spreadsheet program I've seen, and it probably shouldn't be concatenation in anything. That's the solution, having a separate concatenation operator.

the '+' operator provides concatenation of strings in both python and C; in Excel, you use '=CONCAT()'... just syntactic differences, is all.

Yeah, I spend more time doing data manipulation in python than in excel, so I thought that excel would do that as well.

It's not just a syntactic difference. It's a huge UX difference that one operator always does an addition on both numbers and numeric strings, and another operator always concatenates both numbers and numeric strings.

The looser your typings the more it matters, and spreadsheets barely even have types.

In C? How?

sorry, meant cpp! C (apparently) uses a function that more closely resembles Excel

C has strcat but that only does the easy part - allocating and freeing the memory is still the caller's b job.

Tend to agree that choice of + for string concatenation was a mistake - anyone know the history/first widely used example? Did the original BASIC have it? (I think QuickBasic did, but that's unlikely to be the origin).

Edit: found https://retrocomputing.stackexchange.com/questions/20460/wha..., suggests it was ALGOL-68.

+ is not a cancatenation operator in excel. It is numeric addition. It is added the numeric values of which the letter characters are merely a representation then that answer would be the same as if excel treated them as numeric data types.

Alternatively, if you don’t want excel to perform addition on underlying numeric values but instead only on defined data types then the formula could result in NaN which would signal to the user that the explicit data type was not compatible with addition.

I know, complaining like this probably doesn’t make sense. It’s a minor issue to begin with and I’m probably an edge case as far as excel users go. It doesn’t make sense for MS to tailor its product for my type of user.

Databases only have a fraction of the capabilities of a spreadsheet. That's not a reasonable response.

There’s nothing you can do in a spreadsheet that an RDBMS can’t do.

Programmers often object to spreadsheets because they’re too “brittle”. I don’t think it’s an unreasonable objection.

Most spreadsheet users don’t exhibit the rigor necessary to impose a schema on their spreadsheet-based data, nor to use features like named ranges to abstract away the physical location of data in the spreadsheet from its semantic meaning. Data validation is an afterthought, too. Spreadsheets invite silent errors when rows or columns are inserted or removed, data is copied/pasted, etc. It takes more effort, in my experience, to keep the formulae in a spreadsheet working in the face of modifications versus a database.

A database explicitly separates the data from logic (at least, typically— getting into queries that act conditionally based kn queried data blurs that line a bit) and provides a strong schema and enforced validation at the time data is added or modified. You’re not going to get wrong answers from a database query because somebody added a row at the bottom of a range that isn’t covered by a SUM(), for example.

...I logged back into my lurker account just because of how annoying this was- I use Google Sheets all the time to do data visualization and analysis work because it just works, and because I can share my results with other people, and let them explore the data in a format that they understand.

There's a reason lots of MBA types get issued high-performance, high RAM machines- because asking them to learn SQL or whatever AND asking them to learn how data structures and query syntax work instead of just doing things a little more slowly and throwing more performance at Excel makes a lot of sense.

Not saying it's perfect, but please don't hate on Excel et al because they aren't the perfect world way of doing things.

And even if you do teach them proper database queries, that doesn't give you a nice editable view of multiple interlinked queries and parameters, with output organized into charts and graphs. The best option down that road isn't switching to a database program, it's using a spreadsheet program that can do SQL.

Spreadsheets are great for ad hoc work. I think they’re intuitive and agile. I use them for that purpose too. All the time.

Using them for business processes is a recipe for sadness. People don’t have the discipline to use them for that, and the tools have too many shortcomings.

In terms of “exploring data” I’ve been in way too many meetings where different users have worked on forks of the same original spreadsheet and come up with contradictory answers.

Re: slowing people down - I’ve had the fortune to show a few “MBA types” how to do JOINs in lieu of the eldritch horrors they’d created attempting to re-implement JOIN with VLOOKUP. The productivity gains were significant. Excel is using a screwdriver to drive nails in some very glaring cases.

> Using them for business processes is a recipe for sadness. People don’t have the discipline to use them for that, and the tools have too many shortcomings.

> In terms of “exploring data” I’ve been in way too many meetings where different users have worked on forks of the same original spreadsheet and come up with contradictory answers.

You shouldn't use a notepad of SQL statements for processes either!

Once certain things are figured out and planned around for repeated use, they need to be transformed into a proper maintainable form, probably involving careful code with error detection. But that's not a spreadsheet vs. SQL issue. If SQL causes fewer problems it's probably because someone that knows SQL is more likely to have programming knowledge/discipline, not something about SQL itself.

> Re: slowing people down - I’ve had the fortune to show a few “MBA types” how to do JOINs in lieu of the eldritch horrors they’d created attempting to re-implement JOIN with VLOOKUP. The productivity gains were significant. Excel is using a screwdriver to drive nails in some very glaring cases.

It's great to use a JOIN when it's appropriate. But a lot of the time there's still no real need for a database, and you want all the other tools a spreadsheet has. Or there is a good case for a database, but the best answer is having some queries that feed the spreadsheet.

> So, if you have `1234` in A1 and `5678` in B1, `=A1+B1` should by default produce `VALUE ERROR` rather than `6912`?

If they were text, I'd expect it to end up as `12345678`, personally. + being reasonably commonly overloaded to mean string concatenation.

Wat.

Excel uses & for string concatenation. Excel considers + to be an arithmetic operation. Excel is a spreadsheet.

you are so not the target user my dude

Maybe, but it seems weird for it to be completely unreasonable for operator overloading to be a thing, but also expect the user to have a working knowledge of basic data types.

Everyone uses Excel as a database, and Excel should absolutely tackle this. It’s like “Everyone is using Toyota to carry children to schools in underdeveloped countries” so let’s put seatbelts, you know, even if it’s not the intended usage.

The French government believed that Covid cases had stopped increasing, because their federated Excel spreadsheet was silently truncating data at 1 million rows. Accidents like this happen all the time.

Access should be based on Excel, only adding sheet types: single-record forms, SQL reports, pivot tables.

In fact, pivot tables are the proof that Excel is a database tool.

> It’s like “Everyone is using Toyota to carry children to schools in underdeveloped countries” so let’s put seatbelts, you know, even if it’s not the intended usage.

WTF? Are you saying Toyota don't normally put seatbelts in their cars? Like... what???

In the bed of pickups, maybe?

I think the usage to store other type of type is getting popular very late after excel has being invented.

It was created to count money. So it would make sense to defaults field to number unless it obviously isn't.

People are just using a tool that isn't designed for your current purpose.

I think probably the best change it can do is don't alter the data at input. Instead, alter it at display and compute. Then it won't actually corrupt your data anymore.

Then for a wrong field type, you just need to change the type, and the correct data will be displayed as expected.

Your expectation is that it should destructively change what you enter? Why not even just store what I put in and cast when necessary?

Zip codes are not a weird edge case either.

They're weird in that they are text codes rendered exclusively in digits in the US. Most Americans are generally ignorant of this and blissfully input them as numbers and those in the east with leading zero codes have to deal with the blowback.

Storing a zip code in a column of an excel file is by no means an eccentric edge case. If I type 01234 into a cell, it’s not unreasonable to expect excel to save it (and reproduce it in e.g. a mail merge) that way.

I would expect it to store 01234 as a number unless explicitly told to store it as a string.

These days I'd disagree. Once upon a time explicitly column delimited data with leading 0 padding may have been common, but I suspect in most software written in the last 20+ years leading 0s if included are more likely relevant parts of the data rather than Hollerith card default punches.

excel users don't think like this

Excel users use leading zeroes to indicate they are using a punchcard based format? Seems… unlikely

not what i'm saying. the vast majority of people who work with spreadsheets have no interest in redefining it to align with other paradigms. spreadsheets work damn well as they are

and spreadsheets would not break if 02559 were stored as 02559, so what's the problem?

it would break tens of millions of people's expectations. decades of muscle memory for many people. pretty big effect for no objective benefit

When writing a number you never start with a zero. If something never happens you should favor the scenario that happens quite often.

If I type 000012 why would you think I wanted 12?

12 is equal to 000012

If you're in the context of math, yes. Otherwise no. How often are you both doing math and _also_ bizarrely typing leading zeros? Who does that?

I would expect it to not destroy data.

I don't care how it stores it but if I've input 01234 either manually or via a CSV import etc. I do expect it to remember that there was a 0 at the beginning. Excel's lossy data type conversion has caused more problems than I care to remember over the years. One of them caused a significant sum of money to be deposited in the wrong bank account once.

If I explicitly typed a leading 0, I want it to stay unless I'm typing it into a numeric cell. In that case I'm doing something strange and wasting my time but what I want is obviously 1234.

Why would anyone type a leading zero on a number? The only time I _ever_ do that are parts of dates, which this wouldn't hit, and octal numbers extremely rarely in code.

I think cases where I want to preserve the leading 0s vastly outnumber cases where I don't.

The leading +1 on international numbers is a pain too.

Zip codes are easy to fix.

What’s painful is when a file has phone numbers and excel helpfully converts them all to scientific notation.

I always prefix zipcodes with a single quote. It tells Excell “this is text” and the leading single quote does not render.

It's little things like parsing unmatched leading single quotation marks as meaningful non-errors that contributes the "hate" component of my love-hate relationship with Excel.

Reply on news.ycombinator.com