Hacker News Reader

Comment by beagle3 | Hacker News Reader

This is indeed user incompetence, but so far every news source quoted the official PR and said it was an "Excel Problem".

Microsoft PR was caught unprepared - I wonder how they'll re-spin it in the next few days (and for the first time that I can recall, a Microsoft product was wrongly blamed...)

Pay attention, how every time there's a Windows virus or worm, it's a "Computer Virus", but in the (extremely rare) occasions where Linux or MacOS is involved, it's attributed to that system. I don't think that's a coincidence, CMV.

View on HN

I think Linux and MacOS viruses get a special callout because they're exceptions. Similar reporting conventions are used all the time, like a terrorist attack in the middle east vs a terrorist attack in France. It's not some nefarious PR scheme, the story is just more interesting because of how unusual it is.

It's not a user issue, it's a government incompetence issue. Who was the technical genius who thought 'right, we need to store thousands of lines of critical and important data, I know! Excel!'

The common case for stories like this is the answer is "nobody."

Time point 1: "We need a way to map the long-form epidemiology state. It's 2D data... Let's use Excel." "Will we hit limits?" "Pfft, no. Not unless we end up needing to track tens of thousands of patients, and that's not practical; we don't have enough people to do that tracking in the whole NHS."

Time point 2: "This pandemic is an URGENT problem. We need to hire more people than ever before to do tracing. And be sure to keep the epidemeology state mapper updated so it can drive the summary dashboards!"

Have you every worked in a large enterprise?

I have personally tested an Excel based credit rating tool, to be rolled-out globally by a major financial institution.

The fact that one should not do this, is by no means a reason not to do it.

Fun fact: Toshiba has software that generates entire CRUD app skeletons driven 100% by excel spreadsheets. You fill out object definitions in excel spreadsheets with property names and data types etc. It's like an extremely uncool long lost cousin of rails -g

Source: Got one of their engineers to show me after I heard about it and had to know if it was real.

Anyone who's ever worked with or in Japan knows just how far they are willing to torture Excel spreadsheets to get them to do anything and everything.

I've (miss)used excel as a web-service testing platform once. The test data (input/expected output) was anyway in the Excel sheet, so why not. We did not have other software available at the time, so Excel+VBA it is.

It was one of my nicest testing gigs ever. A test session would take only minutes, all results documented to the t. And fiddling around with the test data was so easy. Would be interesting to know if this thing is still in use.

Not sure the customer liked it that much, I regression tested the 6 previous - still running - versions of the service, something no one had cared to do for years. We found bugs both in the Spec and in the Code for nearly all old versions...

>I've (miss)used excel as a web-service testing platform once. The test data (input/expected output) was anyway in the Excel sheet, so why not. We did not have other software available at the time, so Excel+VBA it is.

That's an upgrade on how Toshiba did testing. They did basically the same thing except it was Excel+Humans. No joke. Never saw that one with my own eyes but a coworker did. He also said they had doctors there and each doctor was responsible for X number of staff and every so often made them fill out a questionnaire of which the final question was "have you thought about killing yourself lately?" and if you answered yes you got a day off. Apparently people would ask for transfers to different offices where the doctor responsible for them wasn't physically present, so they could avoid even being asked the question for fear of being made to take time off work. One of the more senior guys on the project kinda just disappeared too. Fun times.

to play devil's advocate, i had a desktop support job in college and worked with an executive assistance. She was _very_ attractive and so i was at her desk "working on the printer" a lot, she knew Excel better than anyone i've ever seen (before or since). It was like watching a magician.

In the right hands Excel is pretty amazing.

I once created a very modern mobile offline-capable web app for a large enterprise company that was powered by APIs backed by excel. Luckily we didn't have to deal with those directly, but it was a bit of a pain for the poor guy doing th backend.

Would be interested in hearing more about that Excel tool.

It started very innocent: Some consulting company apparently was tasked to create a prototype of the credit rating algorithm. Quite appropriately, I would say, they did so in Excel. As you can easily tweak the logic, show input, intermediate results and output. And all can be manipulated by the client who does not need to know a thing about programming to do so. Great.

From there it went downhill: Apparently someone up the hierarchy thought "Wow, that's 80% of what we need, let's just add a little UI and ship it"

They added a UI, but not as a VBA UI as you might hope. No, they did the whole UI in different worksheets. Storing any intermediate data ... also on worksheets. Long story short, it was a mess, and a slow one for that.

Oh, did I say this was a multi-lingual application?

Production rollout was on Jan 2nd, Dec 31st around 4pm I found a bug in the other language, on the one machine which had the this language configured. I believe this was the only time I ever saw a programmer literally run down the office to that machine to debug this issue.

Come work at a bank. Any bank.

Can attest to this. Some of the things I've seen excel spreadsheets contorted to do horrify me.

I once got spreadsheet dumped on me to debug because it wasn't working. A colleague used the spreadsheet to 'generate' interest rates that were then input into a mainframe. Dug into the VBA spaghetti mess, turns out this 15 year old script that took 15 minutes to run originally hit a bunch of Oracle/External APIs and performed calculations was now just copying the rates in from a csv file on a shared drive.

The error was caused by an excel formula ticking over a new year causing it to look for a directory it did not need to access that didn't exist. I thought it was pretty funny until I heard that the entire asset finance business had been unable to write any loans for days because of this.

In my org excel might have been the start, the intent being to move over to a DB once someone got it stood up. Then nobody would stand up the DB - because not urgent, we have the excel for now. The DB would be completely forgotten, and 3 years down the road everybody would be wondering wtf we have this huge excel file for. Then we would have six weeks of meetings to discuss the file until something else comes up and we forget it for another 3 years.

That's why Excel wins. Because it's right there, infinitely adaptable to any change in workflow you may need to make. No need to send official requests through half a dozen layers of management to get the IT people to maybe, in half a year, add a column to the database. No need to involve 20 people for a year to create new software for a new workflow. You just agree with your co-workers on a change, swap a column, add some color, File->Save, done. It wins because it puts power in the hands of the people doing the work.

I mean, yeah, but pretend Excel is, idk, sword arms. You get a lot of work done by whirling around the office, spinning your swords, cutting off a variety of limbs and heads in the process. You get to the end, meet the few survivors, staring dismayed at the trail of blood and dismemberment behind you, and you yell, "BUT I GOT THE JOB DONE!"

You're not wrong, but...

I wish there were some alternative where we instead fixed our ossified, byzantine processes gradually over time, so that we didn't need to break out the sword arm tornado in the name of getting work done and then, in hindsight, say, "well, shame about the negative externalities, but there really just was no other way. Ah well, let's move on, everyone who isn't a pile of flesh and blood and bits on the floor tidy up. Gotta get everything clean for the next sword arm tornado."

But the negative externalities don't seem all that big, considering. For every problem like this, you get countless millions of person-hours and dollars saved in work that wouldn't have happened at all, or would have been delayed, because of the necessity and cost of creating complex IT systems to support it.

If Excel is a sword tornado, it's one happening in an environment where everyone knows to be super vigilant about sharp objects. The alternative then would be a central processing factory that takes several years and millions of dollars to built, and which has to be turned off for a month several times a year, to change the shape of the blades used by the automated cutters.

> For every problem like this, you get countless millions of person-hours and dollars saved in work that wouldn't have happened at all,

I've found errors in every Excel spreadsheet I've ever looked at, and I'm not some master-excel user; I usually find them because - if I care about the results, I rewrite them as a Python script, so I get to go through everything.

The fact that Excel is effectively not auditable is a huge problem. There are a lot of dollars saved, like you said, but also many wasted or embezzled without anyone noticing in time because past a very low bar of complexity, it's really impossible to figure out what's happening without much, much work.

> If Excel is a sword tornado, it's one happening in an environment where everyone knows to be super vigilant about sharp objects.

... rather, no one is really careful, everyone gets a small cut every now and then, to which the apply a bandage and continue like nothing happened. And occasionally, they lose an eye or a limb or a head - and it is only those cases you read about in the newspapers.

I do not have a better suggestion, I'm afraid, but Excel is causing damage everywhere - e.g. [0], the subtitle - which I'm afraid is not an exaggeration, is "Sometimes it’s easier to rewrite genetics than update Excel"

[0] https://www.theverge.com/2020/8/6/21355674/human-genes-renam...

I agree with what you wrote. It does cause a lot of small mess everywhere, and an occasional big mess. But there really isn't any better suite of tools available that hits all the non-database requirements.

Alternatively...

"We need to store some data for a virus" "Where does the data come from?" "Every hospital sends us an excel sheet" "Well, let's merge all of it into a bigger sheet and display it or export it"

"We should maybe start tracking some of this" "Where does the data come from?" "Dunno, a handful of hospitals... maybe?" "Right. Let's just use Excel, not gonna waste time on that now"

... six months later ...

"Hey um..."

From the species that brought you "The Game of Life" comes "The Game of Technical Debt"

>"We need to store some data for a virus" "Where does the data come from?" "Every hospital sends us an excel sheet" "Well, let's merge all of it into a bigger sheet and display it or export it"

you have no idea how right you are :)

Been there, done that, got the sqlite t-shirt...

Wasn't a bad choice, but once you load 20TB into it, you realize you done goofed.

I think you'd be shocked by how many of our financial and social institutions rely on emailing Excel spreadsheets to one another.

I am absolutely not shocked by that. Most people do not know how to insert into a db yet alone spin one up and add tables. Everyone can excel.

Don't forget healthcare! Excel and CSV for tons of data exchange.

I think the healthcare industry is probably the only thing keeping fax machines alive. Interestingly, fax machine are tagged as HIPPA compliant. Lots of spreadsheets get printed and then fed to the fax machine for transmission instead of email/whatever. On the other side, and i'm being serious, someone gets the fax and creates a new spreadsheet and enters all the rows manually. fun fun.

CSV is somewhat okay-ish. Excel, not so much (thanks Excel for discarding the important leading zeros in this field, and for lossily parsing the other as a date... oh, this Excel file is so big it got the "XLSB" treatment which is hard to undo)

Who? Almost anyone who wanted to get something done without going through the red tape of working with IT. Even in a private corporation that red tape is enough to make many a things end up in excel (or for the more tech savvy, access). Government is surely even worse (better?) at having red tape so the pressure was likely stronger.

Yes exactly. An endless battle: IT want to stamp out spreadsheets and centralise everything but it all becomes an ossified bottleneck of development projects. Meanwhile people need to get stuff done so they use Excel.

Yup. What also helps is that the data literally sits on your drive, or your network drive. No SaaS vendor is holding it hostage.

It doesn't feel like that. Even if I save the file (365 subscription and desktop apps) on my disk it steel feels 'cloudy'.

Of course Excel. What else non-technical users can use? They simply don't have any other data tool.

Excel is perfectly fit for that and is nowhere near the actual issue.

It's lightweight, fast, portable, compatible, and just about everyone can use it regardless of (technical) skill.

Except it wasn’t the government. They outsourced the test and trace system to private companies.

Public Health England was responsible for collating the data which came in the form of CSV's - PHE is a government agency.

It won't be a surprise to anyone who lives in the UK that the problem was blamed on Excel, rather the UK government taking any responsbility for it.

Yea, I have a personal vendetta against XLSX due to having had to make tooling replacements for business logic being encoded in spreadsheets in the past, but the title of this article is incredibly slanted.

This was absolutely a user error and the title "Excel: Why using Microsoft's tool caused Covid-19 results to be lost" is really disappointingly click-baity.

Is your view based on data available to everyone else? If so, that would be nice to see. Its far easier for you to convince others with data. Why would anyone want to change your view?

Could also be a lack of budget funding. The newer version of excel supports: 1,048,576 rows

The test and trace system for the UK was ~£12bn.

> This is indeed user incompetence, but so far every news source quoted the official PR and said it was an "Excel Problem".

It's been quoted/reported as "an ill-thought-out use of Excel", when in reality it's "poor use of an older file format".

There's also a lesson to be had here about validating your data conversions. Especially for critical things like this, it's always a good idea to perform the conversion from the old file format to the new one, and then to go back and extract the data from the new one to compare to the source data.

This can also find issues like Excel cluelessly assuming a field is a date, PHP automatically converting a string which happens to only contain numbers into an integer, and so on.

From what I can tell, that sort of thing, assuming proper monitoring (which is a huge assumption), would have detected this immediately (plus however long until someone notices the error e-mail, etc.).

The way I heard it, it was an Excel problem: They had a CSV file that had more than 1 million lines, and Excel dropped the lines beyond 1 million.

I can imagine an automated process that appended to the CSV and then Excel saving back to that same file with the lines stripped.

I hope we can all agree that this is not the way Excel or any other software should behave, whether the user is incompetent or not.

If you read the article (perhaps its been updated?), they were using the old XLS format and they hit the limit at 64,000 lines

PHE had set up an automatic process to pull this data together into Excel templates so that it could then be uploaded to a central system and made available to the NHS Test and Trace team as well as other government computer dashboards.

The problem is that the PHE developers picked an old file format to do this - known as XLS.

As a consequence, each template could handle only about 65,000 rows of data rather than the one million-plus rows that Excel is actually capable of.

Like I said, that's the way I heard it, the way it was narrated to me.

The Guardian reported my version:

https://www.theguardian.com/politics/2020/oct/05/how-excel-m...

CSV would to me make more sense than an outdated binary format like XLS, but either story is plausible.

Did they pick XLS because it's easier to parse or something? I can't imagine why someone would pick a format that was largely replaced 10 years ago.

Moving from XLS to XLSX will break your macros and would require all machines to be running Office 2007 or later.

Office 2007 was like... 13 years ago, what's the problem?

Compatibility. It's reasonable to predict that some of the participating entities are running software that is at least a decade old.

Indeed

If somebody finds it easier to do this process using Excel than with a scripting language, I wouldn't expect them to know the differences between file formats. I would be surprised if they understood the concepts behind file formats.

They may have chosen XLS because they used Excel back when that was default, and now they don't want to "risk" anything by switching horses mid-stream.

Inertia, most likely. They were using XLS 10 years ago and never saw a reason to change.

If the conversion was failing silently, it's a Microsoft software problem, but I bet you money the conversion was failing in a way that fell into a `catch(Exception e) { }` somewhere in their conversion script. Or even a `catch(Exception e) { log e; }`, and the log is stored to local storage on a machine that no human being ever logs into.

There's nothing to convert, Excel natively supports CSV. It just doesn't read beyond a million lines, it presumably won't warn you about that, and if you save the file again (as CSV), the lines will just be gone.

Reply on news.ycombinator.com