16,000 coronavirus cases missed in daily figures after IT error | Hacker News Reader

16,000 coronavirus cases missed in daily figures after IT error

bbc.co.uk

58 pointstrwhite32 comments

Open articleView on HN

Specifically, an excel sheet “running out of columns”: https://mobile.twitter.com/MaxCRoser/status/1313046638915706...

That is actually coming from the Daily Mail, and it's not clear where they get the Excel claim from, it's not sourced.

THe only other report I can find is from the BBC:

"It was caused by some data files reporting positive test results exceeding the maximum file size."

https://www.bbc.co.uk/news/uk-54412581

Still not too clear.

but they have mitigated the issue, by spiting the data into multiple files!

It sounds to me much more likely that they ran out of rows - it would seem fairly difficult to split up the files by column

Other recent discussion: https://news.ycombinator.com/item?id=24685911

I think this link is a better source - and the title isn't editorialised as in the other, though this is one occasion where the journos seem to be missing the main point, perhaps waiting for an authoritative source and confirmation?

That's fair & I agree. I'd simply shared it because it was the _main_ headline.

I've hidden it so that people use the other one (also on the front page.)

Yes, to be clear I mean the link in your OP (BBC) is the better source compared to the other discussion.

I think hide will only hide it for you, I just searched on it?

Ah OK, understood. Admittedly I've never hidden anything before

It was caused by some data files reporting positive test results exceeding the maximum file size

Also being reported in various local news items like that in the other discussion linked.

When I read that, I wondered what clowns built it. Now to hear that it was done in Excel I am shocked. I mean, Excel has its place, but in an operational system in which lives may depend?

16,384 column limit, even in the newer versions according to Microsoft! Mind you, for when things get really bad, it does support "Colors in a workbook: 16 million colors".

Maximum file sizes are a curse.

It's very odd though, as the numbers aren't high enough to hit any common thresholds, right?

Anyone have any thoughts on what could have gone wrong here?

It was the maximum number of columns in Excel ....

That somehow makes it even worse

Columns or rows?

Columns, they were using a column per case instead of a row per case.

Is that definitive?

Nope! Actually going off better info now it looks like they WERE using rows, but it was in the old .xls format instead of .xlsx which limits to 2^16 rows

https://twitter.com/standupmaths/status/1313149987707072512?...

Columns.

Row limit is over 1 million.

Honestly, I don't find this that bad.

A global emergency and the biggest news is one country missed a daily record because their futuristic computerized logging system wasn't as optimal as possible.

Like sure, yeah its funny to us that they aren't using some nice GUI over SQL with a memcache, automatically synced with all hospitals via REST and JSON and the tacked on ability to export to excel.

But I mean, think about it. Its awesome what we have too and how accessible all that is.

Serco have been given (without competitive tender) £345M to run this system. I dunno, I expect things to work for that amount of free money.

One of the hard things about computers is that things often work perfectly right up until they don't.

If you're driving a car a little bit too far between maintenance checkups, it might start making some concerning noises, feel off, etc.

But no one not intimately familiar with spreadsheets would think anything is near failure when they have 65534 rows (showing my age, it's a bit higher now, right? :) ).

This has happened to lots of people that "should know better", like when Slashdot hit a 16M comment limit in 2006: https://slashdot.org/story/06/11/09/1534204/slashdot-posting...

For that amount of money I think standards should be way higher than for Bob from accounting who built a clever Excel sheet. I don't expect perfection, but there are ways and techniques to build and run software in a way that minimizes the likelihood of such problems, and this really doesn't make the impression that they followed any best practices at all. They should (but of course won't) be held accountable for that.

> But no one not intimately familiar with spreadsheets

...should be given £345M to run a vital data intensive response to a pandemic?

345M pesos is too much to pay for someone building a system that uses columns in Excel to manage data (btw that's 16M dollars).

It's serco. Them, alongside, for example, Crapita are one of the big corporate disaster companies the government inevitably gives software contracts to and inevitably they fail.

Given their track records it is beyond incompetence, I honestly believe they only care about profits and barely disguise that fact.

That's more than healthcare.gov's colossal screw up to build a site that couldn't register more than ten people or whatever the figures were.

oh lol. not surprised there but yeah its annoying because the rest of us hone our skills an exchange time for much less.

Ok Serco, you just made The List

Whatever happened to the simulation code that advocated lockdown??

Reply on news.ycombinator.com