Was there a root cause ever published?
This is compounded by the system being delivered and operated by an outsourced vendor which doesn't have incentives that are aligned with those of the outsourcing organization, and the arms-length relationship to the sub-postmasters.
> As early as 2001, McDonnell’s team had found “hundreds” of bugs. [...] One, named the “Dalmellington Bug”, [...] would see the screen freeze as the user was attempting to confirm receipt of cash. Each time the user pressed “enter” on the frozen screen, it would silently update the record. In Dalmellington, that bug created a £24,000 discrepancy, which the Post Office tried to hold the post office operator responsible for.
> Another bug, called the Callendar Square bug [...] created duplicate transactions due to an error in the database underpinning the system: despite being clear duplicates, the post office operator was again held responsible for the errors.
At the high level, though, the controversy here is not any single bug.
The controversy is multiple bugs, and the failure to properly investigate or fix the bugs, and falsely telling subpostmasters nobody else was reporting bugs, and the software development process that allowed for such bugs, and the private prosecutions, and the people being wrongly jailed and bankrupted, and the justice system giving undue credence to electronic records, and the cover-ups like hiring forensic investigators then calling them off a day before they submitted their reports, and the threats to journalists, and the fact there were reports in computer weekly and private eye for many years, and the fact it's utterly inconceivable that the post office's senor management and legal team didn't know what was going on.
So we shouldn't focus on one or two major bugs and ignore the broader failings of post office managers.
[1] https://www.theguardian.com/uk-news/2024/jan/09/how-the-post...
> Jason Coyne, who worked for Preston-based Best Practice Plc at the time, was instructed to examine the computer system called Horizon in 2003. He said he notified the Post Office the data was "unreliable" but he was ignored, sacked, and then discredited.
https://www.bbc.com/news/uk-england-lancashire-67921974
>The 2016 investigation trawled 17 years of records to find out how often, and why, cash accounts on the Horizon IT system had been tampered with remotely. Ministers were told an investigation was happening. But after postmasters began legal action, it was suddenly stopped.
I always wonder if the high-ups who do these kinds of things actually reflect internally and recognize (at least in their own minds) how much of a Bond-villain they are, or if they really think they are doing the right thing by sweeping things under the rug and getting rid of boat rockers.
tl;dr summary is that the root cause depends on your perspective. If you have a CS background you might define the root cause as being the bugs. If you have a management background, you might conclude the root cause was bad/malign management by the board, the civil service and the C-suite. If you have a legal background you might conclude the root cause was the unusual prosecutorial powers, the default assumption courts make that computers aren't wrong and so on.
As this is Hacker News let's roll with the first. There were many different bugs over many years and there was no single root cause. For example at least one case was caused by defective touch screen hardware that would go haywire when nobody was around to see it e.g. overnight, and start generating random clicks. Given enough random button pushes it was possible to ring up transactions that never happened, and thus never had money deposited in the till, creating phantom losses.
In other cases there was loss of transactional integrity. Database writes went missing or got duplicated due to race conditions, not helped by the original Horizon architecture in which the connection to the central DB was asynchronous, due to the existence of post offices in very rural places without good internet connectivity. It's what would nowadays be called a "local-first distributed edge deployment" or something similar. Lots of ways for that to go wrong, and if you don't have exactly-once execution in an accounting system it will appear that cash was stolen because the records say a sale or refund was made, but it wasn't.
This sounds like an exercise in obfuscating responsibility. Post Office management knew from 2003 Horizon was unreliable - they chose to bury the report. They lied in court about its reliability and manipulated expert statements to say the same. They also knowingly lied about remote altering of accounts and buried a report in 2017 on it. Let's not get lost in the weeds - the root cause of the injustice is Post Office management.
Management surely did not intend the system to be unreliable.
But I see your point, it follows the next question after "why is the system unreliable?" would be "why was the system not fixed?" and the answer is "management knowingly allowed it to be unreliable". But that's no longer a technical reason so it's less interesting.
Replicating transactions across distributed systems is very very difficult. Horizon was clearly not designed to handle this based on the litany of errors users documented. Transactions could be silently dropped or duplicated etc. The system was also insecure and records could be silently remotely altered by Fujitsu staff.
The post office does not have unusual powers to prosecute. What it does have are powers to investigate: https://gunnercooke.com/understanding-the-role-of-the-law-in...
> the default assumption courts make that computers aren't wrong and so on.
This is the most disturbing aspect of it to me. Nothing has been done to change this so it will happen again. It probably happens regularly on a smaller scale.
It does. Ross Anderson's Security Engineering describes how UK banks had customers prosecuted for fraud after criminals exploited bugs in chip-and-pin terminals to complete transactions without entering a PIN. The banks also successfully argued that their cryptographic transaction logs (which could definitively prove if a PIN was used) should not be admitted as evidence.
https://www.cl.cam.ac.uk/~rja14/Papers/SEv2-c10.pdf
I can't find the details about the chip-and-pin case. They might have been on Anderson's blog rather than in the book.
I would find it difficult to believe that these wouldn't have been reported by at least most of the postmasters, but I've never seen any mention of it.
https://www.theguardian.com/uk-news/2024/jan/09/how-the-post...
People? Not yet.
Going to make a wild guess like maybe they should have had more than 8 people working on the "biggest non-military IT project in Europe" at the time.