A TV Show Forced Britain's Devastating Post Office Scandal into the Light
nytimes.com
nytimes.com
- Weeks, months, or years of being incomplete and full of bugs. Gaps are filled and bugs fixed continuously, but new bugs are being introduced all the time.
- At the end -- where the end is a predetermined date and not based on the state of the software -- the software is deemed good enough and put into production
So most software is just barely working, at best, and full of gaps, issues, and bugs, both known and unknown.
This scandal is a case of egregious lack of leadership, both at the post office and at Fujitsu.
But it was also enabled by a belief in the reliability and credibility of IT systems, which is profoundly undeserved and inaccurate.
I found it fascinating to learn that UK law explicitly does (but did not always) encode this presumption:
https://emptycity.substack.com/p/computer-says-guilty-an-int...
(This link is to the substack of David Allen Green, an excellent legal commentator in the UK.)
It's easy to see both sides of the decision to presume or not presume that machines work - we should like somehow to explain that on the one hand it's unlikely that without his intent Jim's iPhone sent Gemma a series of messages threatening to rape her, and so we shouldn't need to drag Apple engineers into court to convict Jim - but then on other hand it's far from certain that this bank database which says I made 590 cash withdrawals in sixteen minutes is correct, more likely a bug duplicated a single withdrawal I made, and we should require the bank to produce evidence otherwise if they insist I made these withdrawals.
I can imagine that smart lawyers motivated by some real world examples and assisted by independent software engineers could come up with a general formula for courts, something a bit larger than either the old rule or the new one, but still relatively pithy. Whether a government would ever make time to do that work is another question.
https://www.prospectmagazine.co.uk/ideas/law/64464/post-offi...
Fundamentally, the scandal is not merely a result of misplaced trust in an IT system. It exemplifies a series of injustices, overlooked for over 15 years by complacent politicians. Crucially, it illustrates the extent of deceit possible when bonuses for senior management and technical staff are at stake.
The UK legal system's appalling quality are starkly highlighted once again. Hundreds of postmasters faced wrongful convictions and prosecutions. Individuals were imprisoned for years, without any concrete evidence of their guilt. A single miscarriage of justice is concerning, but when there are over 250, it raises serious questions about the competence of the judiciary.
Astonishingly, more than 700 postmasters were prosecuted without any executive, manager, or statistician recognizing this glaring anomaly.
The longevity of the issue is also troubling. Even low-level Fujitsu employees, aware of the system's flaws, remained silent. Despite widespread media coverage and a string of wrongful convictions, leading even to suicides, not a single person within a large team exhibited any kind of moral responsibility or something that looked like a conscience. Their silence, just makes you believe again in conspiracy theories, and raises concerns about the integrity of modern IT systems and AI that we rely on.
The simple fact that somebody decided to attest to the absence on bugs on an IT system...An achievement unheard of in the history of software development, and the fact they decided to reiterate that under oath, at a court of law, should be enough for their names to be published and banished for ever, from any profession related to technology.
They were fully aware of the problems and deliberately not acknowledging them.
e.g. the ongoing inquiry has just been shown a document showing the Post Office indicated that it would accept a guilty plea on a less serious charge provided the defendant agreed to accept that there was “nothing wrong with Horizon”.
As far as I am concerned, justice will not have been done until all of their bonuses are clawed back, and at least one of the execs has been smeared on tracks by the tube.
>
> The simple fact that somebody decided to attest to the absence on bugs on an IT system...An achievement unheard of in the history of software development, and the fact they decided to reiterate that under oath, at a court of law, should be enough for their names to be published and banished for ever, from any profession related to technology.
Completely agreed. This is why ethics in our profession are important, and yet all too often dismissed.
That's not true.
>Richard Roll is the real life Fujitsu whisteblower behind the character in Mr Bates vs The Post Office who lifted the lid on the faulty computer system.
I was thinking about this last night. I knew there where about 1000 McDonald's in the UK, Googling it there are about 11,500 post offices. At some point you'd have thought someone would have noticed it was a bit excessive.
I don't think this is a real thing. The individual system was vouched for by senior people. That's why it wasn't questioned any further.
"In 1997 the Law Commission published a paper which went into some detail about the use of mechanical and computer evidence in court. It seemed a little too fixated with the effective workings of speedometers, traffic lights and breathalysing devices called ‘Intoximeters.’ It concluded that the present law is ‘unsatisfactory’ because of the necessity for prosecutors to ‘prove that the computer is reliable.’"[0]
The amended law changed the burden of proof from the prosecution proving the system functioned correctly to the defence proving it didn't, without access to the systems being used to prosecute them.
[0]: The Great Post Office Scandal, Nick Wallis
It is mind boggling that someone thought it is OK to put this into law. What happened to the idea of innocent until proven guilty?
(not that it’s fundamentally just with speeding tickets either really! but it's also kind of understandable, you're the 27th person the judge has heard today trying to weasel out of a speeding ticket with the same set of "but the machine could have been wrong!" excuses. And frankly the brits seem to be a lot more "pragmatic" about individual right vs societal ones... the political class there seems to respect the people there exceptionally little even by political-class standards lol)
he breached his bail conditions and was rightly jailed for it
Judges in the UK have been so impartial they publicly made statements of him being a narcissist, not even pretending to even appear impartial. Nils Melzer, the United Nations special rapporteur on Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment, concluded after visiting Assange in prison is treatment was nothing more than torture...
Yeah totally normal...for the UK legal system...
it turns out there's consequences for jumping bail
The fact that the Fujitsu employees had complete and un-audited access to all the remote terminals should have made the computer testimony unreliable, but of course the post office lied about that and denied that anyone had remote access.
In the case of the sub-postmasters the Post Office, as far as I'm aware, never proved where these stolen sums supposedly went. The computer evidence was thought terminating and was the only thing (except false confessions under duress) used to secure these convictions, rather than proper investigative work.
It's crazy auditors didn't spot discrepancies especially with the high base rate of reported frauds and errors, but if finding the receipts for stolen money was the threshold for prosecution any remotely competent thief would be in the clear
If there is absolutely nothing to corroborate the money going missing, no evidence presented for where or how the money was stolen then there shouldn't be enough to send someone to prison.
The amounts of money involved may be small to business owners in other domains but for SPMs many were almost bankrupted by trying to replace the sums out of their own earnings as you say. This wouldn't have been a rounding error to their lifestyle, it should have been provable to any halfway decent investigator. And if not? Then they get away with it and it's the price we pay not to live in tyranny.
Well, yeah. A defendant should be able to argue that because that can (and did) happen and it's bad to routinely wrongly convict people.
I thought when you called it a "tricky one" you were expressing that it might be a bad thing if it were difficult to convict someone based primarily on audit logs.
But if you don't want people to be wrongly convicted, then surely that's a good thing, right? As we know, there's no guarantee a particular audit log is correct.
Think of it like a diagnostic test, like covid tests. That sort of test has 2 measures, not one (anyone who just says "Test X is 95% accurate!" is selling you something) - specificity and sensitivity. Sensitivity is the percentage of true positives it generates out of all positives, and specificity is the percentage of true negatives it generates out of all negatives.
I don't want people to be wrongly convicted, no, so I want legal tests to have a very high specificity. But I could do that easily, by just throwing every case out as not guilty. The hard bit is raising sensitivity at the same time. You can't just say "if you don't want people to be wrongly convicted", because that justifies far too many things.
> As we know, there's no guarantee a particular audit log is correct.
There's no guarantee anything is correct. Three witnesses could have colluded and someone might go to jail for it, but unless there's a reason to think they colluded, we don't assume that. That's the problem I'm talking about: how do we get a feel for software systems without assuming like the Post Office that they work, or like you that they don't work?
Well, how the Post Office commissioned the Horizon system is something that the inquiry doesn't seem to have got to yet.
From what I've seen of the inquiry (not much), Post Office managers seem to have been desperate to protect the reputation of the Horizon system, and still are. But the miscarriages of justice have now been exposed, and the costs of putting them right are now unavoidable; so the reputation of Horizon can't now be just a matter of subpostmasters.
My guess is that defects in Horizon, and damage to Horizon's reputation, will turn out to affect much more than the wronged subpostmasters. I suspect that some very large commercial contracts will turn out to be affected, and paying compensation to the subpostmasters will turn out to be a tiny component of the costs of this catastrophe. The Post Office is already warning that it may not be able to pay the compensation, and that the government will have to step up.
The Post Office is a weird mess of subsidiaries and holding companies, and has contracts with all sorts of 3rd parties. I anticipate that we are going to witness a slooow explosion of the Post Office, over like ten years. It's ludicrous, of course, that a private company should be able to be complainant, investigator and prosecutor as well as being sole custodian of the evidence (Horizon logs).
It's also ludicrous that the Post Office's lawyers, Cartwright King, should describe requests for disclosure as a "fishing expedition". Defendants shouldn't have to request disclosure of evidence that might help the defence; that is an obligation that falls on the prosecution as a matter of statute law.
I'm waiting for the inquiry to find out how the Horizon system was commissioned, and how it was tested; and who was in charge of commissioning.
It was only subsequently that that believe changed to outright lying to cover their own backs.
As can be seen by following the story, this wasn't the case. The only evidence provided in criminal cases was essentially a witness statement saying that the system was working correctly and that was the end of it.
I'm not saying it wasn't. You're assuming a blind faith in all IT is the cause of this witness' belief. I'm disagreeing with you there.
> it was also enabled by a belief in the reliability and credibility of IT systems
The obvious mistake most likely made here is these senior people where not asked , " exactly how they formed this opinion." and what specific evidence they had.
I would have said , if they could NOT produce the evidence , that verified their statement, their is no way they could make the statement as accurate about the system accuracy.
The people must not have had experience in software development and testing.
Fujitsu was a contractor; the contract was a PPI (Public-Private Investment) contract. The deal was that Fujitsu developed the system at their own cost, and ran it in exchange for a cut of each transaction.
I know something about this work; back in the 1980s, I worked on a bid for what was then called Post Office Counter-mechanization. This was probably the third attempt to automate Post Office counters; previous attempts had failed. FWIW, we lost the bid; I think it was won by ICL, the forerunner of Fujitsu. That third attempt failed, of course.
To work on this bid, I was loacked away in an office on my own for 6 months. The requirements were delivered as two metres of lever-arch files, full of paper forms; each form described an input screen, or a report, or a processing operation, or a storage requirement. 90% of these forms were blank, apart from the name of the operation. It was impossible to do a serious job when so little of the required data was supplied. And we weren't allowed to talk to the Post Office about the requirements; that would have risked the entire tender being overturned in court.
If the Fujitsu PPI contract was handled in the same way by the Post Office, and I assume it was, then it would have been impossible for Fujitsu to deliver a working system. But now we see Alex Chalk, the (current) minister in charge of the Post Office, suggesting that the compensation money should be extracted from Fujitsu. It may turn out that Fujitsu have some hard questions to answer; but for now, it looks as if this mess falls entirely on the shoulders of incompetent Post Office managers.
They knew the software had many bugs but decided to use that to bully their employees and steal money from them.
I don't really see how this argument fits. In my understanding the amounts in question were in the range thousands to tens of thousands. It seems implausible that those sums of money, while life changing for the individuals involved, would ever move the needle at all for any of the companies involved. In fact the only people who made money out of it are likely the lawyers.
Presumably those people potentially faced contempt of court charges if they continued to blame the system.
I don't think anyone was trying to do the wrong thing but rather timelines and budgets were tight.
In some industries, like aviation (at least in EU) it appears to be more understood, but in general it doesn't.
> Imagine joining an engineering team. You’re excited and full of ideas, probably just out of school and a world of clean, beautiful designs, awe-inspiring in their aesthetic unity of purpose, economy, and strength. You start by meeting Mary, project leader for a bridge in a major metropolitan area. Mary introduces you to Fred, after you get through the fifteen security checks installed by Dave because Dave had his sweater stolen off his desk once and Never Again. Fred only works with wood, so you ask why he’s involved because this bridge is supposed to allow rush-hour traffic full of cars full of mortal humans to cross a 200-foot drop over rapids. Don’t worry, says Mary, Fred’s going to handle the walkways. What walkways? Well Fred made a good case for walkways and they’re going to add to the bridge’s appeal. Of course, they’ll have to be built without railings, because there’s a strict no railings rule enforced by Phil, who’s not an engineer. Nobody’s sure what Phil does, but it’s definitely full of synergy and has to do with upper management, whom none of the engineers want to deal with so they just let Phil do what he wants. Sara, meanwhile, has found several hemorrhaging-edge paving techniques, and worked them all into the bridge design, so you’ll have to build around each one as the bridge progresses, since each one means different underlying support and safety concerns. Tom and Harry have been working together for years, but have an ongoing feud over whether to use metric or imperial measurements, and it’s become a case of “whoever got to that part of the design first.” This has been such a headache for the people actually screwing things together, they’ve given up and just forced, hammered, or welded their way through the day with whatever parts were handy. Also, the bridge was designed as a suspension bridge, but nobody actually knew how to build a suspension bridge, so they got halfway through it and then just added extra support columns to keep the thing standing, but they left the suspension cables because they’re still sort of holding up parts of the bridge. Nobody knows which parts, but everybody’s pretty sure they’re important parts. After the introductions are made, you are invited to come up with some new ideas, but you don’t have any because you’re a propulsion engineer and don’t know anything about bridges.
> Would you drive across this bridge? No. If it somehow got built, everybody involved would be executed. Yet some version of this dynamic wrote every single program you have ever used, banking software, websites, and a ubiquitously used program that was supposed to protect information on the internet but didn’t.
> You can’t restart the internet. Trillions of dollars depend on a rickety cobweb of unofficial agreements and “good enough for now” code with comments like “TODO: FIX THIS IT’S A REALLY DANGEROUS HACK BUT I DON’T KNOW WHAT’S WRONG” that were written ten years ago.
It's not all programs, but the ones that last (you know, the ones people actually use) get more and more like this over time, no matter how they start.
It's out-of-date though... allow me to fix it:
> ...with comments like “TODO: FIX THIS IT’S A REALLY DANGEROUS HACK BUT I DON’T KNOW WHAT’S WRONG” that were written *twenty* years ago.
(It was written in 2014, after all.)
There are still other scandals out there that aren't getting the recognition they deserve (Infected Blood Scandal[1] being a big one but I'm sure there are others I'm not even aware of). So what can we do to ensure our politicians empower these campaigners instead of extinguishing them?
1 - https://www.theguardian.com/uk-news/2024/jan/10/infected-blo...
I had the same thought. A while ago I heard about a so-called "scandal" in passing, and didn't think of it much. I recently listened to the BBC podcast series, and I felt both sand and outraged. I had me thinking, "does it take a TV drama for something like this to be recognized?". Now every politician is behind it (finally). Where were all these guardians before? If I find myself in a similar situation, should a no-body like me expect a documentary to be made?
This is by design. A representative democracy is not intended to do what the people want. The representatives know the people are ignorant, obedient, comfortable, and scared. So the people just stand there mouths agape while the representatives do (or not do) whatever they want. The only consequence comes from other representatives. No representative fears the people. They're sheep.
This should be something for the HN community to take note, as this scandal is for Fujitsu the chickens coming home to roost for an issue that started in 1999. We as software developers, the maker of the tools and the facilitators of change and advancement have responsibilities, and those responsibilities can have wide reaching consequences.
We saw in Myanmar that not understanding the locality and not having adequate moderation can have consequences, we see with this case that absolute faith in a system can be damaging and mixed with other factors (such as the UK's obscure Draconian laws around the mail service being able to prosecute people) can be catastrophic.
So, friends, when you are approaching a deadline and you feel as though you need to ship even though you know the product is bug riddled, just take a moment to write some extra tests or to document the shortcomings more thoroughly. Someone you don't know may be depending on it.
So none of the criminal trials ever got as far as actually looking at the disputed figures, because the defended would plead guilty or the PO would just lie about how the system worked to "prove" false accounting. As Justice Fraser wrote in the 2019 common issues judgement:
"This means that, even for disputed items going back as far as the year 2000, this litigation is the first time that there will be any independent consideration of disputed items showing in the branch accounts for the vast majority of the Claimants."
Indeed many of the disputes were not even caused by software errors but human error on the part of the SPM. Errors that they attempted to resolve but could not due to the way PO handled disputes.
So this is far more than just a code quality problem.
Honesty comes before all else.
Public Function ReverseSign(d)
If d < 0 Then
d = Abs(d)
Else
d = d - (d * 2)
End If
ReverseSign = d
End Function
See page 17 onwards
The 'server' piece seems to be based around an XML message store (hey it was the 90s dont forget, XML was the big new thing) called Riposte build by Escher Group.
There were some touchscreen EPOSS terminals for the post offices - these may also have been part of the Riposte system. Then there was a bunch of other stuff built around that and written in VB6 (which for the time was a reasonable choice unless you wanted to go Java or C++) running on NT workstations.
Syncronisations from the thousands of post offices to the central server seemed to be via scheduled programs that picked up local message files (probably xml) and sent them to the central server over ISDN lines.
Centrally there was something called the 'cash account' which pulled in those messages and (I guess?) added them to the riposte xml message database. As mentioned in the witness statements, they had no agreed data dictionary or schema for all the XML stuff so no centrally agreed scheme of how to represent all the different messages. So somewhere something was probably written to 'transalte' all these different messages into some nominally agreed standard.
I've watched some parts of [1] on youtube, and it seems the riposte system is on the machines in the post office. if it is also used in headquarters, I don't know. they have messages that have incremental sequence numbers to identify the replicates messages.
messages are put into the message store where each message has a checksum. when a message is read and gives a CRC error, someone raises a call, and they rebuild the message store. which recalculates the checksums, I assume.
CRC errors can be caused by data corruption on the counters. so this implies data is stored on the counters. when they rebuild the message store it seems they can use data from the other counter which hopefully has no corrupted message (but the former software engineers can't quite remember if this was true, at least she says so). so it seems they have redundancy between these counters. when CRC erros happen often, they'll replace the counter.
I also assume they describe the Horizon system pre-2010.
[1] "Anne Chambers - Day 67 AM (26 September 2023) - Post Office Horizon IT Inquiry", https://www.youtube.com/watch?v=q72zvd9Iz3Q
https://web.archive.org/web/20000706232447/http://www.escher...
"Any fear of complete local failure is alleviated by the fact that each workstation can recover from a central location."
And also, er ...
"Riposte protects against fraud by providing a clear and complete audit trail for every transaction."
The concept looks really nice on paper, also the network communication considerations [2]. But there were bugs in Horizon like when a notebook at a post office was shutdown before a certain time, there were no day-end-markers added to the last transaction, and it seems they had no concept to let the system check this on the next day. There were bugs where the start times for transactions where missing, and these transactions were filtered in certain views however these messages eventually were transmitted.
[1] FUJ00080526 - Fujitsu Report Horizon Data Integrity v1.0, https://www.postofficehorizoninquiry.org.uk/evidence/fuj0008...
[2] WITN00780100 - Richard Roll - Witness Statement.pdf >[...]there were several different types of configuration, depending on the type of PO (mobile, single counter, multi counter or main PO with a separate gateway computer), but from the SPM's perspective these would have all looked the same. >General security protocols were in place at the PO's. Secure passwords were required, which were not supposed to be shared but often were. There was a secure link from the PO counter to Fujitsu's servers —the counters would only respond to requests originating from specific telephone numbers —and all data was encrypted. Additionally, the network was completely (logically) isolated form the internet, it had its own dedicated lines[...]
Computerphile video from 2 years ago about the bugs: https://www.youtube.com/watch?v=hBJm9ZYqL10
Fujitsu casestudy PR document from 2004, still on their website! with a bit of high level info and bumf about the system. Has a few photos of the hardware.
https://www.fujitsu.com/downloads/SVC/fs/casestudies/uk-post...
This system audit manual from Jan 2000 looks like it has a lot of detail but I havent had the chance to read it yet
https://www.postofficehorizoninquiry.org.uk/evidence/pol0002...
number = -number
They actually point this out when introducing the code snippet in the report.
And then after the snippet:
>> Whoever wrote this code clearly has no understanding of elementary mathematics of the most basic rules of computer programming.
Second thought: good job posting this, now it will be accepted into the corpus for GPT5
By comparison, this drama was only a few hours long, was aired while people were still in a holiday/relaxing mood after Xmas and New Year, and spun a compelling narrative about the people and communities affected. It had much clearer "goodies" and "baddies" (without being unfair). So it deserves credit in reigniting interest in the scandal.
Fujitsu is just the current name of who owns ICL, International Computers Limited. ICL are British through and through. This entire scandal is "Fujitsu's UK subsiduary", i.e. ICL
ICL were established and got into UK Government IT procurement in the 1960s, because Harold Wilson and Tony Benn didn't want to be beholden to the Americans (IBM) for these new-fangled computer calculator thingies.
ICL would supply both the _computers_ and the _software_ to British government. It still runs mainframes for the Inland Revenue and Department of Work and Pensions.
By the 1980s, they were still running but they were using Fujitsu components and entire PC systems, and by the late 1990s, Fujitsu had acquired ICL... but ICL is still ICL (with a new name) and is still incredibly chummy with the UK government and still wins contracts for government IT procurement. None of this scandal happened in Japan. It happened in Bracknell.
I'm waiting to see a bunch of Post Office senior management in court on criminal charges.
1. Here's a quote from a 2001 (yes, 23 years ago) report by Fujitsu/ICL into how badly developed their Horizon system was:
https://www.postofficehorizoninquiry.org.uk/sites/default/fi...
> Although parts of the EPOSS code are well written, significant sections are a combination of poor technical design, bad programming and ill-though out bug fixes. The negative impact of these factors will continue and spread as long as the PINICL fixing culture continues. This is partly due to the nature/size of the bug-fixing task and partly due to the quality and professionalism of certain individuals within the team. The problem is probably best illustrated [by] examples:
> Example 1: This extract from EPOSSCore.dll has been written to reverse the sign of a number and is equivalent to the command d=-d
Public Function ReverseSign(d)
If d < 0 Then
d = Abs(d)
Else
d = d - (d * 2)
End If
ReverseSign = d
End Function
2. Here are the juicy details about the overall architecture:
> The Fujitsu-designed Epos software on the PCs was written onto an off-the-shelf system called Riposte.
> Our source said the big flaw in Horizon was the way data was being written to Riposte.“Riposte wasn’t really a database, it was a messaging system based on an XML structure where you write messages down into the message store, and then Riposte took care of replicating them,” he said. “The first thing that you should always do with a system like that is design and agree a data dictionary and a message library repository, basically to say: these are the messages that are allowed to be written to the message store and they all provide the following function.
> “You should also have a layer of software that lies on top of the message store that checks that any application above it which is trying to write a message, conforms to the agreed data dictionary. Otherwise, you can just write freestyle to the message store, which is what they were doing. There was no application interface in there, no agreed data catalogue or anything.”
> Computer Weekly also spoke to a former Fujitsu employee who worked in the Horizon Service Support Centre from 2001. The support engineer – who also wants to remain anonymous – recognised this new description of Horizon’s badly built message store, adding: “Our job was to fix these problems as they arose. We all knew the code wasn’t fit for purpose and needed rewriting. The data dictionary was still being added to when I got there.”
> For the first 10 years of Horizon’s existence, transaction and account data was stored on terminals in each branch before being uploaded to a central database via ISDN. Our source says this part of the system simply did not work.
> “The cash account was a piece of software that sat on the counter NT box, asleep all day,” he said. “At the end of the day, or a particular point in the day, it came to life, and it ran through the message store from the point it last finished. It started at a watermark from yesterday and combed through every transaction in the message store, up until the next watermark.
> “A lot of the messages in there were nonsense, because there was no data dictionary, there was no API that enforced message integrity. The contents of the message were freehand, you could write whatever you wanted in the code, and everybody did it differently. And then, when you came back three weeks later, you could write it differently again.” He gave an example of a message stored previously when a customer bought a stamp. It was feasible that a new message for buying a stamp weeks later could be slightly different.
> “When the cash count came along, it found a message it was not expecting and either ignored it, tripped up, or added something it shouldn’t be adding,” he said.
I think we can see that this sentiment has wider support in the software community - I would put many NoSQL database adaptions into this category. Another interesting development that shares this sentiment is frontend developers desire to bypass backend layer and get direct access to the data to "easy" and "speedup" their development.
This is the software equivalent of a The Onion article.
The world runs on badly written software. I have even written some highly incompetent code myself for essential services. Finding terrible bugs and then looking for who caused them has almost humbled me once or twice.
They did not care about data integrity and schema migration. It's probably the most incompetent thing one can do when working with data in such a system.
That's why we define database schemas, or write protobuf schemas, or avro schemas, and parse JSON data in the frontend with zod.
As a rule of thumb you don't even just change the datatype of a field and hope it works. You introduce a new field and migrate over the course step by step from the old field to the new field, i.e. you write the data into two fields, and read the data from the new field or fallback to the old field.
Actually their use of the Riposte message system in 1999 was not that different in what we use today in many cloud-based systems with microservices.
What's wrong with code above is 1) it does a conditional branch (meaning if.. then... else...) which is unneeded. 2) The "then clause" does an absolute value which is a waste of a function call. 3) The "else clause" does `d = d - (2 * d)` which is a waste of a multiplication 4) possibly with an overflow risk and/or floating point estimation risk.
That said, it's admittedly possible that the programming language above, or the number representation in that code above, have some highly atypical needs that we here on HN don't understand.
Even when these code snippets are cherry-picked they tell enough about their review practices.
>have some highly atypical needs that we here on HN don't understand.
Somebody else checked in another thread. You can use unary minus in VBA.
[1] >One member of the development team, David McDonnell, who had worked on the Epos system side of the project, told the inquiry that “of eight [people] in the development team, two were very good, another two were mediocre but we could work with them, and then there were probably three or four who just weren’t up to it and weren’t capable of producing professional code”.
Also I think the code could just multiply d by -1 which would "reverse the sign".
Even that has some edge cases that should be handled around the maximum value of an integer.
But the main thing to me is that as written this code is confusing and overcomplicating the easy part of the problem.
Post office should have used your Turbo Pascal-based system, the post master would not be forced to become a taxi driver.
This reminds me I always wanted to build a fictional cashier system with terminal interface in ncurses, after I saw these still used in early 2000s in DIY stores. Maybe a local post office system would also a great side project for the weekends.
If folks here don't share this view in their day-to-day work, then they are part of the problem. The act of writing software does not exist in a vacuum and software engineers are not too stupid to be able to consider the wider consequences of what they are doing.
In other professions, acting honourably actually matters and there are professional bodies which actually give a damn about the ethics and actions of their members. I'm not sure why it's okay for software engineering to remain in the dark ages.
There definitely needs to be some heads rolling for perverting the course of justice if nothing else. I'm surprised that the court system is happy with the Post Office using them as their attack dogs.
Why specifically them? Because both gave evidence in Post Office prosecutions of SPMs that Horizon did not have flaws, then in the Bates v Post Office Ltd case, their support ticket logs (PEAKs) were finally included in evidence. These work tickets show both of them acknowledging bugs that create false accounts, and manually correcting data in the records and audit logs to balance it out... and also sometimes getting the "fix" wrong without noticing... and noting that they saw no need to inform the SPMs.
https://www.postofficetrial.com/2019/03/the-smoking-gun.html
The very things SPMs said were happening and the Post Office and Fujitsu denied. They fucking knew it. They testified that the things they did day-in, day-out never happened, and SPMs were convicted and sometimes imprisoned as a result.
Obviously there are a lot more people at both Fujitsu and the Post Office that need to see justice, but this is a start.
> The Postmaster reported that the problem at his one specific site had been been happening intermittently all year. He told NBSC who sent experienced trainers out to ensure the Subpostmaster wasn’t doing something wrong. The trainers concluded the Subpostmaster was doing nothing wrong and witnessed the error happening. The auditors, who were incentivised by the Post Office to suspend Subpostmasters with discrepancies (another story), came out. They concluded the Subpostmaster was doing nothing wrong and they witnessed the error happening.
> Chambers went to work. She delved into the system and reported: “have checked the system figures… and can confirm that all the variants reported since then have been calculated correctly. There are no known issues that would result in the variance being incorrect.”
> Chambers closed the ticket with a definitive: “No fault in product”.
> The cause of the defect was assigned to “User” – that is, the Subpostmaster.
> When Beer asked why, Chambers replied: “Because I was rather frustrated by not – by feeling that I couldn’t fully get to the bottom of it. But there was no evidence for it being a system error.”
...
> Chambers conceded: “something was obviously wrong, in that the branch obviously were getting these discrepancies that they weren’t expecting, but all I could see on my side was that they were apparently declaring these differing amounts, and I certainly didn’t know of any system errors that would cause that to happen, or that would take what they were declaring and not record it correctly…. so I felt, on balance, there was just no evidence of a system error.”
> No evidence. Williams pointed out that it surely was unlikely to be a user error if both trainers and auditors had recorded the Subpostmaster as inputting information correctly. Chambers replied:
> “Well, yeah, I… yes, I don’t know why… I’m not happy with this one. But I still stand by there being no indication of a system error and the numbers that they were recording just didn’t make a lot of sense.”
Print-outs of the support system tickets are shown in the inquiry [1] (e.g. the ones with former technical support engineers). For instance, there is one from October/November 1999 with conversation history of people from different levels of support, where you can see they (the support and software engineers) knew the exact amount of money missing and that the caller was "very agitated as he keeps having problems with the system when balancing. He thinks it is a system problem".
I haven't seen support system tickets where they stated the discrepancies were caused by Horizon in the inquiry (obviously in court they proved there actually were), but I only watched a few hours of this so far. And the support engineers stated they did not grasp the implications of the callers having discrepancies.
The support engineers even visited the callers on-site.
[1] https://www.youtube.com/@postofficehorizonitinquiry947/video...
In both cases, if I lied by, for example, saying that I had evidence that I did not, I could and should be charged with a crime, in the very least making a false report and/or making a false statement to the court.
Yet we let companies do this with IT systems all the time. Hertz famously accused people of stealing cars that they did not [1]. The same principle should apply: whoever OKed such a system should be charged with a crime.
All these systems make the same mistake: they want to replace humans. If you want to do that, you should be on the hook legally for the results but really it shouldn't be allowed.
It's completely fine if Hertz designed a system that flagged people for not returning cars (ie stealing them). That would get sent to a person. That person would review the information, check if it was an error or not and, if not, file a police report. That human would be responsible for maaking a false police report in the case of an error. Any errors could then be addressed as bugs with the systems provider.
Likewise, for Horizon, by all means automate the review of financial records. Find what you think are fradulent transactions. That gets spit out to a report. That report may then be investigated by, say, a financial accounting firm who is responsible for determining the facts and, if necessary, taking legal action.
You see how this version aids a person doing this job and the actual versions of these replaces the humans? A human can't necessarily review every transaction of the postal system so automating fraud detection makes sense.
What doesn't make sense is allowing the systems provider and the government agencies that enabled this off the hook because of "bugs".
[1]: https://www.npr.org/2022/12/06/1140998674/hertz-false-accusa...
The subpostmasters were signing statements that their accounts were accurate, their charges were mostly related to these sorts of collateral lies rather than the theft. So in a uselessly pedantic and literal sense they often had performed the wrongful act they were accused of, though this ignores that they were more or less forced into it and sometimes explicitly told by tech support to do so.
In specific cases the financially disabled and often unrepresented subpostmasters made legal errors. For example, instead of defending themselves by responding that the Horizon figures were inaccurate they defended themselves by saying they could prove they didn't take the funds from their own books. Essentially allowing the burden of proof to be reversed.
So subpostmasters signed affidavits to the accuracy of their accounts. The burden of proof should still be on the government to substantiate that those claims are false and there is material harm and not, for example, just a bookkeeping error.
Particularly when a system is new, we all know that it has to be verified. For migrations that typically means comparing the results to the old system and then manually looking at discrepancies. In the case of a new system, it should be the bare minimum standard that before a person is accused of a crime, the system is verified, in this case by human review, probably by an auditor or financial accounting firm.
The crime being the false affidavit (even here you have the government has the burden of proof to show it was knowingly false) rather than any alleged theft is pure theater and a cop out. Like you sign a statement saying you didn't steal money and I charge you with lying. Shouldn't I have to prove you were lying? Isn't that basically the same as proving the theft occurred?
If subpostmasters can be charged and/or sued for making false statements by a computer system, why can't someone be charged for the false statements that computer system made?
At best they were in a weird catch-22 where disputing the records is an admission that they knowingly filed false accounts (itself a crime, regardless of they funds owed or not), and at worst they indisputably entered false accounting records (though for understandable reasons and sometimes at the direction of tech support staff).
Part of the reason justice was possible at all was because Alan Bates simply refused to sign the false reports. He got shut down and suffered tremendous hardship as a result, but he wasn't litigated against-- which left him in a position to drive forward justice. ... and even that required whistle blowers and leaks to force the disclosure of documents that were wrongfully being concealed.
I can't emphasize enough that I'm not saying any of this to attribute fault to the subpostmasters or deflect it away from the post office or Fujitsu. I don't think it does.
But I think it's important to have a more comprehensive understanding of what went wrong so as to understand how situations like this can be prevented.
To that end understanding how the situation made some of the subpostmasters guilty (or at least in violation of their contract) in narrow a technical sense is important, because it highlights how the judiciary and its duty bound actors (the solicitors, barristers, and independent experts) operated in a -- for lack a better word-- autistic and rule bound way that yielded results which were completely incompatible with justice.
The SPMs were forced to sign and take responsibility for accounts that were prepared by the Horizon system, which claimed the SPMs owed the Post Office tens of thousands of pounds of imaginary money. If they later claimed that the accounts were wrong (which they were), they'd be prosecuted for signing them, and if they didn't do that they would be prosecuted for not paying the PO the money they supposedly owed. So they would lose either way, and the actual accuracy of the accounts was never tested in court, because in any individual case the PO did not disagree with the SPM about that.
The point being that this can't be blamed on software bugs alone. It was caused by different departments of the PO having completely contradictory ideas about how the software was supposed to work.
Were bugs actually involved? I could only find this sentence in the article and it leaves it open if it was a bug or an user error.
>Many of the Hertz cases involved customers who had called to extend their rental agreement, but the extensions were not properly reflected in Hertz's computer systems.
Regarding software bugs in Horizon: Some of the bugs are mentioned in the early chapters in the book by Nick Wallis. I first read an article by the Guardian which only mentioned two of the bugs (as far as I remember) but also said many other bugs were discovered [1]. I first thought these were mostly UX issues [2] [3] that caused user input errors, but reading some of the inquiry reports and the book made it more apparent that there were system bugs, e.g. [4] on top of a sub-optimal message system [5]. Of course, you can say UX issues are also some kind of bug, error or defect.
[1] >As early as 2001, McDonnell’s team had found “hundreds” of bugs. A full list has never been produced, but successive vindications of post office operators have revealed the sort of problems that arose.
[2] >“The Dalmellington Bug entailed a user repeatedly hitting a key when the system froze as she was trying to acknowledge receipt of a consignment of £8,000 in cash. Unknown to her each time she struck the key she accepted responsibility for a further £8,000. The bug created a discrepancy of £24,000 for which she was held responsible.”
[3] >"If a bunch of products sit in a basket for long enough on the screen Horizon will turn them into a sale automatically." -- ‘It is an example of Horizon doing what it was supposed to do.’ -- ‘It is evidence of the system doing something without the user choosing to do it.’ retorts Mr Coyne. >"This stance of blaming the users if they were confused by Horizon’s design was taken to an extreme with “phantom transactions”. These were transactions generated by the system but which were recorded as if they had been made by a user"
[4] >In April 2003, Horizon version S30 was released by Fujitsu. It had not been properly tested, and it introduced a bug known as the Reversal bug. If a Subpostmaster inputted a series of transactions, and then for some reason wanted to cancel them, the transactions would have to be reversed out of Horizon. The Reversal bug appeared to reverse a transaction, but in fact doubled it. This was due to a plus sign in the code which should have been a minus sign. The Reversal bug was discovered by Fujitsu after Subpostmasters began to complain about the growing amount of non-existent money Horizon calculated as being in their accounts at the end of the week. When one Subpostmaster reversed an internal £13,910 cash transfer out of one stock unit and into another, he managed to create a £27,820 negative discrepancy in his accounts.”
[5] > "The biggest problem, however, was the information being written to something called a message store.” >“According to Clint, Horizon’s developers had spent the last couple of years ‘firing any old shit’ into the message store as they changed and developed the product. Clint explained that this was a recipe for disaster. If a message was given a new data field, but this wasn’t retrospectively applied, the counter software could retrieve an old message with what it would now perceive to be a missing data field. The reaction of the application would be unpredictable.”
My answer is that information can easily be buried by the media, courts, politicians. The purpose of media is to guide not inform - there is nothing natural about the news.
The more interesting question in my mind, is why is this news now? It can't be that a genuine issue has filtered into the public consciousness via a dramatised TV show.
Is it that the show and surrounding dreams is intended as a distraction? Is something in the drama 'reframing' reality? Perhaps it is intended to show how 'the justice system may be slow, but still calls people to account' ie an attempt to shore up faith in our system? Is the loss of someone's CBE (an honour) intended to be a lightning rod to channel popular discontent down a cul de sac?
This event requires a media analysis.
If the drama enables people to question whether the result their software gives them is correct, then it does a good deed indeed. The justice system being that slow is an indicator that it needs more resources.
The UK honour's system... Liz Truss' most chaotic seven weeks in UK politics and she has the impudence to even consider, let alone release the traditional honours list of a Prime Minister leaving office. https://www.bbc.com/news/uk-politics-67843755
Strongly disagree, for examples: https://www.bbc.com/news/magazine-17820571.amp
(also one underestimates the sheer presence of Toby Jones at one’s peril)
It seems you have missed my point though. Your example strongly supports my thesis that this event requires a media analysis.
From your linked article:
> It was in 1951 that BBC radio and the Ministry of Agriculture created The Archers, an "everyday story of country folk", which encouraged farmers to try new techniques to increase productivity in the years after WWII.
> "It's one thing to read a persuasive argument in print it's quite another to be persuaded by the power of the human voice," says Andrew Crisell, Professor of Broadcasting Studies at the University of Sunderland in the UK.
TV shows, news, the BBC, ITV, papers, etc, etc more broadly "stuff on screens" are used to deliver a message. Media literally mediates reality for you - this is how governance occurs. One is given a message, and then most people respond appropriately. (Obv. psychological 'tricks' are known and applied, eg reverse psychology.)
The question for me is what is the message that the media is attempting to be convey in this case? Answers on a postcard.
Why not? That's exactly what has happened.
Except that is exactly what has happened. The story was widely reported, nobody cared. Footballers wives fighting in court, people care. The royals, people care. Celebrities eating bugs in the jungle, people care.
If a person affected was on love island or dancing with the stars, people would have cared.
1. The public inquiry has begun digging into what actually happened.
2. An absolutely excellent TV drama has brought out in excruciating detail the misery inflicted on subpostmasters, and the unfairness of the processes.
The returned CBE is a consequence of the drama and the resulting public interest, not a cause.
I knew that this nonsense was going on, in a sort of subliminal way; I knew about the miscarriages of justice, from the MSM. The drama has - well - dramatized it. The drama's producer has expressed astonishment at the impact her show has made, but I'm not astonished. It's a really good drama. It pulls no punches. It must have been really hard to make, given that many of the people criticized are very powerful and rich, and that criticizing the rich and powerful can land you with very large bills to defend libel suits.
This event requires a rethinking of responsabilities and the power structure in the Post Office (but probably in all external goal oriented orgs). What a media analysis, in all its sloppiness, will do is make people believe there's a format for exposing scandals that can be used at will.
The software is dogshit as well as the company, their investigators and Post Office. But I think a lot of people were pressured into plea deal's or something, pressured/forced into repaying money they never stole, which complicated things quite heavily from a top-down analysis of these prosecutions. But the idea some random postmasters would collude in this all over the country is kind of crazy and I'm not sure how post offices are that profitable to begin with for each branch to have £40,000 stolen or whatnot..
I note any person could have reported this issue , by sending a few anonymous messages to several relevant people - starting with senior people and mentioning their responsibility's and casually mentioning a few facts that only a limited number internal people likely to know etc .... And if nothing happened "in a while" , send information to a wider circle of people
As A Software test alysist for 20 Years, working in a company that was "bank like", where numerous transactions could involve large sums of money. When software changes where ready to go to production, There would be a audit of the testing. This would take at least a few DAYSs. The (government) customer would hire independent very experienced software engineers / testers. We would have a test system that mimicked the distributed production system. with all (100s) of software versions matching . We also had internal and external auditors .
Our test plans . test cases , test data and test results would be examined. Some tests would need to be reproduced in the presence of a auditor. Or the auditor would ask for tests to be run using their provided data. We also had load testing, deployment testing, rollback tests etc. The software had to conform to certain international standards for security - so such remote access would not have been possible, in our case. If it had existed it would have been discovered through audits . Other measure s where taken in system , such as encrypted of messages and encrypted databases . Access was tightly controlled . Software developers could not deploy to prod. Lots other security controls to
Here's one example: Alan Turing was convicted of "gross indecency" (having gay sex) and ultimately took his life rather than live with the chemical castration ordered by the courts. It took a lot of convincing, but in 2017 Parliament ultimately passed a law pardoning all men convicted of that former crime. It doesn't matter how correctly the court interpreted the law in 1952, it was fundamentally a miscarriage of justice that we recognise in the present day.
It has already been determined by other court cases that many of the court cases that convicted SPMs were unsound, particularly because the court took the Post Office's word for it that their software worked correctly, and we know it did not. Rather than wait years to thoroughly re-examine every case on its merits, which would in itself deny justice to anyone who dies of old age in the process, we should move forward by annulling all of the Post Office's cases.
We don't have a written constitution. The sum of legal presidents, procedures and conventions is our constitution.
https://en.wikipedia.org/wiki/Alan_Turing_law
Although, I don't see why the courts themselves could not overturn all cases that included evidence from the accounting system in question.
I can imagine an appeal court ruling that some accounting system produced unreliable evidence; but each affected case would then probably have to go to appeal.
Interestingly the TV drama doesn't mention Adam Crozier, who was CEO of Royal Mail 2003-10. I wonder if this is at all related the fact that Crozier was also CEO of ITV 2010-2017, the company that made the TV drama? Hmmm. - https://news.ycombinator.com/item?id=38931500
In most industries you need to actually make or do good stuff to have a good career. In the software industry you can (badly) manage failed projects, or lead the development of poor software products, or run internal fiefdoms that make the development of good software harder, and still have a stellar career.
Because you can make almost as much money making it badly as you can making it properly. Especially in 1998. And especially with huge government contracts.
In fact with huge government contracts in the 1990s you could make more money doing it badly than you could doing it properly. Because if you quoted the price it would have taken to do it properly, the government would buy it from your competitor who was quoting the price to do it badly.
Eventually crap project goes live once government has paid 4x the original estimate for it (because its better for everyone involved to pretend its not crap) and the company goes on to use that 'success' to bid for their next project.
Software is neither good nor bad. It's run in a box of sand, and it's a minor miracle that humanity has found a way to make a box of sand do anything at all, let alone the digital empire that we have built.
Good software isn't even describable. "Well, it should be correct." Okay, well, if good software is correct software, then why didn't you use that word to begin with? But what do you mean by 'correct'? Any definition you give will have edge cases that you didn't think of. Your resolutions will self contradict. And when the whole thing is deployed we'll discover that the assumptions of what correct means do not exist in physical reality; our house of cards painstakingly constructed with insane attention to detail collapses into ash.
Jesus then goes onto to tell the rich young ruler to follow the second half of the 10 commandments. ~Yeah, I've been doing that since forever.~ One thing he lacked, though. He was to sell all his possessions and follow Jesus.
This is not a rebuke. Do not stop asking for good software. It should be your demand; it is what we all deserve. But good software isn't a checklist of rules. It is not a simple thing to ask for. It is a lifelong journey where the destination is to get further than those who have come before. Good software is a limit as our striving tends to infinity.
Having worked through the thicket of rules for FCA regulation I am gob-smacked how far things went.
More info: https://en.wikipedia.org/wiki/Private_prosecution#United_Kin...
What the Post Office did have, though, is its own centuries-old investigation team, and much of the scandal hinges on the fact that these investigators are employees of the Post Office and likely knew the accounting system was faulty, and those faults could explain supposed losses, but deliberately disregarded that in bullying and prosecuting subpostmasters into accepting liability for phantom losses.
As is everyone in the UK.
So it's not like the stupid people of Britain woke up one day, turned on the TV and said "Golly gee, post office scandal. I never knew!"
Sounds like a puff PR piece for the TV drama?