Software Bugs Led to 'One of the Greatest Miscarriages of Justice' | Hacker News Reader

Software Bugs Led to 'One of the Greatest Miscarriages of Justice'

cacm.acm.org

32 pointsadunk9 comments

If I recall correctly from the previous threads on this subject, the bugs were just one factor. The more important, and more culpable, was the Post leadership that knowingly covered up the bugs, and lied about not having received any bug reports.

Bugs were just the excuse - the cause was criminally hostile and deceitful post leadership.

Subpostmasters began reporting balancing errors within weeks of the Horizon system being installed, via the helpline the subpostmasters were instructed to use. The Post Office denied the subpostmasters' reports of faults in the system, insisted that the subpostmasters make up any shortfall of money, and in many cases untruthfully denied that any other subpostmasters had reported problems. In May 2002, shopkeeper Baljit Sethi raised concerns with the press that there were errors in Horizon, after his wife Anjana was notified that her subpostmaster contract would be terminated.[48] The Post Office responded that it "totally refuted" that the system was faulty, and that it had "sent experts ... to check it twice". - https://en.wikipedia.org/wiki/British_Post_Office_scandal

They not only lied, they use threats to silence the only external expert consultant they brought in to examine the system who found faults. They then spent a decade prosecuting people and sending them to prison while instructing people to lie on the stand. Innocent people lost their businesses and homes, went to prison, marriages fell apart and victims committed suicide. Post office management should be in prison. They killed people and destroyed families and communities.

It is a little like modern aircraft disasters. A technical failure or combination of failures, perhaps even minor ones, combined with bad/wrong immediate management of the situation when the fault is detected, compounded by some matter of policy. In this case a refusal to acknowledge at the highest level that the system could possibly be wrong. "Clearly it must be the sub-postmasters stealing".

Edit: I would also add that successive governments have shamelessly dragged their feet over this.

This comment is your periodic HN plug for "The Field Guide to Understanding Human Error" - https://www.amazon.com/Field-Guide-Understanding-Human-Error...

At a recent IT event, I heard a revealing story during a sub-postmaster interview. In the audience was a software engineer who, during the Horizon scandal, had been a first-year software engineering student whose family ran a Post Office branch.

Her technical background helped her recognize the system's flaws - this wasn't user error, but a fundamental software issue. Acting on this insight, she convinced her family to implement manual double-entry bookkeeping alongside Horizon, creating their own audit trail.

The family ultimately sold their branch due to the operational burden. Tragically, the next owners, without this technical foresight and parallel accounting system, fell victim to Horizon's failures. The Post Office then attempted to blame my source's family for the 'missing' funds.

The way software is treated in law is one of the greatest problems this has revealed; despite being a SWE I am convinced that, had I been in the positions of these subpostmasters, I would not have been able to sufficiently document and present the issues to avoid criminal charges.

Many of them were accountants. So the government was telling an accountant-type-person that your inferior math must be ignored for our superior math. Would you accept another SWE telling you that?

I feel like it's time to take seriously what we mean when we say 'bureaucracy'. In this case, it's not just the bugs, it's also the organizations.

So what might we be talking about when we talk about bureaucracy? Three things really stand out to me (feel free to add, subtract, disagree or recategorize):

1. Someone else's moat. This can be a good or bad thing, but it should always be part of the understanding. Someone else's moat can mean an industry protects itself by pushing stringent licensing requirements - not that there should be NO licensing requirements, they just don't always make sense. Someone else's moat can be protective - see Accessibility or Clean Air/Clean Water requirements. Understand that someone is benefiting from your bureaucratic pain points - you can feel good or bad about this depending on where you're standing - inside the moat or outside.

2. Going with the flow. There's an interview with a parole officer that really stands out to me. The officer said something like 'I have the ability to throw immense resources at apprehending and jailing a person if they violate their parole, but I have no ability to expend any resources to help that person in a positive way'. If other parole officers just do their job and strictly enforce parole conditions, and you try to help someone out and they mess up badly, then that's your job at the very least. But that No Strikes compliance mentality can really ruin lives - the slippery slope goes both ways.

3. Incentives, incentives, incentives. The real power of senior management/leadership is setting incentives. If they really believe in 'the numbers' and enforce bad metrics, it's can be really hard for an organization to learn and see the problems that will destroy them. See Boeing, General Electric, anyone who believed Jack Welch. Metrics and incentives aren't the cause of the problem, but they can tell a success story that turns out to just be the first chapter of the book. It can be hard to see the contingencies when you feel successful.

This will happen again and on a far greater scale thanks to the consolidation.

Reply on news.ycombinator.com