Hacker News Reader

Comment by bobnamob | Hacker News Reader

> where there was a bug in a system designed by Fujitsu

To clarify, there was not _one_ bug, but hundreds.

https://en.wikipedia.org/wiki/British_Post_Office_scandal#Pr...

From my read of the information publicly available, there was a fundamental lack of distributed/transactional system understanding from the developers at Fujitsu.

This[1] article recently posted here[2] reminded me of the Post Office Scandal.

The lack of respect for established CS theory (transactions and distributed systems) and established accounting practices (double entry book keeping or even the idea of a ledger) is mind boggling.

[1] https://news.alvaroduran.com/p/engineers-do-not-get-to-make-...

[2] https://news.ycombinator.com/item?id=42269227

View on HN

IMO it's not really a lack of respect as it is software vendors and managers not really being incentivised to do the right thing. Therefore even if the engineers know the right thing to do, they are unable to solicit the buy in from stakeholders to invest the development into doing the right thing because there's no recourse for doing the wrong thing but it's more expensive.

And even if you do the right thing, if a competitor comes in and offers to do the "job" quicker and cheaper because they're doing the wrong thing. You may have the moral highground but that doesn't pay your employees' salaries.

What we need is a better framework for punishing bad software, in an ideal world without a bunch of red tape and reducing the burden on non experts identifying what is or isn't bad behaviour.

> What we need is a better framework for punishing bad software

So like engineering licensing and insurance?

I'm pretty ignorant on the ins and outs and trade offs associated with accreditation but yes that sounds like one solution.

It doesn't strike me as unreasonable, software engineers are now able to cause billions in damages and/or cause loss of life via primary or secondary effects.

Sure, one of software's greatest strengths was that anyone could learn it and it'll be sad to lose that but that's not really a reason for structurally enabling things like the Post Office scandal.

You probably don't need accreditation to serve cat pics but things dealing with money and life/death should. But IANAL so hopefully someone else can bring better insight to this area.

> it'll be sad to lose that

People will still be able to learn how to program and actually program. But if they take money or deal with people's private information, then they'll need to comply with the standards that will be regulated.

I think we need to start thinking of an individual's persona which includes all the information about them that is online or in government (non secret) files.

That needs to be considered when designing regarding regulations for software that interacts with someone's persona.

So anyone can set up the equivalent of an art stand in the park (serving cat pics), but if they start either selling cat pics or selling advertising that relies on collecting information about the personas on the site, then that needs to be regulated.

> People will still be able to learn how to program and actually program. But if they take money or deal with people's private information, then they'll need to comply with the standards that will be regulated.

You can even have non-licensed people doing work, and a licensed individual signing off on the end result. They would need to review the designs and work to make sure they agree with how the work was done, because it's their license on the line if something goes wrong because of bad craftsmanship.

It doesn't need to be too dramatic. It's already a general rule never to write your own cryptographic or financial code from scratch, you reach out for established libraries or services written by experts.

The law may require to use the ones that are certified if you are doing something sensitive. It doesn't need to be universal necessarily, but it should apply to public tenders for critical infrastructure at least.

You want some government mandating your dependencies...?

Yes. Legally mandated supply requirements have been an element in every other engineering sector for decades or centuries. If you're building an airplane you can't purchase from any random aluminum supplier, there's an authorized list.

That would seem to result in industry consolidation and declining innovation.

Software isn't made of physical materials anyway, it's speech. How about instead we impose open source requirements to enable public verification of critical systems?

Innovation like calculating numbers incorrectly and getting people sent to jail? Or maybe just another hundred slow, rent-seeking React web apps that'll disappear when the series B money dries up, if we're lucky. We must have different ideas of what innovation is.

So, outsource the core government function of regulation to "the public". Who exactly do you imagine is going to actually do this public verification, and where is their paycheck going to come from.

Some software is critical infrastructure and needs to be treated as such. We are not special. Every other engineering discipline has gone through this same process as and arrived at the inevitable conclusion that government regulation is essential, but only after causing unthinkable damage to the public first

software "engineers" love the cachet but hate the idea that they might also have to uphold the values and responsibilities of engineering.

I say we regulate the word engineer the same way it is in many countries for real engineering. if you don't want to progress beyond code monkey, you can be a software developer and innovate yet another react clone. if you want to be called an engineer, you learn and follow the regulations.

Forcing transparency isn't outsourcing anything.

Enabling public verification is not the same as required verification.

At least when you are building key infrastructure for the government, directly or indirectly through third parties. I don’t think it’s unreasonable.

And it is not as absolute as you make it sound. Only dependencies for specific critical functions may be regulated. And they don’t have to literally force a whitelist of dependencies on you, just whichever has been certified as appropriate for that purpose.

This is how you get FIPS 140 [1], which for those not in the know is a US Federal standard that mandates encryption which is _less_ secure the the current state of the art and has been for decades. (Yes, there's a new version which was approved 5 years ago and which is still rolling out [2]).

[1]: https://en.wikipedia.org/wiki/FIPS_140

[2]: https://csrc.nist.gov/Projects/fips-140-3-transition-effort

At the same time, turning on FIPs mode is the way we discover that some of our modules were using MD5 in security critical places. Because the government actually enforces FIPS, people (primarily Red Hat I think), now actually put in the bare minimum of engineering effort so that when you set fips=1, the system will actually enforce the policy (unless you go out of your way to override it, or use a non distribution provided crypto stack).

Sure, now that the infrastructure for this has been built, it can be configured to require stronger crypto then FIPS does, but that infrastructure would never have been built without the likes of FIPS, and the government mandating it's use. And I know this because even with all of the hard engineering work done of building that infrastructure, there are no commonly used stronger policies; because the only people who actually care are the ones forced to care by the likes of FIPS.

Our electrical standards might not the safest way of wiring buildings, and not what we would come up with if we wrote the standards today. But they are orders of magnitude safer then what electricians would be doing without the standards.

Who maintains the mandated dependencies? Who performs certification?

What prevents regulatory capture?

how does it work for aircrafts? how does it work for anti-money laundering policies?

The way it works for aircraft is that now it's getting prohibitively expensive to design new planes. But new planes are still needed, so Boeing found a solution, and made the 737-MAX which is in theory an upgrade that does not require re-certification, but is in fact different enough that the differences lead to hundreds of deaths.

Perhaps without the certifications lots more people would have died. I'm just an armchair analyst. Just food for thought.

Uh, the MAX was a disaster because Boeing is run by MBAs instead of engineers. Regulation is why air travel is remarkably safe. Those rules are written in blood, something everyone here conveniently seems to forget.

So you write regulations for software development, but software companies run by MBAs instead of engineers can still cause disasters. I guess as long as disasters are rare, it's a win.

Is Fujitsu run by engineers?

lots of people did die without the regulations!

Famously, civil engineering has never solved this problem, either, leading to the sad state of affairs in which we find ourselves today, wherein no bridges have been built anywhere.

I don't understand the confusion, practically everything we interact with in our daily lives is regulated and goes through certification processes, just look at how all that is done.

Food, drugs, healthcare, consumer products, chemicals, cars, planes, trains, buildings, utilities, energy, infrastructure, salaries, loans, investments, accounting... Even media requires some licenses, receives age ratings, and has restrictions on advertising.

It's not rocket science, this is normal for every single other industry.

And that has resulted in massive industrial consolidation and lack of innovation in food, drugs, healthcare, consumer products, chemicals, automotive/aerospace/locomotive manufacturing, construction, utilities, energy, payments, finance, accounting, and media outside "tech," where we manage our own dependencies.

Rocket science is one of the few industries that's actually seeing active innovation.

This isn't (or doesn't have to be) about laws specifying how all software has to be made.

What they need to specify is the standards for software that certain types of organizations can use. Like government agencies, government contractors, medical organizations, construction and engineering firms, and probably some other kinds of large private businesses, depending on their industry.

Basically, if the software your organization uses can cause the level of destruction that Horizon did, it needs to have specific certifications, or you can't use it.

In order for such software to be certified, it needs to meet certain clearly-defined standards of quality, potentially including having all the technical leads of some level (or just all the developers, depending on various factors) be licensed, and have their licenses on the line of something like this scandal occurs.

It's not a panacea, and it would definitely be an absolute bear to get the terms of all of it defined both clearly and in a way that is likely to actually produce a quality product, but IMO it is likely to be worth it in the long haul.

think of it as accreditation profiles. the highest is for life threatening cases like flight/medical, the lowest might be government expenses.

While I personally love that idea, I suspect it's too complex to fly in actual attempts to get something like this in place.

the same could be said for aerospatial regulations, and we still manage to do them. civil constructions are mindbogglingly complicated and we still manage.

all it takes is enough people to die, and/or for rich people to lose enough money and it'll become the rule.

Transparency and competition seem to work better than certifications and credentials.

Just mandate open source if using public money.

How's it working for those postal workers who committed suicide?

That software wasn't open source, was it? I don't understand how that's supposed to be relevant.

If someone prefers solution B to solution A, bringing up a situation that had neither is not a counterargument.

I think it needs to be on the individual-liability level. Blessing implementations from corporations is an environment for monopolies to grow and corruption to set in.

The Post Office scandal at its core was a political/legal problem, not an engineering one.

Software needs to be transparent, a human expert worker needs to be able to verify the result a software generates within reasonable time.

There is no expertise for software that you could delegate to aside from an entity using some form of integration test.

> Software needs to be transparent, a human expert worker needs to be able to verify the result a software generates within reasonable time.

This is becoming more and more important as the AI industry is pushing non-deterministic computing further and further. There will be court cases where the full automated decision chain will be called into question, and a default assumption of "well, we don't have any logs so we should assume the software did the right thing" terrifies me.

The magic word that always pops up to me is "liability."

Even speaking from an IC/non-managerial place, far too many technical problems in businesses are really just symptom-alleviation (or worse, performative look-I'm-doing-something theater) for a deeper problem which involves misaligned incentives for humans.

Some of the sites we work on are insurance based. We save every step of the calculation (inputs and outputs, e.g. rates and sub-totals). An administrative user can see the entire calculation from start to end, including overrides at various steps (e.g. a manual discount), and breakdowns per state or item insured if appropriate. This seems like the acceptable bare minimum to me, rather than just showing a magic number. And it definitely helps to expose bugs.

Sure, it doesn't need to up-end the whole field, but it is rather obvious that critical infrastructure should require significant compliance, particularly if it's being purchased in a public tender process where the cheapest supplier that fits the criteria is hired. Certifications like SoC 2 are already a big thing.

> > What we need is a better framework for punishing bad software

> So like engineering licensing and insurance?

How is engineering licensing and insurance punishing bad software ?

Microsoft is still going strong and all they do is "checklist security".

> managers not really being incentivised to do the right thing

It's more likely that managers are just incompetent. Not being able to distinguish between infrastructure and "decoration" means all their decisions are at best superficially motivated.

The punishment we have for this is unemployability and bankruptcy. It will happen by itself if you let it.

A framework for "punishing bad software" sounds to me to be hiding the ambition to "protect bad managers from consequences of their decisions".

> The punishment we have for this is unemployability and bankruptcy. It will happen by itself if you let it.

In my career, I have never seen a manager see consequences for making the decision that values "money now" vs "avoiding things going wrong later". I do not believe that it will happen by itself.

Well yes, agree this is how it looks now. But my perspective is that there has to be layers of mechanisms protecting them, rather than the natural order. (These mechanisms are tricky to pin-point but seem to be on the themes of mutual loyalty, avoiding embarrassment, keeping face, and so on.)

Imposing some formal framework is more likely to insulate managers even further while making software developers accept legal responsibility beyond their pay.

> The punishment we have for this is unemployability and bankruptcy. It will happen by itself if you let it.

yeah, nah.

"No one ever got fired for buying IBM"

after decades of incompetence the multinational contracting firms are still going fine. IBM still runs fine and they're focus is now even more in the insulting side of the business.

I don't work in IT and I'm not a developer, but IMO a developer should do the right thing even if that thing is not written in the specs/contract (transactions and ledger in this case). It is not the job of the developer to care about the competitiveness of the company (Fujitsu in this case), contracts, money, etc.

You get fired for doing good work, in many places out there.

It seems to me that it's a fundamental failure of software engineering culture.

The number one rule for engineering domain applications is to understand the domain.

I would blame the perennial neophilia and lack of (or inadequate adoption of/respect for) standardized texts in the industry. Though, to be fair, a lot of this does come down to the rapid changes in the technology.

Iterative development is necessary for software, of course, but this should be understood as a necessity due to the medium, not as an excuse for skipping research and design. A lot of these domains (especially something as critical as accounting) should be solved problems.

From what I can see "software engineering" for the most part isn't really "engineering" in the sense that a PE or CEng would recognise?

It is partially on the countries where one cannot just slap engineering on their job title as they feel like.

Where signing off contracts does have some implications beyond "it works on my computer".

Engineering is a professional activity, a legal title is just a moat, it doesn't mean that it is Engineering, just a protected profession of some sort.

In al seriousness, Engineering is about verifying systems to make sure their lifespans and failure modes are known, up front.

This has a legal dimension and a practical one. Legally you can make people liable for unreliable systems. But you can also be liable for failure to maintain properly, or failure to warn about impending calamity. Because it's all documented and verified.

Practical you can live worry free in earthquake and flooding proof buildings, trusting in the diligence of Engineers, and maintenance workers, because they and others have liability imposed on them.

For software this is only the case in a few sectors. For buildings in all cases. Not comparable.

When society depends on software is more than comparable.

Any life can be absurdly destroyed via malware, security exploits, accounting gone wrong, a database deleted in production,....

People also don't put up with faulty products, why should computing be an exception, shitty ship now fix later culture?

And above all, calling oneself "engineer" out of a bootcamp, has nothing to do with Engineering.

This is not a story of a failure in software engineering. Shit happens, no system is 100% reliable. The failure lies in how management handled it (pretend there’s no issue, don’t launch an internal investigation, blame others).

No doubt management also oversaw the development of the system and rushed it to production.

A healthy culture should accept failure as inevitable and learn from it when it occurs. It should also listen to the people who know best: the engineers who built the thing. You know, like the aerospace industry.

The damage that morons in suits do in pursuit of their bonus cannot be overstated.

There's also the issue where public software engineering related projects are too broad in scope and poorly defined. Rather than making small projects that are useful and then expanding upon them the trend is to specify massive nationwide databases and then just throw money at them, often using 'Big bang' deployments. These are so called because you deploy them and then there's a large explosion ;)

I'm not sure I entirely agree in this circumstance.

Normally I'd give the developers the benefit of the doubt. But the sheer number of issues, and how fundamental some of them ~were~ are[1] leave me little room for sympathy.

https://en.wikipedia.org/wiki/British_Post_Office_scandal#Pr...

Transaction idempotency is such a basic property for a financial system that I struggle to believe that Horizon was tested in any meaningful way.

"the engineers who built the thing" (Gareth Jenkins) are also under investigation for perjury.

[1] Horizon is still in use, in its buggy state, with replacement scheduled for 2030....

>Normally I'd give the developers the benefit of the doubt. But the sheer number of issues, and how fundamental some of them ~were~ are[1] leave me little room for sympathy.

The developers were just doing their job. It's management's responsibility to construct a functioning system of checks and balances and understand the limitations of their systems, both of which they failed to do. If it weren't for their hubris the fundamental issues with Horizon could have come to light much earlier.

Let's also not forget that the reason executives are compensated well is for them to take accountability in situations like this.

I think you’re trying to fit this into an exclusive OR when it really is AND. Unless the developers strongly warned management about the lack of idempotency and were ordered to continue anyway, they share the blame for those bugs even if they were not part of the conspiracy to cover them up (this also goes for anyone involved in those hot fixes trying to patch up errors without telling the customer).

More broadly, your idea that this is solely a management problem is how we end in situations where developers are being told to unquestioningly code some design exactly as given, which never works. You don’t get professional judgement if you don’t accept responsibility, too.

It can be more than one thing at a time.

But I would also say that that kind of toxic management is absolutely a part of "software engineering culture". How many horror stories do people on here have of managers who care nothing about the quality of the product, only meeting the deadline so they can get their bonus?

"Software engineering culture" is way, way more than just "how write good code." It includes how we work, how we manage/are managed, how we advocate for ourselves, or fail to do so, and much more.

It certainly includes the very common resistance to unions among programmers, and assuming this was caused by management pushing a known-bad product out the door, a strong union would have (at least potentially) been able to stand up to such demands.

Programmers who get paid asstons of money don't want to admit that these are also fundamentally business failures, partially because if they were fixed, they probably wouldn't get six figures for styling HTML buttons anymore.

> The failure lies in how management handled it

The thing literally went into a criminal court as evidence, and was "presumed correct" in a way that overloaded any technical or reasonable discordance.

The largest failure here was from the judges and lawyers. The software failure isn't even relevant.

I'm so tired of seeing "no system is 100% reliable" being used to excuse the most fundamental failures in this industry. Nobody's asking for 100% perfection, we're asking for the people writing the software to do X to actually do X correctly in the first place. How about, I don't know, 95% correctness? Is that too onerous?

Software engineering culture is still in a very primitive phase: just take a look at the comments of the category theory submission currently in the front-page https://news.ycombinator.com/item?id=42291141

Try suggesting to use a tool like TLA+ to validate some complex design and the most likely scenario is that people will laugh at you, even if it's a critical component for the business.

Most decisions in the industry are based on weak anecdotes and unfounded opinions of underserved "authorities".

The many bugs are still there. Fujitsu should be renamed Bugitsu....

"Majority of Subpostmasters still getting unexplained Horizon discrepancies" - https://www.postofficescandal.uk/post/majority-of-subpostmas...

Reply on news.ycombinator.com