Hacker News Reader

Comment by movpasd | Hacker News Reader

It seems to me that it's a fundamental failure of software engineering culture.

The number one rule for engineering domain applications is to understand the domain.

I would blame the perennial neophilia and lack of (or inadequate adoption of/respect for) standardized texts in the industry. Though, to be fair, a lot of this does come down to the rapid changes in the technology.

Iterative development is necessary for software, of course, but this should be understood as a necessity due to the medium, not as an excuse for skipping research and design. A lot of these domains (especially something as critical as accounting) should be solved problems.

View on HN

This is not a story of a failure in software engineering. Shit happens, no system is 100% reliable. The failure lies in how management handled it (pretend there’s no issue, don’t launch an internal investigation, blame others).

No doubt management also oversaw the development of the system and rushed it to production.

A healthy culture should accept failure as inevitable and learn from it when it occurs. It should also listen to the people who know best: the engineers who built the thing. You know, like the aerospace industry.

The damage that morons in suits do in pursuit of their bonus cannot be overstated.

I'm not sure I entirely agree in this circumstance.

Normally I'd give the developers the benefit of the doubt. But the sheer number of issues, and how fundamental some of them ~were~ are[1] leave me little room for sympathy.

https://en.wikipedia.org/wiki/British_Post_Office_scandal#Pr...

Transaction idempotency is such a basic property for a financial system that I struggle to believe that Horizon was tested in any meaningful way.

"the engineers who built the thing" (Gareth Jenkins) are also under investigation for perjury.

[1] Horizon is still in use, in its buggy state, with replacement scheduled for 2030....

>Normally I'd give the developers the benefit of the doubt. But the sheer number of issues, and how fundamental some of them ~were~ are[1] leave me little room for sympathy.

The developers were just doing their job. It's management's responsibility to construct a functioning system of checks and balances and understand the limitations of their systems, both of which they failed to do. If it weren't for their hubris the fundamental issues with Horizon could have come to light much earlier.

Let's also not forget that the reason executives are compensated well is for them to take accountability in situations like this.

I think you’re trying to fit this into an exclusive OR when it really is AND. Unless the developers strongly warned management about the lack of idempotency and were ordered to continue anyway, they share the blame for those bugs even if they were not part of the conspiracy to cover them up (this also goes for anyone involved in those hot fixes trying to patch up errors without telling the customer).

More broadly, your idea that this is solely a management problem is how we end in situations where developers are being told to unquestioningly code some design exactly as given, which never works. You don’t get professional judgement if you don’t accept responsibility, too.

It can be more than one thing at a time.

But I would also say that that kind of toxic management is absolutely a part of "software engineering culture". How many horror stories do people on here have of managers who care nothing about the quality of the product, only meeting the deadline so they can get their bonus?

"Software engineering culture" is way, way more than just "how write good code." It includes how we work, how we manage/are managed, how we advocate for ourselves, or fail to do so, and much more.

It certainly includes the very common resistance to unions among programmers, and assuming this was caused by management pushing a known-bad product out the door, a strong union would have (at least potentially) been able to stand up to such demands.

Programmers who get paid asstons of money don't want to admit that these are also fundamentally business failures, partially because if they were fixed, they probably wouldn't get six figures for styling HTML buttons anymore.

There's also the issue where public software engineering related projects are too broad in scope and poorly defined. Rather than making small projects that are useful and then expanding upon them the trend is to specify massive nationwide databases and then just throw money at them, often using 'Big bang' deployments. These are so called because you deploy them and then there's a large explosion ;)

> The failure lies in how management handled it

The thing literally went into a criminal court as evidence, and was "presumed correct" in a way that overloaded any technical or reasonable discordance.

The largest failure here was from the judges and lawyers. The software failure isn't even relevant.

I'm so tired of seeing "no system is 100% reliable" being used to excuse the most fundamental failures in this industry. Nobody's asking for 100% perfection, we're asking for the people writing the software to do X to actually do X correctly in the first place. How about, I don't know, 95% correctness? Is that too onerous?

From what I can see "software engineering" for the most part isn't really "engineering" in the sense that a PE or CEng would recognise?

It is partially on the countries where one cannot just slap engineering on their job title as they feel like.

Where signing off contracts does have some implications beyond "it works on my computer".

Engineering is a professional activity, a legal title is just a moat, it doesn't mean that it is Engineering, just a protected profession of some sort.

In al seriousness, Engineering is about verifying systems to make sure their lifespans and failure modes are known, up front.

This has a legal dimension and a practical one. Legally you can make people liable for unreliable systems. But you can also be liable for failure to maintain properly, or failure to warn about impending calamity. Because it's all documented and verified.

Practical you can live worry free in earthquake and flooding proof buildings, trusting in the diligence of Engineers, and maintenance workers, because they and others have liability imposed on them.

For software this is only the case in a few sectors. For buildings in all cases. Not comparable.

When society depends on software is more than comparable.

Any life can be absurdly destroyed via malware, security exploits, accounting gone wrong, a database deleted in production,....

People also don't put up with faulty products, why should computing be an exception, shitty ship now fix later culture?

And above all, calling oneself "engineer" out of a bootcamp, has nothing to do with Engineering.

Software engineering culture is still in a very primitive phase: just take a look at the comments of the category theory submission currently in the front-page https://news.ycombinator.com/item?id=42291141

Try suggesting to use a tool like TLA+ to validate some complex design and the most likely scenario is that people will laugh at you, even if it's a critical component for the business.

Most decisions in the industry are based on weak anecdotes and unfounded opinions of underserved "authorities".

Reply on news.ycombinator.com