Hacker News Reader

Comment by thePunisher | Hacker News Reader

I keep noticing that more and more aviation and space missions fail because of software problems. It seems to me that the new generation of engineers are generally less competent or companies see software as an afterthought which can be outsourced to lower-wage countries.

The Boeing MAX and Starliner come to mind, but the failed Moon missions by Israel and India are also examples of this trend.

Cost cutting in software development is costing companies dearly. Boeing may even go bankrupt because of this.

View on HN

It's a hiring problem. They let go of the good and/or experienced engineers in the 00s, then replaced them primarily with EEs (as a computer scientist I was told I couldn't write code, only test it, at my first job, I did not stay long) with minimal programming experience. These were very compliant people, happy to do 60 hours or more per week (work harder, not smarter). They lacked the historical context of the systems they were maintaining/developing, and the experience to properly model the systems under development [0].

This hiring problem is compounded by the oversight problem. The program managers are similarly inexperienced. Or they came from strictly a testing side with no concept of what software development itself entails (I've seen this a lot). So they aren't bad at managing requirements, they may actually be really good at it, but they absolutely fail to understand that software is a hard problem (especially when dozens of subcontractor are involved) that extends beyond just the technical problem, and to the communication and coordination problem. That's assuming they're experienced, USAF program managers for software (IME) are straight out of college history majors. DoD programs are scary.

[0] Most avionics systems, in my experience, boil down to rather straightforward state machines. Understood this way they become much simpler to write and test. The hard part is hitting your timing constraints, but that's easier to achieve with correct-but-slow-and-maintainable code than with incorrect-but-fast-and-unmaintainable code. Inexperienced developers won't see this possibility, either by failing to spend time studying the requirements or failing to understand how to implement state machines at all.

(The MAX was not so much a software issue as an architecture issue (starting with insufficient redundancy). So that's not a good example of software causing problems for airliners.)

There are two reasons why you can expect software to be more and more the cause of airliner safety issues:

- software is eating the world

- software is getting more complicated

The first is a long-term trend now. Look under the hood of any automobile from before the 80s: no computer to be found. Look under the hood of any automobile from the past 30 years: computers abound. The reason for this is that many problems are easier to address in software than in hardware. Of course, you go from N hardware problems to some possibly smaller set of possibly simpler hardware problems at the cost of gaining a set of software problems -- but this trade-off usually pays off. In some cases this trade-off enables functionality that would be infeasible to create otherwise.

The second problem is also a long-term trend: CPUs, systems, operating systems, and applications have all tended to get more complex. In embedded systems the trend has been less strongly towards ever-increasing complexity, but even in embedded systems things have gotten more complex.

Whether the problem is less competence among today's programmers is hard to establish here. First, we need much more software, which means we need many more programmers, which means the quality of programmers you get probably does decrease, though then again, we do have more programmers overall as more people (competent and otherwise) are attracted to the industry. But more importantly, the increase in complexity of today's systems could very well be enough to make yesteryear's competent programmers incompetent today -- you can't really compare software development 40 years ago to software development today.

(I object to this idea that lower-wage programmers necessarily can't be competent, though that isn't quite what you wrote. It's true that a lax process for outsourcing can mean you get less competent programmers, and it's probably true that higher GDP/capita correlates with availability of competent programmers. But it doesn't follow that there are no competent lower-wage programmers in India, say.)

The problem seems to me that management doesn't appreciate the need for competent (and therefore expensive) software engineers in critical projects like defense, aviation and space. The whole idea of trying to save a few bucks on something as critical as a fly-by-wire system or end-to-end testing seems totally ludicrous to me.

Boeing has been cutting too many corners since the MBA's took over and started reorganizing things to maximize profitability. There's a good chance the company will fail because of this.

Don't underestimate the fact that there is a higher volume of software though.

Many things that used to be done by analog computers or manually done by the pilot are now done in software.

In addition, the Boeing MAX was a largely system design issue; the software was operating as-designed, and had it been implemented in hardware, it would have likely failed in the same manner.

Most modern systems aim to move all 'hard' bits to software.

Theres no surprise that's where most of the failures occur.

Reply on news.ycombinator.com