The Ethical Failures Behind the Boeing Disasters
blog.apaonline.org
blog.apaonline.org
The focus on Boeing and elsewhere has revealed that for a substantial period of time the focus of management was essentially "gutting" the company; replacing highly skilled and expensive engineers and workers with cheaper workers elsewhere and generally removing any impediment to immediate profit - this including stretching to 737 spec to the point that it essentially "broke" rather than taking the cost hit of designing a new plane for a new era.
The thing is, when the paradigm the top puts out is "do whatever it take to make those numbers", you already have an implicitly unethical outlook but one with plausible deniability. The people who push that (whether management, hedge funds or board of directors) are careful not to overtly advocate anything illegal or immoral but it seems logical that when X underling who does Y ethical act is caught and punished, that underling will be replaced by another one who will face the same pressures and quite likely also engage in similar overtly unethical behavior.
Like retail stores with 8 employees on, where some are meant to handle stock and some are meant to serve customers, but you've given them all sales budgets for the day. The stock doesn't get handled, and you have 8 people not meeting their budgets. All the employees are unhappy about that, stressed, and working against eachother to try to succeed.
Competitive metrics just makes it so you have a handful of "star" people, who are great at undermining their colleagues in order to meet their own budget. Rather than an effective team of good people. So in the example, by the metrics, you have 1 good employee and 7 bad ones, no one is happy, your processes have fallen apart and your team doesn't work effectively together. The only people who get fired are the employees, yet the store continues to underperform.
The reported target in the 737 MAX case was that Boeing had promised Southwest a $1 million penalty per plane if retraining was required.
So both Boeing and Southwest created a perverse incentive to make the plane less safe. One or two airline accidents can close an airline, so I guarantee nobody at Southwest really wanted to make that deal in retrospect.
Although Southwest didn't have any 737 MAX accidents and they paid for the deluxe MCAS instrument package, they were horrified to find out that what was delivered, what it was supposed to do, and what was documented were 3 different things.
Source: commercially-rated pilot, followed MCAS fiasco daily. Search for "simulators" in the following link:
https://www.forbes.com/sites/petercohan/2019/03/28/did-airbu...
In context that's 1-2% of the sale price of a MAX (depending on discounts). It's plausible that retraining drove MCAS, but that seems unlikely to me.
deluxe MCAS instrument package
No such thing exists. The AoA annunciator was supposed to be active on all 737 MAX planes.
https://boeing.mediaroom.com/news-releases-statements?item=1...
Both planes that crashed didn't have the optional add-on, so ended up not having the Alert either.
https://arstechnica.com/information-technology/2019/03/boein...
That’s one of the reasons why I am against more nuclear plants and running them like other businesses . They will start out doing solid work but over time management will get greedy, reduce quality until something blows up.
However, actual liability is only accrued by those who act on those policies.
It'd be nice to see liability, once determined at a lower level, forced to follow the organization chart back to the root decision. Possibly balanced for something like direct reports.
E.g. you manage 100 people, 20 of them committed fraud, you have a 20/100 share of the crime
Example of it occurring unintentionally:
1. A corporation sets goals for store managers based upon performance metrics that include payroll. These goals are currently realizable by a competent manager without forcing workers to work without pay.
2. Some fraction of managers realize rather than actually working hard to meet the goals, they can get their numbers up by forcing workers to work without pay. This skews the metrics a bit.
3. The most incompetent managers that aren't committing wage-theft are demoted or fired, being replaced with managers that are either more-competent or willing to commit wage-theft.
4. The payroll goal is now adjusted down because the previous accounting-term's payroll average is lower than before.
5. Competent managers manage to get jobs elsewhere, so the fraction of managers willing to commit wage-theft increases.
6. GOTO 2
Eventually the metrics are so skewed that the majority of managers must either commit wage-theft or lose their jobs. These managers are the only people that will get in trouble if the wage-theft is discovered.
> O’Donoghue’s military experience includes 12 years of active duty as a U.S. Marine Corps fighter pilot and test pilot. He flew operational missions in the A-4M, AV-8A and AV-8B Harrier aircraft, and engineering flight tests on the AV-8B and F-14 Tomcat. In 1994, O’Donoghue transferred to the U.S. Air Force Reserve where he flew the C-130, C-141 and C-17. While there he commanded both the 728th Airlift Squadron and the 446 Airlift Wing, stationed at McChord Air Force Base, Wash. In 2005, he retired from the Air Force Reserve at the rank of colonel.
> O’Donoghue holds a bachelor’s degree in mechanical engineering from the U.S. Naval Academy.
(http://www.boeing.com/company/bios/dennis-odonoghue.page)
This doesn't seem to conform to your theory that it's "cheaper workers" and business majors that caused these problems: sure, they may have created even undue pressure. But it was an all-American fighter jock doing a lot of the actual lying about MCAS.
At what point does some other word become more appropriate?
What you seem to be saying (without actually saying it, so I have to guess) is that, because it's a failure of a group rather than an individual, it isn't ethical any more - it's something else. But if the group structure pushes individuals into unethical actions, isn't the group structure an ethical problem?
Or, for the snarky answer: If companies are corporate persons, then of course they can have ethical issues.
You're going to need overall integrity in addition to ethics, from bottom to top, starting with elements that are known to be lost, or recognized as not being adequate, before you have a chance to make it to the goal again.
Are you saying that a plane that tends to tilt up dangerously under certain conditions that is designed to be pushed down nose first by a software with no limit to how much it'll push down depending on a single sensor would have been an acceptable solution if the workers were different? This is a problem with the design and definition of a large system rather than the implementation.
So, I think in this case, whoever had any knowledge of this at any level and let it go through are absolutely responsible. While outsourcing to cheap labour elsewhere can and often does become a problem,these crashes can't be attributed to that. This was a very specific failure with very specific people in the wrong.
With different engineers and a focus on product versus profit the MAX never would've seen the light of day. With the focus on outsourcing Boeing had a gigantic mess on their hands with the 787 (and 777X) and no resources to focus on getting a 737 replacement to market.
I read this as: engineering ethics were broken because of a toxic management culture.
If we start to put CEOs and majority share owners in prison things will start to shift a bit.
There are lots of very smart people at Boeing. If laypersons can recognize these mistakes then there must have been hundreds or thousands of engineers at Boeing who also recognized these mistakes. But they didn't speak up. I see this as evidence of a cultural problem.
Do you work at a company where the boss discourages you from speaking up about potential problems, and expects you to just do what you're told to do? If so your company could be the next Boeing.
"Anyone who has stood up a service on the cloud"
I don't think I'd call those the same thing.
.
Aside from that, having a single point of failure in some component isn't necessarily unconscionable, you just make sure the system as a whole can handle the loss of the affected component.
That is a thing. It's not ideal, but it's not the end of the world provided it works. E.g., the F-117 cannot be flown without a computer constantly adjusting flight surfaces -- the plane isn't stable. The other decisions are definitely intolerable.
The whole thing smacks of corner-cutting in the worst way.
Reuters had an article at https://www.reuters.com/article/us-boeing-737max/new-boeing-.... This article contained a paragraph that was news to me:
> Boeing had earlier turned over the documents to the Justice Department, which has an active criminal investigation underway into matters related to the 737 MAX plane.
An active criminal investigation? Wow. Boeing's in a lot hotter water than I knew.
Programmers have no such institutional support. If a programmer refuses a job, it goes to someone else that's it. Programmers may have ethics but ethical training a la engineers isn't going to give them any leverage for choices.
Without protection for people refusing to do bad things, you create a system when there's always someone desperate, hungry or unethical enough to do things that shouldn't be done.
Imagine the state of the tech world today if all of the "engineer" programmers at Google, Facebook, etc... practiced at the same ethical level as actual engineers.
This collective action problem is solved by coordination, through the means of the licensing body. That body can impose severe penalties (not just firing you from your current job, but from all future jobs) for anyone who betrays the group strategy, so an individual engineer can feel some more safety refusing orders in the knowledge that the whole profession will back them up.
EDIT: In civil engineering, this system is propped up by the state, which requires plans to be signed off by a licensed engineer. The guild functions in this capacity as a subcontractor of the state, taking on a regulatory burden and allowing rather more severe punishments (barring someone from a profession) than would be acceptable from a purely state organ. In software, this could be enforced by similar means for safety-critical applications - the ACM, for example, could be required to license any software engineer, with the understanding that they would revoke licenses for negligence or malfeasance that didn't rise to the level or criminal liability.
Yeah, there is a handful of cases where software developers have been given bad instructions from their management, and perhaps Boeing is one of them. But the real problem is developers being unaware of the most basic good practices.
They would be held liable even if their boss ordered them to do so.
It's a very different set of incentives than we have in software, but maybe its time we introduce real PEs into software development.
Which seems a pretty bad state of affairs to trust when building things like airplanes.
In terms of liability, Boeing can try push it to the subbies and the subbies can try and push it back to Boeing. Both are trying to bamboozle the non-technical lawyers.
The purpose of the FAA is to cut through that crap and enforce actual, effective change through sanctions or otherwise, and they didn’t do that. That’s what fell apart here.
In my state, Gambling is legalized. I remember my surprise when a friend who worked in the compliance side of the business actually knew what MD5 was (Back in 2007.) She wasn't a 'technical' person either.
She explained that they actually had to audit the slot machines to make sure that the code running on them had a hash that matched a codebase that had been audited and approved by the state regulatory body.
So, the practice for auditing code by a regulatory body is nothing new. If we do it for money, FFS can we do it when there are actual lives involved?
https://www.economist.com/business/2019/03/23/regulatory-cap...
And will likely eventually be held to the standard of "How dare you let this happen?" as they're fired, to demonstrate how seriously Boeing takes safety.
What ethical safety standards would you have them implement?
I assume that you would agree that the control system specification is the ethical responsibility of the control system design engineers.
I'm a generalist, so I make it my business to know a bit of everyone else's business. If I can't look at a spec without seeing issues down the chain that the spec makes no mention of, I end up feeling that it is my duty to make sure to raise the question until I am satisfied with the answer.
I don't always get the most satisfying answer, and I haven't had to put the career on the line by doing so yet; but I'm prepared to do so nevertheless.
I will not be part of the next THERAC-25/MAX fiasco. And if I've learned anything from this decade, it is that engineers as a whole may need to organize against those that would seek to have us do unethical work.
It wouldn't stop the practice, and God help me, I don't want the field locked behind accreditation/licensure...
However, I don't see any other defense or measure that would allow for putting the kebash on bad work. There has to be a price for bad corporate behavior in terms of ruthlessly pursuing performance that can only be met through wink wink nudge nudge style inducement to unethical behavior. At least, no way besides publically outting a company's dirty laundry. That really isn't satisfying though, because that requires a sacrifice of somebody's integrity every time, and no one wants to touch you after that.
I just can't converge to a satisfying middle-ground with the right incentives. Besides maybe anonymous whistleblowing to an appropriate watchdog agency. Even then though, issues are raised in that you are leaving the regulation up to people who feel insecure reporting something when they have everything to lose.
It is a frustrating issue to say the least.
1. Safety
2. Non-Safety
3. "Safety"
For #3 I mean it's "we realize that failure has bigger repercussions than a fail-whale, but we can't afford to do any of the ISO processes that have been proven to work." Sometimes I feel like my only job on those sorts of systems is to bang the "Normalization of deviance is not okay" drum in every meeting.
All failures need to go to the PM and get signed off on, otherwise the PM has a false sense of the actual reliability of the system. If the PM wants to get more budget for safety concerns, they should be able to hand a stack of 100s of pages of papers to whomever controls the purse strings and say "These are the failures in the last N days" If all they can say is "some of my engineers have expressed concerns" then 0 change will happen.
It is the primary responsibility of the control-system, but there is still also a responsibility with everyone who interacts with that spec to speak up if any flaws are noticed.
One of the big things that tight deadlines do is give tunnel vision to the engineers, so "just implement the spec" becomes the goal and the forest can be missed for the trees.
There were probably dozens of engineers that saw the MCAS specification as part of their duties; here's a few possibilities for what happened:
1. Nobody considered the case of improper MCAS engagement under normal flight conditions; this should clearly qualify the system for "Hazardous" classification under DO-178, which would require redundant AOA sensors.
2. Someone considered this case, but didn't speak up (was very junior, or it was way out their specialty).
3. Someone spoke-up, but was told by the person they spoke to disregarded it for the same reasons as #2, so it never made it to the control-system design team.
4. Someone spoke-up, it made it to the control-system design team, and business pressures caused the concern to not be investigated.
#4 would be significant ethical issues for the control-system design engineers, but I think it to be unlikely compared to the others.
#1 can be indirectly caused by time pressure. The certification process is supposed to slow things down, but there is some indication it did not sufficiently do so in this case.
#2 and #3 show ethical lapses outside the control-system design department, and are not just isolated to the individual in question, a safety culture needs to include cultural norms of speaking up about potential problems even when you think you are wrong.
"Anyone can build a bridge that stays up. Only an engineer can build a bridge that barely stays up."
(I mean, it's obviously exaggerated for effect, but still.)
For Software that would be Homeland, FBI, and even the FAA for governing bodies specifically. But software doesn’t have ethically governing bodies because we have enough laws to cover a lot of mandatory auditing.
For example: HIPAA - Medical Data Protection & Compliance Gramm-Leach-Bliley Act - Financial Data Protection & Compliance FISMA - Federal Data Compliance GDPR - PII Data Compliance PCI DSS - Credit Card Data Compliance
Specific example of use: Code that is deployed that involves PCI data being handled requires a code review.
There are heavy penalties for lost of these things, and I promise there are many companies focusing on this. But it could be very much improved with a proper governing body.
All of this is about making money and nothing else. Each quarter make these numbers at all costs. That is what almost all companies are. Amoral beasties that do whatever is needed to make more money.
If they could have killed 300 more people and still kept the planes in the air, the CEO, wallstreet and anyone who held the stock would not have given a single fork.
Boeing's corporate governance failures became life threatening only because FAA certification process failed. Regulatory capture of FAA was the second failure.
Regulatory capture may be responsible for Boeing's recent problems https://www.economist.com/business/2019/03/23/regulatory-cap...
Barbara Hollingsworth: 'Regulatory capture' explains a lot about FAA's failures https://www.washingtonexaminer.com/barbara-hollingsworth-reg...
> What happens to federal employees who ignore safety warnings, cover up incompetent or even criminal behavior, destroy official documents and mislead members of Congress? At the Federal Aviation Administration (FAA), they get promoted.
> That's the take-away from last week's National Whistleblowers Assembly on Capitol Hill, sponsored by the Government Accountability Project (GAP) and featuring famous NYPD whistleblower Frank Serpico and former FBI agent Coleen Rowley.
https://web.archive.org/web/20190509220235/https://blog.apao...
His egregious behavior doesn't get emphasized enough. It's beyond horrible. To back up what was said in the article, here is something from Reuters on the day after the 2nd crash:
“We are confident in the safety of the 737 MAX and in the work of the men and women who design and build it,” Boeing Chief Executive Officer Dennis Muilenburg told employees in an email seen by Reuters. “Since its certification and entry into service, the MAX family has completed hundreds of thousands of flights safely.” https://www.reuters.com/article/ethiopia-airplane-boeing-ceo...
That asshat CEO made those comments the day after the second crash, after 346 people were dead.
That person should be in jail.
https://www.militarytimes.com/news/your-military/2019/03/13/...