United finds loose bolts on plug doors during 737 Max 9 inspections | Hacker News Reader

United finds loose bolts on plug doors during 737 Max 9 inspections

theaircurrent.com

506 pointsetimberg735 comments

I found it interesting that Boeing did proactively tell airlines to inspect 737 MAXs for a possible loose bolt in a different part of the plane (rudder section) at least 8 days before the January 5th event. Example story: https://www.reuters.com/business/aerospace-defense/boeing-ur...

Unfortunately, Boeing did not know they had other issues with the plug door bolts.

Imagine the quality of the manufacturing and QA / final inspection to have that kind of issue.

I expect even at its worst, software development could learn a lot from aircraft QA.

Especially since most shops have pretty much tossed professional career QA out the window.

software development: what is QA?

That's where failed software devs / management go to.

Except, of course, from Boeing's aircraft-software QA... which killed hundreds of people already.

The problem was not really the software in isolation, but that pilots expected the 737 NG to behave exactly like the old version - because Boeing decided it was too expensive to retrain pilots.

The expense for retraining pilots falls on the airline.

Retraining has its own problems. No matter how well retraining is done, pilots still make mistakes from doing the right thing for the previous plane that is the wrong thing for the one they are currently flying.

Adjusting airplanes to fly the same way is a major safety advantage.

Arguably, Boeing hit the uncanny safety valley -- similar enough so that pilots and airlines relaxed, but different enough so that relaxation ultimately killed people.

The emergency procedure for runaway trim was the same for both aircraft types, and was not followed. After the first crash, an Emergency Airworthiness Directive was issued to all MAX pilots reiterating the procedure, which was not followed in the second crash, as well as not reacting to an overspeed warning.

Unreported by the media, there was another MAX incident before the first crash. The crew had no knowledge of MCAS, but did follow the emergency runaway trim procedure, and continued the flight and landed safely.

"Runaway stab trim". It is a memory item, every pilot should be able to perform it from memory.

Turn off the motor, and the trim is manual. There is a crank right there in the cockpit. If it is too hard to turn, change aircraft configuration to reduce the forces required to. Pilot know how to do this. This pilot stuff, they understand the forces on the flight controls and what impacts them.

Boeing made an engineering mistake. The pilots also made an operational mistake. Unfortunately, both mistakes at the same time were fatal.

I pray that pilot training has improved. And that Boeing has made systems level changes to the aircraft that will preclude it happening in the future.

And that is how aviation becomes safer every year; at a significant cost of customers lives.

> And that is how aviation becomes safer every year; at a significant cost of customers lives.

"Significant" might be inaccurate.

It looks like FAA Part 121 accidents over the last 10 years with fatalities have been... 4. [0]

For a total of 6 fatalities.

[0] https://www.ntsb.gov/Pages/AviationQueryV2.aspx; 2018 (1 passenger fatality) https://www.ntsb.gov/investigations/Pages/DCA18MA142.aspx ; 2019 (3 crew fatalities, cargo flight) https://www.ntsb.gov/investigations/Pages/DCA19MA086.aspx and (1 passenger fatality) https://www.ntsb.gov/investigations/Pages/DCA20MA002.aspx ; 2022 (1 ramp fatality) https://data.ntsb.gov/carol-repgen/api/Aviation/ReportMain/G...

That low accident rate is nigh inconceivable. It's an incredible achievement.

The fatal accident count is higher for GA, but I didn't normalize against flight hours or flights, just glanced at it.

I'm sure there's been a study somewhere that attempts to untangle all the factors that differ between commercial carriers and GA, to see which safety is most sensitive to -- continuous highly professional maintenance, highly trained and experienced crew, rigorous airliner certification regime, etc.

One of which (Atlas Flight 3591) was Pilot error:

> The probable cause of this accident was the inappropriate response by the first officer as the pilot flying to an inadvertent activation of the go-around mode, which led to his spatial disorientation and nose-down control inputs that placed the airplane in a steep descent from which the crew did not recover.

Boeing also reduced the size of the manual trim wheels, which let them become impossible to turn sooner than on previous 737s.

The electric trim switches override MCAS. This was explained in the Emergency Airworthiness Directive sent to all MAX pilots after the first crash.

Also, overspeeding the airplane makes it much harder to turn the manual trim wheel. The cockpit voice recorder on the EA flight recorded the overspeed warning horn, which the crew did nothing about (they were at full power, should have pulled the throttles back).

The LA crew restored normal trim twenty-five times before crashing. What they never did was turn it off after restoring normal trim.

If a pilot can't be expected to maintain the pitch of a plane on takeoff, he has no business flying ANYTHING.

What Boeing did (and is STILL doing) is expect pilots to know or remember obscure NON-PILOTAGE (and in the case of MCAS, BURIED) trivia to prevent disaster.

Now... what's the more-responsible approach? Expect pilots to pilot, or expect them to recall an ever-growing list of workarounds to incompetent system design?

The whole MCAS was just unnecessary feature (bug fix). Without it the plane would have worked just fine. The pilots would just have had to go some amount of training scenarios to get the certification on how the MAX plain flies.

Exactly. Unless the "upward-pitching tendency" under high power is extreme, any competent pilot should be able to keep the plane's attitude as desired.

The problem was software that prioritized input from a fauly external sensor, over pilot control, and literally crashed planes directly into the ground. At a certain step in the sequence it was not physically possible for a pilot to pull hard enough on the control element to counteract the software. Could they have disabled the system? Only if they could figure out the specific software trying to crash the plane.

Is that what you meant by "the problem wasn't the software?" Because the pilots should have been trained to unplug the computer to stop it from crashing the plane?

Even more ridiculous, Boeing offered a second source of truth option, but marked it as an upcharge, which the airlines in question rejected. "No thanks, no need for a second AoA sensor, one is none is probably fine!"

Additionally, two feels like a really strange number. I would think three for a tiebreaker would be standard for any sensor with that much impact (no pun intended).

Yes, two would be very ill-advised. I think there was some incident where a plane had only two pitot tubes and of course this caused problems. Or... I may have simply misremembered pitot tubes instead of AOA sensors in the Boeing case.

Pilots are definitely trained how to disable the autopilot, if needed.

Afaic, the fault apportionment was Boeing documentation > airlines >> pilots > Boeing technical design.

Not sure why you’re bringing up autopilot— the MCAS system runs even when the autopilot is disabled.

Edit: Also, how does the fault lie with the airlines? Boeing didn’t document the existence of MCAS in the flight manual or training materials.

Because the comment I was replying to

>> Because the pilots should have been trained to unplug the computer to stop it from crashing the plane?

Yes.

The fault lies with the airlines because I don't for a second believe they didn't put pressure on Boeing to get the MAX certified without mandating retraining.

And then once that was done, didn't dig into the details too hard about what changes were made.

I have a low tolerance for 'I set up all the conditions and incentives to encourage you to break the law... but you should take all the blame when it explodes.'

At some point, the customer has to take some responsibility for what they asked for.

It’s easier to blame Boeing because they made the damn thing its documentation. We know for a fact they are at fault. Some or all of the airlines may or may not have put pressure on Boeing.

Wasn't MCAS designed to activate when A/P is disconnected, also?

This wasn’t related to autopilot and they removed mention of the MCAS system from the documentation to support the main selling point of the 737 MAX, which was that existing 737 pilots would be able to switch easily without recertification. They knew that they’d lose most sales to Airbus if the aircraft were compared on their merits so they were banking hard on their huge pool of certified pilots as the competitive edge.

If you listen to podcasts, these two episodes of Causality are excellent:

https://engineered.network/causality/episode-33-737-max/

https://engineered.network/causality/episode-50-737-max-ethi...

You might enjoy this. I have a pin that blinks "AOA Disagree".

Back when I flew regularly before covid, I was tempted to create a bunch of these and hand them out to the flight crew for the flights I flew on.

Ha, playing hardball! I wonder whether you’d find pilots who are Boeing loyalists who’d take offense, or if those guys are even madder at the current management for letting them down.

Pilots should (are supposed to) disable the auto-trim if it's doing something uncommanded/unexpected. Runaway trim can happen for reasons other than faulty software. MCAS was a new factor and they should have been told about it, I don't dispute that at all.

The manufacturer company put in larger engines than the aircraft is designed for. And they did it to avoid all the homologation licences and design costs involved in bringing a new aircraft to market with the appropriate tolerances, and to compete with another company's aircraft in time (Loss of sales).

They introduced MCAS in the aircraft for to balance by software a hardware issue, a big design negligent issue which can lead to stalling. It is beyond to trim an aircraft, and because of this there is a big difference in the scale of the values that the algorithm manages from a trimming.

It is not my field, but I think it is not a simple factor, and that it should not be put this over the Pilots like if it were a normal aircraft that received a simple update. Every pilot flying that plane should have been warned that it was not a classic plane with a classic update.

If this type of behaviour by aircraft manufacturers becomes the norm, costs over safety, we as passengers will suffer it, as other passengers unfortunately suffered it, while they blame the Pilots. In addition that nowadays the China's aircraft manufacturing industry wants to enter global market. Some days ago I read they want permission (homologations approvals) for to enter in the European Union.

PS: They also cut costs retiring backup sensors, delegating responsibility for a vital system due the MCAS to the buyer as if it was an unimportant feature; disaster was the order of the day. And the spending cuts were not limited to that, as we have seen in recent days.

Where I wrote,

> They introduced MCAS in the aircraft for to balance by software a hardware issue, a big design negligent issue which can lead to stalling.

> Every pilot flying that plane should have been warned that it was not a classic plane with a classic update.

I was mean,

> They introduced MCAS to use software to attempt to balance an aerodynamically unbalanced aircraft with a high stall tendency, in order to avoid designing a new aircraft.

> Any pilot flying that aircraft should have been warned that it was a plane that didn't want to fly aerodynamically, with software forcing it to fly without backed redundancy. It was not mere trimming.

Here we are again, this misconception just won't die.

In the 737 MAX, the only way to disable auto-trim also disables powered trim (the thumb buttons). As grand parent says, at a certain step in the sequence it was not physically possible for a pilot to trim the plain back to stability manually. It simply can't be done.

In the 737 ng, there was a button to do just that. That would have been useful.

And that's even ignoring the fact that all symptoms were very different from those present in a runaway trim situation as described in the manual and learned by the pilots.

The thumb buttons would override MCAS. But then you'd have to disable the trim motors and trim manually (by hand-cranking a wheel). That part was not clearly understood by the pilots, because they were not told about MCAS.

How are pilots expected to disable a malfunctioning MCAS in an emergency, and balance manually by trial and error the aerodynamic extravagances of the angle of attack of such unbalanced aircraft in the middle of procedures?

The user of the parent comment is remarking about time.

The aircraft can be certified without MCAS?

By what I read, MCAS is there for to avoid entering into an aerodynamic stall when the aircraft is approaching a high angle of attack, due it's using larger motors for what classical 737 was designed for. It's balancing an unbalanced aircraft using software to repeatedly adjust the horizontal stabilizer.

It is not my field, but I'm not even sure if it should be called to trim, it sounds like a euphemism for what's going on.

Wrong.

QA was literally invented for the airline industry.

Software QA when actually practiced is more advanced now than airline QA.

> Software QA when actually practiced is more advanced now than airline QA.

...eh, I think "when actually practiced" is doing a lot of carrying there.

What do you mean by "actually practiced".

Outside of the aerospace and healthcare industries, I'm not sure there are many software shops that are doing QA to a level I would like to trust anyone's life with.

what does advanced mean when comparing things so unlike from each other?

also software is the least likely comparison I would have made; software quality is a shit-show on a general level, and the vast public is quite aware of this every time a subway timeboard blue-screens or gets frozen on an AMI screen, or the POS machine that they're forced to interact with at work does something equally as stupid.

Nah, in the software world, the truth is QA is where the people who can't get jobs as programmers end up. I've seen testers go on to become programmers, but I've never seen a programmer become a tester. Maybe it's different for real-time or life-critical systems, sure, but I can confidently say this is how it is in web development.

I work in the defense industry, it’s very much like the aerospace industry in that we deal with human life as a consequence of our work. We have software QA departments that operate very much like manufacturing or aerospace QA.

Software QA provides nothing of value to software development; having it as a dedicated function works against the overtly stated goals of the function and counterintuitively acts to degrade quality within software by mandating strict top down process and brittle end-to-end testing.

Although Software QA is intended to be an independent verification body that provides engineering organizations with tools and resources, in practice they function as a moral crumple zone [1] within the complex socio-technical defense industrial system, being one of the groups that the finger will be pointed to when something goes wrong and absorb shock to the business in the event of a failure. As a result they have a strong incentive to highly systematize their work with specific process steps, to shield them from liability, which can be applied generically to all projects.

Good software teams build quality into projects by introducing continuous integration, unit testing, creating feedback, and tightening these feedback loops. This acts to find problems quickly and resolve them quickly. Software QAs need for high level, top down, generic systemization requires them to work against these principles in practice. Bespoke project specific checks, such as unit testing, is not viewed as contributing to the final product and is discouraged by leadership who see it as waste.

To give an example of how these dynamics destroy quality in software. I once found a bug in software on a piece of test equipment where a logarithmic search function was not operating on a strictly sorted list. When I pointed this out to my leadership I was told that if we changed any part of code, it would require a new FQT, which would be too expensive to conduct and was not in the budget. Although the bug would have been trivial to solve, and was clearly wrong and would not provide any benefits by remaining in the test equipment software, the process required for changes prevented solving the issue.

[1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2757236

It sounds like they are calling something QA but using it as a liability shield. It makes sense that you are upset about that, but naming something QA and having it do something else doesn't mean that QA as an effort is bad. It means that the people doing that are being deceptive.

Fair point, you are correct in your inference that there are some bad actors in my workplace. However, I’ll argue that the fundamental dynamics of bifurcating the responsibility of quality from software leads to a steady state where all QA departments end up as a liability shield given enough time.

This is driven by Pournelle's iron law of bureaucracy [1], which says that people who promote the bureaucracy rather than the mission of the bureaucracy will get promoted within the organization and come to dominate its decision making.

For example, in schools, administrators make more money than teachers. This is despite both groups having similar levels of education and intelligence. The reason for this is that administrators know the laws and regulations of the environment they’re working in and ensure the continuity of the organization. Despite not directly contributing to the organization’s stated mission of education, they are in charge of the organization and take more benefits from it.

Software QA has similar dynamics. A QA department may start out making good faith contributions to the organization. Eventually there are product failures, eventually leadership needs a scapegoat to show they’re doing something, and eventually QA takes the blame. People get moved, demoted, or fired. QA realizes its risk, and takes steps to mitigate it. They create a highly systematized workflow and process, adopt or introduce standards. Then assert that following process equates to good outcomes. When bad outcomes occur, they point to their strict adherence to following process as evidence of innocence.

If the process does not support the work or mission, that is a cost they are happy to impose on other functions to deal with. This is the final state until a system disruption happens.

[1] https://en.m.wikipedia.org/wiki/Jerry_Pournelle

But what makes 'software QA' fundamentally different than 'non-software QA' to give it the problems you foresee?

Because every QA is something new for development barring regression testing.

An equivalent software QA to building planes would be to verify a known process with existing tooling.

Fundamentally different problems.

Manufacturing process builds identical widgets using standard equipment. Widgets are inspected to confirm they are within spec. Frequentist statistics are used to determine when widgets are consistently out of spec. When this happens, equipment is inspected and repaired as a corrective action. This process is well defined and linear.

Software produces bespoke, non-standard widgets to address domain specific needs. At the end of the day, software developers are defining a process for machines to follow. If you want to control quality in the software development , aside from having perfect domain knowledge for a particular project, the only way to do it do it is to define an arbitrary process for developers to follow and track adherence to it. This may have no impact, or be a hindrance. It will never add value because it will never be abstract enough to be appropriate for every domain.

I have seen a case of Software QA taking a very different shape, so I'd like to argue that the outcome you describe is not intrinsic to software QA, but rather to company culture.

The case I'm talking about does not have a separate QA department, but QA people as part of every software team. If a product fails, that team is responsible, so software devs are in the same boat as QA. They focus on learning from these failures, so no scapegoat is needed. Process does get followed, but not as a defense mechanism, but because not doing so introduces noise that is an obstacle to improvement. In case of bad outcomes, people do point out that they followed process because then it is clear that the process is involved in the failure and should be improved.

Unfortunately, companies with that kind of culture are rare.

I can see integrated QA working for teams because QA personnel would understand project specific constraints and degrees of freedom and tailor solutions in a way that top-down QA cannot.

However, there are situations where less QA may be needed, for periods of time, such as when PRs may be low. QA may be seen as overhead by management, and something to reduce. This will lead to QA shared between teams and a push for standardization and top-down process deployment to minimize complexity for these personnel will develop. Complexity to manage the QA personnel will be shifted to development teams.

This situation absolutely is controlled by company culture. A culture that neither values QA nor development will do this. A company under financial strain will do that. Companies wax and wane constantly.

It's like saying communism isnt the problem, but that it s how every single group attempted to implement it that should be blamed.

Sure, maybe, but if nobody ever can implement the theoretical utopia, maybe we should talk of things humans can do instead and ditch the unimplementable idea.

QA cannot be done by a separate team the way you dream: it will always be a political buffer zone staffed by the cheapest half-competent people you can find, expulsing good people into dev or management. Or you merge it into dev/solution design.

The reason is simple: just like contract law, you only care about quality once you are in trouble and need to reverse back the source of the issue to give to the client a post mortem. Otherwise, you care first about velocity, or $ input/hr of effort.

Other fields do QA just fine.

> Good software teams build quality into projects by introducing continuous integration, unit testing, creating feedback, and tightening these feedback loops.

Agreed, for good software teams.

I would content that most software teams at most companies are not good.

Which is to ask, with an average to bad software team is it better to have integrated or separate QA?

Does it make sense to degrade the performance of good software teams because bad software teams exist?

Ideally we’d always have good software teams, but in the real world sometimes you have to build software with bad teams.

Leaders have options, they can do things like reduce scope, increase budget, increase schedule, or full on abandon or cancel the project. These are all options available to leaders, but they require tradeoffs and decisions to be made on a project by project basis.

It is scalable to have a strict process that everyone has to follow, then impose a watchdog to enforce it on a wide scale. It may not be better to have separate QA, but it is easier for those in charge.

It makes the most sense to me to match the org structure to the teams you have.

If I'm trying to build something with undertrained, demoralized, underpaid engineers... it's not optimal to use methods intended for self-motivated, high-performance teams.

And nothing says there must be company-wide mandates. Maybe this area gets a formal, independent QA team, but this other area doesn't.

My experience just doesn't bear out that collapsing the QA function into development always leads to better outcomes.

I've seen the opposite happen too often, and QA be the sole bulwark between idiocy and customers.

> Ideally we’d always have good software teams, but in the real world sometimes you have to build software with bad teams.

Here's a real, perhaps unexpected counterpoint. Say you have a good software team. How do they build good software with bad management?

They quit.

Exactly!

Quit and do it for someone else

> Does it make sense to degrade the performance of good software teams because bad software teams exist?

Consider the classic statistic "most drivers think they are above average".

I posit that the same is true of software teams, almost every team will self-assess as above average, i.e. good. Those teams will then imagine that, being good, they build quality into the process and very little verification QA is done.

I have worked as a software consultant for 15 years now. I've worked with at least 40 separate software teams in that time. Every single team manager would pep talk with "this is the best team I've ever seen". Some of this is obviously blowing smoke to get people to work harder and feel good. But over the years I've had candid conversations with managers and realized that most of the time the genuinely think their team is really good, truly top 10-20%.

Here's the rub. Being a consultant, I'm almost always brought in by higher level management because something is going horribly wrong. The team can't deliver quickly. The software they deliver is bug ridden. They routinely deliver the wrong software (i.e. incorrect interpretation of requirements.)

Often times these problems are not only the fault of the development team, management has issues too. But in every single case, the development team is in dire straits. They have continuous integration sure, and unit tests, and nightly builds, and lots of green check marks. But the unit tests test that the test works. The stress tests have no reality based basis for expected load. The continuous integration system builds software but it can't be deployed in that form for x, y & z reasons, so production has a special build system, etc...

In 15 years I have never once encountered a team that would not benefit from a QA team doing boring, old school, black box manual testing. And the teams that most adamantly refuse to accept that reality are precisely those that think they are really top tier because they have 90+% unit test coverage, use agile and do nightly builds.

So, my question is, do you (I don't mean the specific "you" here, rather everyone should ask themselves this, all the time) think that most bad software teams know they are bad? Including the one you are part of? Would it really hurt to have some ye olde QA, just in case, you know, you are actually just average? :)

I'm curious: in your many years of being a consultant to these bad teams, where the manager really thought they were top 20%, did you get a chance to talk to the rank-and-file team members, and did they paint a very different picture of the team health and software quality than their manager?

Also, did you run across any orgs where they basically refused to use a process like Agile, and instead just did ad-hoc coding, insisting that this was the best way since it worked just fine for them back when they were a 5-person startup?

Not parent, but in my experience as a consultant working with bad teams, the rank and file were 'doing the job.'

You usually had a few personality archetypes:

- The most technical dev on the team, always with a chip on their shoulder and serious personality issues, who had decided to settle for this job for (reasons)

- The vastly undertrained dev who was trying to keep up with the rest of the team, but would eventually be found out and tossed, usually to blame for a major issue

- The earnest and surprisingly competent meek dev, who presumably didn't have enough confidence to apply to a better job, but easily could have made it on merit, work ethic, and skill

- The over-confident dev who read a bit of SDLC practice, and could see every tree while missing the forest

The key is that, aside from the incompetent person, they had all always been working there for awhile. Consequently, there wasn't good or bad health and quality: there was just "the system" (at that company) and dealing with it.

And none of these folks ever worked at 5-person startups. ;) I think it was definitely more an issue of SDLC "unknown unknowns" they should be doing, than willful decisions not to.

> I'm curious: in your many years of being a consultant to these bad teams, where the manager really thought they were top 20%, did you get a chance to talk to the rank-and-file team members, and did they paint a very different picture of the team health and software quality than their manager?

Yes, generally I join teams and work as an engineer or sometimes as a team lead, so I'm talking to all the team members.

Most start up teams are composed of junior developers, often pretty smart people. Usually 5 or fewer years of experience. Many times these are people who have already accomplished stuff they didn't think they could do. So that generally means that yes they think pretty highly of themselves. To a degree it is quite justifiable, they tend to be very accomplished but in a narrow domain. Unfortunately they don't realize that their technical accomplishments in a specific field does not mean that they are experts everywhere. Their managers understand that these are smart people and assume again that this is therefore a good team.

Non start ups that I join are usually just plain dysfunctional.

> Also, did you run across any orgs where they basically refused to use a process like Agile, and instead just did ad-hoc coding, insisting that this was the best way since it worked just fine for them back when they were a 5-person startup?

Usually more the opposite. In my experience I come across teams that are sure they must not need any help because they follow all the rules in Scrum and have great code coverage metrics.

It is really common to see this kind of thing. I call it "the proxy endpoint fallacy". It can crop up anywhere that there is something that can be measured. In that example, it would be confusing adherence to Scrum with having a working SDLC or perhaps confusing code coverage metrics with the objective of having bug-free releases.

This isn't a software only fallacy. In politics, GDP is often confused with societal well-being. Always be wary of your metrics and change them as required to keep you tracking your actual goals.

Depending on the shape of the distribution, most drivers could be above average. Average doesn't imply 50th percentile, that's what the median is for. A minority of tremendously poor drivers could certainly mean that most drivers are in fact better than average, in the same way that my friends on average have more friends than I do.

If your devs aren't good what are the chances of your QA team being good enough to make up for their short comings? The dynamics laid out by the parent comment will just hit even harder. Your best bet is to enforce basic practices like continuous integration, coverage goals and maybe a coverage ratchet as a merge gate. Training and education on areas were the team is weak is also a must.

About 2000, Software QA (and almost all traditional QA activities) were changed. The focus was on process over inspection. "Design in quality, do not inspect it into the product"

Suppliers (to include software) were expected to manage the quality of the product they provided; the purchaser would focus on how they managed the process, not in the compliance of every part.

This had a chance until software process was tossed in the name of "agile".

Here's a stupid question: How do you know your process is good unless you inspect it?

"Hey Bob I know you're a competent engineer, but don't worry about specifying a certain type of bolt or loctite, the untrained assembly personnel will figure it out. I'm sure they won't let 200 people die in a plane crash."

Make it impossible to use a wrong bolt, and train the assembly workers.

I recall a bug I was involved with at a telecoms equipment market in the early 2000s. The bug only showed up in our biggest base stations in high load situations. We diagnosed the bug, and there were a couple of parts to it. Sloppy software design in an optional hardware module (no state machine) was one part - and was fixed. But there was another underlying issue in the way message queues were handled.

Anyhow, the fix for this was created and written. But we never got to put it into production. The reason: the company didn't have a lab test facility that could put a sufficient load on the software to prove it. Even though we were getting field failures because of this issue that were getting a bad rep, we couldn't fix it because even though the old code was known to be buggy, we couldn't prove the new code. So the process said we couldn't ship it.

Another way of looking at that is that within the ability to test, the implementations were indistinguishable, so the process mandated that the older implementation must be used. I wonder if they would have explicitly specified age as a metric if this was considered when designing the process.

> We have software QA departments that operate very much like manufacturing or aerospace QA.

I don’t work in this industry, but this seems fairly ridiculous on its face: software is not at all like manufacturing.

In manufacturing, there’s a design and a manufacturing process, and a critical function of QA is ensuring that the manufactured produce is manufactured to spec.

With software, the software is written, compiled, and then repeatedly copied. And something should verify that it’s copied correctly, but this is straightforward and boring.

So software QA ought to be much more like the kind of validation that happens when designing hardware, not like the kind of testing and validation that happens as products are manufactured.

Ideally there should be a solid spec written and then qa can test against the spec. Maybe there is somewhere that does write solid specs, including accounting for corner cases, but in my 25 years working professionally in the industry, I’ve never seen it.

The only complete and precise specification of software is the code itself. If some other form of specification was complete, we would be able to auto-generate the code.

This is beside the point. The code specifies what the product _is_, not what it "should" be. If you ask for a word processor and I deliver a perfectly bug-free and feature-complete calculator would you really believe it lived up to spec?

This is also beside the point. I think both of you are trying to warn against the dangers that lie on both sides of this coin: people can invest too heavily in a specification and waste an enormous amount of time, and people can immediately jump into coding and code something that does not do what it was intended to do. Like with all things in life, there’s a balance between these two extremes that’s correct.

You need some level of specification so you know what you’re building, but you have to keep in mind that the final code defines what the behavior truly is. Sometimes, that behavior unintentionally becomes part of the specification because users begin to rely on it.

I do like the fact that you both used hyperbole to succinctly illustrate the dangers of veering too far in either direction though :)

A (human language) specification is simply _enough_ information about a system that a human can figure out the intention of the author. The smarter and more context-rich the human, the simpler the specification can be. The dumber and less context-rich the human, the closer the specification needs to be to code.

It's asymptotic. By the time you reach a human who is as dumb as an actual computer, the specification _is_ the code.

I work in MedTech. We do this. A design has to be reviewed by QA, and is then tested, and the test is reviewed again. So just to counter the narrative, there are companies that do that, and it is working. In other jobs I also saw the cargo cult of QA. But in some industries it is just crucial, otherwise the pressure is too high to cut corners to implement something. It is a good mechanism to counter the need to move fast and break things.

I work in software for CPU design/verification. Even here, where in theory there should be a rock-solid spec, there's not. There's a 12,000 page architectural specification, which is very helpful for specifying all the end-user visible state. But the microarchitectural specification is scattered all over different PDFs, visio docs, excel sheets, and sometimes the only spec is the RTL code itself.

> this seems fairly ridiculous on its face

I think that's why people always tell each other to not take things at face value.

Of course there is a big difference between sw and hw QA, in the thing that they test, and how they test them.

But they are also very similar. Any QA department has to think about ways that things can go wrong, and what things to test for, how to test, which testing methods, which standards to handle, keeping certifications, etc. During testing you also need to keep reevaluating if you actually are catching each problem/bug and how to implement changes in your company that decreases the amount of problems or increase the amount that you catch.

I think in that way there's a lot of overlap in thinking about business processes and how to identify problems with them.

Of course once a specific binary gets tested and approved by QA it shouldn't matter if it gets copied or whatever as long as you make sure its the same binary (by a checksum for example).

But still making sure that errors don't reach the customer, is vital in any QA. If errors does happen, QA is the department that can make sure that it doesn't happen again. And ofc be able to proof in court that you did your due diligence if something does happen.

> Good software teams build quality into projects by introducing continuous integration, unit testing, creating feedback, and tightening these feedback loops.

No. Good software teams are led by competent, technical management. Managers who aren't afraid to get down into the dirty details. Managers who aren't afraid to roll up their sleeves and write code if they need to.

The process doesn't matter. The management of what is or is not important does. Agile is just one process out of many.

Imagine an accounting team led by someone who never did accounting in their life: "Just make the numbers work out! I don't care how you do it! My bonus is at stake!"

But that doesn't contradict the parent, does it? I'd say you both make good points.

Well... here's a thought experiment.

Let's say you have a bunch of school children and architects create a skyscraper. I've given both groups the process to design a skyscraper.

So in both cases, I should end up with a safe building?

Why not both? Am I missing something? You can have feedback loops and CI and all that, "good craftsmanship" or "good practices" (not "best" practices because those often suck hah), where of course opinions vary on the details of that -- and then someone who is also good at the craft who spends more or most time on helping the rest work together, i.e. manage/lead them.

I’d bet the children would come out better simply because they have parents who are likely multi-disciplined as a group. A disparite group will (almost) always come up with better results than a homogeneous one (at least in my experience)

> Good software teams are led by competent, technical management.

...or perhaps with no managers at all. I'm less and less convinced of the importance of management in engineering except to give investors an illusion of control.

I sort of agree, and I do think it’s possible depending on the team. But unfortunately developers can be too opinionated and get focused on low priority things.

Sigh... This myth that the only people who can competently manage developers are other developers has been floating round for decades.

For some reason, developers seem remarkably blind to the skills other roles and disciplines require. Only a developer can do that, everyone else is basically useless fluff. Maybe it's a form of arrogance or just deep unself-awareness.

Let's apply your reasoning to medicine. I'm sure you would be completely fine with managers telling your surgeon what parts of the surgery can be 'optimized away'.

Haha, funny strawman. My reasoning is that non developers are capable of managing developers, notably people who have good management skills.

Your contention is that the surgeon should be running the hospital.

Hahah, indeed. So have you seen a law department in a company headed by someone who doesn't come from law background? How about a finance department headed by some schmuck who doesn't know anything about finance?

I've seen plenty of departments managed by people who don't come from the background of the department. My current boss is extremely good and came from a different discipline.

Although I don't deny it can help to have the background, it is not necessary to be a good manager of something. Also seen plenty of good techies promoted to management and failing badly.

This is a lie, and you know it. Even a mere idea about lawyers being managed by a non-lawyer would be laughed at. Same with finance, nobody would be stupid enough to even try it

It's ok to disagree, we don't have to accuse each other of mendacity.

I stand by what I said, although my experience is in the technical domain, not finance or law. Maybe those departments are different, I don't know.

I disagree with you. You are stating it, but you are not giving reasons. Managers who weren’t developers tend to not be able to manage the team. They can’t help with or understand the technical decisions made. The non-technical managers tend to be project managers just focused on dates.

We may be talking about different levels of management here. A manager should not be making technical decisions, they should have team leads and architects who do that. It's their job to manage the team, interface with the business, prioritise work and give cover to the team so they can get on with it.

I guess if you have a manager who is making technical decisions, they are really a hybrid manager/contributor role. Maybe that works better in smaller organisations.

Then what exactly is the non technical manager's added value?

He has no experience to lead the team in high pressure situations. Like production being down.

He can't truly have a first person understanding of the work of the people who he manages. He has to rely upon others to tell him who's good and who's bad. That sets up a pecking order.

He can't help or mentor engineers with design decisions, or provide a historical context.

He doesn't understand the technology so there's an immediate communication and knowledge barrier that has to be overcome between him and his directs.

He doesn't feel the pain of a bad decision, because he's not coding it, and he can't emphasize with them since he doesn't code.

He tends to push feature development without fixing technical debt. Again that's pain he personally doesn't feel.

To me, the role you are describing is a principal engineer or team lead, not a manager.

Simply not true however that a good manager can't lead the team in a high pressure situation. I'd say that exactly what a good manager could do well. Obviously they won't be making overtly technical decisions, that's what you are for. They can make business decisions, provide cover, get resources, communicate to other stakeholders... All the bits that need doing but would be a huge hassle for the techies who are trying to fix the issue.

I don’t see much value added by a manager that you described. What you are describing is more of a product manager role.

Not everything of value is technical. That you don't see the value is either because you have great managers shielding from having to deal with all that, or you have always worked in a place that combines management and technical responsibilities (which I never have except for very small companies).

I wouldn't call what I'm describing as product management, although it's possible they could do general management too.

Sure, but I feel it's easier to teach management skills to technical person, than it is to teach tech to someone that's only been in management.

The head of surgery should be a surgeon, not an accounting manager.

The head of accounting should be an accountant, not a surgeon.

And even at the executive level of a hospital, you would want people who have spent their careers in healthcare, rather than, say, architecture.

How to tell everyone that you never worked inside a hospital ...

I’m going to be using “moral crumple zone” in every conversation I possibly can from now on.

"Hey, wanna hang out with a beer this evening?"

"Nah, I don't need to slouch anymore in this moral crumple zone!"

"We need this new feature in our program!"

"If we implement this it means the management fell in a moral crumple zone"

"What seat would you have to have for your flight?"

"Anywhere, but not in the moral crumple zone, please"

An example from my actual work life, lightly fictionalized—

Them: “Please review this design.”

Me: “Ok, sure, when do you plan to start coding?”

Them: “Oh, it’s already in beta.”

Me: “So you can’t do anything with my feedback, but you’ll say I reviewed it?”

Them: “Well…”

Me: “You’re putting me in a moral crumple zone here!”

High quality comment

I'm not going to argue with the general thrust of your comment, which I think is insightful as to how incentives can compromise objectives. But...

> To give an example of how these dynamics destroy quality in software. I once found a bug in software on a piece of test equipment where a logarithmic search function was not operating on a strictly sorted list. When I pointed this out to my leadership I was told that if we changed any part of code, it would require a new FQT, which would be too expensive to conduct and was not in the budget. Although the bug would have been trivial to solve, and was clearly wrong and would not provide any benefits by remaining in the test equipment software, the process required for changes prevented solving the issue.

I've seen this happen where it was a bad thing, but also where it was a good thing.

It's all about risk.

What risk does the software defect pose to the mission? What risk is inherent in making any change to the software? Noting that even trivial changes can be fat-fingered and thus are a source of risk. I've seen it go wrong this way: a seemingly trivial change was made, but the developer accidentally checked an extra file into source control, causing a further defect.

And then: what is the cost of mitigating these risks? Maybe the software defect is as trivial as its fix. Maybe an acceptable fix would be to write up a workaround in the documentation.

I don't think it's always wrong to say no to fixing issues. I also don't think it's always right that a separate QA department contributes nothing to the organization, even if they act as a handbrake on the software developers (sometimes, precisely because they do that). Human factors are real.

I think you're sort of misunderstanding the role of QA.

You think that QA is a liability shield, but that is only a side effect of the work that they actually do.

The task of QA is exactly that: an entity that tries to assure that the quality is up to some standard. Even in favourable conditions mistakes happen, so how do you make sure as a company that not 1 in every 100 product are faulty and tarnishes the good reputation that your company has spent so much time and money on to build? You hire a QA to make sure problems get caught before delivery.

But if all humans make mistakes, and QA is human, how do you make sure that the QA doesn't make a mistake? A never ending chain of QAs expecting each other?

No of course not. One thing that helps with reducing errors is to have a rigid protocol that is followed to the letter everytime. Pilots, for example, have a preflight checklist that they have to run every time they operate the plane.

The rigid protocol of QA teams is therefore an essential part of their jobs.

Although from your standpoint as a developer it might seem strange that QA is 'preventing' you from fixing a bug, it is actually very reasonable.

Especially since you work in the defence industry, I hope you understand that it is very important that the software that operates radars, planes, missiles, bombs, etc is working exactly as expected. Understandably there is a great deal of effort made to assure that when those things are needed they work exactly to spec.

So in your example it is probably very reasonable that any change you make needs to go through some rigorous process. The fact that it 'only' was about test equipment, doesn't matter because test equipment is just as, if not more important as the stuff it tests.

The reason why QA has the side-effect of being a 'liability shield' is that it gives companies the ability to argue (and proof) after the fact that the company did their due diligence in making sure that the product was to spec.

Especially certification is basically to get an external organisation to approve your QA. In that case if you get sued you can rightfully claim that you did everything that was legally asked of you, and if there is blame, then it is the certifying company using insufficient standards.

Is it plausible that Boeing has "learned" from software/startup/venture-capital culture with regards to tolerating higher risk to minimize costs?

I suspect it's rather a case of parallel evolution between McDonnell Douglas brass and software startup culture, since cost-cutting culture goes back many decades (remember "Chainsaw" Al Dunlap[1] ?) — but I wonder if there's a more direct influence.

[1] https://en.wikipedia.org/wiki/Albert_J._Dunlap

In lots of ways, the "learning" there would just be "capitalism".

It's inherently short-sighted unless forced to do otherwise by legislation. Cutting small corners pays off A LOT until the hammer falls, so there's a massive advantage to doing it / you need to do it if competition is doing it, or you eventually shut down as they take all your business.

It's inherently a race to the bottom. Sometimes that's a net gain, sometimes it isn't.

But all those communist airlines had no problems at all - exceptional build quality and operational efficiency! /S

Yep, Chernobyl being a prime example. Or Komarov's failed re-entry after complaining about the design faults of the vehicle long before launch. Then there was the more uhhm run of the mill backyard blast furnace campaign which contributed to misallocation of workforce which then led to mass starvation.

There are many many more examples. I find it so tiresome to see young people just use capitalism as a catch all for the failure of something. It's such a lazy and uninformed argument.

I'm not carte blanche defending capitalism - its a mixed bag but it sure outpaces the competing systems put forward to date. It does need some stronger safeguards against industry self regulation - that has a bad track record.

I think we're on the same page. Economic systems need failsafes so that they don't suffer from positive feedback loops.

What anti-capitalist sympathizers, in my view, don't realize is that this is due to people being in the loop. These economic systems are merely vehicles, some better than others, but the conductors are people, be they communists or capitalists. At least with capitalisms there is a delayed regulator (negative feedback) in communism it's up to the system to decide if it needs to modify itself.

All major economic systems of all major national economies over the last century have perverse incentives. It’s not a capitalist thing.

Other systems had incentives such as, get it running by such and such date or have yourself and relatives sent to inhospitable place. So people rushed flawed designs into production.

That said, upper management at Boeing needs a shake-up. People need to get fired. They need to do what Intel is trying and that is to get more engineers in charge, or at least grant them veto power on designs.

> All major economic systems of all major national economies over the last century have perverse incentives. It’s not a capitalist thing.

It should be a lesson against dogmatic pursuit of absolutes: capitalism comes in a wide range of flavors, and the worst is if it’s completely unrestrained. Communism produced worse and worse results the further it got from any sort of public accountability, etc.

The two problems that I see is that the concept of nuance is somewhat at odds with having a simple concept to teach kids at school, and there’s always a group which is more motivated to game the system than the average person who really just wants to hang out with their friends, raise a family, etc. rather than play political games. Boeing didn’t start it by any means but they’ve benefited enormously from decades of reduced oversight and elevated pay driven by a sort of cartoon libertarianism where letting people get enormously rich will motivate them to build great things unfettered by “red tape”.

> All major economic systems of all major national economies over the last century have perverse incentives. It’s not a capitalist thing.

They have, but post-Thatcher neoliberal capitalism has taken the existing perverse incentives and made them exponentially worse. We're on a course heading straight to feudalism, just with fancy titles with legal rights replaced by economic might.

"Better to ask for forgiveness than for permission"

Ironically, I believe it was Grace Hopper who said it... Whoops.

That adage is okay, but for it to work not everything can be forgiven — there actually has to be an expectation to be held responsible towards acting on good faith.

Boeing Airliners are much safer now than before they merged with McDonnel-Douglas. (Because basically all airliners are.) And I say that as a regular Boeing critiquer.

The 737 NG from before the merger has a much better safety track record than the 737 MAX.

Here's a Netflix documentary (in the wake of the MCAS crashes) that alleges that after the merger with McDonnell Douglas, the culture of the firm changed. Previously dominated by engineers, it was now dominated by MBAs with a focus on profit and shareholder value.

"With impressive clarity, Downfall: The Case Against Boeing reveals corporate corruption that's enraging in its callousness and frightening in its scope."

https://en.wikipedia.org/wiki/Downfall:_The_Case_Against_Boe... https://www.netflix.com/hk-en/title/81272421

Would you pay at least 2x for your software to have couple more nines of reliability? I’m gonna guess that “no”. At places where it costs $$$ to have bugs shipped to the end customer (e.g phones) or where there’re regulatory requirements they still have dedicated qa.

It depends.

1% of 10000 is 100.

.01% is 1.

If someone came up to me and said, "Hey I can save you 99% of expected costs with 1% of your profit.", I might go for it.

Which is what i said in second part of my comment. For most software businesses the cost of shipping a bug is trivial and/or poorly measured so due to McNamara fallacy it is readily exchanged for well measured cost of having a functioning qa team

I agree.

I’m regularly critical of Boeing Defense (particularly space contracts where I’m a huge Boeing skeptic), but I think people are pretty off base if they think Boeing is just completely incompetent.

Airliner safety is insanely good. Just vast seas of competence, but when there’s a super rare failure, the incorrect impression people get is that Boeing (or Airbus) is just full of incompetency. Almost nothing that humans do is held to the same standard. Not spaceflight, not software, not healthcare, and certainly not automotive.

Flying a 737 Max with a bad door and without the fix to the angle of attack sensor is probably still better per mile than driving. In spite of going at 10 times the speed and miles above the Earth.

You can almost argue it’s held to a higher standard than it should, slowing development of cleaner aviation (and therefore killing more people in the future due to tertiary effects of climate change, etc).

It kind of annoys me when comment sections are filled with people talking about how incompetent Boeing is. It feels like out of shape slobs on their La-Z-boy chairs talking about how incompetent or slow some professional sports players are. Like, airliner safety is just a totally different league than almost anyone else plays in. On the worst day, their better than almost anyone else is on their best.

> Airliner safety is insanely good.

Because I dug it up for another comment, commercial carriers operating under Part 121 (roughly: scheduled passenger and cargo operation) had 4 fatal incidents in the last 10 years. [0]

Totalling 6 deaths.

In 10 years of US commercial carrier aviation.

One of those was literally 'the engine exploded and threw part of the turbine into the cabin (and also shredded some of the wing)'!!

Which resulted in 1 person dying and a successful landing.

[0] https://news.ycombinator.com/item?id=38921664

> Totalling 6 deaths. In 10 years of US commercial carrier aviation.

Okay, awesome. But how much of that was luck with the 737 Max that they didn't crash on US soil by US airlines?

Most of it. Statistically. Its not hard to assign part of the deaths from the MAXes to the US.

How much was a rigorous safety regime and high quality training?

Military training. See my comment above: https://news.ycombinator.com/item?id=38925089

Ya but your sample size is way too small to measure the death rate. Aircraft deaths are rare, but flying is too.

The two MAX 8s that fell from the sky were 100% Boeing's fault and could have happened in the US. If 5% of airline traffic is in the US you can renormalize those hundreds of dead and you get dozens dead.

We know US pilots have been warning about the same issues that led to the deathly crashes later but were ignored. The thing is, one part of US commercial aviation being so safe is a lot of pilots responsible for the jet airliners are ex-military. Someone mentioned Southwest Airlines Flight 1380, yup, captain Tammie Jo Shults was one of the first Navy female fighter pilots. Miracle on the Hudson? Sully Sullenberger was an Air Force captain and training officer. Civilian training, no matter how good, is just no replacement for military training and experience.

I can't find specific numbers but estimates say about one in three has a military background. That's an awful lot.

> I can't find specific numbers but estimates say about one in three has a military background. That's an awful lot.

Not surprising given that pilot training is really really expensive. Airlines love former military pilots because they are a significantly lower financial risk for them. Put them into type rating and off they go, it's rare that one ends up as a dud.

Let's assume American pilots are gods. They were shouting that their crafts were unsafe.

No matter how good they are and how prescient, that doesn't help them if the aircraft computer decides it's stalling, forces a nose down and they cant fight the controls.

But, even if we assume omnipotence from these American pilot gods, and assume they can fly outside the bird and Superman-style catch it, they are still only 30% of American pilots. Just another population to normalize out.

> forces a nose down and they cant fight the controls.

But that's not what happened. According to every report, it is possible to take back control, it's just very much not intuitive and the situation was confusing.

According to the Seattle Times

> However on both accident flights, the angle-of-attack sensor failure set off multiple alerts causing distraction and confusion from the moment of takeoff, even before MCAS kicked in.

> On the Ethiopian Airlines flight, for example, a “stick shaker” noisily vibrated the pilot’s control column throughout the flight, warning the plane was in danger of a stall, which it wasn’t; a computerized voice repeating a loud “Don’t sink!” warned that the jet was too close to the ground; a “clacker” making a very loud clicking sound signaled the jet was going too fast; and multiple warning lights told the crew that the speed, altitude and other readings on their instruments were unreliable.

You can find some of the US reports complaining about MCAS here https://s3.documentcloud.org/documents/5766398/ASRS-Reports-... and it includes

> I manually positioned the thrust levers ASAP. This resolved the threat

Then there's

> B737 MAX First Officer reported that the aircraft pitched nose down after engaging autopilot on departure. Autopilot was disconnected and flight continued to destination

> I called "descending" just prior to the GPWS sounding "don't sink, don't sink." The Captain immediately disconnected the autopilot and pitched into a climb

Another

> Takeoff and climb in light to moderate turbulence. After flaps 1 to "up" and above clean "MASI up speed" with LNAV engaged I looked at and engaged A Autopilot. As I was returning to my PFD (Primary Flight Display) PM (Pilot Monitoring) called "DESCENDING" followed by almost an immediate: "DONT SINK DONT SINK!" I immediately disconnected AP (Autopilot) (it WAS engaged as we got full horn etc.) and resumed climb

> It kind of annoys me when comment sections are filled with people talking about how incompetent Boeing is. It feels like out of shape slobs on their La-Z-boy chairs talking about how incompetent or slow some professional sports players are.

People do this with everything though, and air travel induces a large amount of fear in the populace. Not only are we not generally comfortable flying in the air for obvious reasons, but when it happens almost everyone has to concede control to a few people in the cockpit and on the ground. Driving, even if exponentially more dangerous, affords the illusion of control of one's outcome, given driving or having someone you know driving, and control over the vehicle maintenance, etc, as well as familiarity with the control and mechanism of the vehicle. These things don't exist with airplanes for the vast majority of people.

So, you can see why there is a need to find a human component to air travel problems, because that is something one can fix (fire the incompetent people, fine them, whatever), as opposed to all of the other things which must be accepted or rejected entirely.

It is entirely in line with human nature to do this, regardless of its accuracy or effectiveness.

Being better then driving shouldn't be the standart. Specially driving in the US.

Flying isn't safer then trains I would assume.

Flying has the advantage of being seperated from almost everything else. Most accidents happen when there is mixed traffic, specially cars operated by people with minimal training.

https://turbli.com/blog/the-safest-transport-modes-ranked-by...

So much THIS

> Airliner safety is insanely good. Just vast seas of competence, but when there’s a super rare failure, the incorrect impression people get is that Boeing (or Airbus) is just full of incompetency. Almost nothing that humans do is held to the same standard. Not spaceflight, not software, not healthcare, and certainly not automotive.

And there's good reasons for that. Spaceflight actually is regulated pretty strictly (partially, because any spaceworthy rocket is effectively a missile), and space pilots and tourists both sign up for such missions fully knowing that they will have a very significant chance of dying one way or another - there simply hasn't been enough human spaceflight activity to work out and understand all the failure modes, unlike with other forms of transportation.

Humans, unlike birds, aren't naturally wired to travel by air... they need to be able to trust their lives to a significantly higher degree to someone else behaving like they should, because unlike in a car they have zero control (or the illusion of control) in an aircraft.

Additionally, the inherent security risk of an airliner is very high: what is a widebody airplane at its core? Hundreds of tons of weight, a decent portion of which is fuel, propelled at near-supersonic speed, and only two people in control of it. Anything goes bonkers and you can get thousands of people killed and injured (see 9/11).

In contrast, cars, even trucks, have way less capability to cause damage simply because they weigh so much less. The only thing that comes close is railways, and hell I don't get what the US is doing there, there's barely any regulation compared to European standards (see the videos I linked at https://news.ycombinator.com/item?id=38725988).

Cockpit resource management is also something a lot of industries can learn from. As well as human error analysis. How an error came to be is often much more interesting then the personal shortcomings of the person who caused it.

At it's best, software QA and related methods should be equal to airline manufacture.

Think of railway signalling systems, control-by-wire bits of modern cars, medical equipment, etc. Where the design of the software is formally proven, and the implementation verified to ensure it fits the design.

of course, but most of us aren't working on products where a quality problem would kill hundreds of people. Having aircraft-level QA would be plain silly, you don't expect that level of quality from most other industry like eg guitar manufacturing, do you?

Nope. It is most probably caused by operational stress - rudder assembly is moving, fuselage is also working (compression and decompression cycles on take off and landing, thermal expansion and compression). I bet they don't just put red Loctite on it to keep it from getting loose. My bet is design flaws, not manufacturing or QA.

EDIT: I saw the pictures of bolts with pins and bolts without pins. The ones with pins cannot get loose, the others can. Let's see what happened.

I don't know about you, but in my industry, "QA" also means extensive testing to ensure that part/assembly/etc doesn't break with expected operations. So, yeah, from where I'm standing, this was a QA problem. Something did not get checked or tested as it probably should have.

The 4 restraining nuts and bolts on the door have a cotter pin like mechanism to prevent them from loosening. If assembled correctly they cannot loosen unless the pin fails.

Maybe an inadequate torque specification for the bolt tightening?

Bolts are most likely tightened with a torque wrench or a gun that is set to a torque spec. Over tightening a bolt is as bad as a loose bolt. I speculate these passed QA from Boeing because they might have been correctly torqued to the spec. What happens in field is hard to understand. One possibility is vicinity to the engine can cause extreme vibrations, these can make them loose. Other possibility is the maintenance side of things - maybe a badly calibrated torque wrench could be the reason. Mechanical systems are not inherently immutable.

I would expect lock wire or some other method of ensuring the bolt does not un-torque itself. Especially for bolts that are not required to be removed past final assembly...

When it's happening on a two month old plane it's a production problem.

It's a QA process better than almost every other industry in the world...

Sure, it failed, and it isn't perfect.

But planes have had a long track record of being absurdly safe.

Given everything I've seen so far, I'd bet good money that what happened here was miscommunication between Spirit and Boeing. Spirit started out locking down the plug, then Boeing asked them to just loosely attach it[1] so Boeing could yank the plug for interior/wiring/AC/paint, then someone at Boeing forgot about the "loosely". So now, they get in a hurry (maybe the AC/interior didn't need any access to work on, which makes sense for this MAX variant, it wouldn't need as many hatches to pull wire) and it went down the Renton line as if the plug was fully installed. It's enough to pass high blow inspection and other inspections, but then over time that "shipment config" attachment vibrated out, and pop goes the plug.

Almost certainly systemic issue though, so that sucks. Sucks real bad.

They need to get a Tiger Team or whatever together to look at everything with a shipment config, and make sure those "ship kits" don't leak into the real actual airplane configuration. This is . . ok, this is really manufacturing 101 stuff, but well, things happen.

I'm in the industry, but haven't touched the MAX, so take this with a grain of salt.

[1] Basically a "shipping" or train configuration

They are not related. Probably different types of bolts, for sure different stress types. Rudder assembly is a moving part, these false door panels are not.

I think the point is there were at least two sets of loose bolts at the same time.

> these false door panels are not.

It's not a control surface, but it is a "moving part." That's what's baffling to me, that they spent a lot of effort building this hinge and pin roller system, and designed the door to hinge open up to 15 degrees.

It makes me wonder if there's maintenance procedures that at some point would require the operation of that door to successfully complete. Otherwise, the mechanism itself seems so incredibly overwrought, with lots of additional bolts, castle nuts, retaining pins, and even sprung hinges at the bottom.

Does anyone know why this "plug-type non-plug door" is built this way?

my guess is: to replace gaskets/seals

It's built to be an actual emergency exit.

It needs to be usable depending on how many passengers the interior is configured for.

So it has all of the door bits there. Maybe some parts like the emergency escape slide are not installed.

e: I should be clear that it's not usable as an emergency exit, as configured by Alaska. However the operator could choose to activate it later and install a usable exit.

This is not correct. To the passengers, this just looks like another seat next to a window with a plug installed. It's not a door.

If there was a reconfiguration to a seating standard that required the extra exit, the plug would be removed and a proper door would be installed, with the associated interior pieces.

> To the passengers, this just looks like another seat next to a window with a plug installed. It's not a door.

This is true.

However there's still common hardware in there to allow the plug to be installed and maintained. This is why it's a complicated set of kit vs just bolting in a permanent fixture.

Where did you see that? My understanding is that it's an optional plug door that's used to assist with interior installation. Once the interior is done, it's bolted shut and interior paneling is installed over top. From the inside, you can't tell it's there.

My source is [1], specifically at about 5:05.

Alaskan airlines chose a 178 passenger configuration for their 737-9, and so are not required to have a mid-cabin exit door.

Lion Air's chosen to go with a 221 passenger configuration, and so are required to have an operating door.

Obviously changing up the number of seats isn't done on-demand, you'd need to go for a refit/maintenance cycles.

But if Alaskan decided to change density, or sold the aircraft to someone else who decided to change density - then they could go and do this.

[1] https://youtu.be/nw4eQGAmXQ0?t=305 "The Boeing 737 Technical Channel"

Ahh. Well, in the case of the Alaska flight, it's a plug door and not used as an exit. It's pinned in place with large pin that has a bolt, a castle nut and cotter pin which lock the pin.

> It's built to be an actual emergency exit.

If you are correct, then the implication is that the concern extends beyond door plugs for MAX-9 737s to all emergency exit doors on all models of aircraft sharing this design. This is somewhat reminiscent of the huge problem with the 688 (Los Angeles) class submarines, where the discovery of a faulty weld that had passed inspections raised doubts about all welds.

This is not true. It's designed to be opened when inspecting the fuselage for corrosion or stress cracks at the opening. To open it you have to remove the interior plastic panels and undo the 4 bolts that this accident is about

I recall a running joke from my childhood - from a former communist East European counry - about a certain car saying you should finish the assembly at home after purchase, tightening the screws before first use. Despite being a famously poor quality car - even in the sloppy East European practices - that supposed to be a joke not to follow suit!

That’s “testing in production”, but beast-sized.

I was responsible for safe for flight inspections on military aircraft and the photo included in that post is completely insane to me.

Those bolts being loose (and they are BIG bolts) would mean multiple people in the installation process didn’t do their jobs, and signed their life on the line saying they did.

When I did maintenance, there was someone (QA) there to witness every torqued bolt, inspect every safety wire and installed part.

There is something rotten in Boeing.

The management imported from McDonnel-Douglas?

The same management that drove it to ground?

The fish rots from the head. And these constant problems sure do sound like a new company culture of cutting corners instead engineering first.

Looks like the installer did their job poorly and the QA rubber stamped it without checking. Are the QAs required to make photos to prove that they did their job? Like those food delivery people.

Alternative is wrong bolts, or sabotage.

But more possible - one lazy QA ghosting.

I would doubt one lazy QA, more likely a over worked and time short QA in a company culture that does not allow them to speak up when they need additional support.

Exactly. This is a failure of management.

They're finding problems on multiple aircrafts. This isn't one QA. In fact, forget QA missing this. Bolts on aerospace systems are supposed to be properly torqued and arrested in most cases. How do the assembly people make such big mistakes?

In my experience, no one puts their job on the line over silly reasons like this, unless there is intense pressure (unrealistic deadlines, heavy workload and poor working conditions) that makes mistakes like this inevitable. I wouldn't be surprised if an honest independent review of either company found ridiculous cost-cutting measures and/or emotional overload.

Perhaps DEI hires that don't have to do their full duties because expecting them to is racist or transphobic or something stupid and they'll be protected by their manager who got shafted in his last performance review because he had too many "white males" on staff.

Do you have any reasonable justifications to base that speculation on, or are you just using this as an opportunity to target people racially or based on orientation? This sounds oddly similar to how Boeing tried to save their own skin by pinning the blame on foreign-trained foreign pilots when their own mismanagement in the form of 737 Max 8 MCAS killed hundreds of passengers.

I am extrapolating from the DEI stuff exposed by Ashley St Claire at United Airlines and I'm using this to target patronizing and dangerous DEI initiatives, not individuals.

Boeing should be held liable for hiring based on skin colour and orientation instead of merit, this risking lives.

So you're 'extrapolating' an unrelated incident in a different company to target DEI initiatives at Boeing? If you remember the MCAS saga, you'll note that rot occurred at the exceedingly white executive level while the company was trying to pin the blame on foreign pilots. This is even supported by voice clips and email transcripts. Some of those were outright arrogant. It clearly shows a pattern consistent with profiteering work culture and nothing related to any race.

In fact, this is one pattern I've seen with such racist dog whistles - blame the marginalized groups and target DEI measures without any good justification to save the image of the privileged. It propagates the sinister implication that an employee is definitely bad if they are a DEI hire.

We're talking about a dangerous situation again with the potential to cause loss of life. It warrants serious contemplation as it might affect us some day. The fact that some people will hijack the discussion to advance their racist agenda is beyond disgusting.

DEI is a racist agenda that endangers us all. It should be discarded for a merit based approach that ignores race, religion etc.

> DEI is a racist agenda that endangers us all.

The argument so far has shown how fallacious and unsubstantiated this claim is. The irony in this case is that the exact opposite applies.

> It should be discarded for a merit based approach that ignores race, religion etc.

This so called 'merit' is a distraction meant to hide 'privilege'. There is no merit unless privilege gap is addressed first. If you think DEI hiring counts as privilege, then you haven't seen or recognized systematic discrimination, racist mistreatment and outright kakistocracy. It's like a trained and well-armed soldier asking a farmer for a fair duel over who gets to keep the harvests- there is nothing fair about it.

My original reply had nothing do with race, but you managed to drag it into it for no good reason. The only reason I responded is because I didn't want to lend credence to such crude ideas with my silence. Unless you have any reasonable evidence to suggest that DEI hires are in any way responsible for these mishaps, I'm going to assume that have nothing more to say other than a one-sided diatribe against the challenge to your privileges.

> The argument so far has shown how fallacious and unsubstantiated this claim is. The irony in this case is that the exact opposite applies.

Once you get out of the wacky humanities courses and into the real world this argument falls apart. Look at the aforementioned UA incident.

> This so called 'merit' is a distraction meant to hide 'privilege'

We'll have to agree to disagree on that one.

> My original reply had nothing do with race, but you managed to drag it into it for no good reason

Race is but one component of DEI and the easiest to reach for.

> I'm going to assume that have nothing more to say other than a one-sided diatribe against the challenge to your privileges.

Yes safe landings and safe air travel is a privilege and DEI is a challenge to that. What's more it is but a single industry where public safety and product quality is being sacrificed for the religious dogma of DEI. Its a funny coincidence DEI is Latin for god, but fitting none the less.

Perhaps the installer and/or QA person should be required to be a passenger on the plane after their inspection and before they can mark the task complete.

> The management imported from McDonnel-Douglas?

I've heard this since they killed hundreds of people in the two crashes. Why are they being protected? They have names.

It's not necessarily a question of persons (and the CEO did a resigned after the Max crashes).

It's more a question of culture (oversimplifying, sales vs engineering) and this is harder to change most of the time. Apparently, even the Max debacle was not enough.

> and the CEO did a resigned after the Max crashes

That was a funny one. They replaced him with his chairman... hence, more of the same.

The MD "moles" managed to change the culture of Boeing pretty quickly from engineering- to sales-first. https://www.theatlantic.com/ideas/archive/2019/11/how-boeing... https://archive.ph/vy5p7

Some of these executive quotes read (admittedly with the benefit of hindsight) like satire.

> But the nearest Boeing commercial-airplane assembly facility would be 1,700 miles away. The isolation was deliberate. “When the headquarters is located in proximity to a principal business—as ours was in Seattle—the corporate center is inevitably drawn into day-to-day business operations,” Condit explained at the time.

Oh man, wouldn't want that to happen.

> With ethics now front and center, Condit was forced out and replaced with Stonecipher, who promptly affirmed: “When people say I changed the culture of Boeing, that was the intent, so that it’s run like a business rather than a great engineering firm.”

Indeed.

With a 63 million golden handshake for job well done, on top of the all the money he previously made.

https://www.imdb.com/title/tt11893274/

Because the shareholders like the management? Cutting corners to increase (short-term) profits is more popular with them than focusing on quality. And now that the company has been in the red consistently since 2019 (https://www.statista.com/chart/20660/boeing-earnings-loss/), of course the answer has to be to lower costs even more wherever possible in order to return to profitability...

So should something as mission critical as an aircraft manufacturing co have some ownership/oversight that's not public shareholders? Give the NTSB a few seats on the board

NTSB?? In American style management there's no room for namby-pamby "stakeholders" on the board.

Who do you think the NTSB are?

The NTSB is staffed with people familiar with the industry - it has to be. Thus, a significant portion of these people are former Boeing employees. And historically, many NTSB staff have gone to work for Boeing (and other industry giants) after their stint at the NTSB as well. It's not that large of an industry after all. This is known as the Revolving Door.

> This is known as the Revolving Door.

Regulatory capture

The "MAX" in 737 MAX means "maximum profit", not "maximum safety".

You're joking but it's the real meaning conveyed to their customers, the airliners.

Yeah, that's why the leading low-fare airlines (Southwest, Ryanair) love it - so it's up to the customers to say that they no longer want to fly with an over 50 year old design that was initially a regional airplane but is now being used for transatlantic flights.

  > that was initially a regional airplane but is now being used for transatlantic flights.

And had its engines moved to an unstable location 50 years after the type first flew, necessitating software controls to compensate for which, paradoxically, are there so that the pilots do not need to train on the new aircraft's inherent flight characteristics yet should be disabled by the pilots under some emergency situations.

[REDACTED]

737 Max 9s are made in Renton, WA.

Wby the rush to judgement ("The fish rots from the head."). What was done and not done will be traced and the causes determined with good old engineering and detective work. Responsibility will be revealed and the proper parties punished.

Rushing to judgement merely obscures the true responsibilities.

the point is maintenance personnel might be directly signing off but that doesn't mean they're directly responsible if their management set unrealistic timetables. that's the point, systems are as accountable as individuals but often individuals are easier to indict and scapegoat.

Given Boeing’s success in getting passes directly from Congress, that could be plausibly too rational given the current circumstances. Boeing got to where they are because your outcomes didn’t actually happen.

Bolts can get unscrewed with vibrations. So probably a design error, they did not use the correct type of bolt (the one with the safety pin).

Don't they have many ways to prevent bolts from unscrewing? I know at least a few by doing mechanical stuff on motorcycles, and it seems that other planes don't have such problems (at least not within 2 months after the last inspection).

There are washers, or thread lock glue, but it is still a question of correct execution: has the glue been applied? Was the glue batch used before its shelf life? Were the parts degreased before glue application?

Or did anyone decided to cut corners by using an old batch/skipping degreasing/not putting glue because they were late on delivery?

safety wire and cotter pins are preferred methods on aircraft.

Yes. For mission-critical bolts and nuts, absolutely safety wire and cotter pins should be used.

The engineers at Bell also thought that blind bolts you couldn’t inspect in dailies should be wired.

My understanding is that the bolts did have castle nuts and retaining wires in the design. So either they were incorrectly fitted, or the bolts themselves were under specified with regard to strength.

ah. i've been wondering what the method of securing the bolts would have been and have not seen anyone mention it. torque values or threadlocker is rarely enough for the FAA. it's usually safety wire, castle nuts, lock washers, etc.

Not sure what happened from “the design” to “the field”. The two loosen bolts in the picture are not castle nuts.

The door hinge bolt had a castellated nut and pin, but the screws holding the hinge mount to the airframe apparently had no retaining wire. See leaked photo at: https://twitter.com/ByERussell/status/1744460136855294106/ph...

Those bolts aren’t drilled for safety wire.

Did you torque every bolt every time before every flight?

Every scheduled maintainance that had that on the punch list, yes.

With a seperate follow through by another party to check the work.

That's SOP for US | AU | UK | EU military air mechanical crews.

Every bolt you could see was checked before every flight yes. Every important bolt you couldn’t see during inspection was torqued, witnessed by QA, secured via safety wire or cotter pin, and secondary torque holding was then inspected by QA.

This thing is obviously not just an interior part, look at the meat in those castings, and it’s obviously safety critical, look at the cotter pins on other bolts. Sounds like it was going to be installed behind interior paneling and not inspected every day. For something like that, every important bolt should be secured by secondary methods, torqued and witnessed installed correctly. This looks like a failure in engineering (not having wire on this bolts), then a profound failure in assembly with multiple people not doing their jobs (not torquing, not witnessing, faking logs), risking the lives of passengers.

If this happened at cruising altitude and speed, people would have died. I can’t find the flight number but I believe 9 people died when a jet lost cabin pressure and a piece of the plane while decompressing during cruising altitude over water.

Correct: https://en.wikipedia.org/wiki/United_Airlines_Flight_811

> signed their life on the line

Clearly this is hyperbole.

It wasn’t in the military. Doing this would land you in Leavenworth.

What I mean is: clearly that's not the case in the civilian world.

Though it should be!

I think if someone falsified records of witnessing torque on these bolts when they didn’t, they could easily be held criminally liable had someone died in this incident. I’m not a lawyer but my plain reading of involuntary manslaughter seems to fit this.

From when I worked as an engineer on the assembly line for smaller jets, know that there would be a record trail of exactly who completed the work, who signed off on it and what the work order steps were for anything related to these assemblies and components. This would include the work done at Boeing and their suppliers. Will be interesting to ultimately hear the root cause here.

Let me ask this: assuming they did not tighten the bolts correctly and qa didn't check it, what are the odds they keep an accurate paper trail?

Close to zero, but that doesn’t help them. If the paperwork was falsified to say that everything was tightened and verified, and Boeing stands behind that, now Boeing has to investigate and come up with a plan to prevent these bolts from “mysteriously loosening” (because everything was fine when it left the factory)… which will be an even bigger pain in the ass than just fixing the QA process.

Maybe bring in some QA professionals from... Airbus.

I mean, if they are not doing it in good faith no matter what happens is going to be half assed.

Bolts and machine screws are very interesting. One mistake I, an amateur bike mechanic, made for years was not greasing bolts properly before assembly.

It sounds counterintuitive but without grease a bolt (or machine screw) will bind early with a high torque well before it is correctly tensioned lengthways. The torque is just a proxy for the tension and it is this tension which is needed to fasten your components together as intended.

The grease means that when the torque to turn the bolt reaches the correct value then the bolt is also under the correct tension instead of being because it got stuck in the thread half way.

I keep a toothbrush covered in grease for just this purpose.

It’s also a magnetic one so I can stick it to stuff and it stays put.

> I keep a toothbrush covered in grease for just this purpose.

Speaking from personal experience, this is also useful for brushing a robot's teeth.

Where does one find a magnetic toothbrush?

It’s an old Sonicare brush head (a couple of fairly strong magnets are mounted to the bottom), there are also some kinda neat suction cup toothbrush holders you could use

There are different torque-to-tension equations for lubricated and unlubricated fasteners, of various materials and platings.

Only grease fasteners if specified by the manufacturer, otherwise you're going to apply much higher than specified tension at the same torque value.

I need manufacturers to give "frame spent 3 decades in the salt belt, questionable metal compatibility and won't be taken apart until it breaks again" torque specs.

It's like they think I'm going to take this thing apart and put it back together every year.

j/k, finding torque specs for a decades old steel bike is a lost cause

a good machinists handbook will give you torque specs that take into account fastener size, material, and thread-locking style.[0]

mine is much older than linked version, but i doubt they'd revise that whole section out of it.

[0]: https://www.amazon.com/Machinerys-Handbook-Toolbox-Erik-Ober...

In the car world that torque is measured in ugga duggas from the impact wrench

Yes, that's the way aircraft fasteners are put on.

There's another reason to grease them - the grease keeps the water out which prevents corrosion.

In working on my cars, I always use a bit of grease when assembling fasteners. I've never had one come loose, nor have any rusted themselves on.

A properly designed service manual will specific if the fastener should be lubed. If it doesn't say lube/thread locker should be used, then the fastener should be torqued dry.

In bicycle applications it is common to elide explicit instruction to use grease. Nevertheless it should be used.

It's a complex topic. Depending on the design, lubrication may be required or forbidden. And if it's required, it should normally be on the threads only, under the head is usually forbidden.

And it's very important to understand that in most mechanical design with fasteners, the fasteners provide tension, and friction between 2 faces actually carries the load. Too little tension then fasteners can be sheared off. Too much, then the fasteners may not have remaining strength available for loads that act to add tension to the fasteners.

Using a torque wrench to reach target tension is normally only about +/-30% accurate. Usually design margin allows for this but in very critical applications where margin is not feasible, calibration is done on a sample of the same materials, etc, or a more direct measurement done. More direct measurement can mean hollow fasteners that allow you to measure the amount of stretch, use of ultrasonic measurement to measure stretch, washers with integrated strain gauges, or cleverly designed "tension indicating" washers or fasteners. There are many types so I just those keywords for anyone interested.

I am not saying that this design required such complex methods and sizing these fasteners should not be difficult. There is probably a mistake or lack of control in the assembly process.

Lots of good stuff in this comment but I'm confused about the +/- 30% accuracy quote, in my experience it is relatively easy to reach sub 10% accuracy based on comparing different (good quality) torque wrenches against each other and against a calibrated one.

Is there something I am missing here?

The desire of fastener tension is clamping force or fastener stretch in some cases, not fastener torque. I think what the parent is saying is that a perfectly torqued fastener is only about 30% accurate in terms of clamping force.

I presume his point is that you are measuring torque (potentially very accurately) but what you actually care about is tension. It's the variable factors affecting the relationship between torque and tension that results in most of the +/- 30% accuracy in tension, not the measurement of the torque.

It's easy to calibrate torque wrenches to +/-5% of each other for _torque_. Calibrating a torque value to _tension_ of a fastener depends not only on the accuracy of the tool, but on materials, lubrication, temperature, process, etc. +/- 30% actually depends on having a good process. This only accounts for variations in tolerance, surface finish, etc. If you add lubricant under the fastener head when you are not supposed to, you can easily reach +50 or +100% tension.

Ah, I see your point now, I thought you meant it to be a direct cause of the torque wrenches themselves to be off. That makes good sense and aligns with my experience.

After manufacturing tension tends to drop over time so starting off with a '+' may not be entirely bad assuming it isn't extreme and that it doesn't cause the materials to deform more than permitted. The way I understand it: you apply a certain torque to a fastener in order to get to minimum levels of tension and friction (which still have an engineering reserve) on the fastener itself to guarantee a seal and to stop the fastener from coming loose, so under-tension is far worse than over-tension as long as the over-tension does not result in damage to fastener or the materials, and the allowed tolerances for over-tension are quite large (up to +150% or so normally before any permanent deformation would occur).

Unless you are using 'stretch' bolts which tend to elongate to accommodate any over-tension to end up with something quite close to the intended value. This stretching tends to be non-elastic so you'd have to replace a stretch bolt every time you unfasten it or there is a pretty good chance that it will break and/or that the threads under the ending position of the nut will have deformed so that they end up being stripped if you refasten them because the nut will travel a bit further on every refastening.

Ive been working on cars and motorcycles and everything else other than airplanes for almost 25 years now, from motor swaps to transmission rebuild and custom suspension. The only time I ever use is a torque wrench is on engine internals, or smaller fancy areas into aluminum. Everything else, over the years, you learn by feel.

You pay attention to the size of the bolt, the material it's going into, and it's overall job. Yes you'll mess it up a few times a long the way. Hopefully you're a lot more careful when you see that the cost of making a mistake is a difficult extraction or alteration of some sort. Personally, I think you'll run into bolt/thread failure 10x more from improper insertion, dirty threads, corrosion, and overall entropy than you will from over torquing once you learn a few early lessons.

I've done countless brake jobs, tire rotations, oil changes, spark plugs, valve cover gaskets, shock and bushing changes, without a torque wrench. I've yet to ever had a problem from it after going a little too hard when I was 16 yrs old and learning what bolts and materials can hold what.

The size of a bolt head is directly related to it's target/max torque due to the increased leverage from the larger rotation. This wasn't obvious to me at first but it's helped for rough guesses when I didn't know exact values.

At the Costco where I get my car's tires swapped out (summer/winter tire swap) or rotated, the technician always tightens all the wheel nuts with a torque wrench.

Then another, different technician comes by and checks all the nuts again with his own torque wrench.

When all is done and I get the sign off paperwork, they strongly encourage me to stop by next time I'm there so they can quickly check the wheel nuts again with a torque wrench.

So please, don't "do it by feel". That's how you Boeing. I know I would never patronize your shop again if I saw you being so callous.

A true craftsman double and triple checks his work objectively with other craftsmen, not be proud he can do it blindfolded with one arm behind his back.

I don't work at a mechanics shop.

Costco prior to probably 15 years ago used to just throw wheels on and hit them with air guns without a torque wrench in sight.

Mechanics shops snapping wheel studs was a common problem for years. A decent mechanic would know to never use a high torque impact gun on something with decently low torque threshold, as I said in my OP about experience. But that shows you how many years and millions of cars were/are on the road without caring about exact torque.

The reason Costco does this, and does it twice, is to make sure the lig nuts at tight on the first place... So you don't leave with one loose. The fact that they use a torque wrench is just because they got tired of replacing wheels studs.

Torque specs only apply to clean and clear threads that are free of corrosion and grease -- unless otherwise-specified.

Threads that are this way do not get stuck half-way.

If the threads are already boogered up for whatever reason, then the torque specification is already wrong: The threads aren't clean, clear, and free of corrosion and grease. This can result in under-tensioned fasteners when using a torque wrench as a guide, since boogered threads (rather obviously) can present an impingement that allows a torque wrench to click off before proper tension is reached.

However, grease is not a magical antidote to this condition.

If the threads (boogered or not) are greased, then the torque spec is also wrong: Greased threads are also not clean and clear, and free of corrosion and grease. This can result in over-tensioned fasteners when using a torque wrench as a guide, since grease is (rather obviously) a lubricant -- allowing things to slide more-freely in a way that doesn't allow a torque wrench to click off until somewhere beyond ideal tension.

---

So what do, then, as a home-gamer with a bicycle or maybe a car in the rust belt (but never an aircraft) full of dissimilar metals that are constantly rotting?

You could perhaps kit up to do the Junker test such as in DIN 65151, and make a study of how different greases affects things. You can even make a career out of publishing your studies.

Or: Just make sure the threads are clean, clear, and dry, and then assemble with an anti-seize paste which does not have lubricating qualities that affect final tension yield. (Permatex makes some, as do others.)

---

(And in aircraft, always do what the engineers say. If the engineers are wrong, then: Stop doing whatever it is that you're doing, and consult them.)

In some cases (some spark plugs are what I can think of) explicitly tell you NOT to grease them, as that will let you over-torque them and potentially damage threads (aluminum heads, steel plugs). So using grease is not always the recommended procedure.

Check out this video showing how the plug is supposed to be installed: https://youtu.be/maLBGFYl9_o?t=540

Some of the bolts that would be loosened when the plug is opened during maintenance have a pin to prevent them from turning. That pin is present in this photo: https://x.com/byerussell/status/1744460136855294106?s=46&t=s...

However, the same photo shows other critical bolts that hold the whole hinge on the door are loose, and there’s not meant to be a pin on those.

I’m curious how the decision is made whether to include that pin in the design. Did they idiot-proof the maintenance of the plug, but not the initial installation?

More rampant speculation!

Some quotes from this article: https://www.reuters.com/business/aerospace-defense/spirit-ae...

> As part of the production process, Spirit builds fuselages for 737s and sends them by train with the special door assembly “semi-rigged,” one of the people said.

> “They are fitted but not completed," the person said.

> At its Renton, Washington, plant, Boeing typically removes the pop-out, or non-functioning, door and uses the gap to load interiors. Then, the part is put back and the installation in completed. Finally, the hull is pressurized to 150% to make sure everything is working correctly, the person said.

I can imagine a diffusion of responsibility as to whose job it is supposed to be to tighten those hinge bolts. Spirit is installing the plug in a “semi-rigged” state. Boeing is removing the plug to load the interior, then reinstalling it. I’d hazard a guess that Boeing is not removing the hinges, because the plug can be removed without doing so. What if, when reinstalling the plug, Boeing workers just redo the stuff they removed? They tighten the vertical movement arrestor bolt, put the pin through it, and believe they have done their job? If they never messed with those other hinge bolts, they don’t bother tightening them?

There’s no way that this is what happened. They’re not building IKEA furniture, everything is tightly check-listed.

Evidently not though....

> Evidently not though

Bolts can be loosened by the physical and thermal stresses of use if there is a design flaw. So it’s not evidently an assembly QA problem.

The plane that lost a plug mid-flight was only two months old though. A design problem probably would have shown up in an older one.

> A design problem probably would have shown up in an older one.

Maybe, unless it required really specific conditions that have only occurred for this one or also required some maintenance snafu that also had nothing to do with Boeing assembly QA. We don’t have enough data to establish anything as the most probable explanation yet.

According to the linked article: at least five aircraft. So, not unique to this one. Or the one that blew the door out.

Surely you've heard of reliability bath-tub curve.

Uh, first part of the bath tub curve is driven by manufacturing defects... That's not a counter point at all.

That curve applies to design issues as well, they are more likely to be found in the very beginning (teething problems) as well.

You're talking out of your ass. Watch this documentary about whistleblower complaints as far back as the 737 NG line as they observed workers beating structural parts into place with hammers. Clearly, things haven't improved since then. https://youtu.be/IaWdEtANi-0

I’m no MBA or aerospace engineer, but why sub-contract the fuselage? I get subbing engines, since engine designs can be used on/adapted to other (possibly non-Boeing) airframes, hence opening up economies of scale, etc. But the fuselage is most of the plane by mass and integral to the design of everything else. If you don’t like making fuselages, maybe you should get out of the aircraft business. Was this just a union-busting thing?

Spirit was spun out of Boeing

I don't always love the video format but Mentour talked about Spirit, Boeing, and the 737 MAX a few months ago: https://youtu.be/SmJgweFmoxs

The discussion about snowman hole issues and other problems is really alarming.

Engines are weirder. My understanding is that the airline specs out what the engine requirements are, and any engine manufacturer is allowed to try to meet that requirement.

Customers get to decide which engine to fit and negotiate for them separately from the aircraft.

Sometimes there's only one choice, but other times gives you a chance to have the same (or similar) engine to maintain across your fleet (or avoid that to avoid a total grounding) and whatever other reasons to choose one over the other.

At least that's what the A380 tour in Toulouse said. Kinda makes sense to at least keep the big aircraft builders from throwing too much of their weight around.

>Was this just a union-busting thing?

Yes, and to lower compensation because Boeing’s benefits are probably better than Spirits, and so if Boeing wants to pay the people that make the fuselage less, then having them be employed by someone else helps them pass non discrimination testing for pre tax benefits offered to Boeing’s higher compensated employees.

Also, lower unemployment insurance premiums if they want to reduce the amount of labor they buy, since Spirit would be the one laying people off.

Boeing sold the division that makes the fuselages. Could have been for any number of reasons.

The castle nut design requires the use of a cotter or other locking pin.

https://en.wikipedia.org/wiki/Castellated_nut

https://en.wikipedia.org/wiki/Positive_locking_device

"Loose bolts" is not a ton of detail. It could have nothing to do with any of the bolts discussed in that video.

I'm having a hard time imagining how this failure could occur from just those bolts "needing tightening". They are lock bolts with pins and appear to take shear forces and provide no clamping functionality. Even if the bolts were "loose", or not torqued to spec, how would they come all the way undone? Then the bolts, under shear, work their way out completely? And isn't the lift spring forcing the top pins into the upper part of the track? On top of that the curve of the track appears to be such that outward force on the door would actually cause the pins to go into the upper part of the track.

IDK, seems like there is something else going on. Different bolts maybe.

The bolts themselves don't take the entirety of the shear forces. They provide tension and therefore friction between the mounting surfaces which is where the shear is handled. By being loose, there is no friction to handle shear and also the item can gain additional momentum (more distance to travel) during shakes and impacts which might increase the impact force above the rated strengths strength.

I'm not an aerospace engineer, but it seems those hinge bolts might be better replaced with rivets --- if the hinge isn't frequently removed from the door.

On at least 5 airplanes? Of one airline? Are loose bolts a minor issue, or is it as insane as it reads?

No, it's as insane as it reads.

It means people were careless and sloppy during assembly.

Think of it like seeing "one cockroach in your kitchen." It's not one cockroach. You just haven't opened up the rest of the walls.

Or it means the torque specs were a little too low, or too high and stripped, or the bolt threading was defective/spec’d wrong. Or bad metallurgy. Or the spec didn’t specify which order to tighten the bolts, so which direction you go causes different outcomes.

Lots of stuff is assembled consistently and carefully “wrong”, but as specified.

idk what aviation standards say but shouldnt loctite or spring washers at least be used

Bolts are torqued down to spec and if specified will have a retention mechanism, usually a pin, a foldover or a wire.

These bolts were fitted with a wire pin to prevent the nuts from spinning loose. The nuts were not locking nuts on their own.

Some part of the process -- anywhere from specs to assembly-- had some cost cutting that led to the outcome. The root cause of this problem is something financially motivated.

While I don’t disagree with you that it is possible, it isn’t a very “engineering” approach to declare that is the case without doing a root cause analysis. Stating it as fact is as bad as the MBAs…

You don't know that.

Cost cutting could have been a factor. Or the root cause might have been something entirely different.

It's quite possible to choose to spend more money on a process or method you believe is higher quality, but still discover it has some specific problem that the previous cheaper version didn't.

To be fair it seems there is an image of some of the loose bolts:

https://nitter.net/ByERussell/status/1744460136855294106

I think this should stick out to an assembly person as not quite right.

Note that, as a comment there states, there are tool marks indicating the bolts were both tightened and loosened at various points.

Complicates the issue of who is at fault; where/when were they loosened?

Maybe it was opened for maintenance reasons after it was initially installed. Maybe it was torqued twice to get the lock pin through.

That lock nut may not have lined up correctly for some reason and was backed off a bit to get the pin in? That could make it torqued under spec. But also, it doesn't look like that bolt clamps anything so it may not matter if it was a little "loose"..

Are you looking at the center nut in the photo that's cotter pinned in place, or the two over on the right side that appear to be "loosely threaded on at best, washer rattling along the shaft"?

The center one doesn't concern me. The two that aren't even finger tight do.

I was and missed everyone was talking about those other bolts in this particular case.

Those are much more problematic as the remaining tight bolts could fail leading to a cascade. Though, would expect the flange to still be connected to the post in the aftermath images and the upper pins .. how did they get out of the track?

I wish we had close up pics of all the bits in the airframe opening.

It could also mean their torque wrenches are not being properly calibrated and traced, and have become out of spec. Out of college I worked as a mechanical engineer at a torque testing firm. We made devices with high quality torque sensors which are calibrated against NIST certified weights every 3-6 months, and those sensors are supposed to be used to check every single torque wrench both before and after each work shift to ensure they are accurate. Someone could choose to ignore that process, if they were sloppily cutting costs.

So it's not the worker who ducked up, but there are loose bolts either way because the Walsh gang took over.

A recent example of your points: Toyota recently had wheels literally falling off cars due to mistakes like those.

Just because I'm a pedant and had to check, I can't find anything saying that wheels had actually fallen off, the recall was just that they could :D

Assembly was loppy, management either ignored or, even worse, encouraged it while QA was asleep. Which is really bad...

> It means people were careless and sloppy during assembly.

Maybe. Could also be incorrect torque specs, bad parts, unexpected vibration in that area, etc.

Edit: Yes, none of these are good either. Just saying there are many possibilities.

Those don’t sound very reassuring either. What’s to say that this is the only bolt with incorrect torque spec, or if this part is bad why not others, and if bad parts are not being caught during quality control then what other parts will be next to fail prematurely? And if there are unexpected vibrations in that area, why wasn’t it discovered during test flights? What other vibration issues would they not have accounted for if they weren’t capable of detecting this one?

To be fair, I read recently that another issue with MAX 7 was discovered through flight testing. The issue is that now Boeing wants to exempt plane from safety rules…

The de-icing button issue?

What's crazy to me is that these things do not roll of the factory line in large numbers. If it was a car plant or some other line where large numbers come off per shift, you could find that possibly one operator for one shift set their torque wrench to the wrong setting causing the finished items for that shift to be suspect. But seeing as not one plane rolls of the floor per shift, this is much more systemic like possibly the documentation was wrong or similar where it is persistently done incorrectly. Or maybe just the one plane for that one day that the worker incorrectly set their torque wrench that day.

There is definetly one plabe rolling of FAL per shift. Airbus for example is aiming for a delivery rate of 60 single aisle planes per month.

It can be opposite of your interpretation too. If you make 100,000 cars in a year, each step is done at least 100,000 times and it's justified to have a single wrench, with single non-adjustable torque value, and automated datalogging of applied value for a particular step. Or even a robotic process. Maybe even computer vision to check fastener length.

If you make 10 or 500 planes in a year, you have to rely on multiple people to use multiple torque wrenches, with multiple attachments, to follow multiple procedures to assemble multiple planes using fasteners of multiple lengths, nearby multiple other processes and people. There is a big emphasis on procedures and traceability but there are still so many potential failure points that can go undiscovered for a long time.

Well they left a ladder in the tail of a plane during assembly in the not distant past. It's not like just a couple screws or washers were left behind to rattle around.

An ex worked on a QA-type project related to production for them. It makes me wonder if some of the issues were more fundamental than that project could ever have addressed.

https://www.nytimes.com/2019/06/28/business/boeing-787-dream...

In software this is a reboot or an apology to a client.

In aerospace this is dead people.

Server leaking sensitive personal info due to bug can easily lead to doxxing, which can be amplified to swatting, which can kill.

Hospital system failing to parse patient's allergies can kill.

MCAS

Therac-25

That said, the stuffs I normally work in (or rather what I allow myself to work in) fall in to the reboot and apologize category.

When a junior software engineer drops a table in production, we don’t say the junior engineer was sloppy. We say the protections and process in place were bad.

I don’t like how the sentence in this parent comment blames individuals for process problems.

well, so long as a student's name isn't "Robert'); drop table students;"...

It doesn't matter if the worker is sloppy and should have been fired, or if Boeings training is poor and should be improved.

Boeing has a responsibility to make sure its workers do their jobs as needed.

From Alaska Airlines...

>> "Initial reports from our technicians indicate some loose hardware was visible on some aircraft."

Friendly reminder that 'some' can also mean most or all.

In the same way Apple's "a small number of affected devices" means anywhere from 1 to 99% of devices in the field

It's a cliche thing to say, but if it was a known issue and management papered over it - as the alternative is incompetent engineering instead of just cost-center-managed engineering - someone should go jail?

Did anyone go to jail for the previous Boeing MAX crashes?

At this point there's a pretty clear implication that if some Boeing people had gone to jail that time maybe this wouldn't have happened at all.

not only did nobody go to jail, they didn't even fix the problem:

https://www.travelweekly.com/Travel-News/Airline-News/Omnibu...

Nah. When you have an engineering culture problem, putting people in jail is not going to solve anything. It will scare employees, sure, but it won't fix your culture.

I believe someone at Boeing actually was criminally prosecuted in connection with the earlier MAX 8 problems, but, as in all these situations, it is seriously unlikely that that fixed any actual problems. Most likely, somebody felt they needed a neck to wring and found a convenient scapegoat.

I am curious why the door plug is not a plug door — that is, a design wherein the desired panel would be installed from the inside and sealed by the differential pressure, like a cabin door. This part looks more similar to cargo door; those usually have to open outward for space, but what is the design constraint for this case?

Yep, that's the top level design mistake. They should be locked passively by the pressure differential.

This is the the best update I have seen so far on the incident Alaska Max 9.

The presenter is a 777 pilot and A&P mechanic. Released 2hrs ago.

https://www.youtube.com/watch?v=WhfK9jlZK1o [13:43]

blancolirio is really great at explaining...mishaps. He provides details and an explanation of those details that's accessible. It's the first place I go to "find out what happened" for any kind of air incident.

Not just aviation and not just mishaps, but, almoshaps too!

I first started watching his channel during the Oroville dam crisis in 2017 and he had some excellent coverage of that event. He seems to cover "things which interest him" the venn diagram of which which overlaps with mine quite a lot.

A scary comment from the video:

"I’ve been an aircraft technician for 23 years and we operate the Max9. I’ve opened and closed one of these plugs as well. Keep in mind that other 737 NG’s have these plugs installed in the longer fuselage models, not just the Max. They all work the same way and there’s never been an incident like this. I’m not saying this is what happened, but I can’t see how this could lug could come loose unless the two upper capture bolts, and the two lower bolts through the spring hinges weren’t installed. Even if a set of bolts, either uppers or lowers were missing with the opposites installed, I can’t see how the plug could come loose and depart the airframe. Just my ten cents. " - @jeffropenn

So, about that Max 10 on which you were hoping to get fast-track approval...

Airbus must be having a fantastic week.

Probably gives them some succor given the engine issues they’ve been having.

Airbus is sold out on A320s until the 2030s already.

https://www.bloomberg.com/opinion/articles/2024-01-08/alaska...

Airlines should stop buying Boeing airplanes. At this point they are catastrophes waiting to happen.

Fortunately, most airlines in my country use Airbus planes.

Soothing to read this while waiting to taxi aboard a different 737 model flown by United!

Luckily this seems to only effect one (newer) model of the 737, though you begin to wonder what was missed in the others.

at this point, I'd prefer to be on an older boeing than a newer one...:(

https://www.youtube.com/watch?v=maLBGFYl9_o is a good video explaining how the bolts/doors etc are supposed to go together.

Very interesting video that shows a clear picture of how the plug is held in place. Some interesting spots in the video, 8:44 shows how the upper two locking bolts lock the roller pins into the upper door tracks, and 10:00 show the lower two locking bolts onto the sliding hinge posts. 13:10 shows how the door blew cleanly off, with very little damage to the roller pins and lower hinge posts. Some comments say the same plug have been used on the 737-900 which didn't have this issue.

My armchair speculative guess: there are only 4 bolts effectively holding the door onto the roller pins and lower hinges. Somehow I don't think it's a simple case of someone forgetting to tighten the bolts; since they're using castle nuts, forgetting to torque them down would leave behind extra cotter pins. Those bolts look positively tiny, probably no more than M12 diameter, and are subject to intense shearing forces. In the case of the upper roller pin with locking bolt, they are effectively two cylinders perpendicular and on top of each other, which causes extreme compressive forces to be concentrated on one tiny spot on the bolt. My guess is they cheaped out and switched to an inappropriate/softer bolt, which sheared and/or corroded. If one shears, the load quickly spreads to the remaining 3 bolts which all also shear off. This theory would still jive with OPs article about UA; "loose bolts" may not necessarily mean "untightened nuts and bad QA", but rather, signs that the locking bolts are all beginning to bend or shear.

Shear forces on four bolts, isn't that the same thing that caused the disaster in that hotel with the suspended walkway in the 1980s? With more than a hundred deaths. Is this not a basic case in engineering school, like THERAC-25? Not a licensed PE.

https://en.wikipedia.org/wiki/Hyatt_Regency_walkway_collapse

Longitudinal forces on the bolts there, not shear. A change during construction doubled the force on the nuts and the bolts went through weak weld joints in beams. (Also not a PE.)

>A surgeon spent 20 minutes amputating one victim's pinned and unsalvageable leg with a chainsaw; that victim later died.

Wow

Yeah, the page is a pretty harrowing read.

Most of the pressure of the door should be supported by the stop pads, rather than the bolts. The force on the bolts should generally be from vibrations, not from pressure differential pushing the door out.

other than pressure differential...what about the weight of the plug door itself ... do the bolts support that weight?

or if some other mechanism like the springs on hinges below grew weaker over time .. would that in turn put too much weight on the bolts?

The 4 bolts are there to restrain vertical motion of the door which I presume would not involve large forces. They are meant to keep the door on the 4 cylindrical posts and 12 stop pads and it is those posts and pads that take the pressure differential forces on the door and keep it from blowing out. They are way beefier than the bolts and I believe they are still intact on the accident plane.

do some of these bolts also carry the weight of the door?

what if the springs on the lower hinges (that are supposed to keep an upward pressure on the door, supporting its weight) are weaker than necessary or somehow failed and went unnoticed... would that put too much burden on the bolts and other fastening mechanisms and cause them to fail in turn over time?

Oh wow. I made another comment about how important tension in fasteners is, to provide a clamping load and therefore friction between two faces. I was thinking that there must have been a process failure. Fasteners often shear off if there is not enough tension and therefore friction. But in this case it looks like there is not any clamping load between parts for at least some of the fasteners, so those fasteners basically act only as pins. There are other "stop pad" parts that constrain movement of the plug but as far as I can tell they must be compliant/flexible parts. The video mentions "roller pins" which for a door would be used with a cam mechanism. I think that stop pads plus roller pins/door/cam mechanism holds the door securely, but stop pads plus plug/pins does not.

On paper all 6 degrees of freedom may be constrained with the plug but in principle compliant pads may allow shear forces and/or rattling on the fasteners and they eventually fail.

I will eat my hat if the eventual required solution doesn't involve some kind of mating cup/cone or wedge shaped parts to more positively constrain the plug.

The recurring question is that it was only 2 months old so likely had those bolts underneath the roller thing was not fastened properly or the whole concept is flawed and this one simply had some bad turbulence/rough landings which created the perfect pressure on them.

There were only 2 possible causes of the incident, either the bolts had been replaced with bolts made from an inappropriate material, so they broke, or the bolts had not been assembled correctly, and they became loose.

It appears that the latter is what happened.

Perhaps those who did the assembly of the doors at Boeing did not use the right kind of washers that are needed to prevent the unscrewing of the bolts, or they did not apply the correct torque to the bolts.

It is extremely surprising if such trivial errors can happen during the assembly of an aircraft.

Or the lock wire was left off or done incorrectly

Aren’t there multiple signoffs & inspections to see if the lock wire was being done correctly?

No matter the real root cause, surely multiple layers of precautions must have failed to allow this to happen, right? So given that's a foregone conclusion, how can we discredit a hypothesis based on that?

You are assuming this is problem in the assembly and not in the design?

The same part was used on the 737-900 for quite a while and didn't (as far as we've heard, at least) have any similar issues there.

And you seem to be assuming that the problem is in the design and not the assembly...

The latter seems more likely.

You are both missing the 'faulty process' and 'faulty communications' options.

And:

- "bad parts from vendor" - "bad programming on torque wrench from MRP" - "insufficient training on process" - "tooling was programmed correctly but bad sensors" - "lube mislabeled for 3 days"

This is absolutely not some simple thing. This is why people don't take us seriously as engineers.

For loose bolts, incorrect assembly is by far the most likely cause.

It can be a design problem if the bolts and associated parts like washers have been substituted recently or if the assembly instructions have been changed recently, e.g. by specifying a different torque. If any such engineering change has happened recently, then that would be the likely culprit.

A resonance problem of the aircraft body as supposed by another poster seems extremely unlikely as that would have required significant recent changes to the aircraft body, which did not happen.

The torque or fastener spec could just be wrong in the face of unforeseen resonance or some other in-flight oddity cumulatively loosening the bolts.

I wouldn't be so certain it's installation error or mfr defect of the fasteners...

> There were only 2 possible causes of the incident, either the bolts had been replaced with bolts made from an inappropriate material, so they broke, or the bolts had not been assembled correctly, and they became loose.

Or the ‘correct’ assembly instructions weren’t actually correct, or the design isn’t good enough (e.g. it uses too few bolts, or didn’t check the strength of the bolts when they get cold), or the bolts were of the correct material, but designed too thin, etc.

There's at least 1 million root causes of the incident and while I have my own predictions, this industry is a great example of checking all possibilities before jumping to "2 possible causes".

It's really unlikely that washers are a root cause (spring/split washers are useless and are definitely not used for this type of apllication). "Correct torque" is not trivial to determine and maybe it was calculated right or wrong based on right or wrong materials/conditions or maybe conditions were changed or maybe...

I am 99.9% sure a procedure was followed and a torque or other measurement applied and was documented. This is probably not a trivial error but a cascading failure in design and/or documention and/or integration.

You should contact Boeing with that information, I'm sure they'll find it invaluable.

This is reassuring news to me, as someone who has opted for Airbus over Boeing (and even paid extra for it) whenever given the choice since the first 737-MAX crashes in 2017.

How is this reassuring?! Are you sure you don't mean "vindicating" or similar? Hard to understand how anyone could feel reassured by this.

It's "reassuring" in the sense that I'm reassured I've been making the right decision... but yeah, "vindicating" is probably a more appropriate choice of word :)

What you're describing is validation.

reassuring serving or intended to remove someone's doubts or fears.

can be reassuring if it removed the doubt that he was making a senseless decision. while still being the opposite about flying the thing.

Id feel mostly vindicated (which is worse - no one likes that guy), but also reassured. Reassured that I'm not crazy!

The decision OP took might have been extreme and costly (time wise and effort mostly). People around him might have lightly whispered to him to be reasonable, or "trust the experts". This wears on you eventually and you start to doubt your own reasonableness.

Boeing starts leaving bolts unfastened and all of a sudden you remember the almost weekly stream of pre-pandemic HN articles with new material on Boeing's fall from excellence.

Boeing's marketing dept. was capitalizing that everyone had forgotten about the MAX. The few of us who remembered it maybe weirdos, but we're nut nuts!

I've tried doing this too but unfortunately still ended up on some 737 max due to airplane change after I booked the ticket.

Doesn’t this call into question the manufacturing of ALL 737 MAX’s including the 7 and 8 since they must use the same manufacturing process?

Yeah this isn't a plane issue, it's a guy holding the drill issue as far as I understand it

A guy holding an inspection report issue

If a guy holding the drill CAN cause this issue, that means there's ALSO a drill calibration issue, a bolt inspection issue, an inspection recordkeeping and doublechecking issue, etc.

Humans make mistakes. The whole point of modern manufacturing is to make products better than any human can make them, by layering processes and procedures to catch those mistakes before they get out the door, and continually improve the processes to catch ever more.

Spot on, this is exactly the problem: you can't have 'just loose bolts', it's a whole raft of other issues.

In The Design of Everyday Things Donald Norman mentions the swiss-cheese model, whereby multiple layers in a process need to align correctly for a mistake to lead all the way to an incident: https://en.m.wikipedia.org/wiki/Swiss_cheese_model

Keep seeing this take (just an installation issue), over and over and over again.

I think it's much worse than a design flaw.

Even very carefully-engineered systems can have flaws. Engineering flaws, once identified, can be engineered around, managed, or corrected.

I'm much less comfortable with the idea that the assembly plant for these planes could be a random-critical-failure generator based on how the employees handle a torque wrench.

It calls into question Boeing's entire portfolio because it's all a product of the same management culture.

It's a door design that dates back to the 707.

Sure, but Boeing’s current awful cost cutting management does not date back that far.

If this turns out to be the assembly issue it seems to be, it hints at a deeper, potentially cultural, issue at Boeing. One that could affect all their planes, because why would only be the B737 MAX FAL be sloppy?

Only the max-9 has the door plug, the smaller airframes don't have that door, and the larger airframes always have the door fitted

Maybe Boeing will rethink their plan to claim all variants of the 737 are basically equal.

Boeing already warned airlines on january 5 (before the door issue) to check for lose bolts in a different section of the plane (the rudder). This is not a door plug issue.

It's not really about the door plug per se, it's about how Boeing's entire manufacturing, design and QA process could allow such an elementary mistake to escape notice in the first place.

If they got this wrong, what other bolts didn't they tighten properly? This question is especially relevant since just a few weeks ago, Boeing issued an airworthiness directive about loose bolts in the rudder system.

In any reasonable aerospace organization, the discovery of a systematic problem with the rudder system assembly should have prompted an audit of other processes. Why did that audit not uncover this issue with the plug door?

Presumably, the answer is that Boeing's processes are so incomplete or and/or unreliable so that, even when being given a hint at what kind of a problem to look for, they can't find other instances of the same problem themselves.

It raises the question of what other systems on the plane have problems, and whether Boeing is even capable of confidently answering this question at this point, or whether we just have to wait and see what other parts start falling off.

The crazy thing is that since the old Max fiasco several of us believed that the priorities of the Boeing company were completely wrong: they had decided to cut so many corners to increase profit to the point of negligence. Doing That, in an airplane is just crazy.

Their manufacturing process is just kaput. And there is no force that will make them fix it until their planes start falling mid flight in pieces.

I find it unfathomable that the FAA is basically sitting watching with their arms crossed. They should fine the heck out of Boeing and ground all their planes. But american protectionism is strong in this one.

All the 737 Max of the same type in the incident have been grounded.

Grounding every plane Boeing has ever built would be quite an overreaction and would be disastrous for both passenger and cargo transportation.

Ed Pierson, a former senior manager at the 737 Max factory in Renton, WA, indicated there have been 20 quality/airworthiness issues post-MCAS to date prior to this latest incident that have gone largely unnoticed by the mainstream media.

It's also worth noting that other ex-employees of Boeing have also raised concerns about substandard structural components made by Ducommun and Boeing's own manufacturing processes for the 737 NG (737-600 to -900). There is no way to know which ones are defective because an unknown number of these critical parts were manufactured inconsistently and haphazardly by hand when they were supposed to be CNC'ed and Boeing just slapped them in without much care. The net result is an average Boeing 737 NG is variably structurally weaker than previous models, and prone to fuselage breakup on hard landings, runway overruns, and possibly in-flight during extreme turbulence. Level 2 corrosion on structural components only 8 years old is what happens when nonconforming, trash parts are passed off as "conforming".

Around 2009, the DOJ made a false statement that the NTSB said the NTSB concluded structural components were "not responsible" for AA 331. These were absolute lies drafted by Boeing's lawyer, Mr. Cole, as no conclusion had been yet reached. In a subsequent sworn video statement, the DOJ official admits they spoke Boeing's words largely verbatim.

This is what happens when publicly-traded corporations are run by MBAs who gut a company's core competencies for short-term profits and are allowed to largely regulate themselves as a consequent of regulatory capture and the corrosive influence of elected officials who accept "campaign support" and gold bars.

Is the aircraft industry also adopting the "move fast, break things" now?

Literally yes (at least for Boeing). Pray Airbus doesn't do the same.

"the most scrutinised transport aircraft in history"

Not reassuring.

Not reassuring indeed.

"most scrutinized world leader in history"

Well yeah, keep making things that break or cause issues and people will have to scrutinise more and more.

737 != 737 Max.

but shareholders got rich and that's really what is important.

> Boeing Co. (BA.N) directors authorized a record US$20 billion share buyback program and boosted the company’s dividend 20 per cent -- a sign the planemaker doesn’t intend to stop showering cash on investors any time soon.[1]

[1] https://www.bnnbloomberg.ca/boeing-sets-new-20-billion-buyba...

Why are there bolts and machine screws in use in the first place?

The seating configurations for United and Alaska, the 2 major customers for this aircraft, don't require this plug-door feature. Structurally, for these large carriers, the plug door serves only as a fuselage weakness and failure point.

Why don't they just rivet it permanently to the fuselage and make it non-functional? Per the diagrams shown, it's a complex assembly that serves no purpose at all for these carriers.

(Apparently it was used as a cargo door to furnish the interiors, which seems like a trivial use case.)

Saves Boeing money by manufacturing 1 frame

Yes, certainly. But the carriers could configure it to make it non-functional, since it serves no purpose for them. Or they could spec it as a rivet-it-shut option from Boeing. Just seems crazy to have this big failure point in the fuselage, adding to their maintenance costs, for a thing they don't even use.

And mass.

Aircraft are often sold between airlines, or reconfigured even with the same airline. If United ever wants to reconfigure their 737s to be higher density (which is pretty common), they might need to start using that door as a real emergency exit.

Yes I see, that makes good sense. Given the course of events, it's still a fair question from a design standpoint: Can this be engineered with a reversible option to switch it from functional to non-functional, depending on the seat configuration?

It just seems like a lot of complexity and moving parts, for a feature that's not in use.

> Can this be engineered with a reversible option to switch it from functional to non-functional, depending on the seat configuration?

That's exactly how it's engineered right now, though.

This same design has been used for decades and apparently without problem, so it's probably just a case of someone designed it this way originally, and nobody thought to fix what wasn't (at the time) broken.

Yes, there are two options for that space if you don't need a door there, one which fits most of the door mechanism (bolted shut) and hides it behind a solid wall, and an option that fits a plug with a normal sized window[1], which needs more effort to convert to a working door. Alaska choose the latter.

[1] This can still be swung out for inspection / maintenance, but has no normal latching mechanism, and should be bolted in place.

Was there ever an aircraft more cursed than the 737 Max? Sounds like it was designed by a team of accountants.

Concorde ? maybe? it literally changed the industry online to be cancelled later

The Concorde has been one of the safest airplanes in history, with only one accident in 30 years of service. The story is different for the Tupolev Tu-144, also known as Concordoski, a copy of the Anglo-French plane.

How was it a copy when it made a maiden flight before first Concorde?

How were these not noticed in proper inspections? Is this the first time these planes have been inspected?

They generally don't disassemble the entire plane every time they get inspected...

This required a full disassembly to be noticed?

They do disassemble the entire plane for heavy maintenance checks every 6-12 years. The MAX 9 has been in operation for just a few weeks though...

That’s pretty incredible. After a heavy maintenance breakdown, what percentage of the plane gets replaced? Or is everything already on a routine maintenance schedule so the intention is to visually identify unexpectedly failing parts?

Probably still not every single sub-sub-sub-component.

Things like fasteners all get replaced, other things get replaced or refurbished on defined schedules.

IIRC every bolt's torque value needs to be logged any time it's (re)fastened. At least that's true for aircraft engine maintenance, not sure if it applies to the assembly process of the airframe, I'd sure hope so!

Man, if that's not automated how do they keep the numbers from being fudged? That's a lot of bolts, isn't it?

Most of the fasteners on the airframe are probably not bolts, e.g. rivets on aluminum airframes. But yes it's surely still a lot of bolts.

I'm not sure what the standard is in the aviation industry, but things like data-logging torque wrenches exist, and are used in other industrial applications.

They should put the bolt torque values on the blockchain. That'll fix it.

Which doesn't rule out, say, a design flaw that allows the bolt to loosen over time under certain circumstances.

That's why we have safety wire, which is all over aviation construction.

The Max 9 has been in service since 2018 though, I think if it was a design issue it would have occurred before now. This seems like a Manufacturing QA “escape” to me. Some bolt was either not installed properly or not tightened to the proper spec, QA did not catch it

> The Max 9 has been in service since 2018 though, I think if it was a design issue it would have occurred before now.

That's a bit of a leap. If the bolts are e.g. too soft and have worn out in use, that would be a design issue that would only show up after a period of use.

> IIRC every bolt's torque value needs to be logged any time it's (re)fastened.

So how did this happen if they we checked so recently?

Most likely they were either not installed properly in the first place, or there is some kind of design flaw.

Process issues

Shouldn't the bolts be safety wired after being torqued?

Does wiring bolts actually help? I feel like if a bolt is going to come loose enough such that the wire is all that is holding it in then you have a big problem already. The wire is just delaying the inevitable.

A bolt is either fastened or it isn’t. If the wire is all that’s left to hold it, then it isn’t fastened and your parts are being held together with a very weak piece of wire.

The wire is supposed to prevent the bolt from becoming loose.

The wire or pin doesn't support any of the load. It is there to prevent any vibration from allowing a nut or bolt to rotate.

https://fastenerengineering.com/what-is-a-castellated-nut-an...

The safety wire comes in to prevent a loose bolt from gently unscrewing itself further and completely disassembling itself (which can happen with vibration).

You're much better off with a loose bolt, than a missing one. It'll hopefully make itself known somehow (maybe even just visual inspection) while still remaining somewhat attached, which is better than nothing.

Thanks for the explanation. Does that mean a correctly wired bold should have wires loose enough to allow a loose bolt to give itself away — either through rattling or becoming visibly undone?

When I searched for images of wired bolts just now I saw many examples where the wire is done up quite tightly, which doesn’t seem right.

I don’t know what Boeing do, but you typically see torque markings on many of these bolts on small planes (and cars, roller coasters, other things).

If the painted seal is broken or misaligned, you know the nut has moved.

> Does wiring bolts actually help?

There's different kinds of bolts that get wired, with different kinds of goals and results.

My only experience is wiring bolts according to the rules for club racing motorcycles.

Some things, like oil sump drain bolts for example, wiring them will stop them backing out so far as to fall our, and while it will probably weep oil if it loosened enough to take ups the writing slack, it wouldn't dump all the oil out quickly.

Other things, like axle bolts, if it backs out enough to take up the wire slack, it'll have lost its clamping pressure and that's bad, but not immediately catastrophic. The major axle loads are all in shear so the wheel won't fall off, it'll just get more floppy than it's supposed to be. Long term it'd be bad and cause wear to the point that something would eventually fail, but it'd get you to the end of the race most likely, and hopefully the rider will have felt something odd and gone looking or a post race inspection would have caught it.

Happened to me once when I was racing an R6 6 years ago. I didn't torque the rear wheel nut, but had put my safety wire clip back through the axle so the nut couldn't back off. Got a warmup lap in and started in on a fast lap during a practice session, and it was immediately obvious something was wrong. Limped back into the pits and found the loose nut. I said a quick prayer to the wiring rulebook (and the people who crashed before that rulebook existed), and inspected everything on the bike.

We used it in nuclear power so I can say the procedures someone wrote at least believe they can prevent bolt rotation. We also used a lot of loctite.

Precisely... everyone other "expert" on HN is asking the wrong questions

Hot take, but I'll bet this will become more epidemic as companies hire more for quotas rather than merit like they used to.

> Is this the first time these planes have been inspected?

Most likely weren't checked since the planes left the factory yes.

To quote from: https://theaircurrent.com/feed/dispatches/united-finds-loose...

"The five aircraft were delivered to United between November 2022 and September 2023, according to ch-aviation, and would likely not have been through a heavy maintenance C check that occurs every 4,000 to 6,000 hours or two to three years."

I think parent meant inspected by the factory before sending it to the airline, not the regular maintenance check from the airline.

I read that too, but it says they have 79 of this aircraft. Were all of them new? Or did only the new ones have this issue?

I've seen on Toyotas and also at Universal Studios bolts on rides are marked with paint presumably to indicate that the nut hasn't backed out.

Are airliners marked in this way? Is checking the bolts simply a visual inspection, or do the inspectors need to get out a torque wrench?

Workers are trained and qualified to sign of on their work. Some work is double checked by a co-worker. Some is again checked by QA. FAL work, and loose bolts falling under FOD absolutely, are checked by QA. Or at least should be.

No idea how this can happen, Boeing really has to get its shit together.

This is SOP in manufacturing to indicate that a bolt has been torqued to whatever torque specified in the assembly drawing; as an added bonus, if the mark is applied properly, you can also tell at a glance whether the bolt has worked itself loose.

Unlike on a rollercoaster, these bolts are hidden within the fuselage behind the cabin panelling so a regular visual inspection to see if they've been loosening wouldn't be practical. During assembly the bolts should have been torqued and then checked and the bolts would only be reinspected when the panelling is removed after 3 years or so.

Yes aircraft fasteners are often torque marked, and almost all safety critical fasteners have an additional preload locking mechanism. Often these fasteners use mechanical locking features such as cotter pins, safety wire, or safety cable.

Even better, safety critical bolts on these doors are (supposed to be, at least) pinned.

https://www.youtube.com/watch?v=maLBGFYl9_o&t=523s

Even on light aircraft some bolts will have a safety wire through the head to prevent it from turning. You might even see these on bolts holding the propeller on.

common practice in manufacturing to use a paint pen on bolts in assembly lines to check and make sure nothing is backing out aftre maintenance/repair.

> I've seen on Toyotas and also at Universal Studios bolts on rides are marked with paint presumably to indicate that the nut hasn't backed out.

This is also standard procedure for electrical maintenance inside distribution equipment. Every lug/termination is torqued to spec and then marked to indicate it has been torqued down. This should be done regularly to prevent any loose terminations from causing an arc flash.

Kind of non-sequitur, but am I disconnected from reality if at this point I do not want to fly on a Boeing aircraft?

It is really the 737 - not sure what is going on with the MAX design team - but it ain't good

The 787 and 757 seem to have been stable platforms.

I think you're mostly correct but while it's not been as bad as the Max, the 787 has a not-so-short list of controversies, starting with those electrical / battery fires early in the plane's life [1] and going through recently with stray tools, fuselage issues, etc. [2] Not an engineer or a pilot, but like the poster above, I'm a regular person wondering how concerned to be on at least two Boeing platforms these days.

[1] https://en.wikipedia.org/wiki/2013_Boeing_787_Dreamliner_gro...

[2] https://aviationweek.com/air-transport/new-boeing-787-fix-de...

I'm a firm believer that the 777 is the last Good Aircraft that Boeing made.

I would like Google Flights to be able to filter for aircraft equipment manufacturer.

I have a dumb question: why "plug" the doors at all? Why not burn a row of seats and have an extra safety door? Was this at request of the airlines to "meet" minimum safety standards or was there a reason why it's "better" to have less exit doors?

If the regulators don't require the extra exit door, then the airlines would rather put in an extra row for more profits. The extra safety door is only needed depending on seating configuration.

And this just blew every bit of profit you could make off those extra seats.

That's what I figured.

Less cost and weight, and more rows. Only advantages there.

Assuming that it's up to airline's employees (not Boeing) to remove/replace the panel/'plug as needed - and that 'loose bolts' means 'not properly torqued' rather than 'can't be', the logical conclusion is that the problem lies with the airline's employees (not Boeing).

IOW, it's far too early to be speculating about blame.

Why would the airline need to remove/replace the plug? It doesn't appear to be the sort of thing that requires maintenance. They would have to remove the interior wall panels to inspect or work with the hardware involved.

In any case the aircraft is only a couple of months old, so it probably hasn't undergone any major work by the airline yet.

Arguably it should have, if the reports of pressurization warning lights are accurate and meaningful. But sadly, I can imagine them looking for a problem, not finding this one due to the need to take the whole plane apart ("Unable to replicate"), and concluding that the warnings were spurious.

Shades of March 2019, below.

(CNN) In a blistering attack on Boeing, the Air Force's top acquisition official said the company has a "severe situation" with flawed inspections of its new KC-46 air refueling tanker aircraft, after trash and industrial tools were found in some planes after they were delivered to the Air Force.

Edit, remove: What is going on in Everett?

Literally just flew on a United Airlines 737 MAX 9 one week ago. It seems like the craft I flew on has probably been grounded in the week since. I noticed that we were flying on a MAX before boarding and nearly asked to switch flights, but consoled myself that I was being irrational and that the planes were almost certainly fine now. Guess my confidence was misplaced.

FWIW, it's only the Max 9s which have this issue. The Max 7s and 8s don't have an exit in that location (too few seats) and the Max 10s require an exit door to be installed in that location, not the door plug.

It's only the Max 9s that have the option of the door plug, if the installed seat count is below the threshold where an additional exit is required.

I think the concern is more the attitude that allowed this to happen, despite all the issues Boeing has had.

Yes, this seems like a top to bottom culture rot problem at Boeing sacrificing quality control in exchange for reduced costs.

A few years ago, I had read that certain airlines demanded planes manufacturered in Washington rather than South Carolina, and I wonder if that information actually ends up being a useful signal for better quality planes.

https://www.postandcourier.com/business/airline-surveys-poin...

https://www.wsj.com/articles/production-problems-prompt-broa...

I think the Max 9 are made in WA though, so it would be a much wider scale cultural problem than just SC's non-union vs. WA union factories.

That’s unfortunate, seems like there are no Boeing planes left with the level of credibility they had before.

They’re final-assembled in WA. The plug is installed by Spirit Aerosystems in Kansas.

> A few years ago, I had read that certain airlines demanded planes manufacturered in Washington rather than South Carolina, and I wonder if that information actually ends up being a useful signal for better quality planes.

The one with the blown door plug supposedly came off the Renton assembly line. No one is safe.

7s and 10s aren’t out in the wild yet. Only 8s and 9s.

Correct, but I've seen a lot of confusion that its a systemic problem (ala MCAS) with the Max family. It doesn't look that way. It looks like a manufacturing problem on one particular variant.

The 8 had the great MCAS crashes (boeing solution ? tell the pilot to turn it off when it happens !), the 7 has anti icing that burnout the engine nacelle and make it fall (boeing solution ? being exempted from the certification requirement of your engines not falling off and we will fix it in a few years !), the 9 has doors that fall off (boeing doesn't even have a solution on that one, but they know others bolts are at risk too since they warned to check the rudder bolts on jan 5), ... At this point it's a systemic problem with the MAX overall. The only one without issue is the 10 and it's because it's not out yet, but Boeing want a fast track on certification for it because obviously they're doing so good with these there is no need to fully check it.

And frankly this time we're talking about the worst of the worst, the basic first step of making airplanes : being able to ensure parts of it doesn't fall off for no reason. This is not a design issue, not a cost cutting "they didn't put enough sensors" reason, this is straight up "their manufacturing is bad and their QA is not able to catch it", ... These things have barely started flying and 5 of those are already affected (well, 6 ...), this is absurd.

Sure MBA taking over tends to kill engineering companies ability to make great product, but even Hewlett Packard is still able to make printers that print (they suck, but that's by choice from them)...

Boeing overall.

https://www.thedefensepost.com/2023/08/08/usaf-tanker-aircra...

To be fair to OP, cutting QA to save costs does lead to "their manufacturing is bad and their QA is not able to catch it"

  I've seen a lot of confusion that its a systemic problem

United found loose fasteners in a different locations on each of the five aircraft they found loose fasteners on. This is right on the heels of another emergency AD for the MAX about loose fasteners in a completely different location. That definitely hints at a systemic problem.

If you expand the scope a bit, Boeing's had nasty assembly and manufacturing problems across its whole model lineup. I highly doubt this is a one off problem.

Just had all my connections cancelled. So extra day in San Fran for me which is less than ideal, but probably better than being on the flight if something happens.

It was total bedlam at the airport when I got in this morning however. With almost no flights available to replace the grounded ones.

Another red eye special for me tonight but at least no connections.

I mean, you're here now. Plane did its job.

The safety margins on flying machines is really high, so while Boeing really should figure out what the hell is going on, the plane didn't fall out of the sky after a depressurization incident. It failed to a known state (ideally by design!)

That it didn’t fall out the sky is largely due to the door blowing off at a lower altitude by some fluke or miracle. Had this happened at cruising altitude we’d be telling a different story.

Pretty small potatoes compared to what https://en.wikipedia.org/wiki/Aloha_Airlines_Flight_243 survived. These things are actually pretty well built.

What tells you that was by design vs. pure luck?

Given the MCAS thing and other shenanigans Boeing is trying with the MAX certification my bet is on luck.

At this point, I believe Boeing needs to suffer the corporate death penalty, because it no longer operates as a net benefit to the public. It needs to be dissolved, and its assets sold off. Criminal charges should be pursued to the maximum support of the law against all of the corporate officers.

Those in the FAA and other oversight agencies who failed to properly oversee them should also be prosecuted.

Failing the above, perhaps it's time to revive an apocryphal custom of the Romans. The engineer of a bridge was required to have his home below it, as a means of enticing them to make sure it was well constructed. We should require that Boeing board members, and their immediate family, only fly in Boeing planes of same variety sold to carry paying passengers.

I agree with Boeing board members flying Boeing, and likely they already do, but have you ever floated beings above the aether in metal machines? Do we not live in the age of magic?

I'll be charitable and assume you suggest "assets sold off" includes divisions to support existing planes and infrastructure.

Sure, let a more reputable company buy them intact. If there aren't any suitable purchasers, and it's important to national security, then nationalize it.

How do you feel about car makers? They kill a lot of people. Food poisoning, bathtubs, lightning…

I don’t disagree that it appears that Boeing has acted badly here. And I’m frustrated with golden parachutes, out of court settlements, and other ways of evading corporate responsibility. But this is a moment in time, and I think Boeing could redeem itself.

I feel that any corporation which goes rogue deserves to be dissolved, and the people who are responsible should be punished.

Serving the public good should be one of the responsibilities of a corporate charter. If we're not collectively better because a corporation exists, it needs to go.

If we grant Corporations rights, they should also have commensurate responsibilities.

Which is more likely:

* Lazy production line worker hand-tightens bolts and forgets to torque them, faking the results on the electronic torque wrench (which records the tightening of every bolt).

* A design flaw causes vibrations which cause these bolts to loosen.

* A lazy/time-pressured worker skips applying threadlocker before assembly

Insufficient and negligent quality control systems design permits a single point of failure in manufacturing process specifically designed and funded to prevent such things - calling into question the trustworthiness of the manufacturing process compliance for the entire product line.

Those are not the only options.

* Improper bolt tensioning process where bolt tension is later unintentionally reduced due to tensioning order.

* Design or material failure where bolt tension post-installation drops more than design allows.

* Miscalibrated or otherwise defective equipment.

* Other failure in manufacturing not caused by the worker, e.g. missed a necessary second or third tensioning pass (different torque or rotation angle) without appropriate failsafes or final validation to discover it.

Bolt tension validation is normally done with an ultra-sonic scan. Torque only provides a rough approximation of the tension at time of installation, and is only suitable where precision is not a requirement.

I suggest https://www.youtube.com/watch?v=XLzTB4KLCxU for those curious about bolts - a very specific, but interesting video.

> Lazy production line worker

Because of your inclusion of the assumed 'lazyness' of the first I assume that you think it's the first?

why do you think it's more likely that the line worker is lazy and the one that screwed up? why not the 'lazy' designer who thought it would be strong enough, or 'lazy' supervisors that forget to supervise, or the lazy quality-assurer that forgets to assure quality?

Everyone makes mistakes. If your whole organisation relies on nobody ever making a mistake, then your organisation isn't at risk of failing, but is certain to fail. Or alternatively the organisation has already failed in its duties, but the consequences of that failure just haven't popped up yet.

The mistake was forgetting to tighten them. The offence was faking the torque wrench results to hide the fact you forgot to tighten them.

Were these plug doors mounted by Boeing themselves? Or under Boeing's aegis, which is the same thing. Or were they sold in a different configuration and an external contractor did the modifications?

It is not so clear if the loose bolts are on the door or if they're the bolts that attach the door to the frame. Also given that the door that fell was just found in one piece this makes me suspect the latter

But regardless, not looking good for Boeing

(*door but of course it's more a plug than a real door, just using the term for ease of understanding)

Edit: https://twitter.com/ByERussell/status/1744460136855294106?t=...

Getting to the point where I’d never want to fly a Boeing again after the Max debacle.

Completely shocked they haven’t ditched this plane and owned up that cutting safety for fuel savings wasn’t worth it.

Is Boeing an example of 'too big' to fail?

ie one of the reasons it's not out of business due to the multiple failures over the Max is it's just too important to the US economy?

Not just the US economy but the US military and the worldwide aviation sector.

The Europeans would, if it truly came down to the wire, pay quite a lot to keep Boeing afloat, even though they own its main competitor, such is the danger of a monopoly on large commercial aircraft.

They now know "what broke": https://news.ycombinator.com/item?id=38922366

Flying in this plane in a few days......a bit uneasy to be honest.

I think they are grounded - so it will be something else

You think. The bigger problem is the engineering and manufacturing processes that went into the 737 MAX is garbage. There will be no end to the problems because it's trying to nail mud to the wall. People who choose to continue flying on them are like people who don't wear seat belts: they know it's dangerous but refuse to act in their own interest of self-preservation.

Something strange, isn't it? First, the fuselage panel detaches in the only row of seats with no passengers. What is the probability of this happening? Then, an iPhone is sucked into the hole and is found on the edge of a road after falling 5000 meters in perfect condition with an airline message on its screen. Really weird.

Given that these are plug-style doors even if these were bolts in the hinges it would not risk the integrity of the fuselage seal.

Maybe the plugs aren't flanged enough? If the walls are mostly flat you could see the bolts being load-bearing.

My money is on the window. It was compromised and then the plug/bolts/flanges structure was ripped out of the body.

How would compromise of the window encourage the plug to be ripped out?

My assumption is the decompression event would have put the velocity of that decompression on the door/bolts and they are going to be weaker - and the addition of a window into a plug/door like this adds structural mass to add to that which is being sucked out via the decompression putting them over spec load for that area - but since it was a plug, the surrounding structure was still sound - just that thing couldnt handle the load.

EDIT: Maybe the bolts have a really high shearing tolerance, but not as good of a tensile pull?

I don't think there were any modern cases of airliner windows just being removed like that. They're fairly strong and consist of multiple layers of acrylic that are sandwiched between other components, so the entire window assembly would need to get torn out. Even then, it doesn't seem realistic that the difference in pressure would be big enough to instantly rip a properly-attached door clean off. Of course, only the experts can tell for sure, but my money's on a failure of the whole structure.

See my other post. Not in agreement.

You'd lose your money. The plug is inserted from the outside and retained with fasteners against cabin pressure.

https://en.wikipedia.org/wiki/Uncontrolled_decompression

Look at causes of explosive decompression - look at the mass increase to add windows' struct infra to weight and mass to a plane, look at explosive bolts for emergencies

* Explosive decompression is the most dangerous, and can cause severe injuries or death. It is caused by a sudden and catastrophic failure of the pressure vessel, such as a window blowing out or a structural failure.

* Rapid decompression is less dangerous than explosive decompression, but can still cause serious injuries. It is caused by a more gradual loss of pressure, such as a door being left open or a seal failing.he cowling on the window failed. It was due to poor quality control for the extra mass around the window housings on plugs - the extra mass due to the pull and (IN ADDITION) the patsy ois on bolts.

They dont use explosive bolts in aircraft plugs, apparently, but yeah - this wasnt just bolts. Its a procedural impact, installation impact AND faulty materials QA in componentry.

My bet is on faulty adhesives or faulty QA on bolts.

I am pretty sure BOLTS in a plug door are a ruse UNLESS one can prove that window assembly procedures are either by the same company, the same process, or the same QA from multi-vendors.

Rule that out first.

An explosive decompression will not happen unless a single core seal will break....

Look T THE PASSENGER MANIFEST FOR THIS FLIGHT... look at MH370...

And if you want a mind F - look at Facebooks Husband who died in that flight to mexico....

If you look at the similarities, large numbers of high-tech company execs were taken out when those two crashed....

Who was on this manifest?

AI execs? Who lived.... So just give a list of professions of the passenger manifest. (Yes, aware no deaths - who could have? Who were to folks that missed that flight?)

If a singular bolt was faulty - it wouldnt have been an explosive decomp

It would have been a leak.

Explosive at 10K feet is FF attack level "first priority"

(PS they already ruled IN that they use differing contractors to produce certain parts such as plug doors.

How secure are those?

Lets not forget about Wellstone ,and Poland and so many others. Would you like a Helo ride over the beautiful Amazon?

You are a MORON of Aerial-Assassination, as it isnt your thing. Hope you dont get in a small plane anytime soon - why do you think billionaires dont employ you to watch over their planes, Helios, etc...you not-minion.

>>If a singular bolt was faulty - it wouldnt have been an explosive decomp

You dont know this about me. My Grandfather was the primary nuke designer behind Hanford.

Handed me my first engineering design book before I was 6 years old.

I drew every single design in that book before I was 7.

When I went into HighSchool (I am smoothing over a bunch of details) I had already drawn that book left-to-right.) (ABC's Of Autocad} and I went back and forth.

I am 2nd in the nation of the Cad Olympympics.

Might sound stupid - but imagine that in 1992.

3 Hour test finished in 35 minutes;

I designed the datacenters you feed off you dorks.

I was the principle designer for so many you touched.

I fucking hate how much so many people dont get cred.

Just look at Tiny. - We couldnt ever jump out of a plane without Tiny. A 5' woman fearless who invented the parachute....

Look at Kent (the mips designer) - youll never know about him - but why we use the word "fabric" on networking... (that you may never know... It was he and me on ... (he had a sexual deviency proclivity that was bad for computing vcs) - but when he was at MIPS we designed slot level rack computing in 1996

(If anyone wants to refute this, they can refute my claims of being aware of googles early mother board designs,etc...

Lets not forget when you attempted to sell me a simple domain for $75,000 just because you thought it was such.

We will go with your over-valuated opinion on [who gives a shit]

I found this video illuminating. The name is confusing because these aren’t plug style doors. They can open out like they did here without passing through components https://youtu.be/maLBGFYl9_o?si=Km8qfkEv2YLHDY2G

They're not plug-style doors. They open outward. That's the problem.

Opening outward is mechanically riskier, but more likely to be useful in an emergency. The drill for plug-style over-wing exits: Remove cover from upper handle, grab upper and lower handle, yank upper handle, pull door inside, have 15Kg or more of door land on your lap, turn big clunky door sideways, pitch through hole so it's outside and out of the way, climb through hole. Training video.[1] Few passengers are likely to get that drill right in an emergency, and the flight attendants are in the wrong place to do it.

[1] https://www.youtube.com/watch?v=RCFzEg-t_Bk

As an interesting comparison to your A320 video, it looks like the A320neo overwing exits are now outward-opening instead of plug-style: https://youtu.be/VhiYfyE3Vps?si=FuKeNfpd1pD4sgc_&t=47

Empirically, the things can fall out. So your assumption is mistaken.

They're only a plug when they're in the locked position. These bolts on the door plug prevent the plug from sliding into a position where it can come out.

not plug! https://twitter.com/NTSB_Newsroom/status/1744202059870831016

Looks like 4 large bolts with castle nuts and cotter pins hold the plug in place.

Looking at how the plug is configured, you'd have to really screw up for the panel to fall off.

https://m.youtube.com/watch?v=WhfK9jlZK1o

all of this just to eliminate an emergency exit to fit 2 extra fares on each flight

Honestly I'm surprised why airlines aren't being designed for the slumlord airliner at this point. its pretty obvious from the haphazard window placement on any airplane that it was built to have more legroom but they slid the seats narrower on the seat rails. USB ports never work and power outlets are bent loose to not hold anything plugged into them. Might as well narrow the windows accordingly and ship the plane without cabin electric to save a few cents on initial cost.

And to avoid retraining 737 pilots, I guess.

It might be another fault due to the cultural shift that's been happening at Boeing: https://www.npr.org/2019/10/26/773675393/boeings-cultural-sh...

> Safety and quality were taking a second seat to schedule and cost.

Makes you wonder if there was malice involved?

My grandfather worked for Braniff as a mechanic, and I've heard stories where very strange things would happen that definitely gave merit to some sort of sabotage being a likely explanation. From very specialized tools would be missing not from just one bay, but from all of the bays to other issues that would cause regular maintenance from being able to be completed in a timely turn causing more and more planes to be taken out of service. Lots of things went wrong with that airline, but some of the things just makes you scratch your chin in wonder if it might not be possible.

https://en.wikipedia.org/wiki/Hanlon%27s_razor

I really doubt it is but important to keep in mind that foreign governments running active espionage campaigns against Boeing. The French have dozens of agents. This is from a conversation I had with a person who works in a large counter espionage group at Boeing. This was corporate espionage not defense but I'm sure there's a group for that.

At what point do consumers en-masse refuse to fly on these aircraft, if ever? When there's a few more fatal crashes?

Unfortunately consumers don't have much choice, the airline can change which type operates a flight at any time.

However longer term if Boeing's reputation keeps getting worse that can motivate buying decisions by the airlines. That change will be a very slow one, because fleet changes cost a lot of money in training crews, changing maintenance etc.

For example KLM is changing their short-haul fleet from the 737 NG to the A320neo family, that will take many years to completely roll out.

True, but some airlines probably don't have any in their fleet. If worried consumers find out they could opt to not fly on those airlines.

Most airlines do have MAXs. Delta does not. However, they have ordered some MAXs. So they will have MAXs in the future. That will take quite some time however.

So if it's really that big of an issue for a person, then just fly Delta for the time being.

spirit doesn't have any of these in their fleet, but it's not the most comfortable flight

Delta doesn't have any currently, although they ordered a bunch of the MAX family that will be delivered from 2025 - 2029. It will probably be impossible to avoid them, long-term.

I think it would require a fatal air crash of an American flight with a major celebrity on board.

. . . and the US airline industry has still not had a fatal crash in just under 25 years.

Unfortunately there was one in the Buffalo area in 2009:

https://en.m.wikipedia.org/wiki/Colgan_Air_Flight_3407

Beverly Eckert, Sept. 11 Widow and neighbor of a friend, died on that flight. Still makes me sad for their family.

* 15 years

Still impressive.

Public math. Ugh.

What is considered a US airline, and I'm assuming the minimal fatality count is 1 right?

https://en.wikipedia.org/wiki/PenAir_Flight_3296

Kobe Bryant's death in 2020 doesn't count? I guess because it was a private charter (and a helicopter, but that's still air travel)?

There's a decent list of celebs who have died in aircraft accidents. The attention typically lasts for a single news cycle.

Maybe when airlines start advertising that they don’t use the new 737s.

The more likely scenario is that the Chinese or Europeans will ground the new 737s until some very expensive fix is put into place and not accept any workarounds. That would probably make a difference.

FYI: https://qz.com/1569865/china-grounds-737-max-8-after-ethiopi...

There are mutual recognition agreements between the regulators and if the europeans try something like that, the americans can create counter measures to even the field in a mutually undesired spiral. At this moment, the biggest enemy of Boeing is Boeing itself.

We don't care if people die, just so long as your enforcement is not any more rigorous than ours!

It is more like "our cooperation is more valuable than temporary monetary advantages which could be negated and which could leave everyone worse overall". I'm not sure why deliberately misinterpret my comment, but you are quite wrong. The first 737max precedent is quite instructive for the relationship between the two regulators.

I’m not sure I understand what you are getting at.

I’m assuming that the US is less likely to demand costly changes to aircraft produced by Boeing because of the implications it would have for their own economy. I wouldn’t trust France to preemptive ground a series of Airbus planes either.

See, that, grounding of Airbus olanes, is exactly what EASA (which is European and not French) does. Spontaniously, I can think of fractures in the wings of the A380 when that happened.

I’m just poking fun at the French a bit.

And why are you doing this?

EU literally followa FAA: https://www.businesstravelnewseurope.com/Air-Travel/EU-airli...

"The European Union Aviation Safety Agency (EASA) said it had also adopted the FAA’s Emergency Airworthiness Directive (EAD) to ground this particular. ...

configuration of Boeing 737-9. “EASA took the decision to adopt the FAA’s EAD despite the fact that - to the agency's knowledge and also on the basis of statements from the FAA and Boeing - no airline in an EASA member state currently operates an aircraft in the relevant configuration,” said EASA in a statement"

from a cursory check https://en.wikipedia.org/wiki/List_of_Boeing_737_MAX_orders_... there are not that many 737 MAX in Europe:

* Ryanair (124 planes out of 575)

* TUI (40 planes out of 134)

* Turkish (25 planes out of 400)

* Norwegian (18 planes out of 81)

* Icelandair (3 planes out of 47)

Went to check on Air Canada and I thought they had way more, but they only have 40: https://en.wikipedia.org/wiki/Air_Canada_fleet

and aside 787s, they haven't ordered any more Boeings

it seems healthy to have at most a quarter of each main models

https://news.ycombinator.com/item?id=38917569

I doubt ever, consumers have been proven to select the cheapest airline ticket. The average once-a-year flier likely has no idea what aircraft they are even flying on.

I wonder what fraction of seats/revenue come from people who take one or fewer trips per year.

1% of people represent 50% of the ~~flights~~ emissions (see thread): https://www.theguardian.com/business/2020/nov/17/people-caus...

This about 50% of emissions not 50% of flights

My bad, probably inflated by private jets then?

That's not 1% of fliers representing half the flights though, I think it's more like 10% - it's saying that 1% of the entire world population causes it and only 11% of the world actually flies once a year.

I'd be willing to pay a premium not to fly on such an aircraft but not too big, as even though they aren't quite as safe as they should be, my impression is that the risk is still fairly low. Something on the order of 5%

Statistically speaking it's not worth worrying about. In 2021 there were less than 0.001 deaths per 100M passenger miles. Assuming these aircraft are 10x more dangerous than the alternative, that works out to 0.01 deaths per 100M passenger miles, or 0.00000026 deaths NY to SF flight. For an average american aged 38.9 years[2] with 40 years of life left[3], that works out to a loss of life of 5.4 minutes. I don't think most americans would worry over this, especially compared to other far more deadly threats like obesity.

[1] https://injuryfacts.nsc.org/home-and-community/safety-topics...

[2] https://www.census.gov/newsroom/press-releases/2023/populati...

[3] https://www.ssa.gov/oact/STATS/table4c6.html

>Statistically speaking it's not worth worrying about.

Those statistics don't really work out at all. They take number of miles and deaths, and nothing else. Let's say flown miles and driven miles are exactly the same for arguments sake. If the number of miles flown on planes were traveled by 4 million people, whereas the number of miles traveled by car were from 4 billion people, your likelihood of dying in a plane crash is FAR higher than by car if you're one of the 4 million who fly on a plane.

Miles and deaths alone are pretty poor metrics. There's also the part where I may only drive once or twice a week vs. the average which takes into account people who commute daily. So my odds of dying in a car crash are still lower. The frequency of travel seems just as pertinent as the miles traveled.

> They take number of miles and deaths, and nothing else. Let's say flown miles and driven miles are exactly the same for arguments sake. If the number of miles flown on planes were traveled by 4 million people, whereas the number of miles traveled by car were from 4 billion people, your likelihood of dying in a plane crash is FAR higher than by car if you're one of the 4 million who fly on a plane.

It's passenger miles, so the number of passengers is already factored in.

Excellent analysis.

A better metric IMHO is "crashes involving at least one fatality" per flight. For large aircraft the fatalities are often everyone on board or nearly so. So people care about "what is the probability that the flight I'm getting on has a fatal accident?" This is largely independent of number of passengers or miles flown.

I don't get it, is your point that the "deaths per 100M passenger miles" is underestimating the likelihood of death? Doesn't the metric already normalize for number of passengers on a plane?

It's obfuscating it behind enormous numbers. Most accidents happen on takeoff or landing, so per-flight statistics are ideal for them. Some accidents happen in flight, so I'd maybe include hours in flight somehow. But miles and people counts don't serve any purpose.

Does this matter for the average flyer? Sure, if you're in Hawaii and flying daily for island hopping purposes you should probably estimate your risk as being higher, but the average flyer is still going to have average risk per miles flown (by definition).

People don't change their behavior much when things like this happen. When it comes around to the next big travel weekend, people will continue to book on whatever airline gets them to the right place at the right time for the right price.

Even when people have a direct ability to change what vehicle they travel on, like with car sales, safety incidents/features/records/etc have little impact to no on sales.

This is one of the reasons why transportation safety is regulated rather than being left up to market forces. Market forces don't really have an impact until the safety risk reaches some extreme levels.

As far as I can see, there's no real way to communicate this preference to airlines. I won't book a flight on a MAX, but the airline can change the airframe at the last second and it's highly situational how willing/able I am to just refuse to fly if they swap in a MAX.

I've communicated it albeit very indirectly. When looking at holiday packages for this summer I looked up the route to see if it's a 737 max before booking.

It wasn't, it was an A320 as I expected.

The typical person does not have the information about these failures, won't prioritize better odds of survival over short-term savings, and do not have enough money to choose alternatives anyway.

People are very bad at managing risk. Even with all this, a 737-NG is infinitely safer than getting in a car, and yet people drive every day. Getting into your sedan is several orders of magnitude riskier than a 737. You're far more likely to die en route to the airport.

Obviously that doesn't excuse Boeing, I'm just saying, by the numbers.

[edit] The US has 1.5 deaths per 100M passenger-miles driven. [1]

The 737 NG has hit 10,000,000 flight hours 5 year into operating. That should be around 500,000,000 miles flown. About 180 seats per plane makes that ~90B seat-miles. That's 0.0038 deaths per 100M seat-miles flown.

So the 737-NG models are about 400X safer per mile than a car.

The median American lives 17mi from an airport, so the safety break-even point is 13,600mi flown on a 737-NG, assuming your destination is also a median 17mi away from the airport. If you're flying less than 13,600mi on a given 737 one-way trip, you're more likely to die getting to and from the airport.

Unless my math is off. This is just napkin math.

[1] https://injuryfacts.nsc.org/motor-vehicle/historical-fatalit...

I think HN really overestimates the impact of events like this with the general public.

The very first A320 demonstration flight crashed, killing 3 people, allegedly due to software issues in the new fly-by-wire system, and then involved allegations that Airbus had tampered with the investigation. The demonstration flight was mostly journalists on the plane. [0]

The A330 crashed on one of its certification flights due to an issue with the autopilot, killing all 7 onboard. [1]

Both went on to be some of the all-time best selling aircraft, and meanwhile today in this thread you have people touting Airbus as a paragon of safety. Humans have a short memory.

0: https://en.wikipedia.org/wiki/Air_France_Flight_296Q

1: https://en.wikipedia.org/wiki/Airbus_Industrie_Flight_129

I mention these two specifically because they're examples of high-profile failures attributed due to issues with the aircraft early in their lifecycle, but basically every single popular aircraft type (except the B777 and A350) have had serious issues and loss of life due to manufacturing defects, but still went on to be overall successes. Hell, people still flew on the DC-10 even after its _numerous_ issues like AA 191..

How is Airbus doing meanwhile? I didn't hear about any production quality issues in their aircraft.

> The A320neo family has had four ground fatalities and one hull loss accident as of November 2022.

https://en.wikipedia.org/wiki/Airbus_A320neo_family

A320neo is the direct competitor to the MAX. So yeah, they're doing pretty well.

And that hull loss wasn't on account of a faulty design, assembly or process issues.

Crew and passengers were unharmed after a collision with a firetruck crossing the runway while the aircraft was doing its take-off roll.

Exactly, as I understand all those occurrencies were due to people on the runway for one reason or another.

I am not familiar enough with that incident but I wonder how fast the plane was going and how close it was to actually being airborne, that seems like it must have been quite the impact but given that all crew survived there must be some factor that I'm missing. Blind luck either way.

edit: 235 Kph!

https://www.youtube.com/watch?v=FBEE7bzatKk&t=47s

two 10ths of a second later and it would have been an entirely different story. Ugh.

i'm not sure are you being sarcastic, there were no fatalities on the airplane. but once motorcycle and another time a firetruck crashed with the plane and people died.

I am not being sarcastic. Fatalities were unfortunate, but not due to the airplane design or manufacturing.

Half the A320neo fleet needs to be grounded for 250-300 days over the next year due to engine manufacturing issues affecting their wear (which tbf is entirely Pratt & Whitney's fault). Airbus settled a $2bn suit last year with Qatar Airways over A350 paint jobs so bad they grounded them and refused to take scheduled deliveries. You'll hear more about Boeing than the rest of the industry put together because this is HN...

So Airbus has fault for a paint job? While they should do better, that sounds a lot safer than airplanes losing pieces or crashing down to me.

> You'll hear more about Boeing than the rest of the industry put together because this is HN...

That's not it.

> That's not it.

I mean, it is. The Pratt & Whitney engine issue which is going to ground half the MAX's rival's aircraft for a year because a manufacturing fault makes the engine turbine blades liable to crack garnered two threads and zero comments (Boeing got more for a fun story about paper planes!). Airbus being found at fault but acquitted of involuntary manslaughter by a Paris court last year for the AF447 crash (a 2009 mid air stall with a few MCAS parallels) passed without comment. HN is not a place for aviation news

> Airbus being found at fault but acquitted of involuntary manslaughter by a Paris court last year for the AF447 crash (a 2009 mid air stall with a few MCAS parallels) passed without comment

“Company isn’t found criminally negligent in case where it pretty clearly wasn’t being criminally negligent” isn’t.. news.

I’d be _really_ curious to hear what you think the MCAS parallels are (besides both cases involving a sensor malfunction) if you think that’s news.

Ah...because HN famously never has any interest in tech litigation and definitely won't have anything to say if a court finds in Boeing's failure in several years time!

The parallels are fairly obvious: AoA sensors malfunctioned, the situation was recoverable but the pilots were confused by conflicting and absent cockpit feedback and lack of relevant training, the OEM initially placed the blame entirely on the pilot but the problem was resolved with a tech remedy. Plus a whole lot of scope for speculation about Airbus regulatory capture of EASA and whether a first incident should have lead to grounding etc. Sure, with AF447 the issue was sensors having a (known) proneness to systematic failure rather than lack of redundancy and the plane plummeted because a stabilisation system disengaged at the worst possible time. They're obviously also not exactly the same, and the Qantas Flight 72 (different software subsystem input conflict automatically pitches nose down) near miss was a closer analogue, but they're all related to critical software handling edge cases and how guidance and UX might have mitigated issues. But as I said, you won't get much of a picture of the aviation industry from HN.

That's what happens if your employees don't care about your company.

Industrial sabotage? Grievance sabotage? Quality control?

Hopefully they can track down the cause.

Metrics that don't give employees enough time to accurately complete their tasks? Management whose first and last priorities are quarterly financial projections? Transitioning from external oversight to self-reporting?

I do hope they can track down the cause, whatever it might be.

for the history on Boeing's rot and the issues with this plane, this book is great:

Flying Blind: The 737 MAX Tragedy and the Fall of Boeing

Link?

Also, the rot didn't start with the 737 MAX. The 737 NG has critical problems that went unaddressed that has contributed to multiple fatalities and equipment damage. Here's a documentary about it:

People & Power - On a wing and a prayer (2010) Al Jazeera

https://youtu.be/IaWdEtANi-0

https://www.aljazeera.com/program/people-power/2010/12/15/on...

Gotta love the "We protect our content to protect our future" pop-up message when you try to copy text from the site.

Why don’t they just replace the plugs doors. Those don’t seem to fail.

If it's Boeing, I ain't going.

Such a crappy aircraft - can we just get rid of these already?

Always refactor

Bolts can be loosened, or fail to be tightened on any aircraft. I suspect this may be a problem with people and/or procedure.

The question is not about bolts, but more about what else is hidden there, what else is mechanically neglected that's waiting in there to take lives

Since the aircraft with the incident was extremely new, it is pretty certain that the bolts have not been properly assembled at Boeing, so the problem is with their assembly procedures.

The fuselage of this plane is not built by Boeing. It could be an issue with the contractor, Spirit AeroSystems, who builds it. It could be an issue with some other final configuration procedure. We'll need to wait for the investigation to conclude.

If it’s built so sloppily, will you manage to think of literally every mistake they might have made?

Agreed — while we’re at it, let’s break Boeing up into multiple firms. It has been “too big to fail” for too long.

Not sure that would help - the Boeing subcontractor that likely played a role in the defective parts, Spirit Aero, was previously part of Boeing and spun off in 2005.

If Boeing has insufficient competition, they don't have to care because their customers have limited alternatives. If they don't have to care then they don't have to care if their subcontractors care. They also have limited fear of regulators because the US government will be wary about damaging the country's sole supplier in an important industry.

Break them up and it's easier for both the market and the government to punish bad behavior.

And how would you break up Boeing to create more competition for civil airliners exactly?

Make five of them that each have full rights to the existing company's designs and split their existing physical plant between them. Does one of the new companies have the only plant that makes a certain type of component? That's fine, customers can buy from them like they buy from the existing Boeing, but now the other sister companies have the incentive to build another plant to compete for that business, and the first company has the incentive to keep prices competitive to deter them from doing that. For five years they have to sell to each other for cost, giving them enough time to either negotiate with each other or build new facilities.

They all have the rights to the existing designs, and can create derivatives, but they don't automatically get the rights to the others' derivatives, so they have the incentive to specialize or develop new designs to find a niche, for the same reason the Boeing 737 isn't an exact clone of the Airbus A320 or vice versa.

You realize how type ratings work yeah? An airline wants to fly a common model. They wanted a 737. They would buy a worse plane for twice as much (see: 737 max) over a brand new clean sheet model that’s incompatible with their pilots.

Type ratings work that way because there are so few types of planes, and because the incumbents like it that way. "Oh no, I guess it's impossible for anyone else to produce a new aircraft model because nobody's qualified to fly it, now we'll just have to rest on our laurels forever."

If there were more competitors, they'd lobby to change how the regulations work so they didn't effectively destroy the market for new aircraft types.

The FAA doesn't work like that and I can assure you, the airlines find it to be an enormous pain in the ass.

The FAA is susceptible to Boeing, the huge conglomerate and defense contractor, not any random airline that could go out of business without anybody really caring. But the status quo is good for Boeing because they don't have to pay to design new stuff, or at least they're ambivalent to it. Change that and see what happens.

I've said it here before, but working with Boeing, every project might as well be a different company.

All top 5 defense contractors preferably. And do the same in the EU too. Monopoly (effectively) clearly failed western defense efforts as the war shows now.

Force it to develop actually new airplane, not rely on grandfathered type license from 60 years ago.

They keep trying to do that. This mess of a plane is care of Southwest and Ryanair who wanted a re-engined 737, period.

I don't know that they could. We're talking about Boeing, they're the sort of MBA hive that's embarrassed to have to employ engineers.

I wonder what automotive designs would look like if we allowed that kind of grandfathering. Or maybe we do but people keep wanting new car designs?

We don't really get many new car designs do we? Aren't most cars just built on top of old platforms that only see iterative updates?

I think a 2024 Boeing 737 Max is a lot closer to a 1968 737 original than a 2024 Honda Civic is to a 1972 Honda Civic.

Car platforms change a lot more, a lot more often, IMHO.

I think you would be mistaken. The car’s engines probably have more in common than the two planes’ engines (low-bypass turbofans in the original, vs. latest high-bypass on the newest ones; engines so different that they changed the plane’s handling enough to require the maligned MCAS). Avionics are entirely different and arguably much more vital to the planes function than a car’s instruments. The interiors are also basically not comparable (sure they have seats but that’s about it).

Deliberate or not, the NHTSA and automakers have found a symbiotic relationship where mandatory safety requirements become integral to a planned obsolescence sales cycle. Cars are more reliable and longer lasting than ever, but safety features mean many people feel compelled to buy a new car much sooner than they might have 30 or 40 years ago. Automakers negotiate with the NHTSA to set mandatory safety feature timelines so that features can work their way from luxury models down over the course of several years. By the time a features becomes mandatory, it's already been milked as a sales differentiator, and trickled down into most if not all car models.

I don't know if such a state of affairs is possible with the aircraft or airline industries. Those industries are too rational, whereas the expense of safety features in the consumer automobile market is in part quietly covered by arguably irrational consumer spending habits.

>Such a crappy aircraft - can we just get rid of these already?

It was a beautiful aircraft, destroyed by years of management comittees.

Boeing needs to be broken up and reorganized, full stop. Congress has to act. It is simply too important and vital of a company to US strategic interests to be left in this state.

Maybe Apple, or Elon Musk, or some other person or company with a spare hundred billion could spin up a decent aircraft company?

>Maybe Apple, or Elon Musk, or some other person or company with a spare hundred billion could spin up a decent aircraft company?

SV has tried and failed time and again to realize the enormity of undertaking that is commercial aviation. There are less than 10 countries in the world capable of producing and marketing a competitive modern civil airliner. And of those, a vast majority are state nationalized conglomerates, or in the case of Airbus multi-country efforts. It takes almost the entirety of a country's aviation workforce and industrial capacity to maintain just one of these companies. Boeing cannot be replaced. It has to be fixed.

Elon Musk doesn't have a spare hundred billion. Apple is one of the only entities that does, but it's not really their kind of market. Institutional customers that are going to expect gradual iteration and thorough specifications that allow independent third party repairs etc.

That's more like Microsoft's kind of market, but uh... don't give them any ideas.

I can't even fathom what it would cost to ride an Apple airplane, coach.

Probably not going to happen. If someone was interested, they could have had it easy if they bought up Bombardier's CSeries aviation unit before Airbus did.

> Boeing needs to be broken up and reorganized, full stop. Congress has to act. It is simply too important and vital of a company to US strategic interests to be left in this state.

The MD merger that led to the shift of Boeing management culture was due to the government intervening and forcing a marriage between the two companies because MD was too important for vital defense interests.

It seems there's a sense of malaise falling over everything and everyone in the USA. People at large simply cannot be bothered to care about anything, including torque specs for freakin' AIRLINERS

I know of a pretty famous car tuner in the US (I won't get too specific for various reasons) and despite most people in my community going to him for service over the years, multiple people have come out and proved he rarely if ever torques anything to spec. For the longest time he even allowed customers to watch in the shop space as he worked on their cars and he'd hand-torque many things that the car manual was very clear needed to be specifically torqued for safety and operational reasons.

The fact that people would share this and it didn't curb the amount of business and referrals he got just proved to me what you've said for the longest time.

People don't like to be troubled with details and they'd rather be ignorant of them.

Probably one of those guys that says "click!" whenever they tighten a bolt.

How many times are the torque specs available to me when working on my car? I have no idea what random bolts are supposed to be so they get the good n tight click.

Workshop manuals are your friend.

Buy a service manual for your vehicle? I have yet to see an important fastener without a specification (torque and pattern).

If you are a professional mechanic, it's your responsibility to obtain the specifications and follow them. This is especially true when it comes to a licensed and certified A&P mechanic and not your neighborhood shadetree mechanic.

To be fair, manufacturers are trying to discontinue the ability to 'buy' a service manual and make you pay for x amount of time of access. And IIRC they don't format the page into something that's offline friendly.

It's still doable to get the full manual off, just not easy anymore.

If you don't know what you are doing then better stay away from anything critical. There are 'stretch' bolts, bolts that are definitely not to be overstressed or they'll weaken and bolts that will cause damage to whatever they're bolted into if over torqued. Every workshop manual and every workshop process has specs for all fasteners.

It gets worse when bolts and part are of different materials (say: steel bolt in aluminum part), that's a very nice recipe for trouble (also in the long run, not just during assembly because oxidization will almost certainly occur on the interface and oxidized metal takes up more room than clean metal so unless you very explicitly protect against it the fasteners won't come out without massive damage to the part).

The vast majority of fasteners on your car do not depend on being torqued precisely. Basically anywhere you need to worry about clearance (e.g. bearings). If it's a matter of ensuring nothing comes loose you'll have things like threadlocker or a cotter (split) pin.

Exactly this, old school mechanics work by feel and only need a torque wrench for things like head studs

I've noticed the same with electrical work. How many use torque screwdrivers, even when dealing with high load appliances (eg, a 50 amp EVSE)?

I've worked in stage lighting and general construction at different parts of my career, and I never saw these being used by anyone despite them being very much required for a proper job.

WAGO clamp terminals are a godsend here - no need to take care about wire nuts going loose, screw terminals being too loose or too tight (leading to fracture), ferrules being properly crimped... just insert the cable, lower the lever, and off you go. Unfortunately, standard DIN fuses still come with only screw terminals.

Sounds like Hennessey.

It is even worse than that: those that don’t care will make life intolerable for those that do until they give up trying to help and go away.

You would definitely hope aerospace assembly would be immune to this widespread phenomenon though.

German torque specs: "gutentight!"

Nach fest kommt ab.

My beef is that torque specs are for factory assembly. If you’re using new parts, sure, but once they’re old/re-used parts with some corrosion due to dissimilar metals, age or salt belt, you’re flying blind and probably undertorquing if going by the book.

Given the amount of (soft) aluminum on aircraft, (737 is 80% aluminum), it would be insane to not precisely consider torque.

Steel on steel is more forgiving.

Why would the softness matter for torque, if there has been some shift or compression since original install the torque should still ensure the correct pressure between the fitting, no?

That’s assuming then original install torque spec was correct.

Much easier to strip aluminum threading if you over torque. Steel tends to just bend (and bend back better).

Aluminum also fractures while steel bends/stretch back and forth better.

(This is mainly from my experience working on bicycles, and a little auto)

A major concern is presumably over-torquing a screw and causing partial failure of threads in the material. This could be subtle and difficult to detect, but present as a problem over time.

Soft materials develop wear more easily.

e.g. any slight boogering of the threads of a used fastener could translate into rotational torque that doesn't end up being converted to a clamping force.

(or in more extreme instances, some fasteners are torque to yield, and change shape after their first use, and must be replaced. But I'll presume that's not what they were talking about here)

Aluminum has no fatigue limit, so you really need to do things by the book.

> If you’re using new parts, sure, but once they’re old/re-used parts

Most service manuals require new bolts, fasteners and washers for torque-critical parts.

Skipping that part is even more ubiquitous than failure to properly torque. It takes a lot of effort to buy the correct bolts, only specialists have them to hand for a given make. Even dealers might not have every bolt. So it means you need to procure many more parts you would otherwise (the part you're replacing, plus 20 specific bolts with part numbers).. Most mechanics don't bother unless it's something really big like cylinder head bolts or an axle nut.

That is probably valid reasoning when talking about a neighbourhood mechanic dealing with all brands but it does not hold for an aircraft maintenance shop. Skipping this step makes them liable for any problems caused by failure of related parts so I assume [1] they keep to the book in this respect. Anyone here who has experience with one of these shops want to chime in? When I work on machinery I tend to follow the rules when dealing with stretch bolts since these do tend to fail far more often when re-torqued but for normal (non-stretch) bolts I do often reuse them if they're in good enough shape. I work mostly on farming/forestry equipment and our personal vehicles though, not on airplanes.

[1] which makes an ass (out of) u (and) me

When I was in high school I helped a lot of my friends fix issues caused by quick tire and lube shops. Lots of snapped, stripped, and rounded fasteners.

I imagine at least part of this is that specs (and documentation generally) just suck now. It's nigh impossible to sort out which bits are CYA legalese, and which bits are "no, actually, do this or else <terrible outcome>".

That obviously isn't the problem with airplane manufacturing, and maybe not for car mechanics either. But it's totally endemic in the consumer world.

"Do not operate while driving" on car HUDs. "Do not consume if pregnant" on perfectly safe OTC medications. "Do not continue to ride after a crash" on bike frames.

It's not surprising most of this is just ignored now -- there's no information content. The documentation is nothing more than a list of things for which the manufacturer would like not to be liable, and the marginal cost of adding to that list is ~0. It will grow until we run out of room in the manual / space on the packaging.

"Store between 68 and 75 F." Or what? Is that a "must follow or else death", or a "it may reduce efficacy 0.5%" or a "we've never run a sufficiently powerful study under any other conditions, but there's no theoretical reason it should matter"? It matters quite a bit to me which!

I don't see how we can hope to have good-faith communication under such a heavy threat of litigation. I would not be surprised if /that/ turns out to be relevant to the Boeing issue, even if the rest is unrelated.

Culture problem as we hire people of a generation who were not educated or engrained with responsibility they avoid doing the right thing and this happens.

Yes, because no catastrophic defects have been present in airliners in the past 50 years.

The only cultural problem is in management.

Get this generationalist BS out of here. It isn't supported by the facts. Airline safety has improved with each generation. There were many more plane crashes in the 1970s-1980s when boomers were in their prime. Five times as many!

Heh, I remember back then when they said "Airplane crashes come in threes", while I don't think that's exactly true, the number of large fatal crashes was much higher.

I don't think it's generational per se, but I've noticed a declining level of concern for safety in product design compared to even like 20 years ago. I think people got used to the idea that everything is generally very safe and now there's less of an emphasis, so given enough time we'll start seeing dangerous products again and the cycle will start anew.

I agree. We need way younger CEOs (and legislators!). This current crop is perfectly fine rushing QA to ship defective products as long as it lines their own pockets. In industries where failures won't directly kill people they actually plan the obsolescence to happen.

Leadership from a more environmentally-conscious and empathetic Gen Z would definitely have safer products that last longer.

Also an issue, short term success does not equal long term success. Lots of people in the c-suite management are short sighted making 1 year goals and not more.

For example a company of 400 people laying off vital talent is literally shooting themselves in the foot. If I was a CEO I would take a cut till I could course correct. Losing talent with a layoff is way more expensive.

You end up paying three to six months for that talent to not work. This is a huge problem.

> People at large simply cannot be bothered to care about anything, including torque specs for freakin' AIRLINERS

It may have not been operator error with the torque wrench. Maybe the torque wrench itself was miscalibrated. Maybe the bolt had a flaw in the metal. It seems too early to make conclusions.

An example of flawed bolts on aircraft in an excerpt from https://www.sciencedirect.com/science/article/pii/S136970210...:

A bolt from an aircraft flap control unit fractured in the threaded region of the shank near the shoulder with the head upon installation after a major service. A metallurgical investigation was carried out to identify the cause of failure. The bolt was manufactured from cadmium-plated, high-strength steel. Material checks carried out on the bolt showed that it conformed to the required specification and was found to have an approximate ultimate tensile strength of 1380 MPa.

The fracture surface of the failed bolt was examined using SEM to identify the mode of fracture and determine if pre-existing defects were present that could account for the unexpected failure. The fracture surface exhibited two distinct modes of failure. [...]

The embrittlement in this case was attributed to the cadmium plating, which is applied to the bolts to provide corrosion protection to the steel. Hydrogen is evolved during the plating process, which becomes absorbed by the steel. The cadmium plating acts as a barrier to hydrogen diffusion at ambient temperature so that the hydrogen becomes ‘trapped’ in the steel. In high strength steels (>1100 MPa) this leads to embrittlement. To overcome this problem, high strength steel fasteners, which have been cadmium-plated, are baked at 175–205°C for 24 hours to allow hydrogen to diffuse through the cadmium. In this case, failure of the bolts was caused by insufficient baking after plating, which gave rise to hydrogen embrittlement.

> It seems too early to make conclusions.

In this specific case, I would agree.

I share much of the sentiment of the OP, but I wonder if it is simply my perspective changing as I take on more and more responsibilities. Maybe things have always kind of been this way.

I would be hesitant to say it's all going to hell in a handbasket because that tends to be one of those self-fulfiling prophecies. I prefer to look at things as "challenging, but workable".

>People at large simply cannot be bothered to care about anything

People care deeply about a lot of things, just not this. And why should they? When you have every business decision made either by private equity or public companies with a focus only on the next 90 days it turns out everything goes to shit. Wages are stagnant. Wealth inequality is massive. Our society is sick. Issues like this will only increase. Our rail ecosystem is experiencing the same issues for the same reason.

Random aside: I looked up an Acela ticket this morning.

$1000 for a first class ticket from DC to BOS. 6.5 hours.

Same flight first class with Delta this week is $850. Also 6 hours with layover.

Blows my mind first class rail travel is as much as first class air travel.

Do you mean that the train is _as cheap_ as taking a plane?

In my experience, in the UK, taking a train is more expensive than taking a flight (or driving, for that matter).

Why do people pay more for slower transportation?

Train travel tend to be much more comfortable than bus, car or plane. Especially if the plane takes as long time due to airports being located outside the city, security hazel, lots of stress at the airport and so on.

The train cost more, but you enter the train in the city centre, sit down and relax, go to the restaurant car and eat a nice dinner, relax som more and you arrive in the destination city.

This works better in continental Europe, where you have shorter distances and high speed trains between the capitols, often making the train faster from city centre to city centre than air travel. Traveling from Chicago to SF by train is pleasant, but take a couple days, not a couple of hours.

I'd pay extra to take the train for shorter trips. The entire experience is better, from not passing through a security theater production to the larger seats. I've never had a train sitting on the tarmac for hours, either, my plane trips frequently involve waits and delays.

> had a train sitting on the tarmac for hours

I took the Amtrak a couple times from DC through WVA and have had several delays trying to get from VA to KY due to priority being given to the freight trains. I remember one trip having a one or two hour delay in the middle of no where while we were side tracked.

It is much, much more comfortable than a plane though.

I kinda knew I'd get called out on that, but, yeah, definitely more comfortable. You can even get up instead of, "everyone has to stay seated because any minute now we will be ready." Some trains have a food car. It's great.

> Some trains have a food car. It's great.

Also IME Amtrak's prices are surprisingly reasonable. More like a place around the corner than like an airport food court.

Not OP, but as a simple physics exercise, the fact that it's cheaper to put a metal tube flying in the air instead of hurling it down a pair of low-friction steel rails firmly attached to the ground to get from the same point A to the same point B is mind-boggling. It becomes even more so when you consider that I was easily able to find a ~$500 first class flight for the same route that is non-stop for just 1.5 hours of flight time, so flying is going to be a much shorter door to door time for half the price and probably a better experience to boot.

I haven't ridden Amtrak in a long time, but I would be absolutely shocked if the first class experience on the Acella is as nice, much less worth a premium compared to flying. Regardless, there's pretty much nothing they could offer to make it worth the price premium.

DC to BOS should be a 90 minute flight...?

Presumably the "with layover" is something crazy like a Delta flight that goes via Atlanta. You can get some pretty crazy routing sometimes if you pick the wrong combination of airports and carrier. United has a non-stop with weekday first class tickets for ~$500, so I think the point stands.

Those prices are extreme, I regularly travel 1st class by train from Göteborg/Sweden to Amsterdam/the Netherlands at about €85 for a single with ICE/EC/IC, a ~15 hour trip.

What is the reason behind this thought? I know that most people think like this. Train should be cheaper than air.

I guess it has to do with planes replacing plane, hence plane must be better, hence more expensive.

But the reality is that the plane won because of two things: speed and cost. Air is free. Miles of miles of railroad is very expensive to build and maintain.

Trains are a bit cheaper than planes. But they travel slowly. So cost per mile per passenger is lower for a plane.

Even when subsidized, the cost of rail travel is often more expensive per mile than air travel.

So when Europe and Asia now are building a lot of high speed rail, it's not to make it cheaper, but to lower carbon footprint and to make a more pleasant experience.

Amtrak and plane tickets are priced differently. For each class of ticket, Amtrak has X seats at price A, Y seats at price B > A, Z seats at price C > B, etc. When you check prices or buy a ticket, it will offer you the cheapest remaining ticket in that class. If you book Amtrak early, you'll find it is significantly cheaper than a plane ticket. Wait until a week until departure, and usually the airplane will cost less.

I’d expect rail to cost far more

You're catastrophizing a bit. This isn't an America problem it's a Boeing problem.

American air travel is safer than it's ever been. Going years without a single fatality is now the norm, not an exception. The airlines are well run and do care deeply about safety. The regulators aren't perfect but generally do a good job. The NTSB is world-class.

Boeing will continue to lose marketshare to Airbus and feel some pain which is appropriate. Eventually they'll get their act together or they will continue to suffer real consequences.

Working conditions in the USA have come to a point where many don't care because the company doesn't care about them. People are fired at will after working for 20+ years with zero shits given. There is zero loyalty and in return workers just don't care either.

There are only so many checks and balances you can build in until this situation will catch up with you.

Doesn't amazon have such a huge turn over that they are running out of possible candidate to replace those workers?

Being pushed to deliver more and more packages in shorter time or inspect train cars in less time then required why would one care if a packages gets tossed or a rail car is not properly inspected?

In most jobs the only consequence is to be fired which is already possible for no reason what so ever in many states.

Thanks, Jack Welch. May you burn in hell for making the 21st century worker-employer relationship so adversarial.

Sorry, but habe you ever worked in an aero assembly plant? You're making it sound like it's just like any other warehouse sweatshops job where you need to pee in a bottle while you work to meet your quotas when it's the furthest thing possible from that.

You definitely have enough time to work at leisurely pace and assemble every part right. There's not much rushing going on there. If workers get away with doing dodgy work you have a QA issue.

The report from Al Jazeera in 2014[1] about Boeing's assembly line for the 787 was pretty damning, workers were complaining exactly about quality control issues, Cynthia Cole was at Boeing for 32 years as an engineer and she mentions in the report she wouldn't fly on the 787.

So conditions at the plant in North Charleston haven't looked very good for almost 10 years...

[1] https://www.aljazeera.com/program/investigations/2014/7/20/t...

In 1966, Boeing had 142k employees and thousands more employees working for them indirectly through suppliers. By 1970 they had less than 39k employees.

Let's not pretend layoffs are new.

Are those headcounts from the commercial aircraft division?

If not you've got things like a reduction in manufacture of ICBMs to consider.

https://en.wikipedia.org/wiki/LGM-30_Minuteman

Or use a few drops of threadlocker...

Sagaar from Breaking Points felt the same way in his monologue today.

I commented on another thread[1] but I feel like it would apply here:

> My theory is that this is not limited to Boeing or even aircraft design, it's a much deeper and systemic problem affecting all kinds of fields. We've had a lot of industrial accidents lately.

> When aircraft manufacturing was an emerging industry there were tons of undocumented safety margins and "slack" in the design and production pipeline. Over time, the beancounters start optimizing stuff, so these undocumented safety margins are eroded in the name of efficiency/profit (and sometimes even documented safety margins too).

> Furthermore, workers back in the day had a much better life when it comes to purchasing power (especially when it comes to property), and so could actually "give more fucks" about the job than they do now which is a compounding factor. You used to get a lot of implicit quality assurance back then which you don't get now.

> We've now reached a stage where these undocumented safety margins have been eroded enough that it actually starts to cause issues, and the safeguards that are supposed to catch them aren't good enough, either due to 1) they've never been good enough but just weren't really needed before or 2) they too have been eroded in the same way for the same reason.

[1]: https://news.ycombinator.com/item?id=38896457

The 'malaise' is the rot coming from uncurbed late-stage capitalism. Just about every single company only cares about short-term forecasts which leads to enshittification because there is no long-term planning or allowance for things like quality control.

Privately-owned companies or worker-owned companies are more resilient to this problem because the nature of their existence means a stronger focus on long term goals. Though there are still issues with publicly owned companies exerting enough influence to acquire and subsequently destroy private companies. For example, Rite-Aid acquiring Bartell Drugs then going bankrupt and proceeding to shutter most Bartell Drug stores leaving Seattle with a deficit of pharmacies.

“Show me the incentive and I will show you the outcome.”

Welcome to the outcome. When companies look at employees and employee training as an expense, and when the only way to improve your salary is to go to a new company, expect to see these companies to be hollowed out shells of untrained people.

Is union labor shielded from poor QC?

Are airlines still saving money by “servicing” their airplanes overseas with cheaper labor like my friend told me a few years ago?

After 9/11 a friend who I’m close with was laid off as well as many other maintenance people. United began the process of moving their plane maintenance people to southeast Asian locations to cut down on labor.

Race to the bottom to save money and now loose bolts?

Could AI help firms like Boeing perform more robust and automated safety checks? I'm curious about how much an issue like this can be chalked up to human error vs. poor, semi-automated QA.

GIGO

Oh nice this is the first time I've seen the AI equivalent of "Bitcoin fixes this" in the wild.

This is hilarious, but I think we might be being a bit unfair.

Why couldn't you use something a camera on eyeglasses while doing the work correctly to fine tune a multimodal model, and then infer to a user wearing the same glasses? Audio reply saying "nope, try again."

You would need multiple frames per second, so not today, but not that far into the future, right?

edit: Zero-shot, I just took photos of me "fixing things" around the house, and here is what ChatGPT told me. It does not suck. Do this with fine tuning, many frames per second, and what am I missing?

https://imgur.com/a/H1eSShH

https://imgur.com/a/YXxz8uL

Because those things are a joke ?

Please refresh and look at my edit.

If a missing some major required component aside from finetuning and frames per sec, which will require a few years for everything to be fast enough... please let me know what it is.

Wait, oh it will be understanding of time... the sequencing of the frames. That is missing for now, right?

I think you're giving it too much credit for solving examples that are obviously wrong. Not only a non-expert but even most children know those pairings don't go together.

Given that the discussion in this post is around torquing to the right specifications, I don't know if just fine tuning is enough. It might need more serious training on videos of assembly. Even then, can it distinguish between the right torque or not from video?

I think it's interesting, but you should deliberately try to make it fail to see where the edges are. Like hold a wrench of the obviously wrong size next to a nut and ask if that will work. Force perspective so they look the same size. See if it will prevent you from mixing potentially dangerous chemical combinations. Will it warn you to wear protective eyewear when using a circular saw?

Oh definitely, I was giving it some softballs. Maybe I am also giving too much credit to what fine tuning can achieve.

But in a manufacturing environment, if one labelled wrench sizes with different colors, then things like noticing wrench size would be easier. Also, I imagined that the camera would of known (same) geometry during training and inference in a Boeing implementation.

I also wonder if multimodal LLMs are blowing my mind unnecessarily. It really feels like a huge leap to me though.

They are a pretty huge leap! I just think for manufacturing and assembly where it is critical to get specific values right we cannot rely on neural nets in their current form.

I think there are digital wrenches that will record the torque they applied, you don't need to mess around with colors. But at that point, it's sort of like, why bother teaching the AI to notice if people are doing it right, stick a camera on the wrench and QR sticker on the bolt. Make a more formal verification process that can be guaranteed to match specifications.

The reason why it's bad is that this is an overcomplicated process that introduces potentially unreliable bells and whistles for little upside. At the end of the day, there aren't fundamental issues with a trained employee following a checklist. Machine learning is extremely powerful but tons of issues can be solved in a straightforward algorithmic fashion, so we should really be using it where it could make a big difference.

Thanks. I get it, and generally agree with your point here. Paying for good people and giving them enough time to get the job done is generally enough. I was just trying to be fair to op's question as devil's advocate. I don't think it should just be dismissed as a joke. I have always been a huge cryptocurrency skeptic, and I just don't see the same trajectory for ML.

So back in the day, Expert Systems were a big thing at Boeing. Searching Boeing's job openings today, it's a still a word used in hiring, but I don't understand what they mean by it in terms of manufacturing. Do you have any idea what they mean by that today?

Yeah, I understand your sentiment. I hope I didn't appear to technologically conservative in my comment, it is kind of a minor pushback to people promoting generative AI as a panacea that makes every problem ever easier.

I can't tell you about what Boeing does with expert systems. I've never been employed by them and I never really looked into it or where they use them. It's especially unclear because Boeing does a lot more than just pure manufacturing.

Found the Boeing executive.

Tech L7 who can only program Python but thinks they could be the next Boeing CEO.

Go outside.

Neural networks for quality assessment aren't necessarily unheard of. I'm not sure how they'd be using them in this case, though?

Maybe something like a camera monitoring that they are using the correct tool at various parts of assembly? But I'm not sure how feasible this would be at airplane levels of volume.

I bet it'll be common in car assembly at some point, though.

It still requires a human to be sometimes skeptical of the AI's results.

lmaooooo

Reply on news.ycombinator.com