FAA Audit of Boeing's 737 MAX Production Found Dozens of Issues | Hacker News Reader

FAA Audit of Boeing's 737 MAX Production Found Dozens of Issues

nytimes.com

82 pointsgumby48 comments

Related: 32 year Boeing veteran turned whistleblower John Barnett was found dead of an apparent suicide today. And a Boeing 787-9 went into a sudden dive yesterday due to a 'technical issue' injuring 50 people, some of which were "thrown into the roof".

If anyone else was willing to blow a whistle, I’d think they’d consider changing their tune.

>And a Boeing 787-9 went into a sudden dive yesterday due to a 'technical issue' injuring 50 people, some of which were "thrown into the roof".

Sound more like a pilot error? I don't know. The plane flew immediately after landing to its final destination.

“Barnett was cross-examined by Boeing's lawyers and his own attorney days before he died… The court planned for Barnett to answer more questions on Saturday, but he did not appear as planned. The BBC reported that he was later found dead [on Saturday] in his truck, which was parked in a hotel parking garage.”

Also why they say to buckle up while you are in your seat. Wind shear and such. Rapid descent they can do 4000ft/min controlled with air brakes.

https://www.msn.com/en-us/money/companies/boeing-whistleblow...

> went into a sudden dive yesterday due to a 'technical issue' injuring 50 people, some of which were "thrown into the roof"

Reminiscent of this: https://en.wikipedia.org/wiki/Qantas_Flight_72

The 2008 incident, this time on an Airbus, is interesting to HN readership as an unexplained hardware bit flip in one of the flight systems that calculates the angle of attack generated garbled data. This system was triple-redundant; however, it subsequently triggered a then-unknown software defect in the flight computer's algorithm for discarding such erroneous data. Some sort of edge case that wasn't considered. The result was the plane entered a dive in a manner eerily similar to the MCAS incident.

To me a unaccounted edge case in a tripple redundant system that can be fixed by an update forever feels different than Boeing routinely missing screws and not documenting them.

Don't get me wrong, both things should never happen, but the first can even happen to very good teams with stellar safety and testing culture, while the latter is indicative of a broad problem.

Missing an edge case nobody knew existed is not the same as not doing a very obvious thing that you should be able to do.

Agreed, I was responding to the GP's mention of the recent 787 incident, a bit offtopic but more technically interesting; I don't believe it made the front page.

Wow, how do you even test bit flips from the procurer side (Airbus in this case)?

I work in industrial manufacturing (although not as mission-critical as a plane) and suppliers have surprisingly amount of problems in their parts and we can't (and shouldn't) need to stress-test every single part we buy from 3rd parties.

Just last month we found out one of the bluetooth chips we use is giving kernel panics seemingly randomly once a week. It can only be fixed with a power-cycle. Supplier just shrugged and asked us to run some diagnostics software, we gave up on using bluetooth for this product instead (we had a physical display as an extra addon, but now it is mandatory). Note this is a million-dollar product and we were lucky to have caught this problem before release.

We actually accounted for a similar problem for our network communication messages by adding CRC to our messages.

https://en.wikipedia.org/wiki/Cyclic_redundancy_check

But bit flips inside a microcontroller or memory is beyond our capabilities to detect or simulate.

Note I am not one of the embedded engineers they might have some safeguards for this on top of trusting the hardware error-correction that I don't know about.

>At one point during the examination, the air-safety agency observed mechanics at Spirit using a hotel key card to check a door seal, according to a document that describes some of the findings. That action was “not identified/documented/called-out in the production order,” the document said.

>In another instance, the F.A.A. saw Spirit mechanics apply liquid Dawn soap to a door seal “as lubricant in the fit-up process,” according to the document. The door seal was then cleaned with a wet cheesecloth, the document said, noting that instructions were “vague and unclear on what specifications/actions are to be followed or recorded by the mechanic.”

As always, the issue isn't the _process_, but rather the lack of _documentation_ of said process...

I don't know about that, the process itself sounds pretty fucked to me.

Ah yes, if there was documentation, it would specify that a Hilton key card should be used. Otherwise, a Hyatt or Westin key card being used may cause an incorrect check result of the door seal and cause possible loss of life.

Well it should specify the thickness, length, and width of the keycard, since brands might change vendors

And of course it should specify which way it should be inserted.

And it needs to specify the amount of bend allowable given some amount of pressure. No point in using a bendy card.

Seems like the clearest example of damage from the iron law of bureaucracy here. As @dboreham mentioned in another thread, there is a culture of management suppressing documentation. Not that management is entirely to blame, as documentation is oftentimes not appreciated by anybody except regulators

https://news.ycombinator.com/item?id=39623522

You're defending the hotel card Dawn soap stuff, as long as it's documented. Got it!

Dish soap and baby powder are commonly used in aerospace maintenance for finding pressurization leaks.

That's quite different to using them as a lubricant. They're not applied to internal surfaces for one.

It very much sounds that parts are not manufactured to proper tolerances, and as a result Boeing uses undocumented lubricants to make things fit (badly). If this were anticipated there would be a documented process, but there isn't, and it means nothing good.

Ajax or Dawn? These things matter.

I think you're assuming that those could never be valid tests or processes for the manufacturing process.

I'm not saying they are or defending Boeing by any stretch, but I know my knowledge of manufacturing details for aircraft is limited enough that I couldn't make that assumption.

> I think you're assuming that those could never be valid tests or processes for the manufacturing process.

Given that Dawn dish soap isn't manufactured to any kind of useful tolerance, purity or performance guarantee, I'd say that's a safe bet.

I really wonder what the effects of Dawn dish soap residue on doors that go through huge pressure and temperature cycles is.

This position makes sense. You do not have enough knowledge to assume that the Dawn soap thing is bad, only enough knowledge to discourage folks from pointing out that it sounds bad

I honestly can't judge if that's bad or good to use those things. Just because it seems stupid from an initials laymen's glance certainty doesn't mean it is.

If it's safe for ducks it's safe for planes, right?

Because they both fly.

There's a lot of shoot from the hip emotional/sarcastic responses to this, but the lack of documentation is absolutely one of the main issues.

If mechanics are improvising with questionable practices, what are you going to do? Fire the dozen mechanics who used Dawn soap and now everything's fixed and we can trust all 737 doors for the foreseeable future? No, of course not, because that doesn't solve the underlying systems issue. You need to create a workplace culture of sticking to the documented process, and you can't do that if there isn't a clearly documented process to begin with. Sloppy engineering culture and QA culture trickles its way down, and that's what we're seeing here.

Both using commercial soap as lube, and a plastic card as a gague seem like symptoms rather than the actual root of problems here. I have to assume the same time crunch that caused them to use quicker to acquire materials than what they ought to have used (and likely had stocked nearby) is what cut down on documenting these processes.

Documentation of what was actually done would certainly be another useful layer in the swiss-cheese safety model commercial aviation relies on.

I’ve written tech manual procedures and maintenance instructions. It’s common to add alternative solutions for cleaning or lubricating parts.

I’m sure there’s a more appropriate way to put it other than dawn plus hotel card because that would not pass the approval process for my organization.

> Clean with a general purpose soap and water with a firm applicator card, plastic or nylon spudger, or plastic spackle knife.

Related: in the small GA aircraft world Dawn dish soap may as well be a MIL spec with how often it's used.

I feel like the words "dozens of" in the title were rather important.

Number 7 will shock you

NYT changed the title after I submitted it…but I was able to edit it.

I wonder how that would compare to an audit of other manufacturers like Airbus or Embraer.

Tbh it’s a fair question, but we haven’t seen a rapid escalation of incidents in planes from those manufacturers. So I’d venture the guess “not as bad as Boeing”

https://web.archive.org/web/20240312024717/https://www.nytim...

They used aeronautical grade Dawn dish soap and wet cheesecloth to fit doors.

It's almost comical.

Nonsense. They were obviously supposed to use MIL-SPEC standard MIL-D-16791 Nonionic General Purpose Detergent. /s

Kind of surprised it wasn't WD-40, the universal fluid. ;)

Nice, nice, they finally drived down the street !

Now audit FAA for being beyound stupidity or taking bribes or whatever - clearly not doing their own work in this and previous decade...

Boeing should be put in receivership, all top management fired with bonuses and stock grants clawed back.

The rot needs to be cut out.

Boeing is too important to be run by MBAs and private equity wannabes

At this point Boeing may as well be nationalized with how deeply embedded it is with U.S defense and economy.

Guess we need to put the federal government into receivership then /s

Outside of early 20th-century Italy and Germany, which countries have actively pursued nationalization of industries in a late-stage capitalism phase of development? What were the outcomes? Are there more effective alternatives?

Post-WWII France and UK, France even into the 80s.

US nationalized passenger rail into Amtrak in ‘71.

New York took over the private subway companies in the 40s.

Since the 1980s the US uses a different mechanism (as they did with Boeing) where the state leaves the companies listed on the stock exchange but influences policy by guaranteed purchase orders, forced mergers (else you lose those purchase orders) etc — privatization in all but name.

Wouldn't count the US though. Privat industry regularly sabotage state enterprises via lobbyism.

Just imagine if we lived in such a world.

Can’t speak for “private equity wannabes”, but good MBA programs emphasize quality and processes with checks and balances quite a bit. I can’t think of a scenario where an MBA program material would lead to recommendation to willfully incur this much liability and brand damage. There simply isn’t enough profit in the world from cutting corners to pay for carrying this absurd risk.

More likely, this is blatant lack of competition and accountability, aided by a huge dose of collective incompetence. Very said to see this from such an important company/institution. This might be the personification of repercussions of too big to fail.

Its not the MBA, its the background of those calling the shots.

Having people who were purely finance/marketing/business people be in charge of an engineering company is asking for trouble.

So an engineer/STEM works in the industry, then later gets an MBA, that's a different story.

If Boeing just wants to be a marketing/finance company they should fire all their engineers and contract it out to someone who actually cares.

I don't think this should be all that controversial. People with a medical background should run hospitals/clincs, people from creative backgrounds should run media companies, people with finance backgrounds should run finance companies etc.

Of course, you need to know some finance/marketing/management to get that high up the corporate ladder. The problem is the thinking that those types know how to run every business, just because of their 'Wharton/Harvard/etc. MBA

Nah. An electronic company I know of handles with the US a lot.

Every 4 years there is new management trying to get costs down and the same thing happens every time.

They once forced them to help outsource production to China and get a commission or no commission at all.

After a year of production, they crawled back since the Chinese used cheaper components and did insufficient testing. It just needed a couple of production issues and lots of warranty claims.

That's how GM wrecked Opel. Every few years, management changed, and the freshly parachuted new guy from the US mothership needed to make a mark for themselves to be promoted. Cost-cutting was the only way they knew, because first they were Americans, and second, they had no long-term attachment. Of course, the relentless cost-cutting without consideration for anything else soon ends you with quality issues and unhappy customers. Market forces at work. Forty years ago, they made very decent cars.

Reply on news.ycombinator.com