Boeing finds debris left in new 737 MAXes, now in storage | Hacker News Reader

Boeing finds debris left in new 737 MAXes, now in storage

leehamnews.com

126 pointsrobin_reala117 comments

I don't care how certified these planes are down the track, I never want to set foot on a 737 Max and generally any new Boeing that doesn't have a solid air history. Trust with these guys is totally blown.

I understand the sentiment but there may well be a reporting bias going on.

Boeing are in the news anyway, which in turn makes other Boeing problems more likely to make the news, both in the number of blogs written, and the likelihood they get shared here.

Or alternatively, it really needs a report that demonstrates that eg Airbus have no similar foreign object problems for one's understanding to be more informed.

A Boeing-only story by itself is an incomplete picture.

It’s not even just reporting bias, there are other biases involved as well. It’s entirely possible that other airplane types have high rates of FOD in the fuel tanks but they go unnoticed because those other planes aren’t under nearly as much scrutiny as the post-crash MAX.

If there’s a systemic problem that involves issues manufacturing a plane, it’s highly unlikely that it only involves one single type of plane.

I'd rather fly on a plane that from a manufacturer whose planes might have foreign objects in the fuel tanks than a manufacturer whose planes definitely do have foreign objects in the fuel tanks.

If there’s a systemic problem that involves issues manufacturing a plane, it’s highly unlikely that it only involves one single type of plane.

There isn't though. Evidence for that is Boeing itself - for decades they made excellent planes that had very few problems. The 747 was the safest and most popular plane in the world for years. Boeing have made more than 1000 of them and there have been very few crashes, and most of those have been pilot error or external factors.

Something at Boeing has changed, and that has made some of their newer planes less safe. That tells you nothing about other manufacturers.

Suit yourself, but I’d rather fly on a plane that had problems found, scrutinized, and fixed rather than a plane that may or may not have problems but nobody actually knows because they weren’t pressured to look hard enough.

Re: other manufacturers, if the regulatory environment is one in which Boeing can get by with cutting corners, that means it’s an environment in which other manufacturers can cut corners as well. That, combined with the inherent characteristic of for-profit companies doing what they do, I have no reason to believe that Airbus et al are not doing the same. Hell, the first A320 was infamous for a software issue causing it to crash, too, complete with regulators and Airbus involved in controversy afterwards for trying to alter the results of the investigation.

> Something at Boeing has changed

MD merger.

This is a good article with some background on why people think that may be the case:

https://qz.com/1776080/how-the-mcdonnell-douglas-boeing-merg...

Wow, top right of the picture, they have actually parked some planes in the parking lot (for cars).

Yes, that’s at BFI. Before production was halted, Boeing kept building planes for months and months. They’ve stuck MAXes all over the Pacific Northwest in what feels like any available space connected to a runway.

Your correct of course. The 767 refueling plane has been on hold for almost a year since the Air Force kept finding random parts strewn all over the plane.

Many airlines will not accept the 787 built on the east coast until they are flown to Seattle to be inspected there first.

Really saving a lot of money on those non-union jobs, huh.

Even if this is a systemic problem is doesn't change the underlying logic involved. It's unknown whether other models have this (or similar) issues, so you are still better off avoiding the model you know has a risk/default.

You’re missing the other half of your logic though, which is that due to knowing that this plane has an issue, you also know that the issue will be resolved, so you actually know that this plane does not have the issue. For other planes, as you said, it is unknown if they have the issue or not. I’d rather fly on the plane that I know for sure has had the issue addressed.

But we also know that this plane was poorly designed and the design process followed a path that completely disregarded safety.

You're saying that the best way for a company to get your business is to make a deathtrap and after it kills a bunch of people they'll fix a few of the glaring problems versus just making a good plane in the first place.

So that’s the question: how unusual is this? How severe?

I'm totally with you, but how do you prevent it? Usually you only know what type of plane you fly with after buying tickets. A solution might be to only buy tickets from operators that don't have with 737 MAX planes but this group might be getting smaller over long time.

Nearly all routes are flown with a specific type of plane always. Just use FlightRadar24 to see for that specific flight what usually does that route.

Thats only a 90% guarantee, since plane types are reshuffled during disruption sometimes.

I pick airlines that have (almost) no Boeings and that I can call before booking to see if it's an Airbus. I have done that for many years before the 737 Max issues. I simply find them annoying (very noisy) to fly with, but I definitely try to avoid. Some times I find myself in a Boeing on short flights when suddenly another airline provides the plane and it may be bad luck or damaged hearing or whatever; every flight I get into (and I fly a lot) with a Boeing, the noise is horrible.

Shortly after the crash of Air France 447 (https://en.wikipedia.org/wiki/Air_France_Flight_447#Sidestic...) many people wouldn't fly on an Airbus plane due to perceived fly-by-wire/software issues. Boeing and Airbus are the duopoly of commercial aviation and each has had issues over the years. However, commercial aviation as a whole is an incredibly safe way to travel long distances.

Have you tried noise cancelling headphones? They make a huge difference to comfort on noisy flights.

I have yes and they work, however, when I fall asleep, I end up laying on them and they kind of break...

my experience with A350 vs 787 (both Qatar) is that the A350's are some of quietest planes I've flown.

Must be me for sure; Qatar as well; did not sleep at all as there was a high pitch whining the stewardess said ‘you get used to’; nothing at all in the 350 or 380.

Got bumped from a A350 to a 777 for an upcoming long haul when Cathay Pacfic changed their schedules due to COVID-19. Blegh.

The 777 is the safest of them all, isn’t it?

This is an example of where “vote with your dollar” and “invisible hand of the market” falls down, and why we need strong regulations.

Airliners are very much regulated already. If you want to set a personal standard above and beyond the regulation, there might not be a market for you. Supposing it could be worked out, do you plan to review the safety record of all your potential rides, or just go by headlines?

The issue many people will have on hn is that the design flaws (not just software issues) have been discussed at length along with mismanagement that has shown little to no regard for design flaws. This has eroded trust in Boeings ability to make a new plane.

Time without incidents will be needed to earn many of our trusts back.

You talk of the regulation but posts on HN have shown how the regulators have failed and even why it's happened.

How do you know Airbus' software is better than Boeing? Maybe Boeing is overall better, but bad luck caught them? Maybe Boeing has learned a lesson and while their past software was bad they have spend the last year making significant changes and all future airplanes will be much better?

You are getting downvoted, but your points are fair. It wasn't that long ago that people criticized Airbus for some surprisingly similar reasons.

Sure we can imagine a scenario where Boeing is still the best choice. But if we actually want to make the best decision possible with the information available at this point in time add probabilities to your cases.

Who is more likely to have quality control issues? The company that has had more planes crash or the company that has had fewer planes crash?

They clearly aren't regulated sufficiently. The lack of FAA oversight is what allowed the whole 737 Max fiasco to happen in the first place.

I think the "vote with your dollar" is already working. How many 737 MAX contracts have been cancelled?

Cancelled because the planes aren’t available.

Meanwhile companies like IaG are buying more 737-Max, knowing full well their customers will base their purchase on the cheapest hit on skyscanner.

Regulation can't help when management is willing to sacrifice safety for profits. Ultimately, the penalties for violating those regulations and loss of customers could kill Boeing, but lives have already been lost.

Except that didn’t happen with the Max. The engineers thought MCAS was safe. Nobody in management told the engineers to design MCAS do be dependent on a single sensor with no sanity check and nearly physically impossible to override.

when you buy a ticket from the airline, it shows you what kind of plane it is.

Look it up on kayak or the airline's website if booking directly. The info is available before booking.

Assuming it's a major route, there are probably carriers with Boeing-less fleets on it.

Assuming Airlines are/will be racing to operate/purchase 737 MAX?

Or larger.

Sure, but you never know which way it will go but this is assuming the 737 MAX gets recertified by the FAA and operators are going to buy them because it is cost-effective or whatever. This means that the group that doesn't have a 737 MAX is going to be smaller and therefore harder/way more expensive to avoid.

It's not much different than the NEO, the pitch was that you will not have to send your pilots back to training because "it's just a 737". Since it clearly is not just a 737 the NEO (which is also a variant, but one that didn't inherently change the aerodynamics of the plane) now has the advantage, not just because reputation.

I completely agree, I'm never setting my feet on a new Boing again. The certification they receive is mostly done by themselves anyway.

I might have been lenient if it was one issue but it issue after issue and it's not just design of the plane, it seems the whole company is rotten to the core.

It's the other way around for me. These planes, and Boeing, are under so much scrutiny I doubt that there will be any other plane in the world that will match it for safety.

I also noticed that the last several issues reported were all reported by Boeing and not by the FAA. I like that as well.

If anything the exact opposite is true.

These planes are vetted so closely and strictly that will probably make them the safest plane in existence.

How many Airbus have messed up wiring or have debris in the fuel tank? None? Do you know why? Because nobody is looking at them with a magnifying glass.

That's absolutely not true regarding the wiring constraints. There have been projects going on for years to sort that problem out.

The Daily podcast interviewed a manager of the NC (SC?) Boein plant and he said he reported debris in planes and tried to stop them from being shipped off... and failed. It wasn't the fuel tank, though, it was in the air frame. Sharp objects. That coexist with wires. While listening I must have said "WTF" out loud a dozen times.

Worse, that type of thing is long been known as an dangerous issue and the industry has processes and protocols to deal with it. That's why it's "worse", Boeing knows about all these things and conscientiously decides to don't give a f#$%k.

Airbus ( and I'm sure pretty much all the other manufacturers ) has all the processes for training the workers about this problem, how to avoid it, how to deal with it when happens, check-in and checkout all the tools at the start and the end of the task, inspections, etc. this is so basic it hurts.

I think its probably a lack of supervision and not a lack of protocols, if it were a lack of protocols the 787s assembled in Everett would have the same issues. Also, I have personally observed the use of protocols to prevent this in tours and (a long time ago) working in the Everett factory.

The tricky thing about protocols is that you have to be willing to follow them and deal with the consequences when you step off the normal path through the flowchart.

If the protocol says "(1) Check out tools before using them. (2) Check in tools when returning them. (3) Verify all tools checked out have been checked in before marking the job complete." that's one thing. Sending employees to training on how to do this and seeing daily clipboards by the tool room with initials might make you think the protocol is in use. And if there are supervisors making sure that the clipboard is full of initials at the end of the day, that can give you a nice warm fuzzy feeling.

The problem comes when supervisors want to 'look good' by having no deviations from the protocol. If people are post-dating their check-in or pre-dating their check-out, or checking stuff in that's still in use by a coworker, the protocol is less than useless.

When humans are tasked with following those protocols, they'll make human mistakes sometimes. If there are no mistakes that's not good, that's bad - it indicates the metrics are being fudged. I think any supervisor that fails to find a deviation from the protocol or who can't list an event when the protocol required action outside the norm does not look good.

Similar stuff applies to software development: If you're doing code reviews, or testing before check-in, or signing off on releases, and can't point to a time when the protocol said "nope, go back to the drawing board" you're not actually making your development safer you're making it a hassle.

The tool crib checkout is done on computer at least. The toolbox inspections might be fudged.

It also might be downstream or upstream of the Boeing factories, like at one of the companies that manufactures major components (example: Spirit AeroSystems manufactures the 737 main fuselage and I'm told Section 41/cockpit of many aircraft) or at one of the maintenance facilities.

In the daily podcast episode that QC employee explicitly tells a story of how managers tried to hush him, change the protocols, ignore issues etc.

That same guy also said it had to do with the location of that new factory and the lack of company culture paired with managers who just had the numbers in mind.

I wonder if there needs to be some personal liability here, too. Many individual bankers have been banned from working in the industry for willfully ignoring the laws/rules. I wonder if that is necessary in this industry, too?

I have heard from union members about union pushback on such things, but that may be anti-union sentiment or rumors.

There actually is a process of beeing interrupted while working on a plane, which consists of retreat ( you carefully undo the last step you remember doing before beeing interrupted, checking you have all parts in the location they started out, protected against dust etc.). Then you make a note on the step you expected to do.

You start out three step before your last step.

You repeat/check those steps.

Its time intensive as hell, which is why the usual interruption to egostroke for meddle management does rarely happen in aircraft maintenance and production.

Boeing knows how to do this. The problem is manager shifting responsibility from one mature manufacturing site to a less mature one without properly compensating for it. A problem fortunately unknown to SW engineers.

Which is why this raises many red flags: it is indicative of a company culture that sooner or later will murder innocent people.

Sooner? Two 737-MAX8s have gone down already!

It wasn't a manager, but aomeone who worked in quality control. His managers wanted him to ignore these issues..

The most WTF moment for me was when he told about a whole ladder that has been forgotten inside.

A friend used to work as an engineer at the Seattle plant 15 or so years ago and he always told stories of how meticulous they were about avoiding FOD, how management completely supported the effort and how a FOD discovery, no matter how small the object, always led to a "five whys" style investigation.

I would guess the NC plant was commissioned without much, if any involvement from the Seattle people, if there culture was that radically different.

The podcast I mentioned, the guy was actually from Seattle. He had been relocated there because the NC plant was new and needed people with experience. He basically said it was a company culture issue. Seattle culture was good. NC culture was bad. Management matters, it turns out.

The plant on the East Coast was because Boeing was busting the machinist union that runs the Seattle plants.

It’s also why Boeing has been cutting their key engineering staff in Seattle — they also unionized. In fact, rumor has it that the entire reason MCAS/737MAX debacle happened was outsourcing and contracting shenanigans to break the engineering union.

It turns out moving senior management away from the rest of the company and getting in an ugly fight with your talent before petulantly building a non-union plant elsewhere leads to low quality products and out of touch executives.

Who could have known?

Oh yeah, my friend’s uncle who flatly said when his generation of engineers retired, Boeing would lose the ability to make planes.

And everyone else.

Forget a scalpel, that's like coming out of surgery and finding that the doctors somehow left the rolling stand thing for IV bags inside you.

You're right, he was a QA person. Thanks for the assist.

The other story I remember was he found that defective parts, which had their own area so as to be separate from good parts, so they wouldn't be used, were going missing. When he confronted a manager they told him to not worry about it. The parts were clearly going into planes so they could ship them out.

This is a great episode. He also discussed seeing people take parts marked with red paint (a signal that the parts were deemed unsatisfactory/broken) and installing them just to keep the line moving and meet deadlines.

He was asked at the end of the episode if he'd ever allow himself to fly on one of the planes that went out of the factory he worked in and IIRC his reply was an emphatic "no".

Apparently these problems are very hard to avoid. See here for some stories in context of the Space Shuttle: https://waynehale.wordpress.com/

As a remedy, they use tools that are tracked by RFID and tool cabinets that monitor that all tools are back in place.

Based on the last time I was in the Everett factory (3 years ago), Boeing requires all machinists have foam cutouts in their toolboxes and that every tool is returned to the toolbox at the end of the shift. Tools checked out from the tool crib must also be returned to the tool crib at the end of the shift. This has in practice greatly lowered the amount of lost tools in aircraft, since every tool is accounted for.

I imagine that the FOD in the fuel tanks is most likely lots of scrap from drilling out the holes rivets are placed in, and probably some rivet debris as well. There most likely is a process for clearing that out from where its generated (rivet holes are not often pre-drilled), and there may be inadequate supervision of the machinists doing that.

It’s a known problem with known solutions. “Foreign object elimination” I ended up taking the FOE training after someone left a tool at a radar site. I work in software and never visited the site, but “big company”. FOD foreign object damage is what everyone is trying to avoid.

(Pdf) https://www.astm.org/CERTIFICATION/DOCS/222.NCATT_FOE_Standa...

Any large aerospace contractor has mandatory FOD/FOE training for ALL employees (contract stipulation). Don't forget to not charge for this time spent!

> Don't forget to not charge for this time spent!

Implying an interesting story here, mind sharing?

If you've ever worked for a defense contractor, you have charge codes for everything. They take charging to the correct code correctly very very seriously. (In 6 minute intervals (.1 hr)). Submit your time daily.

We were a Boeing Sub contractor.

Timecard fraud was a fireable offense. The company I worked for was rumored to have fired someone who came into the office on a weekend and badged in his buddy who wasn't actually there. Basically trying to game the door timer system.

My last task at big co, was making a timecard app for ios and blackberry, to make it easy.

Yes that makes sense, but the interesting story would by why required training should _not_ be billed to the client.

I think he’s just saying remember to get paid for it. Don’t do it for free if you are hourly or contracting. Required training in the US is a paid activity.

Also through “sponge logs” which inherit their amusing name from the medical industry.

The article doesn't say what kind of debris they found. Can it lead to the fuel being cut off during flights? Can it be undetected for non-737 MAX planes that are currently flying?

Also, from the comments:

> One Middle East airline inspected their new 787 and found lots of debris and didn’t want 787 from Charleston for some time. They also found a chewing gum covering a hole in a wall close to a door.

Charleston is also the place that makes those military refueling aircrafts that were refused by USAF?

Not sure, but that seems to be a thing: https://www.popularmechanics.com/military/aviation/a26627917...

> New KC-46A tankers fresh off the assembly line were found with “loose tools” and “bits of debris” in them left behind from the manufacturing process.

Charleston makes 787s and parts for some other aircraft.

The KC-767/KC-46 is assembled at the Everett factory by IAM union machinists.

Thanks, stupid me, makes sense to separate military and civilian stuffs...

Its by airframe, not by military or civilian. The military airframes are made in the same factory as the corresponding civilian airframe, its just flown to a different facility (In Seattle or such) to be built out with the required military hardware.

Thanks! This makes more sense~~

No, those were made elsewhere.

I feel like it would be easier to just put a big mesh over the fuel intake inside the tank, and then say "this plane is certified to have objects up to 10 inches in the fuel tank and will still operate as designed".

Planes have two fuel tanks and two engines anyway - I would really struggle to imagine a scenario in which FOD in a fuel tank would lead to something worse than an emergency landing.

A screwdriver is under half an inch in diameter and will cause major problems with your engine.

There already are filters on the fuel intakes, but having a rag in there doesn't seem ideal.

It's a matter of degree, and you aren't only concerned with the engines. Aircraft frequently pump fuel around to other tanks for trim purposes. Debris that can manage to get passed filters into the pumps/lines can cause premature pump failure. Lose too many, and you have a plane out of service.

Given that most MTBF are calculated assuming the absence of debris except that which may come in with the fuel, there is really no excuse for not cleaning out your tanks.

Fuel filters getting clogged leads to this https://en.wikipedia.org/wiki/British_Airways_Flight_38

An emergency landing in mid-ocean?

So there is no need to delay delivery because they've launched a "robust" internal investigation. I'm sure this debris doesn't at all point to the possibility of overall bad quality control and the presence of other undetected problems. It'll be just fine.

All of the rigor of modern software development.

I'll chime in as I was previously an "air frame structure tech". Basically I did everything on the airplane other than mess with engines or electronics. Worked on the Navy P3 Orion fleet (remove and replace wings), the Airforce C-17 fleet (worked various tech bulletins as well as air frame inspection checks), and a brief stent on the the 747 'Sofia' project (cut a really giant hole for a telescope)

Anyway, I was most intimate with the P-3 orion fuel tanks (which is the entire wing) as we regularly pulled the wings on those planes, replaced various spars, ribs, etc and then reinstalled the wings. Then we'd have to fill the entire wings up with the 'anti explosion foam' which is in several hundred cube shapes, all with painted on numbers (yes they all go back in order), ranging from 6" to 18" in size. After that was done, plane was refueled and tested. Normally after a few weeks of checks, the fuel filters and intake strainers would be checked. They commonly had rags in them, but always had tons of metal shavings and wads of human hair (probably hair from your legs).

So yes, 'FOD' is always in the tanks after any construction or maintenance. There would be no tools in the tanks as normally your employee number or social security number was etched on everything you owned.

This isn't unique in the manufacturing of large engineering projects. There are stories of the US Navy doing sonar test runs of new double hulled submarines to see if the submarine is as quiet as expected. One of the things that happen[s|ed] often enough is a tool box being left in between the double hulls that they know what it sounds like.

USN submarines haven't had double hulls for 70 years almost. Hopefully things have improved

We haven't put skids on the bottom of the boats in a really long time either. Doesn't make the stories any less interesting though.

https://www.amazon.com/Blind-Mans-Bluff-Submarine-Espionage/...

I suspect all the boeing planes have this paltry quality. It's just that the focus is very intense on the MAX right now.

When left to their own devices, and when allowed to act as their own oversight and auditing authority, it turns out that huge, impersonal corporations won't act in the public's best interest.

It's like there should be a portion of government funds devoted to an agency responsible for ensuring that these type of things can't happen.

Who knew? That's weird, right?

Does this mean normally these planes were suppose to fly and they had passed their final check?

The head of Boeing's 737 programme has told employees that the discovery was "absolutely unacceptable".

That's fine, but what action did that person take other than saying that? Or does anything else need to be done at that level?

I recall reading about Andy Grove on the topic of management, and maybe the point of people at that level really is to just push down "company values" in a sense. I'd rather have someone up there shouting about this than constantly shouting "shareholder value" or "just deliver", at least for products like this. Maybe the bad culture there is changing ;-)

Notice that he said the discovery was unacceptable, not that having the debris was unacceptable.

like the employees who wrote the "designed by clowns supervised by monkeys" email. boeing took swift action... to fire the employees who made the mistake of documenting the problem in writing.

Having seen a documentary a few years ago about the 787 and the absolute shitshow that was and still is, I'm now convinced Boeing has been building their planes like an AMC Gremlin for at least 2 decades.

Considering how often surgeons leave all sorts of things inside patients (https://www.ctvnews.ca/health/surgical-objects-left-in-patie...) it's not hard to imagine manufacturing facilities doing much worse!

Of course, the issue is knowing about the problem and ignoring it. That's inexcusable.

It seems like Boeing (and the FAA and the industry more broadly) have been hiding behind the veil of the "zero commercial accidents" streak for however long that was happening while aggressively trying to coverup serious problems.

It's great that everyone is finally doing a deep dive to figure out these problems. We need more of this, more critical thought and curmudgeons and people who assume the statistics are wrong.

If it's Boeing, I ain't going.

With the knowledge that A&P mechanics (air frame and powerplant) are required to shadow their tools in their boxes (filling the drawers with foam and plucking a distinct home for every tool to make confirmation that all are returned easy and fast) I find it staggering that tools are left behind on a regular basis at the assembly phase.

Its the same at the Boeing factories, it will be interesting to see where this stuff gets traced to.

What next?

Submitted title is actual title, but the first paragraph actually makes sense, recommend change

Boeing recently discovered some of its stored 737 MAXes have foreign objects in the fuel tanks.

Who needs Netflix? Just read the Boeing news...

Do you think there is an element of sabotage by competitors?

Why would you think so?

I don't think that the competition are that effective or good. I remember that I have seen documentaries about how the factory workers at Boing anonymously complaining about the build quality and especially that management was not taking their concerns seriously. And this was before Max incidents. I will bet on issues because of profit/cost focus and poor management over conspiracy theories any day of the week.

no where it states if this is actually hazardous or not, if its a common thing which might happen to other vendors too or anything. just 'found some stuff which didn't pass the checks.' ok, thats what there are checks for..., good job... such reporting. just post some random piece of information about some buzzword or google trend without any background or context to put it in.

Have a listen to this: https://www.nytimes.com/2020/01/03/podcasts/the-daily/boeing...

Sharp debris in the airframe close to electrical wires and the very wires that control the airframe in planes that left the factory are not a theoretical risk. Imagine your airline gets a "quality controlled" factory new plane and you find they forgot a whole ladder (!) inside the hull — if this shouldn't raise concerns — what would?

Or must it crash first? Oh wait..

The software problem alone was bad, but combine this with a lack of effective oversight and a bad company culture and it looks far more bleak

The other good news is that the wire distances are not up to current standards in the MAX. So when one goes you have a better chance the other critical one also goes.

> FOD is absolutely unacceptable. One escape is one too many. With your help and focus, we will eliminate FOD from our production system.

> We’ve already held a series of stand down meetings in Renton with teammates on the factory floor to share a new process for stopping FOD.

I can't help but think of "beatings will continue until morale improves!"

Reply on news.ycombinator.com