The Disarray Inside Boeing's 737 Factory Before the Door Plug Blowout
wsj.com
wsj.com
But is there any definite source or elements about why the numbers were so down ? Were the projection unrealistic ? And if yes was it because they pulled them out of their ass or because they needed the unrealistic high ones to get the sales ? Or were the numbers realistic but working with so many externalized subcontrators slowing things down ? I think I read something about, eg how they had to refuse so many of spirits' produce that it was slowing everything down, but by that much ?
Another reason could be that the plane was harder to produce than expected but while I could buy that for the 787 I don't see how it can be remotely true for the MAXes, they were supposed to be a known quantities.
During the pandemic air traffic was hit hard and the entire industry (which is much larger than Airbus and Boeing) started to prepare for a slowing down, including firing staff, retiring products and such.
That slow down never really happened, instead there was a massive upswing in demand. Both Airbus and Boeing are currently racing to build as many planes as they can, as their customers are begging them to give them planes as soon as possible and as many as they can.
This is exactly the kind of situation in which mistakes happen. Boeing and their suppliers needed to rehire staff and train them while having a huge pressure to get planes out of their factories. Somewhat funnily both Airbus and their suppliers have a much harder time firing staff, due to stricter EU employment regulations, meaning that they unintentionally were better off when demand started up again, nevertheless Airbus and suppliers are currently hiring massively as well.
I don't think stability is unintentional. You can see how France economically always "lags behind" other economies in good times, but is also barely affected in bad times. On average, over time, it just performs like the rest of the EU.
Also, such labor laws exist in Germany as well which is Airbuses other big country.
The biggest obstacle is not employment laws or environmental or other compliance crap. The biggest obstacle is getting funding, Europe doesn't have the sheer amounts of "dumb money" that the US has due to their pension/401k system.
Wait, that last bit isn't quite right. Internal forecasts are generally flexible, rather than random, to adjust for the mood of the Overlord of the day. So if you get a stretch where the boss is feeling tetchy, forecasts suddenly start turning pretty good. It doesn't help that, at least anecdotally, the forecasting group's all been former executive assistants and people who were kicked out of systems engineering. Generally people on their way to somewhere else.
Working anywhere in the Boeing procurement group is a career death sentence, there is just no scenario where you win that fight. The price is never low enough, the suppliers are never getting enough money, and eventually you're going to be the scapegoat for a shortfall somewhere, whether it be cash or off-spec parts or PWAs stuffed full of tinsel and compressed dander.
It seems the company had some procedures in place, particularly the documentation of "removals," but the line employees didn't follow that process in this case. One reason appears to be the tremendous pressure they were under to get planes out of the factories.
The article also mentions a possible lack of experience, since some of the workers could be very new to the company...though they could also have been quite experienced as well.
It is so, so very hard to evaluate risk and causation.
Slow down production, increase oversight and focus on training. Make sure that many employees stay for a long time and that they are encouraged to share their knowledge with new hires.
Mistakes happen but that’s pretty intentional
It would keep everybody on there toes, every process functional and the regulators working without getting caught.
I'm not sure what a Process Inspector does. Do they work nights and weekends? Are they protected from reporting failures in process? Who holds responsibility for every process failure?
However, this article about him, and the behaviour of Boeing management at Charleston factory, is a pretty good eulogy. RIP Swampy.
https://prospect.org/infrastructure/transportation/2024-03-2...
https://leehamnews.com/2024/01/15/unplanned-removal-installa...
> Current Boeing employee here – I will save you waiting two years for the NTSB report to come out and give it to you for free: the reason the door blew off is stated in black and white in Boeings own records. It is also very, very stupid and speaks volumes about the quality culture at certain portions of the business.
> A couple of things to cover before we begin:
> Q1) Why should we believe you? A) You shouldn’t, I’m some random throwaway account, do your own due diligence. Others who work at Boeing can verify what I say is true, but all I ask is you consider the following based on its own merits.
> Q2) Why are you doing this? A) Because there are many cultures at Boeing, and while the executive culture may be throughly compromised since we were bought by McD, there are many other people who still push for a quality product with cutting edge design. My hope is that this is the wake up call that finally forces the Board to take decisive action, and remove the executives that are resisting the necessary cultural changes to return to a company that values safety and quality above schedule.
[...]
> However, more critical for purposes of the accident investigation, the pressure seal is unsurprisingly sandwiched between the plug and the fuselage, and you cannot replace it without opening the door plug to gain access. All of this conversation is documented in increasingly aggressive posts in the SAT, but finally we get to the damning entry which reads something along the lines of “coordinating with the doors team to determine if the door will have to be removed entirely, or just opened. If it is removed then a Removal will have to be written.” Note: a Removal is a type of record in CMES that requires formal sign off from QA that the airplane been restored to drawing requirements.
> If you have been paying attention to this situation closely, you may be able to spot the critical error: regardless of whether the door is simply opened or removed entirely, the 4 retaining bolts that keep it from sliding off of the door stops have to be pulled out. A removal should be written in either case for QA to verify install, but as it turns out, someone (exactly who will be a fun question for investigators) decides that the door only needs to be opened, and no formal Removal is generated in CMES (the reason for which is unclear, and a major process failure). Therefore, in the official build records of the airplane, a pressure seal that cannot be accessed without opening the door (and thereby removing retaining bolts) is documented as being replaced, but the door is never officially opened and thus no QA inspection is required.
More than two weeks had passed since workers on Sept. 1 flagged damaged rivets on a fuselage that needed fixing. The section had been assembled by supplier AeroSystems in Wichita, Kan., and shipped by train to Boeing’s plant.
Targets for completion came and went before employees escalated the situation to “Tier 3” priority, a move intended to get high-level attention at the factory, according to an internal Boeing log of employees’ push to finish the jet—the same one that would later lose its door plug panel in a near tragedy in flight just months later.
Punctuating how the situation was escalating with dollar signs, an internal message read: “$$TIER-CHG: 2 – 3 $$,” according to a Sept. 17 entry in the Shipside Action Tracker, or SAT, reviewed by The Wall Street Journal. The SAT is like a factory Slack channel for fixing a production problem.
The communications of the Boeing employees working on the door plug, previously unreported, help illuminate why it blew off during an Alaska Airlines flight on Jan 5. The factory was in disarray. Crews were unable to keep a schedule and apparently didn’t follow procedures, and production pressure mounted as delays piled up, according to entries in the SAT, people who have reviewed the logs and interviews with Boeing employees who worked on the plane.
In their logs, workers extended 50 times the estimated time for completing work on the damaged rivets around the frame of the door plug. The delays ranged from a half-hour to days. The work was finally signed off on and officially completed by the morning shift on Sept. 20, after a final quality check was requested the previous day, the records show.
The production breakdown had stunning consequences: The jet’s door plug blew off in flight, triggering an explosive loss of cabin pressure that risked the lives of passengers who could have been sucked out midair. It has sparked federal probes, including a criminal investigation, and hastened the exit of senior leaders including Chief Executive David Calhoun, who had vowed to improve Boeing’s safety and quality.
A Boeing spokesman said the company is prohibited by federal rules from discussing the investigation and referred to recent remarks by executives that they will slow down the factories to focus on quality and will take new steps to prevent problems from being pushed down assembly lines. The company is in talks with Spirit to acquire the company in an effort to rein in the supplier’s quality issues and also recently changed its bonus plan for 100,000 workers to emphasize quality and safety over meeting financial targets.
“For years, we prioritized the movement of the airplane through the factory over getting it done right,” Brian West, Boeing’s finance chief, said at a recent investor conference. “That’s got to change. The leadership team got it in the immediate aftermath of January 5.”
The SAT logs show that a fix for the damaged rivets was in the works for days, and that workers had to wade through layers of management to get the problem addressed. That task required opening or removing the door plug, which was installed in place of an unused emergency exit.
The workers discussed what documentation was required if they opened or removed the plug. “If removal needed, a removal needs to be written first,” one staffer wrote on Sept. 17. Altering airplane parts requires documentation in the heavily regulated and safety-focused world of aerospace manufacturing.
No such documentation appears to exist, however, Boeing has told U.S. lawmakers and accident investigators who have since been probing why the door plug blew off. The National Transportation Safety Board has said workers appear to have failed to replace four critical bolts needed to keep the panel in place.
The SAT records show a manager communicated that a mechanic was opening the door plug, however.
The NTSB has said investigators haven’t been able to interview the manager who oversees the team of Renton employees responsible for door work. The safety board has said the manager is out on medical leave. Starts and stops
When Calhoun, a Boeing director and former GE executive, took over as Boeing’s CEO in January 2020, he promised change by improving safety and engineering. “It’s going to last for a long time, and it’s going to be healthy,” he told reporters at the time. Two 737 MAX jets had crashed, taking 346 lives and grounding hundreds of Boeing planes for 20 months.
Then the pandemic grounded flights and halted factories at Boeing, rival Airbus and suppliers such as Spirit. By 2022, they were straining to meet a surge in demand for new jets as travel boomed. Boeing hired thousands of new staff and steadily increased 737 MAX production, and Calhoun issued financial targets that relied on plans for higher output.
Last year, the factories were still running with stops and starts as the company worked to return to production levels seen before the MAX crashes. Boeing delivered 580 737s in 2018, the year of the first crash.
“Is there pressure? Yes and no. We’re still there to do a job,” said one Boeing veteran who was involved with work related to the door plug on Line No. 8789. “I have a deadline every day, my team has a deadline every day, so if it gets behind schedule you have to get it on schedule.”
Behind the scenes, regulators at the Federal Aviation Administration and leaders of its factory workers’ union tussled with Boeing over its practices. The FAA and union pushed Boeing to add back second-party quality inspections that had been cut.
In September, while the Renton team was working on Line No. 8789, Boeing’s output of 737 jets had fallen to the lowest levels in two years. Boeing delivered just 15 new 737s to the world’s airlines that month. Executives had been telling airlines and investors that it wanted to produce about 38 a month.
The culprit was a separate issue from the rivets involved in the door plug. A defect in the rear of 737 fuselages supplied by Spirit had been discovered by Boeing the month before. Fixing the issue required inspecting and correcting misdrilled fastener holes in a key structural part. It had snarled production at Spirit, which in turn had delayed Boeing’s factories.
To keep things moving, Boeing kept accepting flawed fuselages from Spirit, including Line No. 8789. And the plane maker allowed unfinished tasks to flow through its factory—knocking production out of sequence in a practice called “traveled work,” which can increase the risk of slip-ups.
“Years ago, we weren’t going this fast,” said the Boeing veteran who was involved with work related to the door plug on Line No. 8789. “I’m not saying fast caused the problem. Something happened. I don’t know what it was.”
A second Boeing employee who was also involved with the same Alaska 737 MAX said he has never felt pressured to move too quickly. The company’s attention to quality and safety has improved significantly in recent years, and the mishap with Line No. 8789 has been weighing on the Renton workforce.
“It’s a failure on all of us,” he said. “We all feel it.”
FAA officials have criticized what they have found upon closer inspection of Boeing’s Renton factory since the blowout. They have described an operation that appears to prize production schedules over safety and quality.
FAA chief Mike Whitaker was particularly troubled he didn’t receive a safety briefing upon starting a tour of the factory, people familiar with the matter said. Such briefings are typical in manufacturing settings. ‘Just painted over’
When Boeing’s production system works according to plan, it typically takes a week or two for a 737 to make its way through final assembly at the Renton plant.
The rivet flaw was flagged the day after Line No. 8789 arrived at Renton. After that it took 18 days, 12 hours and nine minutes to fix the damaged rivets and officially resolve the issue.
The SAT entries suggest an at times dysfunctional production environment complicated by delays in getting parts and red tape with Spirit, the error-prone supplier that Boeing once owned and is in talks to acquire.
Spirit workers were on site at Renton and tasked with resolving the rivet issue.
Six days after workers flagged the damaged rivets on Spirit’s fuselage, an entry read: CONDITION STILL EXIST. RIVETS WERE JUST PAINTED OVER.
Boeing wanted new rivets. Days went by, with Boeing workers noting difficulties spurring the Spirit personnel into action. “Damaged rivets are not acceptable and need to be removed and replaced,” read one Sept. 11 entry.
Two days later: “No Spirit work in progress,” read another entry, suggesting another extension. Then, on Sept. 14, Boeing employees escalated the situation to a higher level of attention for greater visibility.
Workers removed the door plug’s bolts to open it, which allowed access to the rivets. But there is no entry in the SAT log calling for the door plug’s reinstallation.
The absence of documentation has shocked current and former Boeing employees. An official “removal” process would have prompted Boeing’s quality-assurance team to check that the door plug had been properly reinstalled—with the bolts to keep the panel in place, they said.
“I was, like, ‘What the hell?’ Boeing does a great job especially on documentation,” said the Boeing veteran who was involved with work related to the door plug. “If you take something out, you’ve got to write a removal.”
Thousands of people work in the Renton facility on three production lines where half-built planes move between stations every evening. Risers lift workers to the body of the 40-foot-tall planes, where crews use specialized tools to complete hundreds of tasks. Below, workers sit in rows at computers tracking the progress. Boeing’s plan to add a fourth line has been put on hold by the FAA in the wake of the blowout.
The team that works on doors and door plugs in Renton fluctuates between 20 and 30 workers. Some have spent decades at Boeing, others just a few months or years. They generally are split between two 8-hour shifts and can work on several planes a day.
Factory workers eventually secured new rivets for Line No. 8789, and by Sept. 19 a final target arrived. “Per Spirit management, this job is committed to be done today,” a SAT entry read.
A Boeing quality manager was to assign a final inspection. The rivet problem was on track for review, and the issue was closed on Sept. 20. The jet continued to be worked on and inspected for another month.
On Oct. 25, Boeing reported financial results. It booked a $1.64 billion third-quarter loss and lowered its 737 delivery goals for the year. It now expected to finish 375 to 400 of the jets, instead of as many as 450. Even that goal required the Renton factory to significantly ramp up production in the final weeks of the year.
Calhoun sent a memo to employees that day, rebutting criticism of its production numbers. He told them that improved quality procedures and a culture that rewarded speaking up about problems meant the company was finding more things in need of repair.
“I have heard those outside our company wondering if we’ve lost a step,” Calhoun wrote. “I view it as quite the opposite.”
Boeing delivered the jet known as Line No. 8789 to Alaska on Oct. 31. Eleven days later, the plane started flying passengers.
This really downplays how catastrophic this would have been at cruising altitude. And in that case the cause might have been way more obscured.
- implemented post-processing to strip ads and tracker beacons
- bought legitimate accounts that they use to bypass paywalls (some of them show up as remnants in the archived pages' code, some of them show up when you request an archive of the site's account management portal, some show up as HTTP traces during the archival process - the crawler will dump every URL it requests on a "WIP" page, and for some, auth information like JWTs is passed in the URL)
- implemented anti-nagwall features (some sites are "vulnerable" in their paywall/"you read X articles, end of free limit" by pressing the ESC key to disrupt the JS of the site)
[1] https://blog.archive.today/post/618635148292964352/what-scra...
This statement represents a fundamental failure of management to do its job— To know the whole picture, recognize systemic hazards (both internal and external), and correct them before they become real problems.
If executives face no risk for their decisions, then why do they get such high rewards?
I thought the deal was high-risk/high-reward.
Perhaps you and I view the world in a different light, but in any organization where there is even some semblance of hierarchy, the manager is, at the end of the day, always responsible.
If management isn’t aware of what’s going on on the factory floor, then that’s definitely a failure of management. If they didn’t receive any signals, management should’ve investigated. If they hired 100% incompetent people they should’ve hired better people.
Thing is, bosses aren't going to lose one cent. Wanna bet? The linked article is gently trying to shift the blame to factory floor workers.
>Management had no information of what was going on the factory floor
Meaning that management failed at their job.