Boeing seeking to reduce scope, duration of physical tests for new aircraft
reuters.com
reuters.com
Full-up testing isn’t always possible, but it shouldn’t be abandoned only because it is costly or difficult.
We deal in formal verification for some aspects of what we do, and that lead to a conversation that went roughly like this:
AV Exec: "Can we use formal methods to prove that our simulator is as good as reality?"
Me: "No. You can use formal methods to prove your simulator implementation more closely adheres to your simulation model."
AV Exec: "Isn't that the same thing?"
Me: "No. Your simulation model is definitely wrong."
AV Exec: (looking at me both disappointed and despondent)
Me: "Look. It's important to understand that simulation isn't creating an approximation with a known correspondence to reality. Simulation is fabricating an entirely new reality with an opaque correlation to our reality."
AV Exec: "Then how am I supposed to use simulation to prove our system is safe?"
Me: "You can't."
Despair on both sides of the table ensues.
I'm also sure that this article could be embellishing the facts. It's entirely possible that the very smart people who think about aviation engineering all day are supremely convinced that some of the digital tests can absolutely replace physical tests that were in place for decades. I'm not even remotely qualified enough to offer an opinion on that, so I won't.
So here is my question. Boeing is a very large company beholden to shareholders, and kept in check by a declining number of other checks and balances. Should we, the public, trust that Boeing will pay close enough attention to maintaining safety as an utmost priority? Or should we be demanding that more oversight and regulation are put in place?
Let me try and reframe my original comment for you. I do currently think that Boeing should be regulated more, but I'm not convinced enough of my own reasons. So I'm wondering whether my perspective would change if I heard compelling alternate ones?
I can't really help you as I agree too strongly - see my other comment elsewhere re. stress testing.
I can't see any possible reason. The only people reliably who might answer that things are ok as-is are boeing themselves.
This is really the foundation of the problem. Members of Congress are not aeronautical engineers, so they're in the same boat as you. Aeronautical engineers work for aircraft makers, so they have a conflict of interest.
Then, because they don't really know what they're doing, they tend to require things that reduce safety, either by requiring measures that aren't cost effective and thereby blow your entire budget including the money that could have been spent on other safety measures with better cost/benefit, or are overly rigid in requiring the specific solution to a problem which was the state of the art three decades ago when the rule was enacted even though safer alternatives are known today.
So why doesn't the government higher some regulators with some relevant industry experience? Well, they tried that, and then we got all of this revolving door nonsense where, to use an example closer to home here, former Verizon lawyer Ajit Pai is now running the FCC as a wholly owned subsidiary of Verizon Communications.
Then we get proposals to prevent regulators from going back to industry when they finish, to try to prevent that. But the government already has trouble attracting talent when they're paying substantially less than private industry does, and the jobs tend to only last for one administration until the next one comes in and replaces them with their own people, so how are they ever going to get anyone good to do a job that will a) pay less than they're making already and b) by law end their private career even though their tenure in government is likely to be less than a decade?
It's possible that we're better off not specifying how to make airplanes safe but instead imposing significant liability on companies and individual engineers who make ones that aren't. (This works in industries where the smallest company has a multi-billion dollar market cap. It's obviously less of a deterrent when the manufacturer has no exposure to the jurisdiction of your courts or is small enough to file bankruptcy every time there is a problem, but then we're no longer talking about Boeing and Airbus.)
Please don’t forget those engineers’ management chain. In my experience, it’s usually business people trying to change things for profit’s sake, not the engineers.
What do you think about reforming how we regulate though? You point out that the revolving door problem stems from the fact that attracting the right people is hard when you're not offering them the right salary and career security. Surely there's ways to make that possible in the public sector!
Given that regulatory bodies really ought to be non partisan and focused on a particular field or industry of activity, perhaps the institutional heads could be selected via a more democratic process (rather than just whichever buddy helped out an elected representative win their campaign) and given guaranteed minimum terms?
And as to the problem of turnover throughout changes of political administration, isn't that perhaps also merely a policy problem? Why does the FCC or HUD need to be gutted and repopulated during a transition of political power? Those departments still have to report to the executive and congressional branches of government, so are their staffing choices something the White House should just be able to change on a whim?
It's actually pretty hard. The first problem is that if you paid them the amount it would take to attract good people, it's easy for opponents to score cheap political points by charging you with wasting tax dollars or comparing the high salary to that of unskilled workers. Meanwhile a high salary increases the perverse incentive to fill the position with cronies as a reward for political support, which is the opposite of helpful.
There may be ways around it if you're creative, like making a benefit of the position that you and two generations of your descendants get free healthcare and college tuition / loan forgiveness forever (provided you do the job for at least two years). Then that's worth potentially quite a lot of money and provides a suitable incentive to attract good people, but makes it harder to take cheap shots because the total cost is indeterminate in both time and amount. Meanwhile you attract just the sort of experienced greybeard you want, who can do the job for a few years as the final leg of their career before retirement and in so doing provide financial security for their children and grandchildren.
At which point it becomes sort of a meta problem, because to bring that about you need honest and creative people in the legislature to make good rules, which was kind of the original problem -- how do you get good and honest lawmakers and exclude incompetent and corrupt ones, when the people choosing them have insufficient time and expertise?
> Given that regulatory bodies really ought to be non partisan and focused on a particular field or industry of activity, perhaps the institutional heads could be selected via a more democratic process (rather than just whichever buddy helped out an elected representative win their campaign) and given guaranteed minimum terms?
Making them directly elected positions might help, but it's more like trading one set of problems for a different one. Now it's an elected position, but are random airline passengers going to care about who gets elected? (Boeing will, of course.) How do you get honest specialists who know what they're doing to even run, and how does a random voter tell the difference between that person and an industry shill prior to putting them in office? How do you prevent the foreseeable result when there is a disaster not long before an election and people elect a strongman to Do Something About It, only to ruin everything with bluster and incompetence?
> And as to the problem of turnover throughout changes of political administration, isn't that perhaps also merely a policy problem? Why does the FCC or HUD need to be gutted and repopulated during a transition of political power?
It's because they make policy. If a Democrat replaces Trump they're certainly not going to keep his head of the EPA. Giving them guaranteed terms that are of similar length to the President's doesn't strike me as something likely to change much in a helpful way -- and you still have the same issues in choosing who to select to begin with.
> how do you get good and honest lawmakers and exclude incompetent and corrupt ones, when the people choosing them have insufficient time and expertise?
I think this is the crux of the problem, but I also think it's actually a problem no matter what. Because if you don't regulate, you're just basically crossing your fingers and hoping shitty people don't climb the corporate ladder at Boeing / AT&T / Volkswagen / whatever.
> How do you get honest specialists who know what they're doing to even run, and how does a random voter tell the difference between that person and an industry shill prior to putting them in office?
Well I guess in my head I figured that if it becomes an enviable electable position, then you're creating healthy competition for it. In the same way we sort of all collectively hope that competent and benevolent people will run for Congress, the presidency, etc. By making the positions individually electable and clearly demarcated, I would hope that then the public is making decisions for that role more based on competency, rather than charisma. Put differently, I'm not a huge fan of representative democracy because I think it inevitably descends into cult of personality, but I don't necessarily have something I can hold up as a better way ;)
> If a Democrat replaces Trump they're certainly not going to keep his head of the EPA.
Yeah but that's my point. The people making policy in the airline industry, or the telecom industry, or whatever, should ideally be formulating public policy in the most bipartisan way possible. If you made those roles independent from the current ruling party, then it would mean that those organization heads would need to maintain healthy ties to the major parties, since they wouldn't be able to effectively carry out their role if they're only pals with Bernie and are universally hated by all conservatives. I'm probably just hopelessly naive, but I think it should be the moderates who are making decisions about how to maintain airline safety, not some hyper partisan cronie of the DNC or RNC.
That's not true at all. He worked at Verizon for 2 years (2001-2003). Almost all of the other 20 years of his career have been at various government agencies. He was nominated for an FCC position by president Obama and confirmed unanimously by the senate. People hate him because he was nominated to chairman by Trump and because of his position on net neutrality.
I know I'll get a lot of flak for this, but honestly net neutrality had no noticeable impact on consumers. Prices haven't gone up. ISPs haven't clamped down on what sites you can visit. This outcome was obvious before the law changed. Internet services improved before net neutrality and plenty of other countries with better service don't have net neutrality.
People seem to think Pai wants to deregulate everything, but that notion is quashed if you actually look at what he's done and said. For example: he has argued in favor of a cap on the rates of long distance phone calls from prisons. From a 2003 memo[1]:
> I believe that the government should usually stay its hand in economic matters and allow the price of goods and services to respond to consumer choice and competition. But sometimes the market fails, and government intervention carefully tailored to address that market failure is appropriate.
> The provision of inmate calling services (ICS) is one such market. Inmates cannot choose their carrier, and carriers do not compete with each other for an inmate’s calls.
Those do not sound like the words of someone who is in the pocket of giant phone companies.
Pai has also threatened to legislate that phone companies switch to a secure caller ID protocol if one isn't adopted by the end of the year.[2] This would make most robocalls easy to block. The problem is that different phone companies have trouble coordinating, as nobody benefits until they all adopt the new protocol. A government mandating adoption of a new technology is a way to solve that coordination problem.
Career bureaucrats are many things, but they aren't what the media portrays Ajit Pai as.
1. http://transition.fcc.gov/Daily_Releases/Daily_Business/2013...
He is a former Verizon lawyer, as stated.
> He was nominated for an FCC position by president Obama and confirmed unanimously by the senate.
The FCC has five commissioners but only three can be of the same party. Pai's slot was thereby slated to be filled by a Republican even when the President is of the other party. Moreover, confirmation of commissioners is rarely contested because the appointments are temporary and because of the way they're selected, rejecting one would only result in their being replaced by someone of the same party.
> People hate him because he was nominated to chairman by Trump and because of his position on net neutrality.
People hate him because of his position on net neutrality and other issues. I expect Trump loses more support for nominating Pai (from people paying attention to such things) than Pai loses for having been nominated by Trump (in addition to Obama).
> I know I'll get a lot of flak for this, but honestly net neutrality had no noticeable impact on consumers.
It seems odd that the ISPs would be so opposed to a policy that has no impact on anything.
> Prices haven't gone up.
This is never a thing it was supposed to cause. The opponents had argued the opposite -- that network neutrality would cause prices to be higher. Yet we haven't seen a significant price reduction either, though seeing any pricing result on a short time frame would be quite unexpected regardless.
> ISPs haven't clamped down on what sites you can visit.
Again, not a thing that was expected. The problem is things like zero rating favored services while imposing metering or usage caps for others, and purposely oversubscribing transit links so that their direct competitors for video services have to pay them monopoly prices for interconnection. These things occur even now.
Also, these companies are highly experienced in government policy. They know perfectly well not to jump right into outrageous behavior from the date of the rule change if they want to keep it. That doesn't mean it's not coming. Expect to see more things like this, from before the rules were in place to begin with:
https://www.nytimes.com/2007/09/27/us/27verizon.html
And again, if they don't intend to violate it then what does it hurt for violations to be prohibited? It's like repealing the law against leaded gasoline and then claiming it's not needed because nobody has resumed production of it yet. If they're not ever going to do it then the rule is harmless and if they are going to do it then the repeal is not.
> People seem to think Pai wants to deregulate everything,
The claim is that he's in the telecommunications industry's pocket, not that he wants to deregulate everything. These are not the same thing -- especially in the telecommunications industry. The incumbents commonly use regulation to suppress competition.
> but that notion is quashed if you actually look at what he's done and said. For example: he has argued in favor of a cap on the rates of long distance phone calls from prisons.
The high rates don't go to the major telecommunications companies, they go to the prisons or their bespoke prison calling system providers. It constitutes ~0% of Verizon's revenue -- and is if anything probably costing them some money in lost business because the high cost paid to other parties suppresses demand for their services.
> Pai has also threatened to legislate that phone companies switch to a secure caller ID protocol if one isn't adopted by the end of the year.
Once again nothing to the detriment of large telcos, which have the scale to implement the requirement efficiently (unlike some of their smaller competitors), and benefit from it by not having robocalls causing customers to cancel their telephone service as not worth the inconvenience. Also apparently not very effective regardless:
https://www.techdirt.com/articles/20190610/07405142367/fcc-p...
But let's suppose it's not a bad policy -- it's possible for the interests of telcos and the interests of consumers to coincide. The issue is that shouldn't be the only time that FCC policy benefits consumers.
Meanwhile he's repealing network neutrality and privacy rules, rubber stamping the T-Mobile/Sprint merger, appointing industry lobbyists to consumer advisory panels and various other problematic policies:
https://energycommerce.house.gov/sites/democrats.energycomme...
https://arstechnica.com/tech-policy/2018/12/ajit-pai-buries-...
https://arstechnica.com/information-technology/2017/09/ajit-...
https://motherboard.vice.com/en_us/article/bj49j8/fcc-falsel...
https://motherboard.vice.com/en_us/article/mb57gx/no-broadba...
Boeing's corner cutting and lack of regulatory oversight seem to have contributed to the crashes and they rightly should have to prove that this testing will be equivalent.
Generally speaking, the government cannot maintain a staff of aerospace engineers since they would rapidly fall behind industry engineers, you know, actually building airplanes with current materials.
However, the FAA could pay for engineers to provide oversight and also test pilots. Either group would have caught the MCAS issues.
Most FAA oversight has always been paperwork-related. You don't need to be a pilot to be an FAA employee.
Oh no.
> For example, when vibrating a fuselage on an enormous platform to expose weaknesses - known as fatigue testing - the vast majority of the time the tool itself breaks instead of the airframe, according to a person with knowledge of past tests. Such work is costly and has reliably confirmed engineers’ expectations, he added.
Is this just hubris, or is this real? That's a lot of confidence to place in a software model.
Finite element methods are such a known quantity that we're able to design nuclear weapons without ever testing them in meatspace. If it's good enough for nuclear weapons, I am strongly convinced it's good enough for something as simple as stress analysis.
As for not testing nukes, I'm damn sure they test the relevant non-nuclear parts such as the lensing (conventional) explosives.
> ...for something as simple as stress analysis
Of entire wings? From the article "..such as using machines to bend the wings to extreme angles and shaking the fuselage until it cracks"
Your job isn't in this area I guess. I'm sure of it.
BTW the model is only as good as its inputs so if you have a defective batch of components (and wasn't there very recently a case where a bunch of deliberately below-spec aluminium parts were delivered to various aerospace companies which cost lots to rectify) then your model had better refelect that defect. But how are you going to find out? With pure maths?
I'm a mechanical engineer. My point is that stress analysis is the oldest, simplest, and most well understood application of finite element analysis. I can't even begin to imagine the multiphysics that goes into simulating a nuclear weapon, but if we are willing to bet our entire nuclear deterrent on it, it seems reasonable to skip some additional rounds of physical tests that are mostly a formality at this point.
Also, defective materials are caught much earlier in the manufacturing process through quality assurance methods, before they become airplanes. And then you apply a factor of safety on your calculations for added assurance.
Okay so maybe this is an apple's to oranges comparison, but I thought of the story how NASA was scammed by a supplier passing off shitty aluminium as space grade [1]. Is it reasonable to assume that if it can happen to NASA, it can happen to Boeing?
https://weather.com/news/news/2019-05-02-nasa-launch-failure...
And the company, Sapa, didn't (AFAICT) manufacture the components, they were a third-party testing service!
They provided exactly the tests you say should have picked up the problems.
I am not a mechanical engineer.
My understanding is that some composite materials used in modern aircraft manufacturing can fail in ways unlike classic aircraft aluminum (i.e. suddenly, without prior visible signs of stress/wear).
Is that relevant here?
And we still get it wrong at times; especially with interactions between disparate parts. See the fatigue problems caused as a result of interactions between the engines and fuselage of the C-133 Cargomaster, or how the F/A-18 had a habit of destroying it's vertical stabilizer due to generated turbulence unaccounted for in the original design. All the FEA in the world gives you a good estimate when you abstract away the details. It doesn't give you the 'real story'.
> I can't even begin to imagine the multiphysics that goes into simulating a nuclear weapon, but if we are willing to bet our entire nuclear deterrent on it, it seems reasonable to skip some additional rounds of physical tests that are mostly a formality at this point.
This is a false equivalence. Nuclear weapons testing has been strictly limited by international accord. You cannot physically test these weapons anymore without "waking up the neighbors" and leading to exceedingly difficult to defuse international scrutiny. That's why the copious levels of "in theory" with regard to nuclear weapons is a reasonable route to take. There is literally no other way to advance the state of the Art. Planes thankfully, don't have that issue. Also, while nuclear weapons testing wasn't restricted, test destinations were done with some degree of regularity; so again, the nuclear weapon point fails.
>Also, defective materials are caught much earlier in the manufacturing process through quality assurance methods, before they become airplanes. And then you apply a factor of safety on your calculations for added assurance.
Just like those leading edge slats for 737 NG, MAX 8 right? You know, the ones that lead to a grounding of aircraft that had already been carrying passengers recently? And who is doing that quality assurance? That bunch of "Quality" people Boeing laid off/relocated elsewhere a few months ago?
Not a credentialed engineer, but know more than enough to LARP as one in certain fields. Testing, aviation/aerodynamics, mechanics of materials, and software being some of them.
You simply cannot know what you're dealing with until you actually test it, and it's generally right after you stop doing that one test that just tells you exactly what you think it should that you get that design with a problem that would have manifested if you just did it.
The only people who should be interested in decreasing testing is the sales, accounting, and business departments. As Engineers, we've had upward of 100 years of design driven 'oopsie' to grind into us the fact we are, in fact, not as clever at accounting for the real world as we think we are.
The fact that as a credentialed mechanical engineer, you don't see it this way is concerning. Though again, I've not had my scrutinous edges rounded off by regular business constrained application either. So there is that.
Depends on the plane I guess, Cessna 172 or F-34 or 787 and the nuke (last gen two stage fusion or Little Boy).
The nuke is harder to build because of the requirement for fuel purification but the actual engineering has to be in the same order of complexity as a modern airliner surely.
I mean we built little boy in the 1940's we definitely couldn't have built anything like the F-35 then.
Sorry for the crappy mistake.
That said the engineering on a nuclear warhead is still exquisite (if such a word is accurate for such a horrible device), requiring insane tolerances and timings to ensure everything works properly.
So it’s a different kind of complexity I guess.
Stress analysis isn’t simple.
In aircraft applications, the analysis has to model not only metals, but composite materials that have anisotropic and nonlinear responses, account for acoustic energy distribution and local concentration effects, incorporate static- low- and high-cycle fatigue mechanisms (which differ in the materials imperfections that initiate them and in how the ultimate failures play out), incorporate time varying and differential spatial temperature distributions ranging from -55°C to ~1500°C (hot section, commercial engines) and on and on.
It’s not simple. I really like physical testing to validate analytical models of complex physical systems.
Perhaps you are forgetting about all the nuclear weapons testing that was carried out in the 40's 50's and 60's ? (The last tests by USA and USSR, took place in the early 90's.)
[0] https://en.wikipedia.org/wiki/List_of_nuclear_weapons_tests
Now probably isn't the best time to put out press releases on this though.
> Beware of bugs in the above code; I have only proved it correct, not tried it. - Knuth.
Boeing's CEO is incredibly still in office inspite of damning evidence of incompetence which is a straight indictment the whole concept of 'shareholder interest' and accountability.
Can anyone provide one instance where shareholder interest has ensured some kind of accountability of management? Why shouldn't Boeings top management be fired for seriously damaging the company and the brand?
This raises even more questions about the stock markets accurately reflecting business sustainability, revenue pipelines and brand damage given 737 Max orders are now essentially over. [1][2]
[1] https://www.bloomberg.com/news/articles/2019-03-14/boeing-s-...
[2] https://www.wsj.com/articles/boeing-737-max-deliveries-fell-...