The 787's Problems Run Deeper Than Outsourcing | Hacker News Reader

The 787's Problems Run Deeper Than Outsourcing

blogs.hbr.org

92 pointshype753 comments

I really wish people would stop speculating about the 787s problems. Yes, it has had serious issues. But why is the author qualified to talk about them? From his bio, it's not clear that he has any aviation experience, and he doesn't appear to have any inside sources.

The article is littered with phrases like "My hypothesis is that McDonnell's mindset from its defense work — minimizing the amount of capital put at risk during R&D — was applied to the 787" and "But, in this instance, it wasn't so much the outsourcing, as it was the decision to modularize a complicated problem too soon". How can the author make such brazen claims without having talked to a Boeing engineer or visited the factory? I assume if he had done either, he would have noted it in the article.

Only one type of person is qualified to comment on the problems: a person working at Boeing, or someone talking to him/her. Otherwise, we're just playing the Apple rumor game, piecing together tidbits of supply chain data to come up with the conclusion we want to hear.

Do you know what the word 'hypothesis' means? Its a theory, something to be tested - its no fact, no statement of truth, just pure conjecture that something has gone wrong with the Boeing group.

You don't get this far into the 787 project, to be defeated by a faulty battery design, unless something is seriously wrong with the engineering processes. Unless management, senior to engineering in the grand scheme of things, has screwed up somewhere. This much is clear.

That the battery catches fire in an operational plane -i.e. its no longer in the test phases, but actually certified and operational - yet none of the billion-dollar Boeing process caught this issue beforehand - is of EXTREME INTEREST to the tech community, and in that context there is absolutely no reason why the issue shouldn't be discussed, openly, as such.

Look: Someone really, really screwed up. Its a Billion-Dollar project, being crippled by a fundamental design flaw - this proves the value of the circumstances to the tech community, for which the motivation for discussing this incident is, mostly, derived from an interest in the process failure.

That the battery is flawed somehow, that the fuel control system is flawed, somehow, that the fire suppression systems don't work so great - this is not so great news, perhaps, technologically, unless we have access to the real data: but that the Process by which Boeing produced a crap expensive product, in spite of billions of dollars of investment, IS very interesting news.

Remember when Toshiba recalled a ton of their laptops for problems with the lithium batteries? They had tens if not hundreds of thousands of those batteries deployed before the problem became evident.

The Boeing batteries are significantly more expensive and higher capacity than laptop batteries which results in longer and more complicated testing procedures. Not to mention needing to test the many various load conditions on the battery. At the end of the day, there is only so much you can do to test a technology before deploying it. There is a chance that Boeing, it's contractors, or manufacturers were negligent in their testing especially after the initial delays of a $32 BILLION program. However, I'm guessing the battery problem is the result of standards not fully prepared to test the battery technology.

I'd like to add a second type of persons having certified its airworthiness and as a third type everybody else wondering whether the tons of flying metal labeled aircraft will stay over their head.

it's a bit of a gray line - it's being published on HBR which is a "Big Name Publication" to a certain extent. that being said, it IS a blog, and blogs are for random musings such as this...

"Big Name Publications" do not get a free pass for speculation, in print or in blog. They gain credibility, so they can say things like "From talking to senior Boeing engineers, we uncovered..." but making up hypotheses out of thin air is shoddy journalism that should be criticized, regardless of the source.

When I read enough poor quality articles from a publication, I never read anything from that publication again. I haven't read HBR in about 4 years.

I did talk to people at Boeing. (And yes, I'm the article's author).

I think the article gets it. I worked as a Boeing subcontractor on a radar system. Boeing really seemed to want to get into the government contract game, a different beast than commercial aviation. They were making more on defense contracts than commercial aviation (although that seems to have changed back the past couple years).

quote: The thing about these Government contracts is that they are paid as development proceeds. This is entirely different — and a lot less risk.

Boeing is in the government contracting game. They're the second biggest military contractor after Lockheed.

Military contracts are low risk for the contractor because the government absorbs all the overruns, but relatively low profit as well since the contracts specify the amount of profit the company is allowed to earn.

Boeing's huge military contract business is good, steady money, but the commercial aviation business is where they take the lion's share of their profits.

That being said, it seems like both Airbus (EADS) and Boeing are profoundly dysfunctional organizations that are really incapable of doing anything without consuming enormous resources. I wonder if both of them have the "military contractor" mindset built into their corporate culture, and that's why their commercial projects are a serial set of disasters.

I think A&B are lucky to have each other as competitors. If a lean, hungry company ever entered the market it'd wipe the floor with both of them.

Stepping up into the biggest end of commercial aviation is a big, big ask.

There are already a number of aircraft manufacturers playing in the next tier - Bombardier (Dash 8), Embraer (EMB 120), Fokker (F50) and others. I can't really see them ever building something the size of a 787 or A380, but I concede it could be possible.

Looking at existing manufacturers, I think Antonov may stand a chance. They already have the AN-124 and AN-225, which might be engineered into passenger configuration.

With regards to new players, I was reading an article about the Indian company Mahindra yesterday[1]. They have ambitious aerospace plans, already some exposure to the sector through their purchase of GippsAero[2], obviously with much smaller aircraft, but also are located in a fast growing market. The article mentions Boeing's expectation of over 1,000 new commercial airliners needed in India by 2020. Maybe this is your new lean, hungry company.

[1] http://www.flightglobal.com/news/articles/indian-automobile-...

[2] http://www.gippsaero.com

Bombardier also build CRJs, business jets (Globals, Challengers and Learjets), does government work and builds fire-fighting aircraft. Oh, and the C-Series is a brand new a/c entering into the lower end of the Boeing market.

That being said, building even these small aircraft is a multi-billion dollar investment. If people here think that software design is hard. It takes a monumental amount of work to design a commercial aircraft and the manufacturing processes around it. Oh, and then there is the test phase. That part is fun.

I wonder about SpaceX. They are also trying to get in the defense contracts game (already have as of Dec 2012).

They also have strong leadership. I have a feeling that as long as Elon Musk is at the helm, he won't let that kind of stuff seep into their operation. After he's gone? It could be anyone's guess. Pulling in defence contracts could affect the corporate culture long-term, but I have a feeling the Elon will try to keep it at bay while he's there. I mean, he has aspirations to be on a Mars mission himself. I can't imagine he would want the equipment to be skimped on.

As a counter point, outsourcing has served the car industry quite well. At one point in its lifetime Ford had it's own rubber tree farms. Nowadays car companies rely a lot on suppliers and heavily involve them in design as well. It's just very unlikely that one company has the skills and the focus to be good at everything.

That said (and the article makes this point as well) suppliers have to be closely involved for this model to work. When Toyota was consistently beating American car manufacturers in terms of quality one of the biggest differences was in how they managed their suppliers. American companies tended to have multiple suppliers for the same parts and determine (sub) contracts mainly on a cost basis. Toyota on the other hand had very long term relationships with its suppliers and worked much more intensely with them.

I don't think this is a counterpoint. They said that it makes sense to be vertical early on. Look at the Ford Rouge Works where iron ore and rubber came in one side and Model A's came off the other.

Now, it's a very mature category where it's all about refinement, not major innovation - so it's good to develop an interchangeable supplier base because you increase competition between them and you don't need to do anything radical.

If you look at SpaceX for example, they make >90% of their parts because there's full-stack innovation going on constantly.

> Now, it's a very mature category where it's all about refinement, not major innovation

I disagree. You are completely right that in that situation it makes sense to develop an interchangeable supplier base. as you mention it allows you to increase competition and put pressure on them to reduce costs.

However, in a situation where you're still looking for innovation, like developing a new car (or plane) it makes sense to work with highly integrated suppliers. These won't be interchangeable and because you're very dependent on them, the opportunities for cost saving are probably smaller. On the other hand, because they have specialized knowledge and skills they can actually help you innovate much faster. At least that's the theory; you're right that SpaceX seems to focus on doing as much as possible in-house and it seems to work for them. A reason could be however, that the type of suppliers necessary to make this model work are just not there (yet) in their business.

I guess in practice it's about finding the right balance. Insisting on doing everything yourself can lead to bad cases of NIH syndrome, while outsourcing everything leads to a loss of control.

Yea, totally agree, but something like cars have far less variation than do SpaceX's rockets. Say, the automatic transmission isn't massively coupled (engineer constraint wise) to the rest of the car unless you're doing a hybrid or something.

That means the local improvement (better gears, weight reduction, etc.) is most of the improvement - not in conjunction with much else. It's why you'll often find struts from a 1999 model that are drop-in replacements for the 2012 even though the cars look very different.

The other element is being cornered by suppliers. If you only have one or two suppliers who can produce a component, you are exposing yourself to risk. Those one or two suppliers can let you develop a design that relies on their technology, then greatly increase the price when you are ready to go to production. Elon Musk calls this "going Russian": http://www.esquire.com/features/americans-2012/elon-musk-int...

I think the point in the article is that outsourcing your design of a brand new product early on is a bad idea. Once the design is done and the components have been standardized, outsourcing would cut cost. For cars, a lot of components have been standardized and don't change much over time. A tire still has rubber outside and steel rim inside.

I worked on two airplane programs at boeing. The first was a derivative of the 747, the 747-400. The main features of this version that differed from previous 747s were going from analog to digital avionics, a slightly longer body and new (larger) wings. The schedule from project launch to first delivery was about 4 years. We were two weeks late on first delivery.

I also worked on the 787. The difference between the 787 and every previous Boeing plane was almost everything. The schedule from project launch to first delivery was about 4 years. We were two years late.

Moral: don't bite off more than you can chew.

Went to see one of the Boeing leaders talk specifically about the dreamliner's development a few years ago at a Georgia Tech symposium.

He made a point to mention how many different international suppliers were involved and how that was a huge challenge. e.g. They had to custom modify several 747s to increase the cargo hold to fit some of the sub-assemblies.

He gave the impression that part of why there were so many different sub-contractors is to appease foreign governments--especially since many airlines are nationalized. The corporate equivalent of pork-barrel spending.

Seems a bit pre-mature for the conclusions of this article. The article is motivated by the grounding of the plane due to battery issues, but its not yet clear what is causing these batteries problems. In otherwords, the article's argument would still be true if there were no battery issues, but I think readers are more interested in the context of these problems.

Generally I agree with the thesis of the article - aircraft are incredibly complex and require both modularization and top-level design management. And the article does make a good point that this kind of framework is needed for this scale of projects - whether module are worked on "insourced" or "outsourced". I think a potential issue with outsourcing is that geographic and organizational isolation probably leads itself to tougher integration.

I'm not sure the author hasn't run too far with his thesis here. Every modern airliner is outsourced to a major and entire systems must be designed by the supplier that provides them. If the requirements are understood this usually isn't an issue.

Major components of an airliner are often designed and manufactured in geographically separate locations. In the case of Airbus each national entity has different processes in some cases and this has caused issues in the past. The 787 is no different in this respect where entire systems are manufactured and built by different entities and then transported for finally assembly. In the case of the 787, if the author is correct that modularization has caused the issues then one would expect that the largest issues would occur with the major elements such as wings as they are built entirely by a subcontractor in japan. Of course, there have been no such issues.

The electrical problems are probably driven more by the extreme demands placed on the power supply than anything else. It wasn't as if Boeing wasn't aware of the risks involved in choosing Li-ion batteries and was just blindsided by a supplier. It made the decision and then sold it to regulatory agencies. If it didn't understand how to make the batteries safe or it wasn't comfortable with the design Boeing should have rejected it.

Yes, we still don't know the root cause of the issues. But suddenly leaping from that to placing blame on McDonnell executives keen a little too keen on outsourcing just based on a hunch is a little strange to say the least.

The article cites a long list of delays and problems, does it not?

I get the feeling the market had forgotten about the delays until this current situation. Planes were shipping in a highly visible fashion.

Delays in the introduction of new models are extremely common in aerospace. If these battery issues had been noticed by the right people sooner the 787 would have been delayed more than it was. Very few airlines would have cancelled orders over that.

from the linked boeing paper:

if outsourcing is to be employed, it is understood to be absolutely necessary that detailed parts and subassemblies be designed with that process in mind. this requires considerable additional upfront effort in planning to avoid the situation whereby major subassemblies do not fit together at final assembly, increasing the cost by orders of magnitude more than was saved by designing in isolation from the work allocation activities.

forgive typos - cut + paste from pdf not working.

there's more good stuff about the timescales over which outsourcing for low labour is appropriate.

https://s3.amazonaws.com/s3.documentcloud.org/documents/6974...

All this speculation about what's happening from people that aren't aware of the facts reminds me of something that happened to me the other day.

A top manager in my company came to give our office an all-hands meeting about management changes. During that, he claimed that my product (Online Calendar) doesn't support timezones and bashed it in front of the whole group.

After the meeting my manager and I confronted him and told him that the problem was with the iOS Calendar application since we're only providing data via CalDav. He was insistent that because we have control of the data that it is, in the end, our fault and a problem with our software.

My boss's boss gets involved and immediately recognized that the problem is that the phone had a setting enabled that removed automatic timezone switching for the Calendar app. A quick change to that setting and the iOS calendar was showing events in the current (Mountain) timezone rather than his home (Pacific) timezone.

So, even someone who's worked in tech for a long time and has a pretty good understanding of things like protocols can still have no idea what they are talking about without all the relevant information.

Odd thesis: essentially that not outsourcing, but outsourcing too early in design is the problem. Well I don't exactly think you can divorce the two...Once you make a choice to outsource larger and larger components, you're inherently losing control of the design and perhaps more importantly exact construction of those components. In so doing you open yourself to design risk. The way the author describes the primary risk of outsourcing as "business profit risk" is odd to me.

Still the article is ridiculous considering the 787's battery problem, if indeed limited only to the battery, seems like it will be a fairly simple one to fix, and an isolated incident.

I like how he lists three 787 problems, two of which are "smoke in cabin" and "fire" as if the latter two aren't likely the same root cause. Sure fuel leaking was an issue too. So we work around it.

This is typical of the internet era where we can nitpick everything and then suddenly a multi-decade project comes along and we don't know how to handle its optimization/fix process.

You might think it's odd, but it's exactly what Boeing had done in the past.

And if you think the 787's problems are confined to its batteries, please read this: http://www.bbc.co.uk/news/business-21230940 "Keith Hayward, head of research at the Royal Aeronautical Society, said that if the issue is no longer about replacing a faulty battery, it raised the prospect of Boeing having to do a major re-design.

"I think people had their fingers crossed that it was a battery fault... it looks more systemic and serious to me. I suspect it could be difficult to identify the cause," he said."

(Disclosure: I'm the article's author)

Cheers

-- james

Oh, please folks, temper your enthusiasm to hang these guys. For all we know it might very well be grotesquely misplaced.

As I stated in another thread, I'd prefer to read commentary from someone who has actually had first-level access to the relevant design and test documentation. Everything else is nothing more than conjecture.

What are some of the failure modes of battery systems, including Li-based systems?

OK, here's an incomplete list that has nothing whatsoever to do with the 787 case because, well, no design and test data has been released. Again, this has nothing to do with the 787. Think of it as the start of a list of things to watch out for if you were going to set out to design a battery pack and related systems.

    Sourcing
        - Latent defects due to manufacturing QA issues
        - With Li-based cells such things as humidity content can
          be critical as they might lead to the generation of methane
          gas during the service life of the cell.

    Assembly
        - Handling and storage problems
        - Mechanical damage, from dropping a cell to any number of 
          imaginable issues.
        - Assembly defects that can lead to (I**2)*R heating issues

    Testing (both at by supplier and OEM)
        - Failure to detect defects (sometimes impossible if internal)
        - A faulty testing process that actually causes or accelerates damage
          to cells, perhaps only cells that have specific problems
        - Inadequate issue escalation workflow

    Electrical
        - (I**2)*R heating of faulty interconnects (inter-cell and elsewhere)
        - Component defects that can lead to (I**2)*R heating issues
        - Oxide formation that can lead to (I**2)*R heating issues
        - Interconnect crimp quality problems that can lead to (I**2)*R 
          heating issues
        - Interconnect crimp quality problems that can lead to other issues
        - Main pack electronics problems
            This can include chargers and monitoring electronics
            - Charge termination failure or inadequacy
            - Thermal sensing failure or inadequacy
            - Pressure sensing failure or inadequacy
            - Fluid sensing failure or inadequacy
            - Inadequate semiconductor rating (for example: operating temperature)
            - Inadequate power semiconductor heat dissipation
                This doesn't necessarily mean bath thermal design.  
                It could be caused by inadequate or faulty mounting of TIL 
                (thermal interface materials).
                It could also be caused by mechanical and environmental issues 
                affecting the TIL or the mating of the semiconductor to the heat
                dissipation mechanism.  For example, vibration.
            - Inadequate or faulty failover modes
            - Static damage to semiconductors during electronics assembly, testing 
              or general handing
            - PCB via or trace fractures, contamination or over-etching
            - BGA ball soldering problems 
                (mostly only visible via x-ray inspection)
            - Bad, cold, fractured or chemically compromised solder joints
            - Dendrite buildup in lead-free solder chemistry leading to shorts 
              in small pitch components
            - Unaccounted race conditions
                Example: pull-pull technologies without adequate dead-band
            - Thermally-induced failures due to changes in operating 
              characteristics of components as temperature fluctuates.  
                Example: The aforementioned push-pull topology where the deadband
                might have been adequate within a certain temperature range but 
                causing shot-through shorts outside of this range.
            - Unaccounted for effects of such environmental issues as humidity.
                Example: Many VCO chips do not deal with humidity very well,
                particularly if it can condensate.
            - Faulty or non-existing conformal coating
            - Inadequate static damage counter measures 
                (inputs, anything expose to handling)
        - Sub-pack electronics problems
            This is applicable if a large pack is made from a multitude of 
            smaller packs, each with it's own control and monitoring electronics.
            Sub-pack electronics can suffer from the same issues as the main pack
            electronics above.

    Software
        - Myriad of code-based issues 
           (lock-ups, endless loops, pointer problems, uncaught exceptions, etc.)
        - Software problems caused by hardware issues
           Example: Static damage to a chip causing a bit to unexpectedly flip.
           If not caught in software this can lead to catastrophic failure

    Mechanical
        - Faulty inspection and certification process
        - Inadequate design materials
        - Adequate materials but supplier quality issues resulted in unexpected 
          materials issues
        - Inadequate design tolerances 
        - Adequate design tolerances but faulty assembly, subcontractor or testing
          process caused them to be violated
        - Faulty fasteners
        - Faulty fastener application
        - Unexpected interference between components
        - Mechanical pinching or chafing of wires
        - Mechanical puncture damage to cells during installation or operation
        - Mechanical abrasion to protective cell wall or packaging
        - Insufficient mechanical rigidity
        - Insufficient venting mechanism
        - Insufficient containment or venting of gasses (methane), 
          flames or heat
        - Mechanical issues once pack is installed
            - Thermal
            - Mechanical (pinching, squeezing, twisting, cutting)
            - Resonance

    Thermal
        - Inadequate thermal management
        - Lack of consideration of thermal runaway problem
        - Lack of thermal isolation between individual cells or sub-packs
        - Insufficient thermal sensing
        - Insufficient thermal overload coping mechanism 
          (both mechanical and electronic)
        - A design that might cause severe differential heating of cells deep
          inside the pack
        - Inadequate electronic and mechanical fail-over measures in the case
          of thermal runaway
        - Inadequate thermal runaway detection
        - Cell supplier issues (quality control, testing, etc.)
        - Thermal issues once installed
            - Problematic heat exchange
            - Heat input from adjoining assemblies
            - Accidental plugging of filters, inlets, etc.

    Final release
        - Electrical testing
        - Mechanical testing
        - Environmental testing
        - Random destructive testing at the cell level

    Operational
        - User fails to follow published procedures
        - Alarms are ignored
        - Mechanical issues are ignored
        - Inadequate maintenance
        - Handling issues

Anyhow, this is not, by a long-shot, an exhaustive list. There's a lot I am ignorant about and, of course, that stuff is missing. Feel free to expand.

The point is that a proper investigation would use --at a bare minimum-- something like this and, more than likely, a book full of specs and procedures before reaching a verifiable and reproducible conclusion.

Maybe it's just me. I don't know. I just don't like it when people do some of what is being done to Boeing. Particularly when they don't have access to the reams of FAA, Boeing and airline data that is required to really understand the problem.

EDIT: Formatting

>>Maybe it's just me. I don't know.

From your comment history it sure seems like you have a vested interest in maintaining favorable PR for Boeing in this forum.

Why shouldn't the industry discuss this in open terms? The fact is we don't have access "to the relevant design and test documentation". Boeing and their partners won't share it. So, how can we discuss this without the data, then?

Conjecture and informed opinion is all we have, and it seems as if you have rather a bit of the latter .. so why not share a technical analysis of the circumstances based on your understanding rather than attempt an appeal at what could be construed, frankly, as a PR move?

What if, Boeing are simply mis-managing the project? It surely is not the first - nor the last - time that a major aerospace company was bit by that bug - in fact, for anyone that pays attention to the big, fat, military-industrial corn-fed cows of the last few decades, its rather the norm than the exception!!

Nothing says "management really, really screwed up" than a smoking pile of brand new tech, being doused with fire-retardant foam, while the world asks questions like "why is our technology failing us?" from afar .. lucky, we're not there yet with the 787 - so this is really the best time for us to be discussing these issues. Lucky no lives are lost .. yet.

Nope. Never worked for Boeing. Never worked directly in aerospace. I did manufacture motor and battery controls that were used in drones and robots maybe twenty years ago mostly in academia. I have no way to know if the motor controls we sold to institutions such as MIT ended-up in Boeing product. I doubt it.

It's easy. Go to my profile on HN. Send me an email. I'll reply back with my LinkedIn profile. Then you can go figure out if I have anything to gain from my comments.

I'm just an engineer with a lot of experience making mistakes who hates it when people run their mouths without data. That's all. Things at this scale are never as simple as some journalist might make it seem. Most people don't have a clue as to how such massive machines as a 787 are designed, built and tested. Not a clue.

Many years ago I had a chance to meet and enjoy long talks with the engineer who had been in charge of the F18 project. It was a fascinating learning experience for me. If I remember correctly, he said he had some 3,000 engineers working for him. Now, how many people would imagine that 3,000 engineers had their hand on the F18 design simply by looking at it? Better yet, how many journalist? And, how many of these people could diagnose an F18 problem --even if qualified-- without access to real engineering and test data?

> Conjecture and informed opinion is all we have, and it seems as if you have rather a bit of the latter .. so why not share a technical analysis of the circumstances based on your understanding rather than attempt an appeal at what could be construed, frankly, as a PR move?

Crap, I didn't catch that one.

Really? That's how you see what I wrote? Amazing.

Here you have an engineer (me) saying: "We don't know enough. We need facts before we can reach a conclusion. Please stop."

And your interpretation is that I am using conjecture for PR purposes?

Are you an engineer? If you are not, maybe you just don't understand my vantage point. And that's OK. Life goes on.

Here's an analogy, imperfect, but, what the hell:

A guy gets arrested and accused of rape.

You say: "He probably did it. Hang him!".

I say: "Let's wait and look at the evidence, which should ideally include DNA. Then we can look at the facts as they may be available to us and make a decision."

These are vastly different approaches to the problem.

Who would you want in the jury if YOU were the accused? Someone like me or someone like these clowns that are flapping their jaws without having access to the evidence?

Look, since none of us actually have the data, what you're asking is that we just simply stop discussing the issues that Boeing is having. What can we do but discuss it, at this point - no proper analysis can be made, only one based on conjecture and experience.

Isn't it clear by this point that Boeing have a problem? Its either a) an engineering problem, b) a management problem, or c) a combination of both.

Its not unreasonable to expect that a company with as much money, and - frankly - unwarranted guaranteed income from the people of the United States of America - to get its act together - and certainly, the means by which it gets it act together is of high interest to the average denizen of hacker news ..

So, why call for people to 'not discuss it unless they are experts', then? There are many lessons to be learned here, and we can certainly use our intelligence - the intelligence of the whole - to make a social comment on the issues revolving around Boeing, and in fact the entire military-industrial machine. It has its frailties, obviously - and we are seeing them - so isn't this of High Interest to the hacker news community?

Calling for an 'authority' to comment on this issue is hardly a way to gain more insight into the problems of technology in this industry. The 'authority' has already been consulted; and clearly, somewhere along the line, screwed things up completely. The 787 grounding is costing Boeing, its partners, and the American people - a lot of money.

Discussing why this happens, how it happens, and what can be done about it, is a very topical use of hacker news, in the first place: so why call for silence-unless-authorized unless there is some desire to reduce the noise from the rabble? The rabble aren't doing any harm; whereas the likelihood that the management personnel of Boeing are doing harm. Only by discussing this can we gain any real insight into just how that can occur.

Clearly, the engineering is at fault: there is a fault. Is this management, is it engineering, or is it just plain hard science involved - unless we discuss this situation, openly and with a free will to apply to the investigation, there is nothing much to be gained from the incident in a public setting.

I certainly hope they get the problem sorted. The means by which they do that, is of high interest to me. I hope to fly on a 787 one day, and know that my safety is a result of real solutions and not a faulty process being forced on engineering by management under constraint of financial market performance duress..

>>Here you have an engineer (me) saying: "We don't know enough. We need facts before we can reach a conclusion. Please stop."

Here you have an entrepreneur saying: "We don't know enough. We need to know more. No conclusions are being made, but the process of discussion sure is interesting - it reveals a lot about human nature when billion-dollar budgets - and human lives - are at risk. Please continue to discuss this issue as a community."

Sure. Whatever you said. It's OK.

I can't say much more than I already have because I don't have any real data to work with. None. You and others are, of course, completely free to visualize all manner of scenarios based on no direct evidence whatsoever. And, again, that's OK. Perhaps a disclaimer might be nice?

I also look at this from another angle. As an American (US Citizen, whatever sounds better to you) I am proud of the accomplishments of companies like Boeing and, yes, SpaceX and others. These are the companies I point out to my kids as examples of what smart people can achieve. Put bluntly, I don't like it when I see a feeding frenzy that could destroy their reputation, cost jobs and do other damage and said frenzy is based on not much more than pure unfiltered bullshit.

Do they have an engineering problem?

Maybe.

How about a manufacturing problem?

Maybe.

How about a sourcing problem?

Maybe.

Testing?

Maybe.

Sabotage? Terrorism?

Maybe.

Negligent operators (airlines)?

Maybe.

I could go on.

All answers which originate from a lack of evidence can, at best, be "maybe's" and at worst complete fabrications.

Now, you could take the time to go through the list I presented on this thread and expand it with research into each of these areas and even expand my list. That could be an interesting discussion of potential failure mode of such systems. This CAN BE DONE without making categorical statements as to what "actually" happened to the 787 battery pack. And, yes, that would be an interesting and useful discussion. So, there you go. Take one item and fly with it. Let's see where it leads.

>>Do they have an engineering problem?

>Maybe.

No! The answer is: YES THEY HAVE AN ENGINEERING PROBLEM.

The planes are not safe to fly!

It seems to me that it is very clear that there are multiple problems, that the information necessary to identify the underlying causes is not available to the general public, and that Boeing, partners, and USG are working on getting this resolved. It seems interesting but not particularly useful to speculate.

An interesting but useful line of questioning would be to look into industries we know and our past companies for what happened and what lessons can be learned in places where we have more complete information.

That said, the topic at hand :)

One question is whether the battery failures represent an unacceptable safety risk. Without knowing anything about the technical specifics you can figure the answer is yes: Boeing is the home town team and the FAA has every political interest in keeping Boeing planes in the air. Still, their overriding concern is safety and they've grounded the 787 in what must have been an unpleasant choice but one viewed as necessary.

Another question is whether there is a management failure at Boeing. Management is ultimately responsible for outcomes so whether this incident is out of line with the industry norm is one reasonable way to answer that question. In this case looking at data on whether grounding of a new model is unusual would be reasonable. NTSB or FAA probably have this in a report somewhere, my guess is it's rare (the only other similar thing I can think of is the A380 wing cracks). So my guess is yes, by a reasonable standard there was a management failure.

Those two things seem obvious and not particularly interesting to discuss. Some things that are not obvious:

What's wrong with the batteries? robomartin's list of potential causes is long and educational but (as he points out) incomplete. This is an interesting problem for which we don't have enough information to reach any kind of strong conclusion. Some really smart people who are also domain experts and who have all of the data will sort this one out. Best read the news, at some point there will be a weighty public report detailing exactly what went wrong.

What's wrong with Boeing? I'm sure this will be studied for decades to come, I don't know and I'm pretty sure the author doesn't know. The idea that markets start out favoring highly integrated designs and trend towards standardization and modularity is well accepted and probably correct but at this point he's just fitting recent events into that framework rather than finding ways to test that hypothesis.

Whatever.

I couldn't agree more with your post and your approach to discussing the problem. Yes, unless we are talking about something like and attempt to sabotage or worst, terrorism --both of which I would like to believe are highly unlikely-- ultimately management and engineering have to look back and see where the process dropped the ball. We are only human. We make mistakes. As you said, actual data will bubble-up to the surface eventually. From an engineering standpoint (and as a passenger!!!) I'd love to understand what happened.

A while ago I enjoyed reading "The Machine That Changed the World":

http://www.amazon.com/The-Machine-That-Changed-World/dp/0060...

It was interesting to learn about the evolution of manufacturing and management practices that made a huge impact on the automobile industry. Lots of lessons from that book can be applied well outside that industry.

Thanks, that looks like a great recommendation. I spent some time at IMVU (online service + software) a few years back and their processes were heavily inspired by the Toyota Production System. The stuff they adopted seemed to work really well so lean manufacturing has been interesting to me since.

That is not always and engineering problem. It can be a manufacturing problem. One case[1] of this is the F-86. It was engineered fine, but a worker on the floor didn't follow instructions and installed a bolt incorrectly. This caused the plane's aileron to lock during certain maneuvers killing some pilots. This was a manufacturing problem not an engineering problem.

1) http://characterqualitystories.com/cqs/node/394

> No! The answer is: YES THEY HAVE AN ENGINEERING PROBLEM.

How are you eliminating problems with maintaining and operating a new technology? Those wouldn't be Boeing's engineering problem.

And this is why stating your occupation as "engineer" is an express ticket off of a jury.

Isn't that the truth! It has happened to me a few times. I actually wanted to serve too. Nope. The minute I revealed what I did: business owner, engineer, software, etc. Done. Excused.

If we're going to report anecdotes, two (aerospace) engineer friends of mine (one of whom professed this purported fact) at the same company got picked for a jury on the same drunk-driving case.

I think it depends on which attorney gets a turn when they question you. I don't remember exactly, but don't they take turns? I would think that, depending on the nature of the case and the evidence, one or the other attorney might prefer to inject jurors into the jury who might tend to take a more analytical approach to things.

there is something deeply disturbing about our legal system when that is true :|

Not much to add, except that you're not giving yourself enough credit. You didn't just say, "Let's wait for the evidence, like X." You said, "There are a lot of factors that could account for the phenomenon we've observed, some of which would implicate the accused. Here's an extremely thorough list ..."

Couple things.

One is that I'm actually thankful for robomartin's comments; they break up some of the more outlandish hero-worshipping of Elon Musk (i.e., http://news.ycombinator.com/item?id=5138430, which is frankly just a ridiculous thing to say).

Another is that this is not new. What is new about the 787 is the sheer amount of media attention, not the fact that there are serious post-service-entry problems. For example:

You mean, the way we all went crazy, posted huge numbers of threads and begged people who don't work on airliners for a living to come save Airbus in one fell swoop after the A380 exploding-engine accident (which, on investigation, turned out to not be a one-off problem, but a general flaw with the plane's engines)? The one where a smoking pile of brand-new tech had to be doused with fire-retardant foam?

This stuff happens. The other thing that's different about it now is that when it happens, people don't die by the hundreds, as they used to when a new airliner entered service and a problem suddenly became visible. What is not different is that it still gets solved by research and hard work, rather than by pop-entrepreneurs snarking away in major newspapers.

Actually that's inaccurate... The A380 has 2 engine types and the issue only affected one of the 2 types, specifically the Rolls Royce engine. The other problem is that the issue was not a general flaw but some of the oil valves in the engine were not cut properly and would eventually break.

A better example of how this stuff hits the A380 are the cracks in the wings.

There have been a lot more problems with the 787 than the batteries, as the article describes. And you didn't address the central thesis of the article, which is that the root of the problems lies in how the project was managed.

I did say the list was incomplete, didn't I?

Feel free to tackle project management issues of a hypothetical project if you feel it might add value.

Reply on news.ycombinator.com