Hacker News Reader

Comment by code4tee | Hacker News Reader

Boeing’s answer seems to say if you accidentally pull the thrust reverser lever before the aircraft is on the ground it just force shuts down the engines.

That sounds like an odd safety override. Surely a better solution would be to just not activate reverse thrust. Unlike car engines a jet engine can’t just be quickly restarted if it accidentally shuts down. It typically takes 30-60 seconds to get going.

This and earlier incidents are highlighting the dangers automation can add to mission critical systems. No pilot wants to hear notices about “hey, so in case you didn’t know our programmers added some code that does this strange uncommanded thing when you push buttons a certain way”

View on HN

Do you have experience with aircraft engine design, or landing protocols, or insight in to the meetings when the engines were designed?

If so, I'll take back my words but considering the manufacturer doesn't know exactly why this happened, and the engineers on the ground said it seems like a "software bug", it seems a bit presumptuous and frankly a bit comical to start saying things like:

"Surely a better solution would be to just"...

Ah well, glad a 5 sentence comment on the internet can resolve an issue in a hardware, software, and social engineering challenge decades in the making.

Reminds me of the quote,

"For every complex problem there is an answer that is clear, simple, and wrong." -- H.L. Mencken

One of the things I have experienced in my career is the mind boggling complexity of the software in systems that are capable of killing people. The more people it can kill the more complex the software.

As a result when I read articles like the one posted I find it tantalizing to speculate about the requirements behind the story. Taking the story at face value that Boeing has thought a lot about it and that is how the software has to be, what prevents the other solution of disabling the reverser? My guess is that you want the reverser to work in the event of software failure so it has to always work, but if the pilots pull it while you are flying that would probably rip the engines off the plane. Perhaps the compromise is to take the reverser off during flight and relight the engines using aerial pressure to spin up the engines. (which has me wondering if airlines still have a turbine they can drop down to start an APU which can then be used to start the engines). Which leads me to wondering if the pilots turn off the reverser right away do they have enough residual engine power to relight? Versus when they got to the end of the runway and had nothing? Clearly the 777 doesn't have an APU running when it lands since they would have used that to do an engine start at the end of the runway or if it does, it isn't running.

It is one of those things that my like minded systems friends could sit around lunch and make a good hour and a half discussion out of.

Relevant article: "Reality has a surprising amount of detail"

http://johnsalvatier.org/blog/2017/reality-has-a-surprising-...

This is very good.

If the reverser could rip the engine off the wing, it wouldn't be safe to engage on the ground. Just thought that was worth pointing out.

A reverser only redirects thrust, and only generates about 60% of max thrust pushing backwards.

It's only on for a limited time on the ground because under a certain speed, you run the risk of blowing FOD into the intake path, putting the engine at risk.

I'm not aware of any reason to outright lockout a reverser in flight other than to hedge your bets against a very, very poor set of configuration choices by the pilot (reversing near stall speed too low to recover). I could foresee scenarios where being able to use reversers in the air could save the aircraft with the right combination of subsystem casualties.

Not a pilot or engineer, but absolutely LOVE aviation.

The cruising airspeed of an airplane is faster than its landing speed. I don't know if that alone is enough to rip the engine off the wing, but it's something that it sounds like you did not consider in making your point.

[EDIT remove "roughly speaking an order of magnitude"]

>The cruising airspeed of an airplane is roughly speaking an order of magnitude faster than its landing speed.

No. No it is not. 787 approach speed is 145-153 knots. It does not cruise at 1400+ knots.

Thanks for bringing more precision to the conversation (really).

What would you like to call the factor? 3? Call me a weasel, but I was hoping "roughly speaking an order of magnitude" would go down to 3. I suspected it was comfortably above 2. Am I right about that?

> I was hoping "roughly speaking an order of magnitude" would go down to 3.

I'd call that half a order of magnitude, considering log₁₀(3) is around 0.477. So you're off by more than a factor of two.

You know, I considered saying that the "roughly" buys me a factor of two in log space. I didn't want to be that person, but I'm glad you were ;).

It's surprisingly non-trivial to pin down exactly what the 'correct' scale[0] for a given measurement is. I do agree about "roughly" generally being up to a factor of two in the appropriate scale[1], although I'm the sort who thinks a 19% increase (or 16% decrease) should be called a quarter of a factor of two.

0: Uniform, linear and logarithmic are obvious candidates, but depending on the domain you can end up with some really wierd scales (eg floating-point ULPs, which can look logarithmic or linear, but aren't either).

1: hence > So you're off by more than a factor of two.

3 is half an order of magnitude (because it is a good approximation to 10^0.5)

re go down to 3: Use different words and see if you think it was reasonable

roughly 10x going down to 3x would also mean it would go up to 17x. That's a pretty wide range, so I don't think that order of magnitude is going to ever really be similar to 3x of something on the basis of what it means.

but what about a rough order of magnitude? There is definitely some qualifiers you can put on "order of magnitude" to make it include 3.

"within an order of magnitude"

It seems unlikely that a jet engine could rip itself off the wing in any scenario based on thrust alone. The forces involved are not that large.

Your wording makes it where you aren’t factually wrong, but a plane was in fact wripped apart due to the thrust reverser engaging on one engine of a 767:

https://en.m.wikipedia.org/wiki/Lauda_Air_Flight_004

The difference isn't that large in the grand scheme of things. The engine mount would have to be able to fail in it's operational envelope in order for the reverser kicking in to be realistic.

In fact, the reversed kicking in would decrease the loading on the aircraft by decreasing it's airspeed.

This isn't a case of acceleration being able to break the mount from the frame. If it ever could, one wouldn't want it on the plane in the first place.

Doesn't mean you couldn't ruin your day with it, but it isn't an instant catastrophic failure either.

> but it isn't an instant catastrophic failure either.

Read the accident report and follow up on Lauda Air Flight 004. Boeing specifically was forced to issue a statement that it was virtually impossible to overcome a catastrophic failure outcome from a thrust reverser deploying at cruising speed.

I stand corrected. Way to go Boeing. Seems like that should have come up in a design meeting at some point.

Apropos, the Wikipedia article (https://en.wikipedia.org/wiki/Thrust_reversal) mentions number of events where an engine reverser was deployed in flight, one involving a 767 which resulted in the loss of the aircraft.

That counts as a pretty serious thing to avoid :-).

Yes, Boeing still puts the RAT - ram-air turbine - electrical backup system on their planes.

Why "still"? Even if the chances of a failure of both engines is almost zero, it has happened that there are losses of fuel. The ram-air turbine allows the pilots to have some instrumentation. That's super useful, for example in this accident: https://en.wikipedia.org/wiki/Air_Transat_Flight_236

> The ram-air turbine allows the pilots to have some instrumentation.

Ideally, yes. I submitted a recent incident where the RAT deployed on a 777 because it's so unusual (and was also not due to lack of fuel):

https://news.ycombinator.com/item?id=19039978

“Still” because if you read the parent to my reply he uses that term in his query.

I always love it when you have worked on something for months and someone new comes by and says "Why don't you just X?".

It's worse when they are right though. I wish I could say that had not happened to me.

Is that from other experts with a very close understanding of the specifics, or random people online?

What I find is you start with several viable approaches, pick one and go down the path enough to figure out the downsides. At which point you need to decide to backtrack or keep going. That’s the hard part not simply coming up with a seemingly simple solution.

Not "random people online", but I've got a few friends I think of as "terrifyingly smart", and one of their common characteristics is how they've all, in areas of expertise I've been investigating/researching/working in for weeks or months and which they have barely a passing interest in, quite obviously thought about a problem I'm describing and thought through a bunch of the obvious options, categorised them, and made conclusions about which avenues are workable and which should be discarded, and come up with either a workable solution or options I'd not even considered yet - all just in the course of a conversation over coffee...

(Somewhat frighteningly, two of those people are doing that at Facebook right now...)

> Is that from other experts with a very close understanding of the specifics, or random people online?

These groups are not mutually exclusive, membership can only be assigned by a domain expert and is finally irrelevant if the other is right.

Constraints define good decisions. A new team member making a viable suggestion is very different from a random person tossing out a wild ass guess that happens to be right. The difference is the random commentator has no real way to judge how viable something is, and thus is simply tossing out ideas.

This can be right when things change over time and people still operate under their initial assumptions. Times change constrains change but on some teams assumptions and choices are not revisited.

I am fine with that. In most cases they aren't though. Or they don't understand the whole environment.

My experience is about 50/50, both with my own mistakes and with others.

Generally when one is very close to the problem, one sees the environment as immutable. Because, well you spent very many hours building that environment, for very good reasons. And your complex solution "has to" work within the constraints of that complex environment.

Whereas, the "mind of a child" that doesn't grok the environment, also doesn't have a fixed notion of it. This is anecdotal of course, but maybe half the time what I see happen is that it's easier/better to change the environment and this can only be seen with fresh eyes.

It's not the spoon that bends, and it's not you that bends around the spoon. There is no spoon.

Had someone do that this morning. They ran across a problem that has been the focus of a multi-year effort involving dozens of people. Fixing the problem is a big part of the literal #1 priority for the company.

Anyway, he hit an example of the issue, and wanted to just put in a bug ticket to one team, and didn't understand why that was not useful or necessary.

My analogy is that it's like showing up at NASA in 1965, and wanting to submit a ticket that says "Your rockets can't actually go to the moon. Fix rocket so that it can go to the moon."

The best part is when your project is running late or having issues, and the non-technical manager plans a meeting with all the other dead-weight people. "Explain the problem to us, and since we know absolutely nothing about it, our stupid questions might give you new insights". Facepalm.

Exactly. "Let's all take a step back" or "I'd like to challenge that". Makes you sound insightful while knowing nothing.

I found that I now distrust a lot of articles as people tend to get the things in my field blatantly wrong all the time.

Things are complicated and rarely as simple as they seem.

On the other hand, I've worked on things for hours before for someone to glance at the problem and solve it. Always a humbling experience.

https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect

I never knew there was a name for this, but I feel like everyone has come across this feeling at some point in their life.

I for one love this type of question, and was the one asking it as I started out. 10-20% of the time, the asker has a valid suggestion. The rest of the time, the explanation as for why it will not work or is not a good idea fosters a better understanding for the asker, and reinforcement brush up for the answerer.

I always appreciate those comments. Sometimes they include something obvious I never thought of.

Yeah, it shows someone is trying to understand the problem. I mean, sure, most often the response is, "because that would require data that does not yet exist when the decision must be made," or, "there are legitimate circumstances where that would be the exact wrong response," but it does get you thinking.

It is less appreciated when people phrase things in a condescending manner, though.

They are right more often than you’d expect. Irrational escalation, loss aversion, confirmation bias, etc play their part everywhere causing senseless projects to be pushed to completion.

> They are right more often than you’d expect.

When the "they" is at least another expert in the same field and not a total stranger completely ignorant of pretty much every relevant detail, and the "you'd expect" part starts at a baseline of nearly 0% for people walking in out of the street unfamiliar with the specific problem at hand, then yes, "they" are right more often than I'd "expect."

In this case, possibly years

HN is the worst for it. Especially when it comes to tech stacks.

If you use anything other than non-SPA, Go, PostgreSQL then it's over engineered and doomed to fail.

Tech is the worst. Fetishizing disruption leads to neophytes thinking they're breaking new ground when they're just re-discovering long dismissed ideas.

My answer to the question is no, but I am a pilot, and my expectation is an invalid input is recorded but is a no op. It is fly by wire so why not just no op the request? The idea such input suggest sabotage, is more paranoid than cautious. And shutting down the engines is hardly fail safe. It's just less fail danger than engaging the thrust reversers.

Quite a few turboprops have beta range that is supported for use in-flight. The purpose is to increase the rate of decent, similar to slipping the plane. So it's not always an invalid input; it's make/model/phase of flight specific.

Welcome to the internet, where everyone's an expert, the problems are real, and the points don't matter.

So the bulletin says that activating the thrust reversers too soon can cause un-commanded high trust. Activating it too soon by itself does not cause a shutdown. But one monitor that could stop the engine would be the overspeed monitor. The un-commanded high thrust might have tripped the overspeed monitor on the turbines and shut both of them down. There are numerous other monitors on the engines that might also play a role. So determining root cause might still take a while still.

This is an issue carmakers have had to deal with, too, since the invention of the automatic transmission: what if you put it in reverse while driving on the highway?

There are a number of YouTube videos with people who have tried it. The answer, in a modern car: pretty much nothing. It just stays in Drive. On one car, it turned on the backup camera display!

Standard transmissions nowadays often include mechanical features as well that make it very difficult / impossible to engage reverse (and first gear) while going forwards at more than a few km/h.

Even a 1950s era transmission will still be very hard to force into a gear if the speed difference is too great. Likewise you'll have a very difficult time shifting into reverse or 1st at speed. That's just how syncros work. You might be able to overcome it by double clutching but it would require conscious effort. Even just shifting to reverse while rolling is difficult.

Still possible though. Years ago a friend and I pulled a manual transmission in a junk yard. He got it home, installed it, and then discovered reverse was completely stripped.

A stripped reverse gear would be indicative of failure to put it in reverse.

If the connecting rods had fled the engine that would indicate that someone succeeded in getting it into a gear it didn't want to be in.

I thought that most modern aircrafts were not allowing the reverse to be deployed if the wheels didn't touch the runway.

Talking about an odd safety override. It's kind of a self-destruct button.

It's not an odd safety override. An overthrust situation is always dangerous for various reasons. One of the more spectacular is the possibility of liberating airfoils from the engine and shooting them into the cabin.

It's not a self-destruct button, it's anti-destruct limit.

As for the thrust reverser, there are many integrated systems on the aircraft. It's possible that the cockpit detected weight-on-wheels, but the flight mode hadn't yet transitioned for the engine controls.

By self destruct, I meant switching off the engines in flight seems to me to be a terribly dangerous behavior.

Switching off is much safer than a turbine overspeed past certain limits. Because an engine failure (one engine) is a situation all pilots are trained for and it has an almost certain safe outcome. Airplanes fly and land just fine on a single engine. We all train for engine failures all the time.

On the other hand, a runaway engine fire or uncontained turbine failure is much much more likely to cause a crash.

So almost all jet engines are designed to have a shutdown (sometimes helped by the built-in fire extinguishers) as a worst case outcome. The quick response drill for an engine overspeed or temperature past certain limits is to shut it down immediately and pull the fire extinguisher handle.

But why not throttling instead of switching off then?

Good question, you could theoretically throttle a runaway engine by reducing fuel flow (unlike a piston engine where you can limit air intake). But in 9 out of 10 cases you've already done that by pulling back the power levers. The next step is to cut off the fuel, because the throttles didn't get it under control.

Overspeed protection takes place once everything else has failed. The engine controls have already attempted to throttle down and such. If that doesn't work then overspeed kicks in.

Adding on to this... things inside an engine happen FAST. Too much energy can spin up the turbines to the point of failure in milliseconds.

Yep, as anyone who has ruined a fan by shooting a canned air duster at it can attest. :(

I think almost every plane can glide safely for a lot longer than it takes to restart the engines. Could be dangerous at low altitudes still.

That and restarting an engine in-flight is faster than starting it on the ground because it's already spinning (by virtue of sailing through the air, like a windmill). This is, aptly, called windmill starting.

You would be surprised at just how safe and 'normal' shutting off the engines on a plane is.

Related, British Airways Flight 268, a B-747, when taking off from LAX had a problem with one of it's engines so they shut it off, and continued flying all the way to London, albeit to Manchester instead of Heathrow, with one less engine. https://en.wikipedia.org/wiki/British_Airways_Flight_268

Shutting down one engine is one thing. Shutting down all engines is what surprised me.

that's why this is news. it's very rare.

I was watching Air Crash Investigation yesterday, where a reverser deployed right after take off and caused the plane to crash. They said now reversers can't be activated unless all wheels are on the ground, as you've said.

Lauda Air flight 004 crashed because of that.

I'm willing to bet that the plane does prevent the pilot from activating reverse thrust when not on the ground.

But what if a thrust reverser self-activated, without being commanded? The previous safety mechanism wouldn't help because it's being bypassed.

So there is a secondary safety system that detects such situations. Something along the lines of "I think the trust reversers shouldn't be activated right now, yet they appear to be activated. The engine is clearly malfunctioning, lets shut it down"

It appears that this secondary safety system has been activated. There was probably a bug, or a sensor malfunction that triggered it.

Thrust reversers can never be deployed while airborne. This engine shutdown system is only enable on the ground. It's designed to shutdown an engine if the engine thrust doesn't match the commanded thrust, so an engine malfunction on the ground cannot cause the plane to taxi out of control.

But it is still possible for the pilots to deploy reverse thrust too soon, after touchdown but before there there is enough weight on the wheels to provide sufficient steering. I'm guessing the pilots deployed too soon, and discovered a new corner case.

Definitely you can use thrust reversers before touchdown at least on B737. I wonder how big is a difference between software on B737 and B787. Aviation industry doesn't have tendency to build software from scratch. Here is example video: https://youtu.be/-RO66a_nvus

Hey, thanks for this video. I was on a flight that did that some years ago when I was doing a lot of traveling. I don't remember anymore where exactly in Europe it was, but no one I later talked to believed me that the pilot would engage thrust reverser before touchdown.

Maybe the 787 won't allow airborne use, but other aircraft allow it. It's useful for steep descent. If the feature were more common, obstructions near airports (like mountains) would be less of an issue. We could build new runways at some airports.

https://en.wikipedia.org/wiki/Thrust_reversal#In-flight_oper...

Is that true even if the turbines are still spinning very fast?

A thrust reverser isn't like the reverse gear on your car.

It's a vent that opens to blow exhaust gases to the front instead of behind.

I think you've misunderstood. The system you're describing kicks in when (a) the aircraft is on the ground, (b) commanded thrust is idle and (c) the engines continue to produce thrust above idle (when there is an overthrust condition). Boeing does not cut power to the engines to prevent reverse thrust application in air - that would be absurd.

On this particular occasion the system is thought to have malfunctioned in some way (e.g. it might have not registered the application of reverse thrust) triggering a shutdown.

I thought that they did as a mitigation for uncommanded thruster reverser application.

A tough call. Your solution would disappoint a pilot who actually wanted to slow down ASAP more than Boeing's solution, and might contribute to a runway overrun.

No, it would not. If the engine is shut down moments before touching down it would mean that there would be no reverse thrust after touching down since it takes a considerable amount of time to re-start a turbo-fan (not 60 seconds as OP wrote as the engine is still spooling but still a significant amount of time and I don't think that any pilot would go through the workload and checklists of re-starting an engine during the landing roll anyway). This leads to a longer braking distance.

The sane choice would be to not engage reverse thrust at all (until the pilot has reset the reverse thrust throttle) or to only engage reverse thrust once the landing gear has weight on it.

> or to only engage reverse thrust once the landing gear has weight on it.

Having reverse thrust depend on a sensor that could fail seems like a poor choice. What if the landing gear don't drop? Is there a situation where you would still want reverse thrust without landing gear?

Sometimes you'd like to have reverse thrust in-flight if you want to descend really, really quickly. Airliners don't have that feature anymore, but the C-17 and C-5 do.

Actually, maybe some ex-Soviet airliners can still do it. DC-8s being used by cargo airlines could possibly still technically do it, but don't use the ability.

Specifically, it wasn't a design goal of reverse thrust, and it's operation mid-flight was directly responsible for over 200 innocent lives lost[0]. Thrust reversers are very much a "nice to have" feature on turbofan aircraft, with every one qualified to both land and initiate a rejected takeoff at maximum weight without using them, though not without some maintenance after.

[0]: https://en.wikipedia.org/wiki/Lauda_Air_Flight_004

Most things in aircraft automation systems depend on sensors that could fail (those sensors could be redundant etc. but the same can be done with the sensors on landing gears). It is sane design.

> What if the landing gear don't drop?

I don't think that there is need for reverse thrust in such a scenario.

> Is there a situation where you would still want reverse thrust without landing gear?

Even if there is, there could be a manual override.

> I don't think that there is need for reverse thrust in such a scenario.

How do we slow down planes that still need to come down despite a stuck landing gear, then?

Indeed, a manual override must exist.

How would that work exactly? Surely the engines will be among the first things to be destroyed on 'landing' if the gear aren't down.

It depends on where the engines are mounted. There was a Tu-154 that landed gear-up in Greece, then took off again, dropped the landing gear, and landed normally.

https://en.wikipedia.org/wiki/Mal%C3%A9v_Flight_262

There are plenty of examples of non-catastrophic belly landings. I'm no pilot or aircraft engineer, so I don't know for sure if you'd want reverse thrust in such a situation. https://en.wikipedia.org/wiki/Belly_landing

Just have multiple sensors like they do already.

It's also a sensor that you can simply put on each wheel to determine if there is weight applied.

> or to only engage reverse thrust once the landing gear has weight on it.

Or if the manual override is turned on.

hundreds of people are going to die.

Only if the people in this thread are designing aircraft. Presumably the actual aircraft designers have more experience reasoning through these kinds of problems.

It's weird to call it a "danger" when the automation is making a suboptimal response to override a human error that could crash the plane.

Reply on news.ycombinator.com