Hacker News Reader

The purpose of telling pilots thrust was to save money and fuel. It’s a “least viable” optimization.

>“The goal is to lower the power used on takeoff,” he said. “That reduces engine wear and saves money” on fuel and maintenance.

>Flights to Hawaii are typically full, with lots of baggage and a full load of fuel for the trip across the ocean. The planes are heavy.

> That morning, a software bug in an update to the DynamicSource tool caused it to provide seriously undervalued weights for the airplanes.

>Peyton added that even though the update to the DynamicSource software had been tested over an extended period, the bug was missed because it only presented when many aircraft at the same time were using the system.

> Both planes headed down the runway with less power and at lower speed than they should have. And with the jets judged lighter than they actually were, the pilots rotated too early.

>The data “confirms that the airplane was safely airborne with runway remaining and at an altitude by the end of the runway that was well within regulatory safety margins. Both aircraft got airborne well within safety limits despite the lower thrust.

It’s also probably lucky that two Hawai’i bound flights both didn’t notice the discrepancy, or two bumps wouldn’t have bee as suspicious. Interesting that someone cited divorce and sleep examples as emotional pleas, reassuring us they are only human.

Like all complex failures at least 5 things had to happen. Software update that morning, some kind of hard to simulate resource issue in processing, overweight flights, pilot not pricing plane data appeared light, pilot turning early despite low speed, two in a row.

> The purpose of telling pilots thrust was to save money and fuel.

No, it's more that using lower thrust during take off saves on engine wear and noise levels around the airport. It doesn't really have that big an impact on fuel use, at least that's not the primary purpose.

The article concentrates far too much on the thrust setting. The important bit is the speed where the plane should be rotated to take off, which is known as V_r. That depends on the weight of the plane. On the takeoff roll, the pilots watch the airspeed indicators and pull back to lift off when that speed has been reached. If they have been told a V_r that is lower than it should be, then pulling back will rotate the aircraft without it taking off, and that's when you have this danger of the tail hitting the ground. When that happens, the plane must be inspected, because a tail strike has the potential to weaken the pressurised container that is the fuselage, and this could lead to an explosive burst and depressurisation when flying at high altitude. See https://en.wikipedia.org/wiki/China_Airlines_Flight_611 for why this is a Bad Thing.

> saves on engine wear

Which is saving on money

I'm cracking up at the thought of an airline telling their pilots "fuck it, run em however hard you think you need to."

There's a difference between dumping toxic waste into a river to save money and keeping a plane engine in safe operating conditions to save money.

It seems like a slower rotation might be useful, to wait until you feel it leave the ground before aiming higher. I'm sure there are reasons for the procedure as is.

Trying to "force the airplane off the ground" by rotating prior to Vr has a few safety downsides. One of them is that it makes tailstrikes more likely, as you'll have to carefully moderate the angle before lift occurs. Another is that it removes loading from the wheels more quickly, which extends the "awkward zone" in which the airplane is still rolling on the ground but has so little weight on the wheels that it will skid very easily. This can lead to aircraft being blown sideways off the runway since you can't yet roll into the wind.

A somewhat less direct concern but a related one is ground effect lift. the aircraft can remain a short distance off the ground (roughly a wingspan of altitude) at lower speeds than it can actually "fly," due to ground effect. Smaller aircraft might routinely spend some time in ground effect gaining additional speed before they begin climbing, but airliners have so much thrust they usually rotate pretty directly to their climb speed. This makes it more of an issue though that if rotation occurs too far before Vy climb speed the ground effect period will be prolonged and increase the amount of time the aircraft spends at risk, flying but slow with poor control authority and too close to the ground to have much of a recovery opportunity. The ground is a pretty safe place to be, the sky is a pretty safe place to be, but that first couple thousand feet between the ground and the sky is rather hazardous and jetliners get out of it as quickly as possible.

>> The ground is a pretty safe place to be, the sky is a pretty safe place to be, but that first couple thousand feet between the ground and the sky is rather hazardous and jetliners get out of it as quickly as possible.

Then the problem here is trading safety for something - cost apparently.

BTW, in a small plane I found soft field takeoffs quite fun. Nose up early, wheels up early, fly in ground effect to gain speed. I suppose that is a safety tradeoff and might not want to do it in some conditions.

Rotating at a slower speed is a bad idea. Indicating a lower weight, resulting in a lower calculated rotating speed, is the root cause of the problem. There is no "slower rotation", one does not just pull the yoke slowly, raise the nose wheel and wait for the plane to take off by itself when it will be fast enough, that you can do on a light prop plane where rotate speed is so low some planes will take off by themselves, but not on an airliner where you have a high chance that raising the nose too early results in speed not increasing or even decreasing.

I interpreted "slower" as "later," ie wait longer before rotating.

That means rotating at a higher speed than calculated Vr, this is also bad as you put more wear in the tires and use more fuel (rolling on the tires vs flying in the air) to take off. The Vr is an optimal speed, below is dangerous, above is wasteful.

I think the more important factor is that it eats runway and takes longer to reach safe speeds. If there is a problem at rotation you want as much runway available as possible. If you delay rotation not only are you further down the runway before "really rotating", but you're traveling faster. A 10-knot increase in speed means 20% more energy to disipate. And you're even further above the brake's certified performace. And you've dawdled in getting to safe climb speeds; you accelerate more slowly rolling than flying.

Edit: Actually, that probably doesn't even come into it. Not a pilot, but don't imagine "barely rotate so the nosewheel is in the air but still have negative angle of attack so no lift" is very easy, or very stable if possible. Even if you could do it, you'd risk slamming the nosewheel back into the runway.

V_r is almost always above V_1, which is the decision speed. If you're going to abort a takeoff, that decision needs to be made before V_1, otherwise you're likely to overrun the end of the runway.

So, once you get to the stage where you're ready to rotate, or even late in rotating, you're committed to getting the plane off the ground.

On a big airliner yes. V1 depends on the runway length so a smaller aircraft will easily reach Vr before V1. Because most runways are made for the biggest planes.

>Interesting that someone cited divorce and sleep examples as emotional pleas, reassuring us they are only human.

Huh? FAA spends a huge amount of time and energy focusing on human factors, tasks saturation rates and crew resource management. Nothing to do with with argumentum ad passiones, just prudent risk mitigation.

Right. Literally every pilot in the US has been taught the IMSAFE mnemonic to assess fitness to fly, even before their first solo in a single-engine piston: Illness, Medication, Stress, Alcohol, Fatigue and Emotion.

I thought a lot of safety thinking was built around acknowledging that humans don't do everything perfectly all the time.

It does, and one class of mitigation is to improve automation, which airlines (and their regulators) do all the time.

But another class of mitigations is to make the human less error-prone in the first place. This is especially important in preventing failures that were caused by improper management of the automation. Hence the emphasis on rest periods, training, and other mitigations on the human side.

In the "swiss cheese" model, a disaster needs to slip through both the human's defenses and the automation's defenses to actually happen. It makes sense to improve both of those layers of defense against disaster, while acknowledging that humans will never be perfect. But at least they can be awake and knowledgable.

I just don’t see how it fit the article. Unless they are surmising the pilots poorly sanityized the data going into the computer because they were having an off day.

I mean, yes. That is literally exactly what they’re saying.

The full quote:

> …if there’s a glitch, naturally some pilot somewhere is going to miss it.

“Not everyone gets eight hours sleep the night before. Someone is going through a divorce. Someone is not so sharp that morning,” he said. “The sanity check isn’t perfect every day of the week.”

Based on my experiences reading aviation accident and safety reports, air transport pilots have a _lot_ of divorces.

This could also mean divorces played a role in accidents of course.

But it could also just be statistically true. Being in a position of power, surrounded by beautiful coworkers, lots of overnight hotel stays..

Without further investigation a conclusion can't be drawn though.

> Now, do I think telling pilots what thrust they should use to take off is wise? I dont know - I'm not a pilot, and I'd want pilots feelings on the matter before commenting further, but it feels to me like a loss of authority that would make me uncomfortable

Pilot here :) the journalist got it a bit wrong, performance data isn't primarily about how much thrust to use. It is part of the calculations, but crews are free to deviate from reduced thrust calculations and will do so.

This incident is much more about the calculated speeds to use and aircraft trim settings. That's what caused the tail strikes, trying to lift off at a speed that is too low for the weight.

Those numbers are required by every airliner taking off. And getting them from software is standard everywhere. The alternative is doing the calculations manually using paper charts, every pilot is required to know how to do that, but nobody does it that way for commercial operations.

How do you feel about the pilots getting blamed for relying on the software?

> “Still, the mishaps point to the need for more vigilance by pilots in checking automated data.”

In aviation incidents there isn't really a "blame", it's about finding all contributing factors and working together to eliminate those. The intention of investigations is never to assign blame, because that stands in the way of finding all factors and improving them together. So the conclusion "pilots should check the automated data" is a correct one. It's not the only cause, but it is one of the factors and thus it needs to be discussed.

In commercial aviation there can never be a single point of failure that could lead to catastrophic outcome. The performance calculations are such a thing, so there must be validation steps.

In many airlines for example if the performance calculations are done manually (or by an app based on manual input) they need to be done by both pilots independently and the results cross-checked. Otherwise a single typo could have a big impact.

Thank you for the thoughtful comment!

I thought I was missing something in the way the story read, I'm glad to know I was.

This is a failure in the development of the software that good process in the purchaser’s handbook saved them from.

That there were no tests for this system under high load indicates a poor engineering practice at DynamicSource. They don’t deserve any kudos for a post-hoc patch that a test in production discovered the bug for when the system is a safety critical system like this.

My completely uninformed opinion is the design is flawed if the calculations across planes can interact. Is this some single tenant cloud based system with poor multithreaded safety?

Like why would the increased system load change the results. The calculations should be independent and designed in a way where they can't interact.

They may have thought their test coverage, covered it adequately - who knows, the success was on Alaska's part.

Also process, procedure and more importantly people are always the last line of defense against failure.

Making excuses for the failures in the first line of defense is the first domino in catastrophic failure. Normalization of deviance.

I don't think OP was making excuses. It is true that the software vendor's missing test case indicates a poor quality development environment and that humans are the last line of defense. No development process is perfectly resilient against errors, even good quality processes.

What do you propose they should have done? Keep testing until they were certain they caught every bug? :)

The bug is in the weight of the payload of the plane. I don't believe pilots have any way to measure the actual weight of the payload of the plane, they rely on that sort of estimates to determine how much fuel they need, take-off speed, etc. It's not really a loss of authority.

Bad weight estimates have resulted in crashes in the past.

I wonder if it would be possible to put a weighing pad at airports for a double-check. Like on the access ramp to the runway, meaning you'd only need a handful per airport and planes pass them naturally.

> I wonder if it would be possible to put a weighing pad at airports for a double-check.

A few modern pickup trucks have a system for measuring how much weight is in the bed or being transferred through the tongue of a trailer because of how common it is for people to overload their trucks.

AFAIK these systems just use the same sorts of suspension height sensors that have been used for auto-levelling for years and are factory calibrated to know that if the rear end squats by X distance then there must be Y load in it.

Seems like something along those lines would be technically easy to retrofit to large aircraft, possibly even as a software update if they already have some relevant sensors.

If it makes you feel any better, the people providing these calculations to the pilots are FAA-certified airline dispatchers. That means they've passed the same exam that airline pilots have to take, minus the flying. They may not have personal experience flying the airplane but they do have to know it inside and out. And if the dispatchers weren't running the numbers, the pilots would be running them using the same software and/or tables, so there really isn't much difference.