Boeing's starliner docks at ISS after five thrusters unexpectedly shut off | Hacker News Reader

Boeing's starliner docks at ISS after five thrusters unexpectedly shut off

gizmodo.com

118 pointsheywire81 comments

I'm not certain I counted right, but I believe it was eight failed thrusters in its first uncrewed test in 2020, and later four failed thrusters in the second one, in 2022. (Combining the failure count of the larger orbital maneuvering thrusters with that of the smaller reaction-control systems).

- "The NASA source said eight or more thrusters on the service module failed at one point and that one thruster never fired at all."

https://arstechnica.com/science/2020/01/nasa-and-boeing-are-...

- " Two of the 20 main thrusters on the spacecraft's service module, used for orbital maneuvering, failed shortly after Starliner separated from its Atlas V rocket. They were not recovered during the flight. Two smaller reaction control system thrusters also failed during the approach to the space station, but they were recovered."

https://arstechnica.com/science/2022/05/nasas-verdict-on-sta...

Hurray for the software that managed to work even with the failed thrusters.

An astronaut took over manual control after automatic docking system started popping failures.

In reality a Python programmer at the mission control center modified the scripts and told the astronaut what buttons to press.

Sounds like they think it's actually a software problem rather than a hardware fault.

Their first demo flight was plagued with thruster "valve mapping errors" which could have been catastrophic (eg, it could have caused it to collide with the ISS).

>"Boeing described the new software problem as “a valve mapping software issue, which was diagnosed and fixed in flight.” According to the company, “That error in the software would have resulted in an incorrect thruster separation and disposal burn. What would have resulted from that is unclear.”"[1]

My understanding was that was found by happenstance only after they were reviewing the code after an unrelated comms error. Makes you wonder how many untested latent bugs are in the software.

[1] https://spacenews.com/nasa-safety-panel-calls-for-reviews-af...

Will be fixed in the next release. /s

V0.0.2 Patch Notes: fixed a bug that occasionally caused it to explode

> As a result, NASA astronaut Butch Wilmore had to assume manual control of the crew capsule for a while before it went back into its autonomous mode

Is there a video of this? Bet it was nail biting.

Kudos they pulled it off given all that went wrong.

He trained for hundreds of hours to fly the capsule manually and the docking process is excruciatingly slow so it's probably not that stressful. Even when everything is working they let the astronauts have a little manual control as a treat.

I don't know. I pictured it like the landing scene from airplane and I'm sticking with it.

You can try it yourself here -> https://iss-sim.spacex.com/

Protips: Watch your instruments and take it super slow!

Seems pretty easy by orientating first and then just translating into position with some patience.

Would be good if there were modes where it randomly breaks n thrusters so you are forced to get creative.

> Would be good if there were modes where it randomly breaks n thrusters so you are forced to get creative

Or rates you on your time to approach vs fuel efficiency.

Don’t doubt they were well prepared but I can’t imagine what it would be like to learn to drive virtually and then do my first test drive on the edge of the earths atmosphere

> He trained for hundreds of hours to fly the capsule manually and the docking process is excruciatingly slow

_COME ON TARS!_

Why not try for yourself: https://iss-sim.spacex.com/

You know you can always rely on a guy named "Butch Wilmore" to take control when shit hits the fan.

Butch "Space Cowboy" Wilmore

McDonnell Douglas knows they can continue to drain Boeing of money because the US govt will step in to bail Boeing out when things go really bad.

Reminds me of the cynical calculus around power companies here in California.

> the US govt will step in to bail Boeing out when things go really bad

Even an optimistic bail-out scenario involves the stock being wiped clean. (For example, what we did with the automakers.)

Given the bipartisan frustration with Boeing, I'd fully expect calls for clawbacks that make failure personally painful for Boeing's leadership.

Oh there will be political theatre on par with the WWF, no doubt. For those people it falls under the heading "Nice work if you can get it."

These are the same thrusters that are on Orion [1][2]. (They were delivered to Boeing 6 years ago.)

[1] https://www.spacefoundation.org/2018/03/15/aerojet-rocketdyn...

[2] https://www.compositesworld.com/news/aerojet-rocketdyne-to-p...

Sounds like they worked through it, so docking was successful. I imagine that much like SpaceX in a similar situation, there are lessons learned that will be applied to the next iteration.

The reports while it was happening suggested that they were almost down to no redundancies for the thrusters. Which would have been bad enough to disallow the capsule to approach the ISS. They managed to get enough thrusters working to still dock, but it seems that this was rather close.

So even though it worked in the end, I would assume that this is still a quite serious incident and will require some examination.

Also it seems that these problems are not well understood, which is not the kind of problem you want to have on manned spacecraft:

> Some notes from the post-docking news conference on Starliner: Engineers found a fourth helium leak after capture. The thruster issues were similar to those observed on OFT-2, and are still not fully understood.

https://x.com/SciGuySpace/status/1798827111454827003

Ideally you want the iterative lesson learning to wrap up before you put the humans inside.

That's why they're called test pilots. They're not putting the general public into spacecraft.

FWIW, like many previous astronauts, both crew members are US Navy test pilots. They've literally made a career out of solving complex problems and navigating ultra-sticky situations.

The function of test pilots is to validate things that can only be validated by manual flight testing, like aircraft handling characteristics—not to risk their lives to do QA.

Sometimes the tests aren't green.

sometimes in the messy real world, things go wrong even when the tests are green.

Hmm. I hear ya, but this isn’t what I want test pilots spending effort on.

Test pilots are closer to an integration test in software. You expect the unit tests to pass green before you expect the related integration tests to work.

Multiple failures of maneuvering thrusters is technically a mission failure (in the flight plan sense), despite having nothing to do with the parameters of the mission, nor anything to do with adding humans. Maneuverability is the difference between a craft and junk.

I wouldn’t find this acceptable in a project I was managing or funding. Just sayin! Still impressed as hell.

You're not funding it yet until the government either nationalizes or bails out Boeing

The government is paying Boeing, no?

Yeah, that would be ideal. But it seems uncommon. Lots of complexity in a spacecraft.

Generally you actually want 0 "lessons learned" in human spaceflight. Test flights especially and unmanned missions have a much higher tolerance for error. A bunch of thrusters failing with people on board could have been catastrophic and is well into the territory of "enormous problems which never should have happened".

The astronauts may have to be "rescued" with a different capsule returning the departing astronauts to earth.

I agree, zero would be ideal. Maybe I'm just too accustomed to the iteration we see all the time when SpaceX equipment fails. This is arguably more a traditional space program and less of a 'go fast and break things' process.

IIRC, SpaceX also had some sort of glitch when docking with the station, on a crewed mission. Related to the thrusters, too, if memory serves.

Found this https://www.space.com/spacex-dragon-international-space-stat...

Which was a problem, but it was a backup in case of a leak that broke, it had 0 effect and it was a cargo mission not a people mission. Starliner's failure today did have a significant mission effect and had people.

SpaceX had a bunch of rocket landing failures on real missions, but that didn't matter because landing was a bonus, basically nobody had done that before and everything was just fine if a rocket blew up trying to land.

SpaceX has also done a bunch of test launches with new vehicles, this also doesn't matter if they fail, they're specifically launched to find failures instead of expending tons of engineering effort which is more expensive than blowing up test rockets.

Those things are quite separate from "there's people on board and several thrusters broke requiring unexpected manual pilot intervention", this was not expecting or allowing for failure by design but a serious anomaly that has to be researched how the problems weren't caught in human certification.

SpaceX destroys a lot of hardware, but during almost quarter of a century of operations, they had AFAIK just one work fatality - a worker was blown off his truck in McGregor in 2014.

No doubt they were lucky several times, but so were others, and most space institutions have much longer fatality lists, especially when adjusted to total flight volume.

Fatalities aren't the whole story.

> The 2023 records, newly disclosed by the Occupational Safety and Health Administration, also show that injury rates at some SpaceX facilities grew worse than those the company had reported in 2022. At its manufacturing-and-launch facility in Brownsville, Texas, for instance, SpaceX reported 5.9 injuries per 100 workers, surpassing its rate of 4.8 injuries in 2022 and topping a space industry average of 0.8.

That's 7x the industry average.

Ref: https://www.usnews.com/news/top-news/articles/2024-04-22/exc...

Can you elaborate on what constitutes the “space industry” in this sense? Sometimes the OSHA groupings don’t align as well as one would think. A wet lab testing material compatibility and a rocket test stand can both be “aerospace” but have very different risk profiles.

Could it be a reporting issue?

Fatalities certainly aren't the whole story, but they do have one "advantage": they cannot be hushed away, at least not in the Western world. (In China or Russia, well...)

That is why, in an analogical situation, crime and public safety of individual countries is often compared using murder levels. It is expected that most developed countries cannot really suppress/fudge murder statistics, even though they can do a lot of shenanigans around registration of lesser crime.

I can see a similar situation with individual corporations and their diligence when it comes to reporting injuries.

Even murder statistics aren't foolproof when comparing crime rates across different countries and time periods, because of differences in medical care standards. Someone can be shot several times in the modern West and recover, but 200 years ago or in a poor country (or maybe in a more remote location) the same person would not survive and it would count as a homicide.

I imagine these numbers can get padded by including people that have no chance of injury because nothing new is being done (ie human exploration has always been risky).

It'll be super ironic if they have to come back via SpaceX capsule. :)

SpaceX has a little advantage in that their next iteration doesn’t cost a billion dollars…

>there are lessons learned that will be applied to the next iteration.

Maybe that's why there's 20 thrusters instead of 5.

Hard to believe that NASA sent people to the moon 60 years ago, when they can't even get basic reaction thrusters working after over 10 years of developing and building the Starliner...

NASA is just the customer. Starliner is built by Boeing.

Boeing used to mean quality. Now it means cutting corners for the sake of stock price and dividends.

...and the other trendy word that starts with a D.

Come on, say it for everyone.

Deaths (of passengers)?

Disassembly following premeditated lithobraking induced by lackluster system architecture?

Dead whistleblowers?

Figured he was going to say 'diversity' as if that has something to do with any of boeings issues, but your D words are on the nose!

There is a reason why it says Boeing's Starliner and not NASA's

You should read more about failures than happened during Apollo program. Apollo 1, Apollo 13 being most famous, but even during Apollo 11 multiple things failed.

I am aware, but they went from Apollo 1 to Apollo 11 on the moon in like 2 years. This is now after what, 10+ years? With billions of cash and 60+ years of additional know-how and technology?

On the contrary, there's a knowledge about spaceflight that was lost because the funding dropped so low in between, and the older generation retired without having a steady flow of projects to train the new generation(s) with.

Also, with this round of space enthusiasm, everyone is trying to improve on what was done before, to make the cost of ongoing operations finally more sustainable. For example, EVA suits are built more maneuverable and agile, while at the same being more modular, repairable, and no longer custom fitted to each individual.

I think given Boeing’s focus on profit over quality and safety lately, this really shouldn’t come as a surprise.

They should be focused on profit, because they haven't made a profit in almost 5 years now and Boeing is a very important company to the US. If Boeing ends up going bankrupt, the US government will 100% have to bail it out.

The reason that they're losing money is that nobody trusts their planes - because they're badly designed and prone to falling out of the sky.

Only their new McDonell Douglas planes. The old Boeing planes are still fine.

Question is, if the space division is the former or the latter.

Sure, but it's kind of a moot point when the issues have such an impact on consumer confidence.

That's inevitable. Management knows they're too big to fail.

Will these leak and thrusters issues affect the return trip?

Absolutely. Even if zero additional problems crop up while they are at the ISS, the Controllers will need to have fall-back-back-back-and-further-back plans on tap, in case of yet more failures in the Service Modules' propulsion systems.

Not surprisingly, they already have these backup plans. The potential for RCS jet failure was known ahead of time and was determined to not be a flight safety concern.

There’s a bunch of info about it here: https://spaceflightnow.com/2024/05/24/nasa-says-boeings-star...

It sounds like they only recovered 4/5 thrusters.

To be clear, I believe there are a total of 28 RCS thrusters, 5 of which had issues

KSP mojo.

Moah struts! Uhh, I mean thrusters.

> Starliner’s docking has not gone smoothly, the spacecraft developed trouble with its reaction control system thrusters

I wonder how this affects the certification.

I bet the actuaries responsible for the astronauts’ life insurance just bumped the premiums up a few tiers.

"It's not possible".

"No, it's necessary".

“If it’s Boeing I ain’t going”

Came here to post this!

What a joke: An accurate and all encompassing critique of the vehicle and the company who made it.

"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."

https://news.ycombinator.com/newsguidelines.html

Has SpaceX had any thruster problems when docking with the space station?

They had an issue with a leaky valve, in a backup system, on a cargo mission.

https://arstechnica.com/space/2023/08/valves-are-a-regular-c...

Crew Dragon C204 docked with the ISS and returned without problem without people. It was suppose to do so again with people, but instead it blew itself to smithereens on the ground due to a thruster problem.

Which is to say "not exactly (at least that I remember), but problems in a similar vein on hardware that did similar things".

Reply on news.ycombinator.com