Hacker News Reader

Comment by Mikhail_K | Hacker News Reader

> Challenger: no gasses were supposed to make it past the o-rings no matter what,

> but when it became clear that gasses were escaping and the o-rings were being

> damaged, there was a push to suggest that it's an acceptable level.

Interestingly, the article<https://docs.google.com/document/d/1ddi792xdfNXcBwF8qpDUxmZz...> by heat shield expert and Shuttle astronaut Charles Camarda, the former Director of Engineering at Johnson Space Center, asserts that it was *not* the O-rings:

"The Challenger accident was not caused by O-rings or temperature on the day of launch; it was caused by a deviant joint design which opened instead of closed when loaded. It was caused by mistaking analytical adequacy of a simplified test for physical understanding of the system. The solution, post Challenger, was the structural redesign of the SRB field joint and the use of the exact same O-rings."

I find that highly surprising, because "it was the O-rings" explanation seems universally believed and sanctified by no lesser authority than the Nobel prize laureate Richard Feynman.

View on HN

It's the same explaination. When the SRB joints flexxed the o-rings were meant to stay in place, but the joints were defective and NASA knew the o-rings were moving. However NASA also believed the o-rings could still take the abuse, because although they were moving they were getting shoved deeper into the joint, in a way that wasn't intended but was nonetheless at least marginally effective at stopping exhaust blow-by shortly after it began. But when the o-rings were cold and stiff... they didn't move the same way, exhaust blew by them longer and cut right through. At that point the SRB turns into a cutting torch (the SRBs didn't actually explode until after the shuttle broke up and range safety sent the signal to kill the boosters.

> However NASA also believed the o-rings could still take the abuse, because

> although they were moving they were getting shoved deeper into the joint,

Why would they be "shoved deeper," when the problem is that the joint opens wider under load?

See here: https://www.researchgate.net/profile/Scott-Post/publication/...

What would happen "normally" (i.e. the normalization of deviance) was that the rotation (from the SRB joints bowing--essentially "ballooning") would create a gap, and the O-rings would get blown into that gap and ultimately seal in there

With Challenger, it was too cold, so the O-ring rubber was not malleable enough to seal into that space (like the O-ring towards the right of the diagram), so the hot gases were allowed to blow by and erode the O-ring. If they had sealed in (like the one on the left) it would have just taken the pressure but not worn away

> What would happen "normally" (i.e. the normalization of deviance) was that the rotation (from the SRB joints bowing--essentially "ballooning") would create a gap, and the O-rings would get blown into that gap and ultimately seal in there

But data from previous Shuttle flights showed that even that wasn't happening, at temperatures up to 75 F. And the Thiokol engineers had test stand data showing that it wasn't happening even at temperatures up to 100 F. In short, that joint design was unacceptably risky at any temperature.

It is probably true that the design was somewhat more unacceptably risky at 29 F. But that was a relatively minor point. The reason the cold temperature was focused on by the Thiokol engineers (who were overruled by their own managers in the end, as well as NASA managers) in the call the night before the launch was not that they had a good case for increased risk at cold temperature; it was that the cold temperature argument was the only thing they had to fight with--because NASA had already refused to listen to their much better arguments the previous summer for stopping all Shuttle flights until the joint design could be fixed.

> It is probably true that the design was somewhat more unacceptably risky at 29 F. But that was a relatively minor point

This was a critical part of the danger:

> Temperature Effects

> The record of the ... meetings ... on January 27th, the night before the launch of flight 51-L, shows ... limited consideration was given to the past history of O-ring damage in terms of temperature. The managers compared as a function of temperature the flights for which thermal distress of O-rings had been observed--not the frequency of occurrence based on all flights (Figure 6).

> In such a comparison, there is nothing irregular in the distribution of O-ring "distress" ... between 53 degrees Fahrenheit and 75 degrees Fahrenheit. When the entire history of flight experience is considered, including "normal" flights with no erosion or blow-by, the comparison is substantially different (Figure 7).

> This comparison of flight history indicates that only three incidents of O-ring thermal distress occurred out of twenty flights with O-ring temperatures at 66 degrees Fahrenheit or above, whereas, all four flights with O-ring temperatures at 63 degrees Fahrenheit or below experienced O-ring thermal distress.

> Consideration of the entire launch temperature history indicates that the probability of O-ring distress is increased to almost a certainty if the temperature of the joint is less than 65.

https://www.nasa.gov/history/rogersrep/v1ch6.htm#:~:text=Thi...

---

In fact it is also a case study in data visualization: https://williamwolff.org/wp-content/uploads/2013/01/tufte-ch... (more briefly: https://www.residentmar.io/2016/02/07/space-shuttle-challeng...)

For completeness: the engineers' rebuttal: https://people.rit.edu/wlrgsh/FINRobison.pdf but I don't think the back-and-forth takes away from the larger point that there are more- and less-effective ways to visually convey data

Two directions.

Let's examine a slice of the booster. Going vertically you have one segment, then the joint, then the next segment. The O-rings were in that joint and had some ability to move horizontally.

As designed the joint would always be in compression, the O-rings sandwiched between two big pieces of metal. If they moved horizontally in the space they had it made no difference, their job was simply to keep the 1000psi inside the booster inside it. Going inward there was a layer of putty that could stand up to the heat but was useless for sealing.

Unfortunately, when the engines lit the whole booster stack twanged a few inches. A joint meant to always be in compression was suddenly for a moment in tension--the two pieces of metal moved slightly apart--gas could now go above/below the ring. If the rings were pliable enough they got slammed against the outside of their groove where the pressure against the joint stopped the escape of gas--examination of the boosters showed blow-by but it cut off soon enough that the mass of metal was enough to absorb enough heat to avoid catastrophe.

But that night was very cold. And it was very calm--the boil-off from the LOX tank was simply dumped overboard and the booster that failed was downwind. The point of maximum chilling was between the booster and the tank, the lowest segment joint got the worst of it. And that's where it failed.

When the stack twanged the ring didn't slam against the outside quite fast enough--some exhaust leaked past and tore up the ring. But the gas still had to go out the joint--and the shuttle fuel used aluminum. The ring wasn't sealing the joint but enough aluminum solidified out against the still-cold metal of the joint that it sealed the gap and Challenger roared into the sky. But as it went faster and faster the vibrations grew stronger--and eventually the really sloppy weld let go. Even that didn't doom the mission, there was enough fuel to tolerate the pressure loss. But the leak was pointing at a strut and the tank with a whole bunch of LH2 in it. Neither was designed to stand up to that.

There was also a second failure that got little attention: the putty. As intended, it should have covered the entire gap, the force would have been evenly applied and it probably would have made it. But the putty was spread and the segments placed together--in atmosphere. Air was trapped and compressed--and the putty gave way letting it out. What had been an even layer now had holes in wherever the weakest spots were--and that concentrated the escaping gas from the booster. And why wasn't that caught? Because in the static testing someone had gone inside and made sure the putty job was good. Easy enough in a booster laying on it's side, but the Shuttle was stacked vertically.

>>I find that highly surprising, because "it was the O-rings" explanation seems universally believed and sanctified by no lesser authority than the Nobel prize laureate Richard Feynman.

Essentially you are mischaracterizing what Feynman did or say, although this is also Feynman fault :-), by doing the famous public demonstration, with the ice water in a glass [2], although even there he only said it has "significance to the problem...". In other words, we should not simplify, even for the general public, what are complex subtle engineering issues. This is also the reason why current AI, will fail spectacularly, but I digress...

Feynman documented the joint rotation problem in his written Appendix F, but his televised demonstration became the explanation...[3]

Camarda is correct here. There was a fundamentally flawed field joint design, meaning the tang-and-clevis joint opened under combustion pressure instead of closing. This meant the O-rings were being asked to chase a widening gap something the O-ring manufacturer explicitly told Thiokol O-rings were never designed to do. Joint rotation was known as early as 1977, a full nine years before the disaster.

The cold temperature made things worse by stiffening the rubber so it could not chase the gap as quickly, but O-ring erosion and blow-by were occurring on flights in warm weather too and nearly every flight in 1985 showed damage.

The proof is how they fixed. NASA redesigned the joint metal structure with a capture feature to prevent rotation, added a third O-ring for redundancy, and installed heaters but kept the exact same Viton rubber. If the O-rings were the real problem, you would change the O-rings. They did not need to.

The report [1] is public for everybody to read...but not from the NASA page... who funnily enough has a block on the link from their own page, so I had to find an alternative link...

[1] - https://www.govinfo.gov/content/pkg/GPO-CRPT-99hrpt1016/pdf/...

[2] - https://youtu.be/6TInWPDJhjU

[3] - https://calteches.library.caltech.edu/3570/1/Feynman.pdf

That's valuable, detailed explanation, thanks.

Yeah--people don't get it that while it was the failure of the O-rings that doomed that flight that they failed because they were subjected to forces they were never designed to take. The fact that they got that many flights before it blew actually says they were doing an admirable job of covering up the design flaw.

Without being too familiar with the subject - another commenter referred to the "swiss cheese model": the O-ring design, the temperature etc. weren't the single cause, they were contributing factors, and the more contributing factors you eliminate, the more certain you can be that you won't have a repeat accident. AFAIK there weren't any more Shuttle launches at such low temperatures after that anymore either?

That's right, the accident launch was by far the coldest. They also added joint heaters.

My recollection is that a rocket design was scaled up from one that worked, by people who didn't consider how an o-ring should be loaded in order to function properly. They inadvertently changed the design rather than simply scale it. I don't think Feynman got this wrong either. His demo was because the justifications for flight were based on the fact that failure had a temperature correlation, and they had a model representing how damaged the o-rings would be.

The o-ring failure was a measurable consequence of the joint design failure. The data behind the model didn't go down to temperatures as low as that at Challenger's launch date.

For more inappropriate extrapolation to justify a decision: the data for the heat shield tile loss model was based on much less damage than sustained by Columbia (3 orders of magnitude IIRC).

Now they are looking at the same style of fallacy and don't even have a model based on damage sustained in flights.

Another parallel I haven't seen discussed here yet, though I haven't read all comments: I recall Feynman feeling like he was on the investigation panel as a prop, that the intention of the investigation was to clear NASA of any wrongdoing. They used a model, considered risks, etc. Feynman recognized the need for a clear and powerful visual to cut through an information dump and pull it to front page news. The invitation of Camarda to a presentation with a pre-determined conclusion has the same feeling. I don't know what Camarda can do to put it on a (non-HN) front page today.

> "it was the O-rings" explanation seems universally believed and sanctified by no lesser authority than the Nobel prize laureate Richard Feynman.

If you read Feynman's account in the book What Do You Care What Other People Think?, you'll see that he realized afterwards that he was prompted to make the demonstration he made at a NASA press conference--putting a piece of O-ring material in a glass of ice water, clamped with a C-clamp, and then taking it out and releasing the clamp to show that the material did not spring back--to get public attention focused on problems with the joint in a way that could not be ignored. But, as has been pointed out downthread, when the joint was redesigned, the new design did not change the O-rings at all. So the specific issue that was shown in Feynman's demonstration was not the issue that actually needed to be fixed. It was just a convenient way to show the public that there were problems with the joint, with a simple demonstration that everyone could understand. Trying to show the actual problem--that the entire joint design was fundamentally flawed and needed to be changed--would not have worked in a context like that.

Using the same o-rings afterwards is surprising, I've heard that the manufacturer was surprised that they were being used for that purpose because they weren't rated for that.

Also I'm not sure the assertion is correct. If the sealant and O-Rings were adequate, the joint would not have failed. It was suboptimal, and increased risk, sure, but it in itself wasn't the reason for the accident. It was the joint and the o-rings in combination. The holes in the swiss cheese model lined up that day, and a lot of small problems combined into one big problem

> If the sealant and O-Rings were adequate, the joint would not have failed.

That assertion requires some reasoning and evidence to back it.

The sealant and O-rings were meant to keep the hot gasses inside. Simply making a joint slightly wiggly will not keep hot gasses inside. The hot gasses did not stay inside. The sealant and O-rings did not succeed in keeping the hot gasses inside (evidence: Challenger). They were not adequate

> The sealant and O-rings did not succeed in keeping the hot gasses inside (evidence: Challenger). They were not adequate

No. The whole assembly --joint, sealant and O-rings, -- failed.

"They were not adequate" - yet, after the redesign, they kept those same O-rings and declared that boosters are safe to fly, in manifest contradiction to your assertion. So your reasoning is clearly flawed.

>"They were not adequate" - yet, after the redesign, they kept those same O-rings

presumably "redesign" means some stuff changed. why is it not possible that the O-rings were inadequate for the old design, but adequate for the new design?

Exactly. They re-designed the tang and clevis joint so that the metal parts of the joint did not spread under gas pressure and the o-ring did not lose compression. They added a heater to ensure that the o-ring remained in it's usable temperature range. And added a superfluous third O-ring.

Speaking of which, has anyone ever adequately explained why Challenger's Right SRB joint temperature was measured as -13 deg C using infrared pyrometers, when the lowest ambient temperature that night was -5.5C, and the Left SRB was measured -4 C? What subcooled the right SRB?

Allan McDonald's "Truth, Lies, and O-Rings" is mandatory reading for anyone who wants to discuss the details of this particular bit of corporate and government malfeasance. It's 600 pages of technical detail and political intrigue. He suggests that a plume from a cryo vent could have impinged on the field joint and cooled the o-ring to lower than ambient temperatures. No proof though.

>why is it not possible that the O-rings were inadequate for the old design, but adequate for the new design?

Boneheads getting lucky, happens to the worst of them more often than lots of people want to admit :\

I came from Florida and am not a fan of cold weather.

That morning of course nobody knew about defective engineering at NASA contractors when it comes to o-rings. I got in to work, and the office people had turned on the seldom-used little black & white TV in the office manager's room so they could watch the Challenger launch. That was about the only time anybody watched TV at work, except for baseball playoffs when they occasionally occur in the afternoon.

It was 19 Fahrenheit at the launch site so I never thought for a minute that they would go through with it. It was simple common sense. You don't even try anything "normal" during the one day per decade when it gets that cold, and that would be in north Florida. You wait years for it to get below freezing at 32 F, especially on the central Florida Atlantic coast. And no matter what, you never have to wait long for it to get above freezing. I just naturally couldn't imagine anyone not fully on board with living to wear shorts another day. I was thinking about the rubber seals that must be there to keep the crew hatches airtight, for one thing, but aware there were countless other variables which I didn't have a clue about that could also be cold sensitive, like electronics.

I went into the back where my lab office was, thinking they were surely going to delay the launch, at least to later in the day. I didn't get back to the front office until a little after liftoff time, where I expected to find out how much of a delay or reschedule there was. It was very quiet. I asked what happened and they said "it blew up!" I actually thought they were kidding me because I missed the liftoff. Then I saw the tragic replay that was enough to make anybody sick.

Eventually, the o-rings were pointed to, and publicly disclosed and it was stupidly worse than I imagined.

A few years earlier I had experienced a dramatic o-ring blowout on some high-pressure apparatus that one of our engineers had designed at a previous employer. That was an engineering lab, and I'm no engineer but it turned out they needed more help than just chemistry lessons for experiment design. Since I was the one who had taken a reading within the blast zone minutes before I went back to my desk, I took over the redesign of the heavy-walled high-pressure custom cylinders, going over every little thing from alloy properties, dimensional characteristics, reinforced thread strength, etc. It was helpful that I had worked in a machine shop before, but I was the only one there who had any full time experience at metal fabrication. Well constant overtime really. When I got to the critical o-ring design parameters, that alone required more engineering effort than the rest of the project. Each standard o-ring has its own precision design parameters, highly dependent on the durometer hardness of the rubber among many other things.

Without considering durometer, here's a very simplified chart of some key parameters (primarily US inch units):

https://d2t1xqejof9utc.cloudfront.net/pictures/files/186532/...

There's way more data than this and most of it was gathered over decades of serious destructive testing & analysis.

And here's a pretty good article about the Challenger fiasco:

https://clearthinking.co/the-teleconference-before-the-chall...

Plus a color diagram that may be a little clearer:

https://onlineethics.org/sites/onlineethics/files/Challenger...

Never did look into the Challenger o-rings this much until now, all I knew was that defective o-ring design is more likely than not, and you would be a fool to use any o-ring that was not standard size without the equivalent of decades of destructive testing yourself.

All I needed to know was these o-rings circled the entire booster, so that alone was a no-no since it was nowhere near standard. Now in the clearthinking article I see the nominal measurements, 38 feet in circumference but only 1/4 inch thick. Yikes, what were they thinking? No wonder they used two o-rings, it was plain to see that one would never be enough :\

Look back at the d2t1xqejof9utc.cloudfront chart. Notice that a 1/4 inch thick o-ring is not expected to have nominal reliability outside the tolerances listed.

Notice the Groove depth and the gland depth are two different things but actually need to be as close as you can get in practice, within 3 thousandths of an inch altogether across the entire (38 foot!) diameter, or half of that when measured at any one point on the arc. This requires some precision machining and quite rigid metal substrates or it will never come true. This is precise enough that large temperature swings would always be a factor, but more so the greater the diameter of the substrate. And the maximum eccentricity of the groove relative to its substrate must be within 0.005 inch. The widest tolerance on this little chart is the "squeeze" of the rubber to be between 0.040 and 0.055 which is not for the machine shop but depends on the o-ring thickness being within its own design specifications. Not surprised to find out they were Viton rubber which is widely known to be some of the most chemically resistant for a non-teflon compound. Probably would have been better if Thiokol also was aware how "good" Viton is for its intended purpose, strong resilience at temperatures 200 F and above, below which it doesn't seal as well as ordinary rubber. Viton is just too hard and non-tacky at room temperature by comparison.

After all these decades, now I'm even more convinced it was always an accident waiting to happen :(

>> Using the same o-rings afterwards is surprising, I've heard that the manufacturer was surprised that they were being used for that purpose because they weren't rated for that.

Not surprising if you understand what the real cause was: https://news.ycombinator.com/item?id=47585889

Surprised? One of the engineers was literally on the phone with NASA the morning of the disaster begging them not to launch. He was overruled by management.

Surprising for the management. If you are a spoiled brat who always got what it wanted if you just asked/cried you don't expect reality to come and hit you.

Actually, not surprising.

The engineering was clear: don't fly. But given political realities had they said that they probably would have lost the contract to build the rockets--and that was a big part of their business.

They made the human choice: chose the option with a chance of success vs the option that was a certain failure.

They had already failed earlier and are just too selfish to accept it. This is not human, its a disgusting subset of the species.

The failure was pretty much forced.

This was a cost plus contract, if NASA didn't want to dig into the blowby issue they weren't really in a position to do so.

Both things can be true. A better O-ring with the same joint might have prevented the disaster. A better designed joint with the same O-ring might also. Feynman knew that a little theater would go a long way. The O-ring explanation, albeit a partial explanation, made for good theater.

There’s a pretty good explanation here: https://exrocketman.blogspot.com/2021/12/the-seal-failure-in...

Reply on news.ycombinator.com