On a side note, the whole first paragraph is just plain insulting to anyone who is making a life or death decision. If that is the attitude of anyone in the meeting, then they shouldn't have that job.
On a side note, the whole first paragraph is just plain insulting to anyone who is making a life or death decision. If that is the attitude of anyone in the meeting, then they shouldn't have that job.
You want to blame her for it and shake her out of it as you hear her say it, but it wasn’t malicious. I believe the mission manager even had a spouse who was an astronaut, so it’s obviously not like they don’t care. I’ve always found it fascinating how organizational structure and pressure can take really brilliant and motivated people and beat them into making such poor decisions.
It’s really important to remember that these people are not idiots. It’s literally a room of rocket scientists and space shuttle mission managers. It’s so easy for us all to say here that it was so easy to see it happening and that anyone who didn’t ask the right question was a moron, but these structures take on a life of their own. If I had it to start all over again, studying that would make a fascinating and potentially rewarding career.
You begin to develop a false sense of security after nobody really does from a gunshot wound. Then, someone septic comes in, and "seems" fine, and they're dead in a few hours.
I'm not sure what this kind of logical fallacy is, but I suspect it's similar in a government environment, where you're constantly at RED ALERT. The risk of danger just seems overstated, even when it isn't.
The way I fight it is by explicitly reminding myself that just because something worked yesterday, that doesn't mean I can skip a step today or let my guard down at any point.
It really does take deliberate thought though. Funny how the brain works.
(The upside is that every successful takeoff becomes a delightful surprise!)
- get distracted while tying the rope to your harness and leave the knot unfinished, fall 20 meters from the top of the climb (Lynn Hill, by sheer luck only broke her foot and elbow)
- use a slightly unusual rope setup and when preparing to be lowered, tie in on the wrong side of the anchor, fall 14 meters onto rock (Rannveig Aamodt, broke her spine, pelvis and ankles)
- have your partner point out damage on your harness, shrug it off because there's plenty of safety margin, continue climbing for 3 days in a manner that puts repetitive abrasion on exactly that part of the harness, have it snap while rapelling and fall 150 meters to your death (Tood Skinner)
A little example from my life is forgetting to secure the buckle of my motorcycle helmet. Once I've done everything else and gotten my gloves on, it's a pain to take them off again to buckle it up if I forget, and a few times I just shrugged it off and rode anyway.
Then an instructor suggested he always follows exactly the same procedure each time, to the point that he always even puts the same glove on first. It made me wonder if I should do that, and in thinking it through I realised if I changed my order to always buckle up before putting my glasses back on, I'd never forget. There's no way I'd ride off without the glasses (because I can't see without them!), so if that step can only come _after_ buckling up, then there's now no way I'd ever not buckle it up either.
What these disasters typically reveal is that the factors accounting for them usually had “long incubation periods, typified by rule violations, discrepant events that accumulated unnoticed, and cultural beliefs about hazards that together prevented interventions that might have staved off harmful outcomes”. Furthermore, it is especially striking how multiple rule violations and lapses can coalesce so as to enable a disaster's occurrence.
By default of your situation, you are facing these life and death decisions on a constant basis. The NASA and Boeing managers do not. I can't imagine they are part life and death scenarios very often, if at all.
Critical thinking failed them that day.
This lead to managers ignoring engineers' warnings about the foam strikes on Columbia, and also rejecting requests for high resolution images.
This is exactly the same culture which ignored engineers' warnings about the O-rings on the SRBs.
Linda Ham, the mission manager who rejected these requests left the space shuttle program after the Columbia disaster and was moved to other positions at NASA. Not firing or disciplining these managers will cause similar disasters in future. https://en.wikipedia.org/wiki/Linda_Ham#Columbia_disaster_an...
I think the quote is something like "Why would I fire them, we just lost X lives and dollars training them?"
> "Ham's attitude, and her dismissal of dissenting points of view from engineers, was identified as part of a larger cultural problem at NASA.[2] After the report's release, Ham was demoted and transferred out of her management position in the space shuttle program."
Recently I've seen a product owner trying to push us on a two week sprint where we needed to complete 100 story points worth of tasks while on the previous two sprints we only completed ~30 story points. To my protest, the SCRUM master sided with the product owner in saying we should go ahead and commit to the goal.
These are not stupid people, they were under pressure to deliver; unfortunately the incentives are not to listed to reasonable people, but to pushers; when the shit hits the fan, they make a big fuss and obtain even more resources to push even harder and they usually deliver -- whit 3X the costs.
Very rarely have I seen reasonable and smooth-sailing managers get to the top, usually it's the die-hard/hard-pushers/busy-appearing types that get to the top.
I think it's curse of modern society with its unlimited resources; this shit would never fly in the times of Sun Tzu or Caesar precisely because limited resources would prevent such smart "idiots" -- they'd succumb to the elements or their subordinates would do them in their sleep for being so detached from reality.
I've encountered a few people with this trait, and the general trend I've noticed is an excess of optimism and a hint of complacency.
"I have the utmost faith that my team can do 100 hours of work in 25!"
"We did 75 scheduled hours of work in 15. That means we can easily do 100 in 25." (completely overlooking that the team - by that point - had been working from 7am until 9pm for the entire week, ordered food in and had to take several days PTO to recover)
Curiously these are usually the same managers who demand crunch-time, but "have a prior engagement" when the team asks if they'll be helping too.
A space program that demands better than six-sigma risk for example is unlikely to ever get off the ground.
Just because their mistake didn't push the failure rate past the acceptable limit doesn't absolve them of anything.
I get your sentiment, but there are more stories to tell in NASA events. Robert Trivers wrote a book [1] about his research on human behavior, specifically, how self-deception plays a role in the event.
[1] https://www.amazon.com/Folly-Fools-Logic-Deceit-Self-Decepti...
Are these accessible publicly? Could someone link to them, if available online? It's not easy to get a deep view into organizational decision making. The least we could do is to learn from mistakes.
I have no pity for such lack of professionalism. You are correct in saying that it is not so easy and that these structures take a life of their own but this shouldn't be an excuse and we see disasters happening over and over because people don't act as professional.
Anyone on the room seeing such mistakes and who was capable of standing up, should have, even if it meant they could end up jeopardizing their careers by doing so.
* I'm not asking for a hero and I understand that failure is part of the human nature but we should have [professional] respect where it is due.
It's satisfying to point to a handful of people to place blame, but it isn't terribly scientific. We should be asking why they asked irresponsibly. The folks making these decisions didn't just act in a vacuum, they were a product of NASA, systems engineering at large, a bureaucratic institution, and our own societal norms.
It's likely that others who went through similar training and operated in a similar environment would have made the same decision. But even if the bad call was due to a few bad apples that inserted themselves into the decision making process, we should ask how we allowed them to get there.
Punishing an individual may or may not be warranted in this case, I suspect that the guilt they must live with is punishment enough. However, what's clear is that punishment won't be enough to prevent similar problems from arising again.
This is a question of justice first of all. Punishing for the sake of punishment is not the way forward. It is true that people will think twice [before dismissing such kind of complains] if they see that you can't get away with [unintended] murder but this should be a secondary, positive side-effect only. If you try to punish to send a message it might be unjust, send the wrong message (like, silencing whistleblowers that have doubts or people from fixing their own mistakes), or both.
No it's not. This is a question of optimizing spaceflight for safety.
The PowerPoint displays extremely clouded thinking. The title is: "Review of Test Data Indicates Conservatism for Tile Penetration." What the heck does that even mean? An unclear title is a very strong indicator of unclear thinking about the point being made. Further, there is a relatively precise estimate of the size of the foam, and its orders of magnitude larger than the size used for the tests. The slide clearly states that the penetration velocity depends on "volume/mass of the projectile"--i.e. one would expect a much larger projectile to penetrate at a lower speed. So why is the lead message not "we have to try the tests with bigger foam, because we don't have the data we need to reach a proper conclusion?"
Remember, while this is happening, a million other things are going on. The officials have other risks and trade-offs to deal with. In that scenario, one of your jobs as an engineer is to clearly convey your point, and perhaps more importantly, the limitations of your analysis. It's not the job of the decisionmakers to tease that information out of you. Ideally, of course, a decisionmaker faced with unclear messaging will try and get to the bottom. But in a high-pressure scenario, that doesn't always happen. For one thing, how does the official even know which issues need to be run to the ground and which do not? Who was the person who best knew how significant this slide was? The official reading it (who probably saw a hundred other slides in the same meeting), or the person who wrote it?
- The title is extremely unclear: are they supposed to be conservative in their beliefs about the risks (i.e., to not believe that the tile got penetrated), or are they supposed to be conservative in their behavior about the consequences (i.e., reduce risks by not re-entering the damn shuttle)?
- wtf does "overpredicted" mean? That's the second largest font, and it has no visible meaning?
- They don't even attempt to estimate the speed that the foam was traveling. They just said that it depends on speed and mass, but they make no suggestion of the speed.
- Nor do they attempt to draw boundaries on the possible speeds. Could they have fit a line on the test data and at least given a range of speeds that were clearly dangerous?
I don't think this is merely bad presentation design, however. This feels like someone was afraid to stick their necks out in a bureaucracy, and hence didn't give the managers enough information because there were some error bars around it.
Honestly, it reads to me as being along the lines of "overestimated" - and in this sentence would mean "reality wasn't as bad as our predictions, so we have a lot more leeway than we thought".
The title has nothing to do with power point. This is no different from how academics title papers. The use of unnecessarily big words & passive tenses (rather than active) increases with level of schooling.
"Don't bury the lede" - Journalists and writers are taught this. It means, save readers some mental clock cycles. Pore through your own news and figure out the most important fact. Then put it in the title and/or on top.
Highly schooled, intelligent and "honest" people don't do this. When journalists over simplify or make logic leaps in their writing, we call it clickbait. We also prefer the use of the words MAY, and COULD (probability isn't intuitive to most people) - which allows people to ignore the information.
Successful salesy types are intentional about their use of words. They'll use active phrases, avoid ambiguous words, use words that denote certainty to force action in choice areas, and vagueries to hide or down play others.
The officials were surely served papers before or after the meeting and made their decision despite the slideshow.
Summary: Engineers need copyrighting and sales training as well.
Waaait. Your comment essentially established two things: that journalists receive training on how to write properly, and that successful salesy type know how to bullshit their audience to get what their want. I think the conclusion doesn't follow.
Observe that journalists universally don't apply their writing training - in fact, modern news reporting is one of the worst kind of writing out there, with the lede buried under 30 meters of gravel, and spread on a hectare of land (we call it clickbait only when headline is manipulative). So this shows your training matters little if incentives on the job are in total opposition to it.
As for the sales angle, commercial copywriting isn't exactly a paragon of clarity either. Effectiveness in sales isn't measured in how clearly you communicated costs and benefits, but how excitingly you communicated the benefits, and how effectively you've hidden the drawbacks. Engineering communication shouldn't be manipulative like this.
The way I see it, many engineers could use some communication training, but it should be focused at presenting things clearly and truthfully, and on effectively ELI5-ing things to managers. But beyond that, incentives within organization needs to be adjusted, because it's hard to get an engineer to explain things clearly when their job depends on them not doing it.
There are technical communications courses, in fact, it's a required course at Cornell Engineering. Agreed, it's not at all the same as copyrighting or journalism, but the general level of technical communication is so poor, that even courses in those would be an improvement.
They could benefit from clear, concise communication that gets to the point
https://www.nasa.gov/pdf/2203main_COL_debris_boeing_030123.p...
```` Contingent on multiple tile loss thermal analysis showing no violation of M/OD criteria, safe return indicated even with significant tile damage ````
That's going to be difficult for management to push back against.
It isn't really that unclear. There are formal words, but - unlike typical managerial presentations - there's also content behind them.
The part I'm grappling with is how they came to that conclusion despite the "flight condition is significantly outside the test database" acknowledgement as alluded to in the original post. To me, this sounds very much like Challenger in terms of drawing conclusions without hard data to back it up. Easy arm-chair quarterbacking in hindsight, I know, but it seems the through-lines are psychological in nature, not engineering or technical problems.
Especially given the sorry state of the presentation in general, I really doubt this is the case, though.
Ambiguous, poorly presented, and technically dense are three different things, though they can all be present in the same place. Only two of them are necessarily bad.
Ironically the slide you point out ("Damage Results From 'Crater' Equations Show Significant Tile Damage") seems to skew our interpretation of the next slide's title and content in the wrong direction. The former slide states that "'Crater' indicates that multiple tiles would be taken down to densified layer". That sounds bad. However, it also tells us that the "program" (assuming this means the Crater model) that generated this alarming prediction was "designed to be conservative". This would seem to downplay any concern generated by this result. Furthermore we are told that "Crater reports damage for test conditions that show no damage", further casting into doubt the predictions of the model.
And then, on the next slide: "Review of Test Data Indicates Conservatism for Tile Penetration". On the previous slide the word "conservatism" was used to tell us that the results from the "Crater" model may be on the high side, i.e. showing a problem where there is none, and that the test data show a much smaller degree of damage. This slide is about test data ("Review of Test Data"). On both slides we are told that compared to test data, the Crater results are inflated: "Crater reports damage for test conditions that show no damage", and "Crater overpredicted penetration [...] significantly".
Where does this leave us? The context of this additional slide makes the presentation even more misleading than the "asshole author", to use your delightful term, thinks it is. The title is not merely ambiguous; we are explicitly nudged toward the wrong interpretation of it. I have to thank you for bringing this additional context to my attention.
The author may in fact be an asshole (though I think your language is inappropriate in context), and his analysis may lack depth (I think it does to some degree), but in the simple matter of this slide being egregiously awful he's totally right. I don't know why you'd choose this hill to die on.
One of the key skills in being a senior engineer, is knowing how to communicate clearly with managers and decision-makers who don't have the same level of technical knowledge that you do. Communication is an extremely imperfect and lossy medium - figuring out how to get the most salient message across clearly, is a hard-to-master but vital skill. This article shows exactly why.
A PowerPoint presentation did not kill the astronauts. Hell, judging by the full slide deck (elsewhere in the comments), the presentation wasn't even that bad - it was detached, the way scientific papers are detached. Could it be better? Yes. Should it be blamed for this? Not really.
A manager, or really anyone who desires to lead, needs to be able to remove the confusion surrounding information. If someone was deliberately not giving information then that another problem, but the slide had the critical phrase at the bottom showing this was way outside the test parameters (600x with some basic math).
I don't deny that the slide was bad and the engineers who put it together failed their profession, but the person at the top needs to be able to get by this.
>> NASA managers listened to the engineers and their PowerPoint. The engineers felt they had communicated the potential risks. NASA felt the engineers didn’t know what would happen but that all data pointed to there not being enough damage to put the lives of the crew in danger. They rejected the other options and pushed ahead with Columbia re-entering Earth’s atmosphere as normal.
This is it. Not any slideshow or wrong communication of risks, but lack of responsibility.
It's easy to go back after an accident and find multiple points of failure and find things that could have been done better.
My guess is that most people thought it was a minor or acceptable risk. Unfortunately, they were very wrong.
But that doesn't mean the PPT slide is at fault.
I think it's a bit of poetic license to say that the PPT is at fault. But didn't help. Many mistakes lead to an outcome like this one. But this PPT is one of those mistakes. The basic information is there for a very significant point--we know penetration can happen with a sufficiently large or sufficiently fast-moving piece of foam, and this piece of foam is 600 times bigger than anything we tested before. Had this PPT clearly conveyed that point, the chances would have been higher that some decisionmaker would have realized its significance.
Poetic license in journalism doesn't help either. What we got here is pure clickbait, literally accusing something of killing people, when it was just one among many things that were suboptimal and just "didn't help".
A plain txt file here, would have been better.
In text this means puting that at the beginning, in the title and in bold. The same is true in PPT, you have to outline the important stuff. It has to be visible at a glance.
File formats don't solve lack of communication and listening skills.
I wasn't saying a plain txt file was a solution; I was saying that even plain txt, a very low-power format, would have been better than this, and since the ONLY PURPOSE of a PPT is to help you format things, this was a failure. Certainly no format will solve lack of communication, but I see better communication in Hacker News comments than I see in PowerPoints, and I think that says something about the value of PowerPoint.
Why on Earth did they veto even doing a spacewalk to inspect the damage, before concluding it was safe?
Our managers aren’t technical wizards, and especially not in the field of healthcare. They are mostly doctors with a masters in management. Their job is to listen to what everyone has to say, and then make a decision based on that.
Our job is to make sure they don’t miss important points related to subjects they may not fully understand. Burying a “may cause significant damage” on a text-heavy slide like that is simply put poor communication.
It should’ve been the only words on a slide, it should have been in bold red, and, it should have said “if you ignore this, people may die”.
A more correct communications format might be the traditional longer-form report. A main document frames facts, possibly with graphs and other features used to reach the conclusions and explain why different points are salient. An executive summary preface presents the conclusions of skilled individuals and their core reasoning, which lets non-experts reference the larger report for a better understanding and/or ask questions if they are still unclear.
But the managers should have carefully read every word that was placed in front of them. They should have asked questions if anything was at all unclear. This was, ultimately, their responsibility and their decision. They chose to rush it, and people died.
And I'm just baffled as to why they didn't send someone out to inspect the damage visually. The Columbia wasn't carrying a remote arm to quickly and safely maneuver an astronaut to the observation site, but the crew were trained and equipped for emergency EVA. It would have been a minor risk, but nothing compared to the danger of reentry failure.
Powerpoint didn't stop NASA employees from asking "What are the chances of a catastrophic result?"
In times of critical issues, you ask plain and direct questions. You will always be given caveats, muddled answers, and "but ifs". As a decisionmaker, it's your job to 'cut the crap'.
But the bureaucratic morass that Powerpoint embodies does cause people to stop asking questions like this in all forms of industry. You have probably seen it - at much lower stakes - in business, finance, or IT.
The post is picking at the nits of Powerpoint's layout, which I agree with you, seems silly. But the broader picture to blame meeting culture is quite accurate.
How so? The whole practice of using Powerpoint, whether in business, other enterprises, education or elsewhere is literally designed around its nature as a persuasion technology that makes true shared deliberation impossible, by ensuring that everyone in the audience has to expend their mental effort to focus on what the Powerpoint slides purport to say. Even Tufte is rather clear about this, and consistently critical about Powerpoint use. There is a very real sense in which Powerpoint stopped the audience from asking the right question.