On the Term “Blameless Postmortem”
tylercipriani.com
tylercipriani.com
However... this has become one of the most powerful changes in our extended group's engineering culture. The focus on what happened and how, rather than the who. Each time, it is a chance to examine how things went kittywampus and how we can break that chain in the future. For the most part, I don't even think we include the names of the people involved. The process grew on me, the more we used it. Folks would fess up, rather than hide it and creating a snipe hunt for root cause. We were able to categorize where our pain point were and what we could do to stop it - and it worked. It really made the folks who could analyze what happened and how we could do better with what we have, shine.
The chain that got us there... Developers had drop access to the prod system - all of them. That got fixed. The account that they were using had more permissions then should be used. Reviews were not done, because rush rush and vacations, the developer did some stack overflow that had unintended consequences. Tests matter. A few other bits that were slipping became checklist items for the change control.
It was far more important to fix the processes and gaps than say 'Bob' screwed it up. The other bit was when folks were chewed on, they hit the mistakes. It consumed so much time, when someone would log in and 'fix' something with a system account, and we had to guess what broke the system.
It takes time. I was so, so very skeptical when we started. It sounded like someone just avoiding responsibility. First few, were. Looking at what broke, why it broke, and how do we avoid it in the future.
At the end of the year, I'll also use these writeups to see where our emphasis areas should be. 2021's top - humans suck at keeping x509 certificates up to date when manually added.
Paul Reed has made a great case for ‘blame-aware’ (https://medium.com/@jpaulreed/why-blameless-postmortems-migh...)
Rein Henrichs also explores the topic very well through a unique lens (https://youtu.be/KXrsvLMqF1Q)
As Cipriani points out, there is the term and there is the concept(s) the term is intended to convey. It does seem to me more folks know the term than have read the origin (my post above from 2012). This is fine, as long as productive dialogue continues in the industry about what the term was made to convey.
Also impressive he’s had an HN acct for 12 years but only 5 karma!
Thank you for posting this, and, of course, for the original concept :)
Aviation uses the word "investigation", by the way. But they can only omit the "blameless" part because there are very strong guarantees that it will be blameless.
It’s also easier to do those on a weekly basis, so there’s less of a Pavlovian association of “bad thing happens” then “synonym for postmortem happens”.
The article even mentions root cause so not sure how they didn't find that term in their searching.
When doing a root cause analysis where one or more of the identified causes is a person failed to perform an action adequately, the "blameless" bit is supposed to be about digging further to understand why they didn't.
Was it lack of training or experience? or maybe they were overloaded with work (fatigued, distracted)? or was it due to pressure to get the thing done by a deadline and therefore steps were skipped? etc
A retrospective, to me is more about continual improvement. Looking at what worked and what didn't in the last project/sprint. A root cause analysis is a different thing that you do when something has gone wrong that shouldn't have that had major consequences.
Yes, it's a simple mistake but how was a system allowed access to our global environment that this edge case was never calculated? In many of the meetings, the common issue is communication even between co-workers on the same team, and between internal platform providers. One case was an outage on the storage backend and realized after a long meeting that the internal SLA was much greater than we expected (and which the systems would timeout). It only worked for so long as storage utilization was extremely low.
Programming culture has almost football field level of "No time for weakness" attitude.
If I see a possible failure mode of a system, and bring it up, someone's going to tell me to stop being a clicky click windows idiot and learn to be careful.
Trying to prevent human error in software isn't seen as a priority so nobody does it. They are concerned with the most reliable code rather than the most reli4 code-user-hardware-task-schedule-conditions system.
Programmers need to accept software fixes for human and hardware failures. It's a lot easier to add a confirmation dialog than it is to somehow become 100% reliable at not clicking the wrong thing.
As you can imagine, we all became instant members of the “we need a better name for this process” club. Thank you, OP for giving us a completely viable alternative. I’m calling it a “disquisition” from here on in.
Did your group somehow started using 'postmortem' instead of 'afterparty'? Because if every event you organize require scheduling a postmortem then it is no longer a postmortem and just a regular activity.
So technically, we only hold one if things went wrong.
If we ever held an event where everything went right, then yes, we’d totally call that one an afterparty.
> Tearing in to what went well, what didn’t go well, and suggesting process changes for the next event
is the regular [performance|postfactum|event] analysis, but it IS NOT the postmortem!
From your previous comment you understand what the term means "after death". Did you really think "after death" and "after event" are the same?