Hospital checklists are meant to save lives, so why do they often fail?
nature.com
nature.com
When surgical or emergency-department teams believe they're participating in an experiment to learn whether certain practices improve outcomes, do the outcomes improve?
Airplane pilots use checklists. It's illegal to start the engines unless the approved checklist book is within reach of the pilot in command. In training it's drilled into us that the response to panic is: get out the checklist and follow it. It works. duh. It's all too easy under pressure to forget to lower the wheels or turn on the carburetor heat, or whatever. The drilling in training is what makes the difference.
(Aviation gasoline cools as it vaporizes in carburetors. Under certain conditions that can make the carburetor fill up with ice. If that happens the airplane will land soon. Not good. So carburetors have heaters. )
Checklists have far less success in medicine. Heck, almost two centuries ago Dr. Ignatz Semmelweis (https://en.wikipedia.org/wiki/Ignaz_Semmelweis) found that patients lived a lot longer when doctors and nurses washed their hands.
But, even in the 21st century, it's a struggle to get compliance with hand-washing rules in hospitals. Maybe something about medical training makes people resistant to fixed procedures. Maybe it's the age-old practice of eminence-based medicine rather than evidence-based medicine. It's a serious problem.
It seems so very obvious to me that any time you start asking, from a 40,000 foot level, why people can't follow simple instructions, the answer is going to be, not that they are all idiots, but that they are trained and required to ignore them selectively based on complex ever shifting criteria. That is basically what makes any job require a human and not a computer in the first place.
The more rigid you want people to be, the better the process has to be.
I think it's more likely that there is an immediate feedback loop to disregarding the checklist (i.e. the plane crashes and the pilot dies), whereas not washing your hands has a lot more delayed consequences in which the causation isn't readily known. I think this is the general problem with Outcomes Based Health - we have the stats to back up its effectiveness, but don't have the immediate feedback loops to promote it.
I do this, and always teach others to do this, during incident response. The reason I think it works is it reduces your burden for making decisions during a stressful time. No matter who you are, how good you are at managing stress, or how experienced you are, you’ll almost always end up making worse decisions under pressure, so removing the need to make decisions as much as possible seems to generally lead to better outcomes.
Ever talked with a nurse or a doctor? The reason is simple: way, way not enough staff at hand combined with maximum allowed/planned time for a certain procedure (e.g. 10 minutes for a full assisted shower) which only works in perfect conditions but falls apart under the slightest change (e.g. patient wet themselves), and combined with ruthless MBA-style C-level execs actually firing people for not keeping up with the numbers.
Medicine, especially highly critical / urgent care, needs to be socialized and shielded off from all negative effects of modern-day capitalism if this is to change.
Medicine is heavily regulated everywhere and people always complain about the same problems in the US and Europe, and these problems are the same kind of problems caused by regulation and guild-like organizations all around the world, in all industries at all points in time.
Even if you were to let someone else pay the bill, it's idiotic to have things organized as they're now.
If anything it’s existing shielding is the problem. The supply of doctors is artificially controlled and limited which means you can’t lower pressure through capitalistic approaches. Without fixing the supply (more doctors) or demand (don’t treat patients) issue these problems won’t go away.
At least in Germany, there are massive amounts of open job postings for nursing and other care staff. The problem is that the wages are way too low, as the county/city-owned clinics have to adhere to the budget rule of maximum efficiency and minimum taxpayer subsidies (which led to massive outsourcing and other cost saving efforts in Munich [1]) and private-owned clinics are under financial pressure from their owners (you gotta make those 18% EBIT [2] somehow), while at the same time the amount of money that the public health insurances pay for procedures (fixed rates across the country) is not enough to cover these costs.
[1] https://www.kma-online.de/aktuelles/klinik-news/detail/staed...
Just some clarifications:
- It's not illegal to fly without a checklist under Part 91 (non air-carriers.) When I fly with military pilots, they use a flow for starting small airplanes:
http://fsims.faa.gov/PICDetail.aspx?docId=8900.1,Vol.3,Ch32,...
- the airplane manual is a book, and must be on the airplane. A checklist is a list, not a manual. The checklist can be electronic, as in the HondaJet.
- it's a good idea to follow the emergency procedures either from memory, or a checklist if there's time. Typically recovering from a stall or spin, or feathering a bad engine is done from memory, for example.
Source: commercially-rated airplane pilot.
Wash your hands 12 times a day for a week and you'll understand why people might not want to comply.
I want my doctor to wash his hands, but I can at least sympathize as to why he might not want to.
Hippocratic oath must be window dressing these days.
Personally, I rinse my hands (i.e. with only water) pretty frequently, but only wash with soap when the occasion calls for it (when arriving home, when my hands are greasy, or after sitting at the toilet), which probably averages 2-3 times per day.
On a separate note, my family doctor used some kind of antiseptic and moisturizing lotion on his hands. He said the antiseptic would tend to dry the skin, so adding a moisturizing element to it was a genius idea and whoever came up with it was probably making millions.
I simply don't believe you.
That would imply that you spend almost an hour per day washing your hands if you are following proper hand hygiene.
With twin infants and a toddler, I'm pretty sure you simply don't have that kind of time in a day.
She described a few surgeries to me and her role in the room as the resident performs the surgery. Her brain is the checklist for the surgical procedure. The culture seemed to be, you should be good enough to memorize these complex procedures.
Meanwhile I'm thinking, why not just have someone else in the room reading and confirming the next step to the surgeon?
I suspect the culture of medicine, and especially the culture of surgeons does not jive well with checklist culture.
One day he was going through the landing checklist, and was interrupted by a radio call. He resumed the checklist, then was told by the tower that he was attempting to land with the gear up. The interruption had caused him to skip that step on the checklist.
After that, whenever he was interrupted in a checklist, he started it over again.
Early on I was pretty lax about following them, and inevitably something bad would happen. I'd forget to pack something and not realize it until I had driven an hour. I'd forget to check for loose bolts and lose one. Or I'd completely forget one of the objectives for the day and have to go back out.
I've tightened up my process quite a bit, and do exactly what your father does: if I get interrupted, I start over. It's mentally hard to do when I'm already feeling late, but I've come to the conclusion that that's exactly when I need to be more disciplined about it; that's when you forget things!
There's a well known story of one of the best climbers in the world getting approached by someone while she was tying the rope to her harness right before a climb. After a brief chat she proceeded to climb the route, lean back on the rope, and promptly plummet to the ground.
After being interrupted she had forgotten to finish tying a knot she had done thousands of times before. It's a good reminder to avoid letting yourself get interrupted in the middle of simple but extremely critical tasks, it's just so easy for the brain to skip over part of the process it knows well and consider it done.
I had a similar incident many years ago where my partner only tied into one harness loop. Luckily it was in the gym and she wasn't on lead, but after that, I always make it a point to thoroughly check my partner's equipment.
The reason was pretty simple. He'd take the time to pour over the maps and intelligence to find the safest route in and out, and they'd get back with the fewest holes in the airplanes.
I.e. when it counted, the squadron would entrust him with their lives.
I don't think it's possible to have a greater honor.
[1] https://www.newyorker.com/magazine/2013/07/29/slow-ideas
Checklists were a new tool that required you to submit that you’re stupid and should depend on inanimate paper to do what you already know.
What do you mean by this? As written, it's total nonsense; germ theory is universally accepted.
Checklists are very useful anyway because they let you focus on and remember the parts that are actually different. However when someone is highly trained it is easy to think you already know everything and not use the checklist.
Too many experts, in every field, stuggle with the idea that they too will make mistakes sometimes, and that this is perfectly normal no matter how good you are at what you do if you doo it often enough.
I find it takes a disproportionate amount of mental energy just to track even 5-10 items. Committing those items to a written list makes it easier for me to take that limited resource and be creative instead of constantly wondering if I completed step 3.
The doctor who did this is still practicing last I checked, as are the staff involved in covering it up, and the hospital he worked for was not informing patients of his history.
Another shockingly similar case: https://www.deseretnews.com/article/412903/HOSPITAL-SETTLES-...
Just two cases I remember reading about. Over a quarter million or more per year according to most sources.
Medical mistakes are the THIRD leading cause of death in the USA. Actually, conscientiously using a checklist to reduce that isn't too much to ask.
The checklist is basically the physical equivalent of asking "are you sure you want to remove this limb?" every single time, and people will just "click OK" before long.
But "wrong site surgery" is a tiny subset of medical error, and I'd be amazed if that was more than 1000 per year.
Most medical error is in the form of medication error. Medication error is very common, and I'd be surprised if it was as low as 250,000 per year. I'd guess that number was the number of people harmed by a medication error.
These errors are serious and we need to work to stop them, but we also need to put them into context. There are over 800 million physician office visits in the US per year. There are over 140 million ED visits per year.
yet, pilots make mistake too. examples were given when they rush and ignore the checklist or the second in command for example. rushing is usually the biggest mistake. and ignoring a lower ranking pilot is another big one.
I wasn't in a position to verbally ask but it seemed common.
> “There's no point in having an item that says, 'Have the antibiotics been given?' if there are no antibiotics in the hospital,” says Dixon-Woods.
The checklists that my wife uses are apparently filled with these types of irrelevant checks.
Not to say that they aren't useful, they are, but as this article points out the implementation of checklists often leaves a lot to be desired.
> checklists often leaves a lot to be desired.
You say yourself that checklists are great, and it is proven that they save lives. So if there is fatigue in using them over and over, and they are not perfect, well... get over it? Sure i can understand that it's boring to go through the same checklist 5 times a day, but come on, there are lives at stake here. If one of your wife patients gets an infection and dies because she forgot some important, simple, step because of "checklist fatigue" how would she feel?
That's true with pilots, too, especially fighter pilots. But they rigorously use the checklists.
The pilots also know that many crashes have been traced to skipping an item on the checklist. For example, John Denver's fatal crash was due in part to failing to fuel the airplane before flight.
I'd forgive them for skipping the checklist if their airfield is under attack and they have to get their crates in the air or die on the field.
Nobody ever is. But if their contract negotiator ever opens with that line...
That in combination with training. It's just the cultural thing to do, everyone uses checklists, a pilot from the moment they start training is always doing check lists.
It also helps that a checklist for a plane is always relevant where a checklist for a procedure is more variable.
It means that the more you keep having to skip over irrelevant items and the more you still depend on remembering other things that aren't on the checklist, the less likely any human being is to reliably follow the checklist -- because they accidentally skip over an item thinking it was the irrelevant one, or jump back to the wrong item (skipping others) because they got "off" the checklist to do steps that weren't on it.
The point of a checklist is that it's supposed to be a single idiot-proof source of truth in a specified area, reducing mental complexity and therefore reducing errors.
Once it stops being that because it isn't perfect, it can easily increase mental complexity which requires more brain use and increases errors. That's the fatigue.
So it's not a question of just "getting over it".
If you have skip items on a list, then human error pops up again
So far, the biggest contributor to 'checklist fatigue' among people using the system is from them not being able to modify the checklist to suit the task at hand. Checklists really need to be living documents, so errors and usability improvements can be made. The article does mention that the more proscribed from on-high the checklist was, the more the performance of the checklist was degraded.
There is the attitude of "get over it" and follow the list - that does NOT work. What does work is exactly that - make the thing useful and people will actually love it. That means local adaption, ability to update and refine.
What works wonders is if you can introduce policy changes and improvements through the list. Ie, the list leads rather than follows and gets outdated.
Especially in govt compliance areas - a lot of folks have attitude of parent, get over the fatigue because this is "important". I once went to a presentation where they showed someone in handcuffs because they didn't follow the procedure. There are 100s of irrelevant items (yes - 8 track data standards compliance in some docs still) on some of these lists and people know it is total BS.
The authors of the article make a great point, local adaption and implementation. The checklists that work are carefully curated and honed. I currently work in a space with lots of checklists, and I take that approach, do we still need this item? Things change. If you make the checklist useful MUCH higher compliance. That means updating and refining it so it's the reference guide for what should be happening not an endless list of all possible things one could do.
The checklist had come after the software so any software that was better than legacy software couldn't displace it if it wasn't better in every way. So if you fixed one bug and attempted to deploy you couldn't because your new version had to pass the checklist.
Fascinating stuff.
I was listing to an interview with the author of The Checklist Manifesto, Atul Gawande, and he cited this as one of the main reasons why attempts to use them fail in some organisations. He seems a big proponent of the people using checklists being the ones who should write and imporve them or at least be involved, and that they be kept short and focused. They shouldn't be stuffed with needless detail, they are not instruction manuals.
Checklists imposed from above without input from those who use them is just another version of beauracratic organisations imposing processes on people.
Another thing with obvious benefits I wish we would adopt from east Asia is wearing a face mask when having a cold to prevent infection those around you.
1. Every prominent error that occurs prompts administration to decree a new checklist item be added to provent it in the future. But these added items will not be root-cause fixes so will proliferate to the point that there are now several dozen of negative checks to prevent these one-in-a-million mistakes. And so the lists will grow and grow, but no one will ever dare to take an item off. Because then if that mistake ever does reoccur, the admin that authorized its removal is up for a hearing in front of the firing squad.
2. Some administrator somewhere, noticing redundancy within a given checklist will reinvent control structures. You'll have lists with loops and conditionals. And since the people creating these lists don't know what is "considered harmful" they will almost certainly add GOTO's. In fact, I'd be very surprised if GOTO's don't already exist in surgical checklists somewhere.
But then this addition of control structures could increase to the point that a surgeon could, potentially, be put into an infinite loop! So committees will be formed to ensure every path through a checklist actually can be completed. And someone, somewhere in a hospital conference room, when asked to PROVE the lists all terminate, will take a breath, sit back, and rediscover the halting problem.
As offices go digital, some are doing it right and recreating the forms in the signing application. Other, lazier firms, have you fill out a scan (gag me with a spoon.)
The idea of a loop is a funny one, I wonder if anything checks for it?
Fortunately, Fixsen says, the lessons of implementation science are “completely generalizable”, and all programmes could benefit by noting the importance of engaged leadership, local adaptation and user buy-in. “It doesn't matter how good the innovation is, it doesn't matter how much has been invested,” says Fixsen. “If we don't have the implementation savvy, we're going to get the crummy outcomes that we have seen decade after decade.”
True role of checklists in healthcare: save management in case of legal action.
Consequence: the checklist becomes longer and longer, and is done more and more frequently. To the point of being quite impractical.
Goal: in case of legal action, take refuge in checklists. At least one of the items is bound to be missing so you can divert responsibility to someone else.
Source: I do checklists. That's my job.
It's pretty disappointing to hear that that's how checklists work in your org. I feel like it's doubly negative: not only does it pretty much guarantee that they won't be followed because they're not particularly useful in that form; it also helps to ensure that the folks that have to use them will push back against checklists in general at their next job!
[I suppose that's what happened with me and JIRA too... it might be a decent product, but every org I've used it in has done a very bad job with it]
The workaround for that is for the practicioners to have their own good checklists in addition.
Honestly, he probably is. We need to have laws because most people do it carelessly. Not necessarily because all people do it carelessly. Naturally, those who know they do it safely aren't going to just lump themselves in with the majority that don't.
It’s not that anyone is always careless, it’s that everyone eventually has an off day, and in those off days our careful habits become careless ones.
It doesn’t matter if you are flying economy or 1st class you get the best in class safety, which is collectively shared across the industry. We pay a premium for the service and not to have better chances of surviving.
In an alternate universe, we could have had cheaper airline tickets which had not so great safety record and the good ones only in reach of the super wealthy.
[1]: I’m aware it may not be the case in some countries with completely free healthcare, but this applies to the majority of the world. Even in those countries with free healthcare for all, it is not uncommon to see huge wait times for some important procedures and being able to skip it by going through a private hospital.
It's not an equal model, but it is more just than what we have in America today.
https://www.bloomberg.com/news/features/2019-03-26/the-world...
However, the price difference between the base pricing across these hospitals vary a lot and you tend to go looking for that famous surgeon who practices in a hospital where everything is more expensive (Narayana Health and a few others may be an exception).
Also, the other aspect of the airline industry which we should get in hospitals is sharing of best practices. The protocols and procedures which can be effective in saving lives should not be a differentiator, instead it should be standardised and made mandatory.
The differentiators should be non-medical as much as possible.
Of course, only the wealthy hospitals may be able to afford the latest equipments but the expertise of the medical staff and best practices are easily transferrable in a cheap way and shouldn't be guarded like an individual's or the organisation's IP
If you're interesting in chatting about this, feel free to email me (in bio) or grab thirty minutes on my calendar: https://calend.ly/labib
Also, I think you can definitely pay more to go on a safer plane - I would imagine more expensive airlines have newer planes and better maintenance - but the risk of death is so small it isn't worth the extra cost.
I feel like many people would be willing to trade some security for better ticket prices.
Of course there are differences with the car industry, one of them being the economic value of the vehicle and thus the loss in case of accident. I don't know if it would actually be profitable for them.
This sets a floor on airline safety--it has to be safe enough that the airlines can readily find pilots. Some passengers, especially those who rarely travel, might accept more risk for a better price but I suspect that pilots, who might be making hundreds of flights a year, would not be so willing.
My expectation is that there should be exactly zero surgical instruments that get left inside the bodies of patients, when they are sown up, or zero cases of the wrong organ being removed.
And yet, there's >1,000 cases a year of the former, in the US alone.
Pick almost any profession other than pilot, and checklists are something that most people will ignore, often for the same excuses doctors have.
It seems like it should be comparable, as both medical errors and aircraft crashes are taken seriously, but there's a huge difference in magnitude.
It’s a habit I had to work hard at, item one on my list is usually don’t skip any steps.
- we've implemented some simple changes that make things better!
- everyone, let's all do this thing!
- there! We've tried! It doesn't work
- but you've bastardized the original idea beyond recognition
- doesn't work!
[0] https://ronjeffries.com/xprog/articles/jatbaseball/
edit: formatting
Now I am kicking myself that I never thoguht about applying point-and-call to driving. I can absolutely see the benefit, but would need to think about the details. Do you point out only traffic signs and signals, or things like potential threats too?
I will look-and-call for lane changes, pretty much as you would imagine look over the shoulder and "Clear Left," then "Turning." Same goes for right. For me it is more action oriented, to make sure I've assessed the area for issues. I haven't done anything like call outs for threats, but now I'm kicking myself for not thinking of that. I already do that religiously in FPS games, I could see how doing that would be good to get and stay in the habit of active scanning, "Pedestrian crossing, two o'clock." Might have to try that one out too, after all my friends will love that development :)
If it works it ain't stupid. That's good enough for me.
> Even a small fender bender changes my driving rapidly for a good period of time.
That's always a good thing. Not sure what helps long-term other than commitment to a wonky habit.
Like a good engineer, I do a post-mortem after any incident or near-incident. Crashes, tickets, brake-slams, etc. Unfortunately a lot of near-incidents are of other drivers that no amount of preparation can prevent, e.g. drivers turning onto your road severely under speed, drivers stopping short (10+ car lengths) near lights to be nice and let someone turn into or across the road.
I do point and call a recipe ingredients when I try to cook. I would love to make a point and call routine for packing travel bags, but their 3D nature makes it hard.
I have a little "dance," slap left-pocket (key / clip), slap right-pocket (phone, wallet), slap left-pocket again (double check keys), slap right-knee-pocket (knife). I should throw in a foot slap to make folks think I'm doing a Schuhplattler on my way out the door.
> I do point and call a recipe ingredients when I try to cook
You must have a nice spice shelf. I have 30+ spices in a narrow cabinet. Finding the right 4 is a chore unto itself.
The controversy is usually about wether the movement direction is any good.
I wonder how many people would actually follow the checklist if they knew their compliance was being continually monitored and audited?
Since it is intended to be in a hospital, that's a $40K Alexa that must be created by somebody with the right connections. But all that you need is an Alexa.
I know in the last ten years I've seen new processes with forms to fill out for mundane tasks and its PITA.
Most QA systems have a source of responsibility and multiple levels of review.
I wonder what would happen if there was a QA team (with sufficient education) at a hospital that had complete control over checklists and associated procedures. Would that improve the outcome when utilizing checklists?
--- Anecdote ---
I've been in the hospital many times, and I've been present for pre-surgical checklists a few times. It was ALWAYS a nurse handling it, and it was never a nurse that I was familiar with during my care. All but once the interactions started with trying to figure out who I was... without asking me.
I can recall a number of times where there was a question, such as "Which body part is being operated on?", except I was told the answer. The "question" turned into "Looks like your right kidney is being removed", with a slight pause for me to object.
It always made me feel like the staff thought the purpose of the checklist was to complete the checklist, rather than to QA the system they're part of.
--- Anecdote Extended ---
I suffered an unpleasant complication from this during a procedure.
I do not respond to anesthetics the same way as most people. It takes a MUCH larger dose to knock me out, and to keep me out. (years of desensitizing medications + big (very tall) person + MC1R gene).
I know this. I can communicate my past procedures and the doses administered. USUALLY they are supposed to ask if you've had any prior procedures and potentially follow up with questions about allergies, anesthetics and other questions related to sedation.
I was not asked. I didn't think to speak up because I was already being inundated with stimulus.
The result? The first "count down from 10" resulted in me counting to 0 and asking "What next?".
I vividly remember the abject horror on the face of the nurse. She fumbled around, whispered to the anesthesiologist, whispered to other nurses. People shuffled around and nobody said a word to me.
I was scared. It felt like forever before I was asked to count down from 10 again. It worked that time...
Until I woke up in the middle of the procedure!
When I woke up in post-op, I asked the nurse about it and she told me that I wouldn't remember any of it. She told my wife that I wouldn't remember the couple hours after post-op.
I remember every single nurse's and doctor's name involved and the discussions my wife had with the doctor in post-op (which she confirms my recollection).
It could have all be avoided if someone just said, "Hey, have you had a procedure where you've been under general anesthesia before?"
I bet that checkbox was ticked though.
Separately, maybe it would work better to give the checklist to the patient and have her check off the items.
In light of some comments, it appears I wasn't clear. My point was that the problem is having BOTH those items on the list. Marking the surgical site is definitely worthwhile. Having it on the same checklist as introducing yourself dilutes the importance of it and the other important items on the list.
Amputating the wrong limb, removing the good kidney instead of the bad kidney...
The introduce yourself by name item is less crucial, but ensures everyone knows who everyone is, especially in larger hospitals. In a stressful environment, it might be the difference between fumbling and not. Apparently it also increases the likelihood that someone will speak up if there is an issue noticed.
It was because on the form, the "right" checkbox was on the left side, and the "left" checkbox was on the right.
(He immediately caught it and corrected it.)
However, there is a reason for the latter item. There have been many plane crashes in which at least one cockpit crew member had some inkling that something was wrong [1], but their warnings were ignored, often when there was a big authority gap. Crew Resource Management [2] was developed to address this.
Introducing yourself by name is designed to emphasise that the group is now a team with a common purpose, and everyone, from the lead surgeon to the nurse, can and should speak up if something looks awry.
Quite some thought and research has gone into this. Obviously, this doesn't mean that the ideas can't be criticised, but a facile dismissal like yours is unwarranted and might be one reason the checklists don't live up to their potential.
[1] Some examples:
The Tenerife disaster, 583 fatalities: 'On hearing this, the KLM flight engineer expressed his concern about the Pan Am not being clear of the runway by asking the pilots in his own cockpit, "Is he not clear, that Pan American?" Veldhuyzen van Zanten emphatically replied "Oh, yes" and continued with the takeoff.' https://en.wikipedia.org/wiki/Tenerife_airport_disaster#Comm...
United Flight 172: 'The captain focused on the landing gear problem for an hour, ignoring repeated hints from the first officer and the flight engineer about their dwindling fuel supply. Only when the engines began flaming out did he realize their dire situation.'
https://en.wikipedia.org/wiki/United_Airlines_Flight_173
Korean Air 801: 'The crew noticed that the aircraft was descending very steeply, and noted several times that the airport "is not in sight." Despite protests from flight engineer Nam that the detected signal was not the glide-slope indicator, Park pressed on.' https://en.wikipedia.org/wiki/Korean_Air_Flight_801#Accident
Seems doubtful that everyone saying their name would have fixed any of those problems...
Though I am a long term patient with a chronic condition so that might make a difference.