Hacker News Reader

Comment by ekianjo | Hacker News Reader

This is described in the book called "the checklist manifesto". Very good book by the way.

https://www.amazon.com/Checklist-Manifesto-How-Things-Right/...

Agreed, the book was very well written and the audiobook had a great narrator.

Because of that book, I'm not surprised that the checklist reduced deaths, but its a bit shocking that its by a third. Then again, the base rate was very low to begin with, showing that surgery teams were already doing a seemingly good job:

"The death rate fell to 0.46 per 100 procedures between 2000 and 2014, analysis of 6.8m operations showed."

Still, over 6.8 million operations, is that 15,000 survivals? Wow.

One big difference, I've always suspected, is that a plane crash is very visible and affects hundreds of people at once.

Somebody sick dying in hospital, on the other hand, is just something that happens, and draws very little attention - even though way more people are affected than in plane crashes.

So, the clustering and visibility of plane crashes leads to excellent check list discipline and other best practices in aviation, by and large (CRM, Crew Resource Management, is another thing Atul Gawande brings up in The Checklist Manifesto, and also eminently transferable to surgery), while I suspect that in surgery it is easier to drift away from best practices again without anyone noticing.

That's why these large studies and the educational efforts of Dr Gawande and others are so important.

edit: typo

Yes, that’s probably the reason because mistakes in health care is an absolutely massive killer. In the US alone it’s responsible for about as many deaths as if two jumbo jets collidee with each other every day. The number of people dying from the simple fact that health care practitioners are humans and humans get tired, drunk, angry, careless or just plain stupid for random reasons is mind boggling.

If we saw a midair collision of jumbo jets every day however the public would never accept it.

https://catalyst.nejm.org/medical-errors-preventable-deaths/

Read this: https://www.newyorker.com/magazine/2007/12/10/the-checklist

Same author as the book, but earlier. Contains all the meat, but none of the repetitions and the tenuous analogies.

Gawande had enough for a long-form article, but not for a book, and IMO it shows.

The book isn't bad. It's just that the essay is better.

There seems to be minimum book size that forces inflation in a whole genre of self-help books that genuinely have good advice. I've read books where all of the advice I need is in the end-of-chapter checklists.

On one hand, it's good to err on the side of too-long a book, as it's easier to skip fluff than to seek out more details, illustrations, and alternative phrasings that aren't in the book. On the other hand, it's annoying to have the extra weight in volume in one's book collection.

I was lamenting this exact thing to a friend the other day. Books are just too long for most self-help advice and it ends up being 80% filler.

The worst is when the first three chapters are about how rich/smart/successful you'll become by reading this book! (Except for the first three chapters, of course, because those are just advertisements for the book you already acquired.)

To solve this I subscribe to blinkist where they have summarized every self help book into a 15 minute read. It costs money but the summarization of content is worth it. I've read hundreds of self help essays (books) with blinkist.

Does not the book also give more insight about how applying the principles to someone everyday life ?

I also didn't think this. I've read the book but not the article, and while the conclusion seems fine, a lot of the material in the book is stretched.

The one that bothered me the most was the inclusion of the India soap program as support for checklists -- those are entirely different models.

The soap model involved researchers handing out free soap, and teaching people that they should wash with the soap in any of a set of specific circumstances:

    - once a day (full body)
    - before preparing food (hands)
    - before eating (hands)
    - before distributing food to anyone (hands)
    - after defecating (hands)
    - after wiping an infant (hands)

This made a big dent in the prevalence of disease in experimental neighborhoods. Great! This was a good idea. But it's not an example of a checklist. The concept here is that, every time you do anything at all, you see whether it's one of those five hands-washing circumstances, and if it is, then you wash your hands. This is the opposite of a checklist, where you perform a series of verifications whenever you take a specific particular action, not whenever you take any action at all.

(Obviously, people can't handle the mental load of "before doing _anything_, check to see..." and instead would have added "wash your hands" to the appropriate five behavioral sequences. That reduces the mental load from (1) a constant mental drain on any activity of any kind to (2) learning five things. But if you make that switch, you stop having any relationship to the checklist concept at all.)

Why do you think so? I don't.

The article in The New Yorker is superb, you should really take a look. What could be missing there?

The medical community uses checklists. They've been using checklists for a while. A number of studies found that they only help for a few months, while they're new.

That said, "the medical community" is not a homogeneous monolith, and you can absolutely find regional variation in what checklists are used for, how detailed they are, how closely they're followed, how people are accountable for keeping to them, etc.

"The Checklist Manifesto" chose to overlook the studies about how transient the benefit of checklists is.

> A number of studies found that they only help for a few months, while they're new.

This 2014 review looked at 34 studies that looked at the effectiveness of safety checklists:

> The main findings were improved communication, reduced adverse events, better adherence to standard operating procedures, and reduced morbidity and mortality. None of the included studies reported decreased patient safety or quality after introducing safety checklists.

> Safety checklists appear to be effective tools for improving patient safety in various clinical settings by strengthening compliance with guidelines, improving human factors, reducing the incidence of adverse events, and decreasing mortality and morbidity. None of the included studies reported negative effects on safety.

So I'm going to ask for a citation on that claim.

https://www.ncbi.nlm.nih.gov/pubmed/24116973

Could like point out the studies you mentioned?

The studies concerning long-term impact of surgical checklists I found all claim an improvement.

> Sustained use of the checklist was observed with continued improvements in process measures and reductions in 30-day surgical complications almost 2 years after a structured implementation effort that demonstrated marked, short-term reductions in harm. The sustained effect occurred despite the absence of continued oversight by the research team, indicating the important role that local leadership and local champions play in the success of quality improvement initiatives, especially in resource-limited settings.

https://www.ncbi.nlm.nih.gov/pubmed/25806951

The way they run these studies is crazy to me:

>Only 3 of 22 operating stations (13.6%) had a functioning pulse oximeter prior to the intervention; accordingly, a pulse oximeter (model 7500; Nonin Medical Inc) was installed at each operating station as part of the implementation effort

The addition of essential medical equipment surely won't bias our results!

One could argue the essential medical equipment was missing due to lack of checklists ensuring its existence. That said, yeah, it would have been better to do 2 separate studies then: one on adding the equipment and then one on checklists.

> A number of studies found that they only help for a few months, while they're new.

Interesting, that kind of surprises me. Without going to research it right now, I wonder how/if the efficacy is affected by things like going through them in tandem with a partner, or doing them out loud or with exaggerated physical acknowledgements (like the Japanese train drivers who point out signs/notices along their route as a way to maintain focus on those notices).

A speculative hypothesis related to this (based on my own experience as someone who has read the Checklist Manifesto and is a big advocate for checklists):

Over time people internalize the checklists because they unwittingly conclude the reason for the checklist is to learn a new procedure rather than foolproof their procedures and avoid errors. As a result, they start to skip the checklist or get sloppy with it.

There are a couple other points related to this that I believe Gawande does address in his book:

1. Doctors (more generally experts or egoists) will be resistant to checklists because they find them unnecessary (given their experience or expertise).

2. Checklists need to managed with regular review and updates.

You can boil Gawande's book down to: "Start using checklists because they're really effective." But his book goes deeper than that and addresses some of the underlying human factors involved in getting an organization to use checklists effectively.

> Doctors (more generally experts or egoists) will be resistant to checklists because they find them unnecessary (given their experience or expertise).

At the end of Chapter 7 of Gawande's book (The Checklist Manifesto) is this revealing tale:

<quote>

Nonetheless, some skepticism persisted. After all, 20 percent did not find it easy to use, thought it took too long, and felt it had not improved the safety of care.

Then we asked the staff one more question. “If you were having an operation,” we asked, “would you want the checklist to be used?”

A full 93 percent said yes.

</quote>

So basically people stop doing checklists and then claim they don’t work. Checklists are great, but they have to be used. Instead of treating them as a learning process, I treat them as something I explicitly do not have to learn. It frees my mind for other things.

I don't know that this is the case. I was just speculating in response to the claim upstream that Gawande "chose to overlook the studies about how transient the benefit of checklists is." I too would like to see the citations. If there was a study showing this regression, I would be interested to see if it controlled for this sort of factor.

But I agree with you wholeheartedly. I don't want my mind cluttered with stuff that can be handled by simple script.

"Dropping protections because the failure rate dropped" is a common failure mode, as seen in: Checklists, vaccines, financial regulations, and I'm sure many more arenas.

That was the explicit justification for the Supreme Court finding that the Voting Rights Act wasn't necessary, from memory - states weren't doing the things it banned, so it wasn't necessary to ban them any more.

Makes me feel like there's a similarity between implementation of checklist methodology and implementation of agile methodology. Depends so much on culture and processes and people.

The other problems doctors have is there are many cases where people are different and so you deal with a situation not on the checklist.

While we're guessing, I would figure it just becomes drilled into people as automatic & a given, the same way you buckle your seatbelt without thinking about it.

> "The Checklist Manifesto" chose to overlook the studies about how transient the benefit of checklists is.

It's been almost a year now since I read this but I'm fairly certain Atul touched on this, either in this book or in his book "Better", that after those implementing the changes left, the departments often fell back into old habits.

Could this be the case?

This isn't just with checklists, but a typical behavior of organizations (at any scale). Organizations are systems, systems tend to fall into a steady state of performance, quality, etc. after some period of time. If you draw the people's attention to something (checklists, quotas, threats, praise, etc.) you can see a temporary boost in performance, but unless the culture changes the performance tends to return to that same steady state (maybe slightly enhanced or worse).

In order for the benefit to stick, you need to actually change the system. Otherwise, complacency or other things will return when the pressure and other early benefits wear off.

In other words it isn't having a checklist, it is actually checking that people follow the checklist. Note that follow the checklist is different from checking all items on the checklist which is an obvious work around to the requirement that you follow a checklist.

That's an interesting point, can you provide a link to those studies ?

Are these paper based? Any checklist apps out there for our daily use?

GitHub and GitLab support task checklists in Markdown and also project boards which add and remove labels like 'ready' and 'in progress' when cards are moved between board columns; like kanban:

- [ ] not complete

- [x] completed

Other tools support additional per-task workflow states:

- [o] open

- [x (2019-04-17)] completed on date

I worked on a large hospital internal software project where the task was to build a system for reusable checklists editable through the web that prints them out in duplicate or triplicate at nearby printers. People really liked having the tangible paper copy.

"The Checklist Manifesto" by Atul Gawande was published while I worked there. TIL pilots have been using checklists for process control in order to reduce error for many years.

Evernote, RememberTheMilk, Google Tasks, and Google Keep all support checklists. Asana and Gitea and TaskWarrior support task dependencies.

A person could carry around a Hipster PDA with Bullet Journal style tasks lists with checkboxes; printed from a GTD service with an API and a @media print CSS stylesheet: https://en.wikipedia.org/wiki/Hipster_PDA

I'm not aware of very many tools that support authoring reusable checklists with structured data elements and data validation.

...

There are a number of configuration management systems like Puppet, Chef, Salt, and Ansible that build a graph of completable and verifiable tasks and then depth-first traverse said graph (either with hash randomization resulting in sometimes different traversals or with source order as an implicit ordering)

Resource scheduling systems like operating systems and conference room schedulers can take ~task priority into account when optimally ordering tasks given available resources; like triage.

Scheduling algorithms: https://news.ycombinator.com/item?id=15267146

TodoMVC catalogs Todo list implementations with very many MV* JS Frameworks: http://todomvc.com

Reusable Checklists could be done with a simple text document that you duplicate every time you need it no ?

For sure. Though many tools don't read .txt (or .md/.markdown) files.

GitHub and GitLab support (multiple) Issue and Pull Request templates:

Default: /.github/ISSUE_TEMPLATE.md || Configure in web interface

/.github/ISSUE_TEMPLATE/Name.md || /.gitlab/issue_templates/Name.md

Default: /.github/PULL_REQUEST_TEMPLATE.md || Configure in web interface

/.github/PULL_REQUEST_TEMPLATE/Name.md || /.gitlab/merge_request_templates/Name.md

There are template templates in awesome-github-templates [1] and checklist template templates in github-issue-templates [2].

[1] https://github.com/devspace/awesome-github-templates

[2] https://github.com/stevemao/github-issue-templates

I really want slack to adopt these as well...

I'll often be on call with customer and create a checklist on MacOS Notes on the fly. Then will copy paste that in slack or github for simple tracking.

Mattermost supports threaded replies and Markdown with checklist checkboxes

You can post GitHub/GitLab project updates to a Slack/Mattermost channel with webhooks (and search for and display GH/GL issues with /slash commands); though issue edits and checkbox state changes aren't (yet?) included in the events that channels receive.

Kanban boards e.g. Trello, Kanbanflow, Microsoft Planner, and others. They all support checklists, and you can categorize/organize as you need.

“A simple, free and powerful way to manage your team's recurring checklists and procedures“ https://www.process.st/ Paid features include 3rd party integrations. (Disclaimer: neither a customer nor employee of the company.)

Do any of these let you re-use your checklists, though? Planner at least is pretty useless for this, as I don't think there is a 'reset all items to un-checked' feature, or even basic stuff like re-arranging your list.

Click the "..." and Copy Task. It will generate the checklist again without anything checked, then just mark the old as complete.

You'd need some system to mark when one list ends and the next begins, KanBan just treats them as individual tasks.

> even basic stuff like re-arranging your list.

All of them let you re-arrange your list. Is this a mobile specific issue?

Everything in my comment was describing Planner. You cannot re-arrange checklists on a task in Planner on mobile. And I'm not clear if you were describing Planner in your comment, but I just tested the workflow described and it results in a second copy of my task with the same checklist items checked.

> This is described in the book called "the checklist manifesto".

You might notice that the book was written by Dr. Gawande, who implemented the checklist in the fine article!

I second that, excellent book.

I've also read it, for anyone thinking about buying it, i think it can be completely summarised by:

'checklists are very very good. You should really use checklists'

Now if you require further info on why they are good, by all means read the book, i just found it was a blog-post, if not a tweet, expanded to a book, YMMV.

That's exactly what it was. Atul Gawande wrote a column in the New Yorker [0] and was presumably offered a deal to expand it into a book.

[0] https://www.newyorker.com/magazine/2007/12/10/the-checklist

> 'checklists are very very good. You should really use checklists'

If you stop at that you miss a big part of the book where he describes that not all checklists are equal, what makes checklists useful (i.e. you should not state the obvious in a checklist and he goes at length to explain why), and how you spread the usage of checklists in an organization. That's a lot more important than "checklists are very very good".

Plus the whole spiel about CRM (crew resource management), eg that the entire team should introduce themselves, that the nurse is as entitled to speak up and ask questions as the Prof. Dr. surgeon MD Esq., etc. etc.

That's the basic gist of all productivity and business how to/advice books. They're all a fairly simple set of ideas with 100 examples and testimonial style stories about just how revolutionary it is or how much it changed executive XYZ's life.

I think that you're right, yet the book may still be valuable. I have, for a while, read summaries of business (and other) books by getabstract.com. These cover the main points of the book, but what happens is that you read it, it all makes sense, and a day later you've completely forgotten of it.

The book, on the other hand, yes, expounds on the very same points again and again, with different examples, under different circumstances, highlighting different aspects, and then, having had the same message hammered home so many times, there's a faint chance that a month later you remember some of it.

(Great books change the way you view things forever, but that happens rarely, and never with business books, I'd say :-)

Yeah I'm not really saying it's useless just that it's the basic format of those books. I think the best way to read those is to skim the various testimonials to get an idea what they're talking about and get to the author's point then read them as they spark your interest or seem relevant.

I get why they're in there, it's one of the few ways to argue the actual effectiveness of whatever advice the author is giving. It just seems excessive sometimes how much of the books are just 'I swear this works look at these important people it worked for.'

Yes, and that's a feature. Human brains are swayed by narrative, not by rote facts. If you want to get a positive change implemented, you need to tell a story.

> I've also read it, for anyone thinking about buying it,

Or perhaps see if your local library has it.

I've reduced my book purchases, and generally only consider buying something nowadays if I still enjoy it after the second/third reading (from the library).

Buying a book before reading it is almost always a bad decision, even if it has rave reviews. And even if you love the book, owning it serves no practical purpose unless you constantly re-read it.

The only good thing is that they serve as visual reminders of the books that you've read; otherwise it's easy to forget the titles and authors.

> owning it serves no practical purpose unless you constantly re-read it.

I disagree. I write a lot on the margins of the books I own - notes, personal observations, questions, criticism. I highlight interesting passages. It helps me understand the book deeper and retain its ideas.

Note that rereading a book doesn't have to mean cover to cover. If you look things up in the book once in a while you should have it.

It doesn't need to be you who reads the book either, it could by your personal library and you lend it to friends interested in the same subject. Public libraries will get rid of unpopular books so having a personal library of obscure subjects you care about is useful. (The internet does not have everything)

> The only good thing is that they serve as visual reminders of the books that you've read; otherwise it's easy to forget the titles and authors.

It's possible to create an account on Good Reads or Library Thing to keep track. There's Delicious Library as well (for Mac?).

One additional thing I got out of it that wasn't totally obvious to me beforehand about checklists is that they work better if they don't attempt to be exhaustive. Focus on the major items that should be "checked".

I kindof agree, but he also had other interesting nuggets such as having all the medical staff introduce themselves and why they are there in the surgery room before any cutting begins.

It's interesting to think how often dumb errors can be caught just by making sure everyone in the room is there for the same surgery (i.e., repairing a joint, not cutting off a limb). Correcting that confusion before cutting makes me feel better.

The only addition I would have to that summary is:

"And yet, most people think checklists are beneath them. They are insulted that someone with as much experience as they currently have would require a checklist."

Here is a Hidden Brain episode on it: https://www.npr.org/2017/10/30/559996276/the-trick-to-surviv...

Reply on news.ycombinator.com