Debugging the Doctor Brain: Who's teaching doctors how to think?
bessstillman.substack.com
bessstillman.substack.com
Link doesn't work in Europe, redirects to https://eu.m3.com/.
From the transcript:
> But before you go looking for the oldest doctor you can find, you know, hold on for a second. Turns out that what David and his colleagues found in obstetrics may not apply to all fields of medicine. Let me give you an example. A few years ago, we looked at a similar question among internists who provide care to hospitalized patients. These doctors are called hospitalists and nowadays if you’re hospitalized with a general medical condition like pneumonia or heart failure, chances are you’ll be treated by one of these types of doctors. So, my colleagues and I looked at whether the age of a hospitalist physician was tied to outcomes among elderly patients. We looked at about 730,000 hospital admissions by nearly 19,000 physicians and we found that patients treated by older physicians had a higher rate of mortality within a month of their admission date — it’s called “30-day mortality” — compared to patients treated by younger physicians. More patients died under the watch of doctors who were age 60 or older. And interestingly, doctors who were just five years out from their training lost the fewest patients. That finding appears to contradict David’s data showing that obstetricians continued to improve even 30 years into their career.
Let's say someone is not treated that well by his doctor for a chronic condition. He gets to the emergency in a different hospital and probably dies. Does his GP get to know about it?
For example, many doctors in my country believe that Left Ventricular Hypertrophy (LVH) on an ECG is nothing much to worry about, as long as there is no anatomical LVH showing up on an Echocardiogram. Yet, study after study online concludes that ECG LVH is a serious marker of cardiac pathology distinct from (but related to) from anatomical LVH.
How do doctors who operate on this assumption going to learn from experience, if they mostly don't know that their patients have cardiac events as a result?
I disagree with this. If a patient dies from a surgical complication, it is often weeks or months later, on a nonsurgical service because there are no surgical options left for the patient.
> Let's say someone is not treated that well by his doctor for a chronic condition. He gets to the emergency in a different hospital and probably dies. Does his GP get to know about it?
Yes, they get to know about it. But ascribing cause and effect in a chronic disease is difficult.
Eventually everyone will die, even if they get perfect treatment. I'm not saying there's no such thing as medical error - in some cases there is clear and obvious error - but what's much more common is a situation of "Did I do the wrong thing, or did I do the right thing but they were so sick that they died anyway?" And there are often many years separating cause and effect, which muddies the picture even further. That's why learning from specific patient outcomes is tricky and why doctors lean so heavily on evidence based medicine, which means learning from large medical trials with rigorous statistical controls.
> For example, many doctors in my country believe that Left Ventricular Hypertrophy (LVH) on an ECG is nothing much to worry about, as long as there is no anatomical LVH showing up on an Echocardiogram. Yet, study after study online concludes that ECG LVH is a serious marker of cardiac pathology distinct from (but related to) from anatomical LVH.
There's a difference between serious pathology and serious pathology you can do something about. I agree that LVH on EKG is a bad sign, even if the ultrasound is normal. But what is your GP going to do about it? There are many test results that are abnormal and/or correlated with bad outcomes, but only a subset of those can be labeled with a concrete diagnosis that is well understood medically, and only a subset of those can be treated.
All your GP can do for an ECG finding of LVH is advise blood pressure control, cholesterol control, exercising frequently, and other things that are generally good for heart health.
On the other hand, if there is anatomical LVH, then the next question is whether there's hypertrophic obstructive cardiomyopathy. That's a concrete diagnosis where we know a lot about the underlying mechanism, which leads to specific advice like avoiding strenuous activity. And some patients with HOCM can benefit from a septal ablation. That's why anatomic LVH gets more attention from doctors.
I’m curious. How do they get to know about it? Maybe the process differs between countries. I’m from Ghana, by the way.
> All your GP can do for an ECG finding of LVH is advise blood pressure control, cholesterol control, exercising frequently, and other things that are generally good for heart health
Mostly true, but taking ECG LVH more seriously helps the patient understand how important it is for them to improve their general heart health. It also makes LVH regression (which is possible in more cases than doctors believe) a therapeutic target.
Genuine question, and I can see totally valid reasons for each course of action.
The feedback loop tends to be stronger on the inpatient side. In academic settings, it's common for each department to have a "morbidity and mortality" conference once or twice a month. This is a meeting where one or two physicians will present a case that they think went poorly and the rest of the physicians in attendance will give feedback on what could have been done differently.
> ...but is there any feedback to GPs about the effectiveness of their work?
Specialties too. esp for any misdiagnosis or hard to diagnosis conditions.
90% of what doctors do is completely routine and they all know and keep up with what the "standard of care" is for common conditions and in many cases it's the same today as was the standard of care in 1984.
Sure, if you have something unusual you might want to go to a younger specialist or a specialty clinic where they focus on leading-edge care for that condition.
A 60-year old GP will do fine for your annual physical.
An annual physical is such a small and simple subset of the total medical field so not sure why you bring it up.
(This is my understanding as a layman, so I’m happy to be corrected if wrong.)
I had a 55+ year old dentist who wanted to pull out my tooth and replace it with a bridge. The tooth was cracked and 1/3 of the tooth had fallen off. Luckily he worked only half time and was on vacation a lot so difficult to get an appointment so I got to see a young dentist who could fix the tooth by repairing it (building up the missing part). She said it could last a month or forever, but if not lasting then the next option would be a titanium implanted tooth. Making a bridge would have been considered malpractice nowadays she said. The old dentist was probably a master at putting in bridges, but if that method is bad and he never practiced the other technique, which you probably cannot learn in a 1 week yearly course, then tough luck.
Not the perfect example maybe, but imagine you have learnt assembler and C in school and programmed in that for 30 years doing say embedded applications. You maybe update yourself regularly in that very field of assembler and C. Then one day you are asked to program a web app using HTML, javascript and CSS and complete it in the same time and with the same quality result as someone who has learnt it in university and with a few years work experience. Quite hard.
Fast forward to today and the NIH director and leading scientists believe this has backing proof.
https://www.ucsf.edu/news/2024/03/427241/covid-19-virus-can-...
https://www.science.org/content/article/long-covid-trials-ai...
It has taken me almost three years to convince my doctor on the rationale of using therapeutics acting as COVID antivirals to treat my chronic symptoms. I now have to wait until clinical trials present data hopefully this summer to at least try.
So I hope this is a story to show that patients are helping doctors think. And it is very slow going.
The other major challenge is that certain approved drugs in other countries can be held by shipping due to FDA approval.
> Paxlovid targets virus that is rapidly replicating—which may or may not be the case for the virus lingering in Long Covid cases.
Viral persistence(2021): https://www.frontiersin.org/journals/microbiology/articles/1...
Persistent symptoms(2020): https://pubmed.ncbi.nlm.nih.gov/32644129
How many times have we all heard from a doctor "we don't like to do so many tests each year because of costs"
I’d guess a massive % of the us population is having a different experience than you.
I guess what I am getting at is that insurance sort of makes things worse for everyone here. When a person comes in with good insurance the doctor has incentive to abuse it, artificially driving up costs, making it harder for people with crappy/no insurance to get the treatments they need because insurance won’t cover it. It’s a messy system.
A large part of me would prefer that we abolish private insurance entirely, but I fear that’s not likely in the states.
If the cost would be low there would be no need for an insurance.
No, not at all. Insurance exists in plenty of places, without significantly influencing doctors' treatment.
In other words, an insurance market that does not influence doctors’ behavior seems to be a pretty inefficient insurance market.
It is one huge reason I work as a federal employee -- FTCA helps me feel free to do what I think is right for my patient without worrying nearly as much about getting sued. In my line of work, a tremendous amount of absurd testing is done because there could be a 0.01% chance of missing something serious, and if you see 1,000 pts per year and plan to have a multi-decade career, it's easy to justify almost certainly unnecessary testing.
These all seem like valid, but unrelated complaints. On one hand, there is a complaint that doctors are judged subjectively, rather than objectively. But in the next breath those same objective metrics are skewered as deductive and maybe even greedy.
The current attending physicians, according to the author, have a population-wide cultural problem where they don't believe they should have to teach. The author's solution to this problem is to hire more attending physicians.
Many of the issues raised in this article merit investigation, but this piece feels too eclectic in its thesis to serve as a launchpad for policy direction.
It taught how to memorize stuff: spaced repetition and semantic encoding. Spaced repetition is reviewing the thing at increasingly spaced intervals of time. Semantic encoding is coming up with connections to the idea. The wilder the better as that tends to be memorable.
The class taught some strategies for creativity such as use of analogies and trying to combine disparate ideas.
The class also taught cognitive biased, like loss aversion.
This class was life changing. It was also easy because the teacher applied the best practices she was teaching.
I would say something like that class is the basis for teaching people how to think.
Unfortunately it seems mostly to be a money grab in many specialties, with passing rates in the high 90s for recertification exams, fees of over a thousand dollars, and often with multiple retries if one fails (as long as you pay the fee for each attempt).
This is one of those assumptions that I didn't notice I was making till it was pointed out.
It is hard to predict capacity for mastery. It seem natural to adopt initial understanding speed as a heuristic, but I think it is the kind of heuristic that while effective, also tends creates a significant overestimation of it's own effectiveness through confirmation bias.
Don't get her argument that someone willing to show up to a 7:30am class is necessarily smart enough for o-chem. Tendentious
On the one hand, the author is on to something. Most people learn much better with intensive, one-to-one instruction than with large lectures. The real issue, as with all educational programs, is the cost in time and money to teach. More personalized, intensive training from more specialists in teaching a specialized subject simply costs more to get the to the same goals as bulk lectures. Nobody wants to say "There was a very rational cost-benefit analysis and I lost out". Instead, it's all framed as a need to dedicate more time and money at an already demonstrably drawn-out and expensive process.
On the other hand, a person's ability and willingness to show up at 7:30am are probably irrelevant. No matter how relevant it feels to a person who wants to show their dedication, passion, and drive.
The author is essentially arguing at length for a greater emphasis on benefit and less on cost. Not just in o-chem, but at every stage of medical training.
The point was that learning the fundamentals was prioritised, instead of "deep learning". However there's a ton of research that suggests that in many cases, learning the fundamentals is one of the best ways to get students to start the process of deep learning. Obviously there's a point where overtraining the fundamentals is no longer a good thing (e.g. trying to memorise every possible edge case and combination) but that's a rare edge case. In most cases, simply getting the grips with the basics quickly, then thinking for yourself (e.g. looking at hard cases rather than asking how someone can teach you how to think at a higher level) is what works.
Education has a long-running holy war between implicit vs. explicit instruction (though "implicit instruction" has a lot of name changes as it always seems to lose credibility and need rebranding). Saying "we need to stop teaching students what to think, but how to think" isn't deep, it's a cliché, and it needs a lot more than vague criticisms of explicit instruction to be worth listening to.
Probably the worst thing about the "deep learning" crowd is that so many of them are in medicine, where it kind of works. You can teach medical students badly, and they'll figure things out. Giving them more independence and teaching badly (while pretending to be wise) can perversely work, in some ways, for medical students who've survived the weed-out classes. But then a few academic studies on how to teach medical students better (apparently they haven't yet learnt how to learn, or how to think critically?) is then used to convince politicians, education academics, and other people who understand little about teaching that it's the best way to teach reading in underprivileged elementary schools.
LLMs should help with that.
In my opinion a well built system will replace all doctors in non-research, medical functions.
The notion that somehow one needs "vibe" to recognize something is wrong when Gladys in the example "vomited a huge quantity of blood" is concerning, and wee bit self-aggrandizing. The doctor reacted because there was clear evidence of an issue, not because supernatural abilities.
> A doctor’s foundational clinical mental models are built during residency...
And, sad to say often stays that way for the rest of their careers. Anecdotally, I have seen this clearest with dentists and ophthalmologists. They will fossilize into their own thinking and unable or unwilling to pick up new methods and procedures (unless sufficiently incentivized by the selling company).
another problem is frailty. I walked out on Monday dental appointment when I could tell the dentist had a "rough weekend". I do not need a person with a hang-over digging in my mouth with sharp spinning objects.
A system that is updated near real time with new research, new medical advances, not getting tired or distracted is more preferable to me than a human. Maybe the actions are not performed by the machine, it just prompts the human to do and feed the information back, maybe it will also perform the actions. All is possible in the future.
When it comes to research and emotional interaction, yes, doctors still hold an edge.
One bit puzzles me though, why would human doctors have an edge in research? Wouldn't the "person" who can read a trillion times more, remember everything, cross-reference it, and monitor billions of realtime data points (from billions of patients) for patterns, have a slight advantage?
Don't get me wrong I think ML systems will be great helper systems but I don't see them replace doctors any time soon, if not for the simple reason that a person is much more likely to trust another person than an electronic system and that is actually part of both diagnostics and treatment
I'm an anesthesiologist, so I don't have a clinic at all. I meet you just before surgery and we go. Vibes and spidey-senses matter a lot, because I don't have time to run down a two-week-long investigation of your problem - I have to fix it right now or you're dead.
Could AI improve me? Probably. Can the AI observe the way humans can? Not yet. Can they intervene like humans can? Not even close. When you have an autonomous robot that can successfully run down the American Society of Anesthesiologists' difficult airway algorithm, you're getting there.
For the record: The doctor stayed in the room due to a “spidey-sense” or “vibe”. The impressive part was predicting the vomit, not reacting to it.
Btw anyone knows what happen in Korea?
1. There are some attendings who have a subconscious inbuilt assumption that there are "good" residents and "bad" residents and that any team will contain a mix of both, even if it's a team of just two residents. God forbid that you should be just a "good" resident paired with a superstar resident, because then you are the one who must get the "bad" label by default. If this happens early enough in residency, it may be the first piece of feedback that your program director gets about you, which puts you at risk of being labeled "bad" by the program at large and finding yourself being pushed towards a remedial track. I feel that residency programs are shockingly bad at identifying which residents actually need remediation. Residents with serious knowledge or work ethic issues can get ushered along through the program, while very bright residents who don't understand how to play up their own successes get put on remediation plans.
2. More generally, confidence is often misinterpreted as skill, and introversion is often misinterpreted as a lack of skill. Many attendings are shockingly bad at differentiating the two.
3. Departmental culture has a huge effect on residency education. I trained at a busy county hospital and now practice at an ivory tower type place. Much to my surprise, I think the EM program here (which gets residents who were the cream of the crop of their med school classes) does a worse job of treating patients than the county hospital (which failed to even fill its residency class a couple years ago). The difference, at least in my view, is that attendings at the county hospital were extraordinarily hands on, which taught residents that a "normal" ED physician is extraordinarily hands on. As another example, the ED at my current hospital is very aggressive with CT scans, to the point that probably 20% of the patients that I admit have been panscanned (CT scan from head to pelvis). I attribute this to attendings here being more risk averse in cases with a ~1% possibility of malpractice. And it trickles down to the residents too, because again, that's what their reference for "normal" is.
4. Basic science education in med school is a joke and needs to be overhauled from the ground up. As it stands, 1st year is "pure" basic science, 2nd year is learning about the library of various diseases, and 3rd year is learning how to treat those diseases. I still remember reading the nephrology section of my 1st year textbook, a large chunk of which was a breakdown of why cystatin C is a better indicator of GFR than creatinine and a description of a procedure in which contrast is injected into the renal artery and sampled at the renal vein to measure true GFR, worked out step by step with mathematical equations. What was never mentioned: What the hell is GFR even used for anyway? Why would someone want to measure it? These pieces don't fall into place until 2nd and 3rd year.
Imagine if you learned symbolic algebra in 3rd grade but the concept of a word problem wasn't introduced until two years later. That's essentially what medical school basic science education is.
The faculty who design med school curricula are not blind to this issue, but their attempts at fixing it are laughably bad. There is "case based learning" where 1st year students are told a vignette about a hypothetical patient and are asked to generate a differential (a list of possible diagnoses) before they dive into the basic science component. This turns into a comedy skit show because 1st year students barely know any diseases, they are only vaguely familiar with the symptoms even for the ones they do know, and they haven't had much, if any, explicit teaching on how to generate a differential.
Why not take a more vertical slice approach to teaching? Introduce a limited number of bread and butter diagnoses in 1st year and gradually layer on rarer and more complicated diseases in 2nd and 3rd. Put more weight on the basic science aspect in 1st year, put more weight on aspects that require knowing a large number of diseases (like generating a differential) in 2nd and 3rd year.
I want to force every software engineer and manager and MBA and anyone else to read this until they've internalized it.
The business side, though, is going swimmingly, so there's that, at least.
Medicine perpetuates a hazing culture of overworking residents thanks to a jumped up medicine pioneer who took too much cocaine. They refuse to come up with a way to train doctors at scale and instead restrict it to physical residency spots, keeping their compensations high. Train doctors like engineers, break up the medical unions and lobby groups with the anti trust act and most of the problems will be solved.
The only choice you really have is to leave medicine. After dedicating ten years of your life to it's study, a fucking huge debt and no guarantee of finding a better life elsewhere.
You want lots of sick people to treat a lot. Doctors being short on time etc, is fine - it's business.
In Canada most fully-trained doctors are paid per patient (AKA fee-for-service)[1] and so there is a huge incentive to rush through as many patients in a day, which results in overwork for the residents (who are salaried) and nurses (also salaried), and no time for adequate education of medical students and residents.
[1] https://www.dr-bill.ca/blog/billing-tips/physician-payment-m...
The failure to divorce your health insurance from your job is what created the mess we have today. It arose from wage caps during WWII, and instead of fixing the defective legislation at the time... Congress just let it fester until it became the life-ruining disgrace that Americans live with daily.
With the strident support of fools who believe that they're getting "free" insurance from their employers... and anything else insurance companies tell them, legislators, lobbyists, drug companies, big corporations all line up against affordable health insurance. Big corporations get workers chained to dead-end jobs by their insurance. Insurance companies and big pharma get windfall profits because the real costs to consumers are buried behind "free" insurance that is profoundly not free.
And nobody in government or politics has the balls to call this out. Subsidizing this rip-off is, in some ways, even worse. Just like subsidizing the obscene rip-off that college has become.
This is straight up fraud and should be treated as such.
My wife wanted to be a nurse her whole life, went to school and got a good job, after 3 years at the hospital she had to quit for her own sanity. Every nurse and resident in her circle had a similar story.
U.S. hospitals are for profit assembly lines. Managers with MBAs that have never worked a shift on the floor in their life see staff as fungible. Throwing a pizza party when patient quotas are met, instead of you know, hiring more to relieve overworked and stressed employees.
Seemingly every motivated and caring person in this field is systematically chewed up and spit out.
In a non-profit, the board uses arbitrary criteria, the board is incentivized to waste all money on executive compensation, because that means the execs throw them excellent parties, galas, soirees, and other social events where they can network with more people to make more profit in their for-profit ventures. Ask me how I know.
Similarly I have a friend whose spouse is extremely talented with a PHD (not MD), and has worked in admin roles at a big nonprofit NYC hospital as well. They have at times had pretty similar compensation, one working at a hedge fund and the other working at a nonprofit hospital. Interesting that. Each side can argue they need to attract talent, and one can argue that hospital admin is closer to doing good for society. It is just surprising to see nonprofit and very high compensation.
In no way does this mean no non-profit should ever pay someone well.
> Every nurse and resident in her circle had a similar story.
Most don't quit after 3 years.
> U.S. hospitals are for profit assembly lines.
Absolutely true, regardless of the actual IRS profit classification.
Running a residency program takes resources, staffing, and people. If you expect everyone in the doctor education and residency chain to work for free, there wouldn’t be much of a residency program to speak of.
It’s a reality of life: doing things within a business takes money. I know some people scoff at the idea of health care being a business, but even in government-run programs there are still budgets to be managed and costs to be paid.
> U.S. hospitals are for profit assembly lines. Managers with MBAs that have never worked a shift on the floor in their life see staff as fungible. Throwing a pizza party when patient quotas are met, instead of you know, hiring more to relieve overworked and stressed employees.
Your view is very US-centric, but this is a common story in other countries too. Healthcare is a very complicated and demanding field, and it has high turnover rates outside the US as well. The harsh reality is that these organizations are operating like any other: They compensate as necessary to reach the supply/demand equilibrium. As long as new people continue to seek out and take the jobs at a high enough rate to keep the organization running, they’re not going to arbitrarily increase compensation.
That’s always the answer: It’s supply and demand equilibrium. We can complain all we want about who “should” be paid more or similar musings, but as long as the jobs are filled sufficiently at the current rate and the system keeps chugging along, that’s how it will be.
They have to cover overhead for that person. In e gineering that adds 30 to 50 percent. I think doctors have additional overhead, like malpractice insurance, and since this is an educational environment more oversight. 2x doesn't seem bad.
It's like that in almost every field.