Comment by tpoacher | Hacker News Reader

Comment by tpoacher | Hacker News Reader

Both the article and some of the discussions here share a lot of commonalities with doctors taking a medical history. There is a certain skill in guiding the conversation towards useful outputs, while not dictating the exact outputs to a patient who is eager to please with their responses.

E.g., medical history taking protocol always says to start with open ended (albeit structured) questions, and converge towards more closed/specific ones when you're sure you've extracted the broader surface and you now want to close in on a differential diagnosis.

If you start open and go with the flow but then just let the patient talk without any structure or subsequent attempt to converge, there's a risk that the patient might spend 60 minutes taking about their fluffy dog at home, which wastes time, and doesn't get you anywhere nearer the diagnosis. But, if you skip the open questions and go straight to yes/no diagnostic questions, you will definitely miss the fact that they have a dog at home that they're worried about, and that they'll be self-discharging against medical advice in the next hour to go tend to their dog.

So while to an outsider, the conversation might look effortless, in reality the doctor requires considerable skill to be able to strike a balance between open vs closed prompts, as well as the ability to critically sift through the outputs, and decide which outputs are relevant to pursue further and lead to a fruitful direction, versus those that can be safely discarded to remove potentially distracting noise from the conversation (and all while attempting to keep this interaction within a limited number of prompts due to operational time constraints).

Counterpoint, some doctors will zoom in on the most likely problem and misdiagnose. This is in part due to pressure on the health care system (where I live anyway); you can only get a GP appointment for 10 minute blocks, which really isn't a lot.

But when a 30-some year old shows up at a rheumatologist with joint pain they will likely go to unusual (at that age) but not unheard of rheumatism/arthritis, not hypermobile spectrum disorder. When a woman goes to a GP with period pain they will be prescribed mild pain killers or anticonception pills and fobbed off, until a decade and much suffering / many more issues later they get diagnosed with endometriosis.

I think endometriosis is common enough that it would be caught earlier ("Oh, does your joint pain get worse right before you menstruate? Tell me more"), but your point is reasonable.

I counter with the platitude that common things are common - especially in fields like primary care, the amount of wasted effort one would expend in pursuing unusual explanations for every presenting symptom is considerable. We thus have to examine patients over time and trust that they will tell us if things have indeed not followed the course of the initial diagnosis.

While this is true as a platitude, I do find that medical colleagues seem to treat this platitude rather overzealously at times, often completely disregarding the posterior probabilities involved. I like to call this the prior-over-posterior fallacy/bias (though one could argue this is simply a variant of the normalcy bias)

E.g., I get the whole "if you hear galloping think horses not zebras" adage, but I guarantee you, if someone comes and says "hey when I was in Africa I saw a black and white striped animal galloping really loud", I bet you an uncomfortably large percentage of the "horses not zebras" crowd would still favour a horse over a zebra diagnosis, despite the overwhelming posterior.

Combine this with our (otherwise reasonable) tendency of avoiding the information bias of seemingly unnecesary tests, and it becomes a big problem, whereby uncommon conditions are treated as common, thereby often escaping detection altogether, and driving down their apparent prior probability even lower, causing a diagnostic vicious cycle.

I hear the complaint, all I can say is that everyone I speak to about this wants to diagnose things correctly while not wasting anybody's time or energy on bad diagnostic workups.

> Combine this with our (otherwise reasonable) tendency of avoiding the information bias of seemingly unnecesary tests

There isn't a way around this: if you order the test and a value is abnormal, you are now committing yourself to treat a thing. We should not be ordering tests if we aren't ready to follow their results to the logical conclusion. So I would disagree that this is a problem in the way you've framed it.

I'm not disagreeing with what you say here, but I think we're arguing about subtly different things.

E.g. you'll note my zebra example was not about whether one should additionally request a photo of the animal just to make sure it is indeed a rare animal. It was arguing that given existing differentiating information that points to an uncommon condition in the first place, one should not dismiss this on the basis that horses are still more common than zebras in the general case regardless. Under this uncommon information, the prior of thinking about horses should go out the window, and one should absolutely focus on zebras (at least as an additional differential). I assume you would also agree with this conclusion.

But of course, in real medicine things are not as simple as this contrived example. So the point I'm making is that, from what I have observed, there seems to be a bias towards decisions based on "prior" rather than "posterior" probabilities, even in the presence of additional differentiating information which should have prompted additional differentials to be considered. But this is different to saying one should constantly seek additional evidence to include or exclude additional differentials that are unlikely in the first place. That, I agree, would be a waste of time and resources (and potentially harmful for the patient).

Having said that, I somewhat disagree with the phrasing that we should not be ordering tests if we aren't ready to follow their results to the logical conclusion. This is a bit like saying, "I don't want to check for X because if I do and it checks out it will derail my current management plan"; but then this is putting the cart before the horse, since it's the findings that need to dictate management, not the other way round. I do think it is reasonable to say that one shouldn't be ordering a test if the expectation that it will change management is sufficiently low to justify the time/cost expended for it -- and where this expectation is a function of both the likelihood of the finding (given current information!), as well as its potential to change management. But this doesn't mean that if you do find an unrelated inconsequential incidental finding you are now required to divert all resources to it.

Conversely, if an incidental findings does prove worthy of treatment, then I don't see the problem with committing to treat it, as long as you don't lose sight of the original complaint either. Obviously that doesn't mean one should go looking for incidental findings willy-nilly though.

I hear your hypotheticals about testing. I think this comes up often, by way of example, when somebody has been in the hospital for a long time. Long stays aren't always medical, they're sometimes due to social issues (adult children are out of town until Tuesday, so grandma won't be safe elsewhere and stays in the hospital).

We don't have a prior that checking her labs daily will have a medical benefit after the time we decided she should have been discharged. And there are some abnormal values in daily labs that are, at once, not uncommon to encounter and also too abnormal for the day of discharge. So we sometimes end up keeping grandma an extra day or two to give her more potassium while we have no idea what the day-to-day variation of serum potassium would be for a 'healthy' person similar to grandma.

This is a clear example of when we shouldn't order testing: there's no expectation of marginal benefit to us while there's a risk that we'll be forced to act based on the result. This is also an example of a time where a person might say, "Why aren't you checking her? What if she develops [x], [y], or [z] and you don't see it until she has symptoms?"

You'd think, but I have 4 friends who have been diagnosed, and they all have horror stories. This is in 3 countries, none of which are the US, but from reading Reddit, it seems the US has the same issue.

Indeed, I didn't mean to imply that all doctors possess that skill in great amounts (or that, in any case, circumstances allow for full utilisation of the skill even if they do) or that medical history taking is therefore always of the highest standard. Obviously some doctors will be better at it than others, or more at liberty to use this skill than others (as you say, I could imagine a GP who only has 10 minutes per patient won't be as keen to start with fully open questions as a junior doctor in a ward who can clerk patients somewhat more freely).

Note that this isn't too different from, say, how software engineers are expected to be good at, and make good use of unit tests. But most probably don't (either because they never really cared to fully develop that skill, or their organisations applied contrary pressures leading to tech debt). But it is a recognised skill.

My main point was that, it is, in theory, a skill that doctors are expected to train (or at least pick up on during their practice), and therefore the same prompting principles that seem to apply here in the context of LLMs also interestingly seem to apply to medicine and history taking when "prompting" and interacting with humans.

You can’t run expensive, time consuming, and potentially harmful tests on every patient. Someone comes in with a headache, you tell them to go home and take an aspirin. You don’t send them for a a CT scan.

If it doesn’t go away and they come back, you start considering more serious issues.

It’s expected that uncommon non-emergent diseases will be diagnosed over multiple visits.

Reply on news.ycombinator.com