Hacker News Reader

> I spent 10 years in mammography software, and I saw first hand how many outside factors can impact a physician's decision to biopsy or not a given artifact on an image.

This is slightly tangential, but I'm curious about your perspective on a classic example of medical statistical illiteracy. Whenever surveyed, the strong majority of doctors vastly overestimate the odds of a true positive mammogram (half of them by a factor of 10!!), due to flawed Bayesian thinking and the low base rate of breast cancer.[1]

Does your anecdotal experience contradict this data? If not, wouldn't two minutes of stats education (or a system that correctly accounted for base rate) utterly swamp intuition-driven analysis of tiny artifacts? Or is it simply that, through folk wisdom and experience, they implicitly adjust other terms in their mental calculus in order to account for this huge error in estimating one factor?

[1] https://blogs.cornell.edu/info2040/2014/11/12/doctors-dont-k...

The doctor diagnosing a patient isn't solving a puzzle of the kind posed here.

They are doing, as the previous comment said, interpretation. In practice, much of their thinking is profoundly rational and bayesian.

Human (and animal) thinking isn't primarily cognitive, ie., explicit reasoning. It is the application of learned concepts to sensory-motor systems in the right manner.

We don't look to dr's to formulate a crossword puzzle when a patient arrives; we look to them to be overly attentive to yellow skin when the patient's family has a history of alcoholism.

I'm not convinced this just couldn't be a large personal data set into an algorithm.

Doctors barely have any data as it is. I think personal bio testing and monitoring is going to be a huge market and medical paradigm shift.

would you rather have you heart rate and temp constantly monitored for months , or get it checked once a year by a GP to see if you have hypertension or any negative markers

With infinite amounts of perfect information and infinite computing time, all question-answer problems are trivially a hash-table lookup.

> In practice, much of their thinking is profoundly rational and bayesian.

Right, this was the third option I mentioned; I'm certainly not leaping all the way to the conclusion that one shouldn't listen to a doctor about the best course of action after mammogram results[1]. If their explicit understanding of mammography's false positive rate is so incredibly flawed, there is presumably an implicit counterbalance in the calculus that's built on experience (both their own and their mentors'/institution's), or an _order-of-magnitude_ error would show up in patient outcomes. I'd guess that this and the other instances of critical thinking failure that plague medical culture have their rough edges sanded over time, through decades of what is effectively guess-and-check iterative evolution, combined with institutional transmission of this folk wisdom.

Though I disagree that I would call this "profoundly rational", as IMO leaving explicit reasoning tools on the table instead of intentionally combining them with intuition/experience/epistemic humility is optimal. Iterative evolution is not an efficient process, and adding an attempt to explicitly model the functions you're learning can be a powerful tool. It's very difficult for me to imagine that a doctor explicitly internalized the basics of Bayesian reasoning wouldn't make them at least marginally more effective in terms of patient outcome, etc. Medical history is full of blind alleys built by medical hubris like your comment's "doctors know A Deeper Truth in their soul that defies explicit logical articulation". (Though I should note I don't claim to have a pat answer to this problem: one can theorize about improving a specific doctor's effectiveness, but scaling it to the whole culture is another, and can bump into everything from supply problems to downstream cultural impacts with negative consequences)

[1] Though with knowledge of flaws in such basic reasoning skills in one subpart of the total calculus, a patient can't rationally escape updating in the direction of checking their reasoning more thoroughly. Medicine is a collaborative endeavor between patient and doctor, both bring flaws in their reasoning to the table, and stronger evidence of a huge flaw in reasoning should lower confidence in the overall decision (though at a much lower magnitude, for the reasons we both describe here). This is the same logic that doctors use to rationally discount patient's opinions when they don't perceive them as coming from, eg, overly emotional reasoning.

Although we are tempted to think "automate away doctors jobs and take their salaries", it's more achievable and more valuable to provide existing doctors with better UI/UX which helps them make better decisions.

Bayesian probabilities could be displayed, other factors presented, and ML could be designed in an explainable way which helps the doctors. But that's too hard, requires too much talking to people, and too much respect for existing institutions, and not enough disruption! Sigh...

Yes, I agree. I was responding narrowly to the parent comment's premise, about full replacement (in theory). It's still useful to think about as a theoretical threshold, even if the path to commercialization is almost definitely through assistance.

This isn't really surprising to me considering that it's not usually a doctors job to estimate the odds of a true postive - false positives are far less of a problem than false negatives when doing an initial scan for cancer.

Biopsies aren't cost- and risk-free, including from a patient health and mental health perspective. On top of that, the decision space isn't simply binary: there are multiple types of biopsy procedure, and understanding (even implicitly) the probabilities in play can guide those decisions.

The Cult of the Doctor never ceases to amaze (and frankly disgust) me. I think it's absolutely a noble profession, and that doctors are generally smart and conscientious. Even some of the classic complaints are understandable. Eg, my incredibly kind, humble, and open-minded brother-in-law occasionally betrays some extreme disdain for his patients, a common complaint about medical culture. I continue to be a little put off by it, but I understand that he has to deal with some incredibly stupid things from most patients, and it's easy to see how taxing maintaining the welfare/autonomy balance can be. I try to keep this in mind whenever a doctor has an initial defensive reaction to my attempts to be an engaged and informed patient.

But, the lengths to which people bend over backwards to make flimsy excuses (like your comment) for massive errors in reasoning is just astonishing. I can't think of a single other context in which people so desperately clutch their pearls at any suggestion that a professional culture or institution could be improved.

Doctors are people, and people have flaws. They go through a training and acculturation process, which may also have flaws. This shouldn't be controversial. The only reason I can think of is that patients are no better at dealing with uncertainty than doctors are; the idea that you're being treated by a deity levitating in absolute epistemic certainty is more comforting than the idea that you're dealing with a smart, conscientious, trained human that nonetheless has human limitations (and that the field of medicine itself has plenty of uncertainty).

In my field (space) we have an unspoken mantra that autonomous systems should aid human decision making.

It's just so much easier to build a system that allows a human to focus on the tough calls, than it is to build an end-to-end system that makes all the decisions. Only in the most extreme examples does full autonomy make sense.

If there were one doctor in the world, I'd build an autonomous mammogram machine and have him periodically audit the diagnoses. Otherwise, better tools is the way to go.

I noticed this when visiting the OBGYN for sonograms to check the development of our children. The tools are really good. You can easily measure distances and thicknesses, visualize doppler flow of blood, everything is projected from weird polar space (what the instrument measures) to a Cartesian space (what we expect to see), and you can capture images or video in real time.

Sure, the cabal factor is real, as is the curmudgeon doctor, but I think we should be building tools, not doctors. We know how to build doctors.

Building tools to aid humans seems like the best of both worlds. This is already happening in some radiology subspecialties: The autonomous systems can highlight potential areas of interest but it’s up to humans to make the final call. For some cases it’s a quick and easy call, but for tougher calls the radiologist can bring other factors into consideration.

A counter example from your field: Would a human add value adjusting the boosters to vertically land starship?

Not a counter example, but a legit question in spirit since autonomy vs human will always be a reasonable choice for some parts of a system. I just don't think real-time control of a system that's never been flown by humans is a good example of where that choice is hard. You can describe it with equations, solutions are well understood, there's no available humans that can do it, there's a well-understood need because fly by wire was necessitated to control some aircraft, and will be necessary to control more air/spacecraft in the future. Classic computer problem.

(But I'm guessing Starship will have human controllers in/on the loop to do interventions before or during landing if required.)

Doctor diagnosis is a good example of when that choice is hard to make. Hard to describe with equations, hard to model, not a well-understood need since professional doctors exist, decades of experience has been accumulated, huge swaths of industry are dedicated to helping doctors, etc etc.

What I meant by "Allow the human to focus" definitely should be interpreted to mean removing small, multi-dimensional, fast, annoying things like real-time stabilization during landing.

I see the same thing in natural language processing. A lot of important details come from outside the four corners of the document.

Ironically, I often find myself in the unenviable position of being the machine learning person who's trying to convince people that machine learning is probably not a good fit for their problem. Or, worse, taking some more fuzzy-seeming position like, "Yes, we could solve this problem with machine learning, but it would actually cost more money than paying humans to do it by hand."

Part of why I hate hate hate the term "AI" is because you simply can't call something artificial intelligence and then expect them to understand that it's not actually useful for doing anything that requires any kind of intelligence.

You are absolutely correct. In fact, most NLP software ignores the formatting of documents which conveys a lot of information as well. For example, section headings must be treated differently from the text that makes up the body of a section. Its very hard to even determine section headings and then its hard to take advantage of them since the big transformer models simply accept a stream of unspecialized tokens.

There's an old AI joke that all actual problems reduce to the artificial general intelligence problem---everything else, by definition, doesn't require intelligence.

There's some truth to that. But I'd also argue that there's a tendency to try to bill every single kind of spinoff technology that the artificial intelligence community has produced as artificial intelligence.

Which a bit like characterizing mixing up a glass of Tang as a kind of space exploration.

But isn't this just what ML should be good at - taking a huge number of data points and finding the patterns? Or are you saying it's not an issue of ML working poorly but rather one of there not being a good enough data set to train it on properly?

One of the main tasks that doctors do is take patient's vague and non-specific problems, build up a rapport with them, understand what is normal and what is not, dealing with the "irrelevant" information present at the time, and focus the results into a finite tree of possibilities.

In principle, this would be a great task for an ML algorithm. It's all conditional probability. But every such system has failed to do that well -- because the "gut feeling" the doctor develops is funded by a whole host of prior information that an ML algorithm won't be trained on: what is "normal" for an IV-drug addict, or a patient with psychosis; how significant is the "I'm a bit confused" in the middle-aged man who was cycling and came off his bike and hit his head? Do the burns on the soles of this child's feet sit consistently with the story of someone who ran over a fire-pit barbecue "for fun", or is it a non-accidental injury? It's a world of shades of grey where, if you call it the wrong way, ultimately someone could die. Doctors do Bayesian maths, and they do it with priors coming from both their own personal experience as a member of society, and professional training. That is, in my ignorant opinion, the main distinction between what I do -- oft-called "academic medicine" or "academic radiology" -- and clinical medicine. The former looks at populations. The latter looks at individuals.

In other words, I don't think it's even possible to codify what the data the ML algorithms should be trained on -- they're culturally specific down to the level of an individual town in some sense; and require looking at huge populations at others.

> In other words, I don't think it's even possible to codify what the data the ML algorithms should be trained on -- they're culturally specific down to the level of an individual town in some sense; and require looking at huge populations at others.

This is inciteful. ML is by definition generalizing, and should only be used where it's ok to generalize. There is an implicit assumption in use cases like medical diagnosis that there is a latent representation of the condition that had a much lower dimensionality than the data and that the model is trained in such a way that there are no shortcuts to generalization that miss out on any information that may be important. The second condition is the hardest to meet I believe, because even if a model could take in lots of outside factors, it probably doesn't need to to do really well in training and validation, so it doesn't. The result is models that generalize, as you say, to the population instead of the individual, and end up throwing away vital context to the personal case.

I also believe this is an important consideration for many other ML applications. For example those models that predict recidivism rates. I'm sure its possible to build an accurate one, but almost certainly these models stereotypes in the way I mention above, and do not actually take the individual case into account, making them unfair to use on actual people.

On codification:

I actually disagree with this, but only slightly.

Imagine if instead of face to face, doctor transactions were via text. The questioning of the doctor can be monitored and patterns in decision trees observed could be codified, and weighed against the healthcare outcome (however defined).

What is missing, however, is the Counterfactual Reasoning. The "why" matters. The machine cannot reduce the doctor's choice of decision trees from all possible combinations, only that which it observes the doctor perform.

Tail-cases like rare genetic disorders would often be missed.

Tail-case like rare genetic disorders are often missed by doctors too. I have several friends who had Lyme disease with fairly serious complications (in the Northeast) (Not that Lyme disease is that rare - it's actually much more common than is expected). Each of them got misdiagnosed for multiple years by multiple different doctors until finally getting the correct diagnosis/treatment. So every system is fallible.

Personally, my exlerience with doctors is neither rapport nor nuanced understanding of my specific situation.

That is really not what they do, are trained to do or have time to do.

"There's not enough good data to train it on properly"

Bingo: You're in _very_ data poor environment, compared to something like predicting a consumer's preferences in videos or identifying and segmenting a bicycle. The external data is also very qualitative and hard to encode into meaningful features.

Right, but "the training data is bad" is a very ML centric way of looking at the issue. It pushes all the difficult parts of the problem into the "data prep" sphere of responsibility.

Note that there are different ways in which data can be bad (i) image resolution not good enough, too many artifacts and noise (ii) its woefully incomplete, doctors collect and use information from other channels that aren't even in the image, regular conversations, sizing up the patient, if the doctor knows the patient for a long time then a sense of what is not normal for the patient given his/her history etc., etc.

Some of the issues that have been discussed in the thread can be incorporated in to a Bayesian prior for the patient, but there is still this incompleteness issue to deal with.

The first step would be to build an information collection pipeline that is in the same league as the doctors. That alone will be a monumental effort because doctors have shared human experiences to draw from and they are allowed to iteratively collect information.

I'm just complaining that it seems fantastically reductive to call the absence of such a pipeline "bad data" because developing such a pipeline would be a thousand times the effort of implementing an image detection model. Maybe a million times. It will require either NLP like none we have seen before or an expert system with so much buy-in from the experts and investors that it survives the thousand rounds of iterative improvement it needs to address 99% of the requirements.

Comparing issues like low resolution and noise to such a development effort seems like comparing apples to... jet fighters.

How else would you describe the issue?

Structural. The problem hasn't even been correctly formulated yet -- and it will take an enormous amount of work to do so.

The gap is interpretation and application of those patterns. Building expert systems is expensive, but ML hits the low hanging fruit of showing patterns to experts out of the park.

ML should be good at drawing basic conclusions. End users are misunderstanding the boundary between basic and advanced.

Or, to put it another way, everyone agrees there's a difference in value and quality of output between an analyst with 1 year & 10 years of experience, right? So why are we treating ML like it should be able to solve both sorts of problems equally easily?

I have faith it will get there. But it's not there yet, in a general purpose way.

Because people like Hinton are outright saying that it already is there.

Incorporating more variables into the model wouldn't be sufficient. You would also need to get the input data for those variables into a form that algorithm could consume. Often the raw data simply isn't recorded in the patient's chart, or if it is recorded it's in unstructured text where even sophisticated NLP struggles to extract accurate discrete clinical concept codes.

Mammography is one of the most difficult to interpret. You need more data like the age and family history to decide on the next step.

Radiology is huge. I am sure ML can help in some of the specialties (it does not need to be all or none). The reason it is not is because of the medical system refusing to give in.