Hacker News Reader

Comment by tedivm | Hacker News Reader

This is actually why it isn't ideal for AI to tackle. The fact is there's no market for the product.

Right now much of the mammogram reading is extremely straight forward. Radiologists can fly through those exams with high quality, especially when you consider the macros they have setup that allow them to write most of the report with with a two second voice command.

This is one of the biggest things radiologists complain about- from their perspective most AI is solving problems that aren't actually useful to radiologists. I think the companies that will do best are the ones that augment radiologists with a focus on making them more accurate and efficient, rather than trying to cut them out of the loop (I'm a bit bias though as my previous company, Rad AI, is taking the augmentation approach and it has worked out rather well).

View on HN

> This is actually why it isn't ideal for AI to tackle. The fact is there's no market for the product.

There's a difference between: not ideal to tackle and not ideal to build a business around.

It is absolutely ideal to tackle which will hopefully build confidence in a system that can then be expanded to solve harder problems. If you can't even solve this problem I don't see how you're getting public confidence in the harder ones.

Yeah this also third world needs this kinda solutions where you can replace specialist with algorithm.

Why? The defining feature of the third world is cheap labor.

The third world might need more effective training, so they can convert more cheap labor into specialists, but I don't see why per se they would prefer to use a computer sold at Western prices to a local specialist.

"Cheap effective training;" Pick two.

> The fact is there's no market for the product.

That's empirically not true; the first such product approved for clinical use entered the market in the late 90s, and they've had small-to-medium success since. Definitely none of them changed things radically, but it's pretty common CAD functionality now and has both detractors and champions.

> Radiologists can fly through those exams with high quality,

This is also not entirely true, one of the biggest problems for screening (not diagnostic) programs is that the false negative rate for radiologists is high. They do fly through these scans, but that's to make it economically feasible. Also, the average radiologist performance on screening mammo is not very good - most have to do it regularly (probably 100s per month) to do it well.

You are right that augmentation is the more plausible path to clinical success, but that's been what most of the clinical systems aim to do anyways, provide a sanity check on the radiologist.

I have no idea why everyone tries to solve the hardest problem (true positives, no false positives) with machine learning, instead of solving the much easier problem (true negatives, no false negatives) that often delivers similar value.

E.g. If a system could tell me which 967 results out of 1000 are normal, and send the other 33 for human review.

True positives require solving every problem in the solution space. True negatives only require solving the most frequent problems.

If I remember correctly this was tried in Europe ages ago, liability concerns made it impractical in US.

Why do I feel the AMA was a bigger problem than liability?

Plus--wouldn't the software company be liable? (I don't think the tech is ready for office use, but when it is--I still think it will be a battle.)

Kinda like all medical devices. Whenever there's a problem, the equipment is looked at first?

Personally, I don't think the AMA wants computers taking away their doctor's precious income, and will only acquiesce when the tech makes the doctor more money, or is so good politicians/insurance companies start demanding it.

Insurance companies will be the ones to push it. And they'll push it when the government (through Medicare) mandates it / cost reductions.

But IMHO medical law in general has a big device / software blind spot. It's become decent at figuring out liability for humans, while permitting normal operations.

But all that machinery and case law doesn't really exist for black box software systems (no human in the loop). And if there's one thing that medical providers and insurance companies hate, it's unbounded liability.

Just commenting to agree with this. I've done a lot of ML use case ideation work, and what happens so often is the ML folks latch on to something that ML may be able to do, but that doesn't move the needle at all in terms of doing the work faster or better.

Often, the ML provides an extra layer that slows things down, even if it's working at state of the art AUROC or whatever we're trying for.

I'm really surprised by the pushback you're getting. I'd be curious the background of the people that have come back with replies about how it's an ideal use case, etc. This misunderstanding of the problem domain is exactly why these projects always fail - people come up with a "solution" that solves something that isn't the problem, despite whatever clever horse analogies people can come up with.

Is there an argument to be made that decreasing the workload on radiologists could improve their results in more difficult cases? I don't think it's a stretch to imagine that less tired/busy doctors would be able to perform their job more effectively.

There's also the fact that radiologists get paid a pretty decent salary. If I could devise a system that can do a highly paid worker's job faster, cheaper, and at a similar level of effectiveness, why wouldn't I? It also opens up that skilled worker's schedule and allows them to tackle more difficult to interpret scans.

> I don't think it's a stretch to imagine that less tired/busy doctors would be able to perform their job more effectively.

You are suggesting something that's against the groupthink for that profession. You can't graduate from an MD program while holding these thoughts.

I'm being 100% serious here.

> You can't graduate from an MD program while holding these thoughts.

By your logic I must not exist then.

Like who do you think is labeling your training images?

Look at the authors of pretty much all cited studies in that meta-analysis. Sure see a lot of MD and MD-PhD there.

That argument is exactly the one made by my former company, Rad AI. Rather than focusing on trying to "solve" radiology they're focused on augmenting radiologists. Their main goal is to reduce the amount of time a radiologist spends on each report while increasing accuracy and reducing errors.

The point I'm making is that focusing on mammograms is not the way to do that. It's already a very optimized specialty. If AI researchers worked more closely with practicing radiologists (not academic ones) I imagine there's all sorts of areas where existing technology today can improve things.

I think my point is that mammograms are a (relatively) easy to solve problem using today's technology. Why wouldn't it be worthwhile to solve that problem such that it has very limited human involvement?

It's the reason professional baseball is experimenting with electronic strike detection, despite the fact that umpires are very accurate (~94%) calling pitches.

I get that there are useful applications of AI that could augment a radiologist. I'm simply pointing out that if one could completely remove a radiologist from the equation, that could be a super helpful tool. I suppose my view is that removing/limiting the human element in routine inspections is a very worthwhile goal.

There's also the side benefit of increasing trust for these technologies. I think a lot of people are rightfully scared of AI technologies; showing that they can be as effective as a radiologist for routine screenings would be very beneficial.

"If AI researchers worked more closely with practicing radiologists (not academic ones) I imagine there's all sorts of areas where existing technology today can improve things."

That's almost certainly true, and surely there must be people in the medical industry already working on this. In hospitals, there are internal organisations that vaguely attempt to link up technology workers with practicing doctors to improve medicine using technology. One of my most competent former developer colleagues retrained as a doctor and worked in medical digitisation, as well as being a practicing doctor.

However:

* Often coming at a problem from another angle and ignoring existing professionals can be useful too. Existing workers are very often stuck in their ways and unwilling to accept the way they look at a problem may not be the best. I have heard from many in medicine, that doctors are extremely distrustful and hesitant towards technology, as well as fiercely protective of their jobs.

* The medical industry is internally highly dysfunctional and trying to work with practicing doctors can be extremely politically difficult. The hospital bureaucracy often actively tries to prevent practicing doctors from working directly with technologists, preferring instead to silo them and direct all communication via senior management - who are often neither technology nor medical experts. Consequently it seems more likely that major changes will come from outside.

* AI has occasionally (rarely?) shown great success in solving problems without needing domain experts. See AlphaGo.

> AI has occasionally (rarely?) shown great success in solving problems without needing domain experts. See AlphaGo.

Games are always orders of magnitudes easier then analyzing reality. They are highly constrained to the narrow domain and space of possibilities. Reality doesn't. There is work on understanding physical laws by the AI but it's very long path and they are much less successful than their artificial reality colleagues.

In just a few decades we've gone from 'chess is ai complete' to 'go is ai complete?' to 'games are too easy to be meaningful.'

In game domains we have the ability to generate infinite meaningful training data and have easily measured outcomes. Waymo is generating near infinite data for driving, and doing quite well with it. In medicine, we have piddly ass datasets because of privacy concerns. 'Reality is hard' had nothing to do with it.

Reality is hard is still a thing. Google has the ability to spends billions (or 10's) on something like Waymo and they still have to spend billions (or 10's) more to properly commercialize it.

This isn't going to be a commonly done thing.

> The medical industry is internally highly dysfunctional and trying to work with practicing doctors can be extremely politically difficult. The hospital bureaucracy often actively tries to prevent practicing doctors from working directly with technologists, preferring instead to silo them and direct all communication via senior management - who are often neither technology nor medical experts. Consequently it seems more likely that major changes will come from outside.

This is one of the areas where working with radiologists actually makes things easier. Radiology groups are normally independent practices that contract with multiple hospitals- they link in with hospital systems but mostly run their own thing. They also tend to have really solid IT departments and enough funding to make things work. Working with hospitals is a nightmare though, as you say.

problem is unless this AI is 100% perfect it will not provide any value to a radiologist. it is their reputation on the line with each interpretation.

to have them read an AI's diagnosis before having to assess a mammogram themselves to confirm the AI's finding is adding an unnecessary step to their workflow.

> This is actually why it isn't ideal for AI to tackle. The fact is there's no market for the product.

I think you're disagreeing with a point that the parent comment didn't make.

It's worthwhile in general to continue AI research. The properties of this application make it ideal for testing and progressing AI technology. But if AI doesn't end up being used in this particular area in the short or medium term, that's another matter entirely.

Hey I've also worked in a previous company that was doing exactly this for DK, UK, etc. As far as I remember one of the bigger things there was tissue density and masking effects where it might become not as clear cut for a radiologist. We saw a lot of opportunity in that area and in prognosis- the whole health economics aspect of frequency of visits.

> tissue density and masking effects

But those are also harder for software based systems.

There are a bunch of countries outside the US where access to high quality radiologists is a significant problem.

But yes, augmenting human intelligence instead of substituting it often does work better.

> Right now much of the mammogram reading is extremely straight forward. Radiologists can fly through those exams with high quality, especially when you consider the macros they have setup that allow them to write most of the report with with a two second voice command.

That to me sounds like an ideal automation use case. You have an entire class of highly skilled professionals performing a repetitive task. Sure, they may be able to do it quickly, but I imagine they must have to do a bunch of them each day.

You could imagine a world where hospitals didn't have radiologists on staff, but rather they acted in a consulting capacity for difficult cases.

> That to me sounds like an ideal automation use case. You have an entire class of highly skilled professionals performing a repetitive task

Walking on variable surfaces is also a highly repetitive task and automation is yet to master it.

Very few hospitals have radiologists on staff

Good point. Similar to intellisense while coding.

But coding is a creative task. Breast cancer screening is more like classifying breeds of dogs or determining if there's a fire hydrant in the picture.

Reply on news.ycombinator.com