Hacker News Reader

Comment by dontreact | Hacker News Reader

I have talked at length with several people in the FastMRI project and in my opinion it is actually very dangerous. There is no feasible way to validate that such a model will not hallucinate normal tissue in the presence of a rare abnormality. The argument often used is that advanced reconstruction techniques such as compressed sensing have not required validating against very rare abnormalities, however when deep neural networks get involved you have a nearly universal function approximator whose behavior is nowhere near as bounded.

The FDA will probably set the low bar of validating that the reconstruction algorithm fares well in the face of a few abnormalities and call it a day, instead of the proper (admittedly infeasible) validation of testing against all abnormalities that the reconstruction algorithm will encounter. Facebook will probably happily jump over this low bar, and patients will get hurt.

View on HN

I never understood these arguments. Isn't it like claiming self-driving cars via neural networks would never be possible because you can't test that the neural network would take the correct decision in every situation?

I view the whole issue stochastically, with the immediate aim being to make (significantly) fewer errors than the current approach, which is having a human decide. I don't claim that I can design an experiment which could selve as an indication whether we are improving upon human judgments, but I think this should be the goal.

Reflecting upon my view, i think it comes from the experience of training ml-algorithms. You are always minimizing errors, but you goal is almost never to make 0 errors, because often your data is noisy and you are probably overfitting. I know the medical enviroments are more sensitive, but I can't really wrap my head around how we could design a learning algorithms that does not make any error and works on all abnormalies. I think it will always missclassify.

Rephrasing my argument: I think the approval should be given if an significant expected improvement over the distribution of real-life abnormalies can be detected and not over the uniform-dsitribution over all abnormalies.

EDIT: detecting out-of-distribution samples is hard and I don't think this is a solution and leads to a false sense of security.

It isn't. It obviously is possible to drive a car. The problem here is you're trying to reconstruct image from less data than what is fundamentally possible. Compressed sensing is already as low as you can get, unless you prove it is possible to reconstruct the image accurately from even less data, then it's probably pointless to try use neural networks to decode it, and it's actually a bad idea anyway.

The problem with neural networks is that they can reconstruct something that looks "normal", not necessarily something that is accurate. The more abnormal the scan, the more likely it will get reconstructed as something that looks perfectly fine even when it's not.

I really don't know anything about MRI and their usage, i was mainly criticising that his arguments sounded like a lot of the arguments I've heard against other usages of ML-Algorithms.

What I meant: There's probably a medical reason why you want such a product and if a more readily available MRI saves (really significantly) more lives than the chance that it might miss some abnormalies which could lead to death, then I think we should allow it. That's what I meant with a stochastic view. If we, for example, only have a few scans per hospital available because the chance might exist that we missclassify something and lots of people get worse or delayed treatment because they are not high enough on the priority list to get access to a super-resulution MRI with a fidelity they don't really need (again, i don't know anything, just to illustrate my point), then I think something is wrong.

His argument just sounded dismissive without giving a, to my uninformed point of view, valid reason.

the distribution of abnormalities should be weighted by the severity of a miss. Unfortunately there are many abnormalities with lethal outcomes if missed. So it’s very hard to get a sense if a reconstruction algorithm will adversely affect outcomes. I would appreciate even a more detailed look at trying to estimate this, especially in light of how much more a deep learning system can be. However that is not what’s happening. Everyone is happy to just jump over the bar that has been set for previous reconstruction algorithms.

I don't have strong feelings about this, but a few thoughts come to mind regarding exploring accelerated MRIs via neural networks.

1. If an MRI can be done 10x faster with the same results except in exceptionally rare cases, might that not still be a win? Order of magnitude reduction in time may translate to substantial reduction in cost and increase opportunity for applications. It seems like it is worth considering whether these benefits might be worth compromising the accuracy of the imaging.

2. How accurate are radiologists at diagnosing / detecting these rare abnormalities that validation might miss? If radiologists are actually pretty mediocre at this, might it be OK to make the scan slightly imperfect if the next stage (the human) is already very imperfect?

1. Maybe, but that should be a tradeoff that is made consciously with some analysis and care, instead of just jumping over a low FDA bar, which is what everyone in this space seems to be doing. I think there are enough abnormalities that can be fatal which a net would just fill in (e.g. aortic dissection?)

2. There are thousands of different abnormalities. From what I understand about the FDA validation process for this sort of thing there would be only 10s of abnormalities. There are likely, many, many of them that are quite obvious to radiologists. And once again, this would be a question that should be studied carefully when people's lives are at risk, instead of just assuming that it will be fine then going ahead to "move fast and break things"

It's my understanding it's sometimes exactly those obvious abnormalities that are missed by radiologists.

Adding another lossy step will only make even more anomalies get missed, or worse, misdiagnosed as something else (I'm thinking about the photocopiers that sometimes change numbers in documents).

The argument that "our system is already bad so we should just merge this new bad component because it doesn't make it worse" is bad in software, and unacceptable in medicine.

I don't think that is a fair characterization of what I wrote.

If the scanning process has a sensitivity of 99.999%, and the next stage in the signal chain has a sensitivity of 50%, and we consider what happens if the MRI sensitivity drops to 99.9%, that's well in the noise in terms of diagnostic value. Use some of the extra money freed up by doing the MRI 10x faster to pay a radiologist to look at the scan for 10% longer, and perhaps the net accuracy has improved.

What is unacceptable about this kind of holistic reasoning about the system? High performing systems are frequently not composed of perfect components.

It's fine to talk about statistical accuracy when the distribution is somewhat normal. ML systems we've seen so far don't typically degrade in nice, statistical, predictable ways.

Omitting 90% of samples (based on 10x speed claim) leaves an awful lot of room for bizarre errors.

One advantage of software is that it can't really have a bad day. The radiologist can be affected by external human factors. Perhaps an angry wife and a dinner getting cold at home. He might not take the extra picture a bit further down the lower back where the golfball tumor was growing (story from my mother-in-law).

> And once again, this would be a question that should be studied carefully when people's lives are at risk

I agree 100% with this, but how can this question be studied of this research to develop these systems doesn't go forward? You earlier characterized the FastMRI research as, in your view, "actually very dangerous." As I see it, the research here is potentially extremely valuable, and the danger comes not from the research, but from somebody deciding to deploy it without considering these questions. Typically the road from research program to wide deployment is quite long, and I disagree that we should discourage the research because of potential flaws in the productization process. (Unless there is ill will on the part of the researchers, which I'm assuming there is not).

I think it’s the lack of will to study these sort of things that is dangerous. There just doesn’t seem to be this sense of caution among the people working on this. They seem to just want to jump over the low bar set by the fda and move ahead to patient care.

To be honest I am highly skeptical that FB would be able to have the sort of institutional capacity to launch any sort of medical device health product. It takes a non trivial amount of effort and time to fully productize it (in terms of system integration/ product testing/ FDA submission). Unless there is involvement with one of the major MRI manufacturers, I don't foresee this going far. Many more players in the game that actually have a decent shot of getting something like this to market.

The thing is that because the highest-ROI application of machine learning is online advertising, unquestionably the two deepest pools of machine learning talent are at Google and Facebook. So this may be able to overcome the obvious culture mismatch between "move fast and break things" and the FDA.

Google has a long long history of failing in the medical space. They tend to overengineer their products in a way that makes sense if you’re a computer engineer, but not if you understand anything about the healthcare system. Facebook has no legitimate history developing these products either. It’s more involved to develop health products than it is to sell ads. And when it comes to deep learning, optimizing your ml for 2D rbg is not the same beast as volumetric 3D data.

And besides, let’s say tomorrow they have a method to do it tomorrow: how do they prospectively scam patients? How do they deploy the algorithm in a clinical setting? For any of this as a product to work, it would have to be integrated into a mri controller. Unless I’m eating my words at RSNA this year and deep mind is presenting their work, I’ll remain highly skeptical that this is going anywhere beyond a PR story that’s been sold to the media.

Deepmind has projects related to 'formal verification' and conservative bounding, presumably for these very reasons.

From what I understand there are tons of MRI applications. If you are looking for something exotic of course you will try to get a human, probably best in the field.

But if you are trying to bring this to masses one can start with with something "simple" like bones or some some relatively trivial organ. Not that MRI prices themselves are dropping much - gotta wait for room temperature superconductors.

nobody expects these systems to be perfect, which is why they have less validation against extremely rarely observed problems- the total impact is small enough that we don't prevent good systems from being approved.

Rather than saying "very dangerous", it would be safe to say that some individuals woudl not be well-served, but ideally, the overall health of the population would increase for a reasonable expenditure.

Reply on news.ycombinator.com