IBM Watson correctly diagnoses a form of leukemia
siliconangle.com
siliconangle.com
Which is funny because it is Google / Facebook / Microsoft which collect and hold everyone's personal details and data. For better or for worse, IBM never seemed to have gotten into that business (yet?) of recording and selling every American's data for ads.
In general, I think healthcare is more important than self-driving Teslas or robots making us dinner and scratching our backs. So at least they seem to have focused on the right thing.
It's a fat target for automation because much of the expertise is available in diagnostic protocols and few humans can hold all that information in their heads and keep it up to date. MYCIN only used a few hundred rules and it could keep track of which antibiotics to use better than MDs.
Well self driving cars in theory should save a lot of lives too.
- Why was Watson being used for this? (That is, whose idea, whose plan, whose test) - How many times was Watson wrong? (many others have asked this within the comments) - Is there a statistical significance to Watson's diagnosis? - How did Watson narrow it down? That is, did it provide a single conclusion of "it must be this form of leukemia" or was it just the most heavily weighted of several options?
The original ad-walled article from NDTV doesn't add a lot of information either on the specifics, though at least the siliconangle article is correction pointing out the limitations on this (needed a huge DNA repository to work with that would expose a lot of extremely private data, probably wouldn't work well for rarer ailments due to lack of data and inability to understand what it's actually looking at).
It is very good and I am happy that the lady was diagnosed correctly and promptly by Watson, and I do think this is important as something like Watson can cut down immensely on the grunt work that a lot of doctors go through. Already most jump on terminals in the patient room and pull up the relevant information from the hospital's internal libraries, so this just seems like the next natural step for modern healthcare; speed up those libraries.
But the cynicism comes from the lack of information, with how news like this gets portrayed (truthfully, I must credit to SiliconAngle for having a much more reserved approach to the article than the source they relied on), and IBM has been touting Watson as a miracle machine before, and they're certainly not curbing outrageous claims made by less careful publications and journalists.
That is the source of cynicism, and I really do think you'd find it as a result of just about any announcement like this from any person.
I think they learnt their lesson on this particular issue shortly after the holocaust.
There isn't a 'unified approach' to anything here beyond the use of various open source machine learning libraries with huge amounts of medical data either licensed or acquired over the past 2 years.
These deals are honestly mostly about trust. Do they trust IBM / {vendor} has the expertise to deliver the project? Given that it's mostly bespoke integration, branding the entire IBM AI/ML area as "Watson" isn't too disingenuous.
Maybe not Americans, but they did pretty well from doing it for Europerans from 1936-1945. [1] and rather than ads, the Hollerith cards IBM supplied by the million tabulated the census data from 1930s Europe into Jew / Not Jew rather nicely.
When the death camps were liberated there was a special unit assigned to makes sure the leased Hollerith machines were returned to the USA safely.
Thomas J. Watson was one of four Americans to receive the Order of the German Eagle [2]
John Welch, Rick Wilson, Tim Ley and colleagues showed how to do this several years ago: http://jama.jamanetwork.com/article.aspx?articleid=897152
I'm curious whether this case was confounded by cytogenetic complexity, which was part of the problem in that case. Ley later diagnosed Lukas Wartman, a fellow in his lab (or, more like, they worked together to figure out what to do), too:
http://oncology.wustl.edu/people/faculty/Wartman/Wartman_Bio...
Lukas did not look nearly as healthy as he does in that picture when I saw him last. He is a great scientist and what he had (adult acute lymphoblastic leukemia, ALL) is a nasty malignancy to treat. Children with ALL tend to do well, but their disease typically seems to arise from different underlying causes than adults, who do poorly.
(Not-so-ninja edit: Here is Lukas' own writeup of his experience. He was in a much tighter spot than I recalled: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850894/ Even children in second relapse are viewed as bad news. Adults in second relapse are generally considered dead men walking. Anyways, read the paper if you're curious.)
If Watson caught that the woman had (say) Ph+-like ALL and suggested the right TKI, that will impress me quite a bit, because I've seen someone die from a wrong guess on the latter. On the other hand, if it was APL, they fucked up the differential somehow and are probably kicking themselves.
(Double not-so-ninja edit: It wasn't APL-vs-AML, or ALL subtyping. Another poster kindly pointed out the primary source for this news, and something doesn't add up.shrug)
After all, winning the lottery is a lot less impressive if you've bought a few million tickets and this article says absolutely nothing about any kind of controls or whether or not this was a one-off performance or if it would work at scale.
FWIW, here's Lukas' writeup of his experience.
They're definitely not free; doctors have to evaluate what Watson says, and if Watson is wrong a significant fraction of the time, then doctors will spend a lot of time and energy barking up the wrong tree.
That's cynical.
Edit: And when the hell did cynical become a bad word in silicon valley? I've had a number of posters on here respond to things "oh, stop being cynical!" as if that was some kind of contribution...
I've got a word for people who are cynical.
Decent scientists.
Feynman would be rolling over in his grave...
"A skeptic doesn't believe anything without strong reasons, which is why it is also associated with doubt, especially when something hasn't been experienced yet. Cynicism is believing the worst of something or someone. It has nothing to do with evidence. It is an outlook on life."
If you understand Bayesian reasoning, then we all have unjustifiable prior beliefs, perhaps due to innate personality, or due to our own experiences.
Calling someone a cynic is often just a way to bash someone who has different priors.
The better thing would be to ask where those priors come from, no?
> Its not cynical if you've actually had any dealings with IBM and Watson.
Maybe GP knows something you don't and he's talking from experience?
The applicability of either moniker towards anyone in this debate, or any other, is up to the participants to hash out.
No, that's history. IBM has quite a series of Watson 'breakthroughs' that ultimately went nowhere, to the point where when I see 'Watson' mentioned in an IBM press release it automatically gets discounted if there isn't a part where the subject has some statistics to go with the inevitable hype.
I see it as IBM trying to stay relevant in a world that needs it less and less and that's why they focus on anything that will grab headlines.
Winning jeopardy, curing cancer, what's not to like about IBM?
http://webcache.googleusercontent.com/search?q=cache:GaC_y_3...
Cached link supplied because the original has departed.
I thought the 20 million oncology studies Watson cross referenced was the control and I arrived at the opposite conclusion as you about the usefulness at scale.
For example, I immediately thought that under the Affordable Care Act, the number of patient Electronic Health Records the Government collects eclipses what Watson had to work with in this case so at scale the potential is huge. Admittedly, until reading your comment I did not consider there were likely a number of times Watson misdiagnosed (which seems obvious now). Therefore, it would be interesting to rerun against the Electronic Health Records the Government has collected so far and test if Watson could diagnosis correctly in less attempts than with the 20 million record it used (or what I call the control). Though at that point, I think it would be the patient that is the control.
I don't know, I'd still be impressed, wouldn't you? Buying a few million lottery tickets is impressive by itself, both logistically and financially. And even still, odds. It's a lot of risk for anybody to dangle.
Shit, buying every ticket is even more impressive. (I grant your point, you just posed a funny metaphor as comparison.)
Tests like these are all about the statistics and without statistics this result - while important to this particular individual - is almost meaningless.
One possibility is that Seishi Ogawa's group has figured out that something off-label can more effectively treat cohesin mutants and it is under review. But I'm not sure how Watson would have known about that. Interesting.
Probably my reading is flawed, but this doesn't make sense.
https://en.wikipedia.org/wiki/Mycin
"it proposed an acceptable therapy in about 69% of cases, which was better than the performance of infectious disease experts who were judged using the same criteria."
"The technology" has "certainly been there" since at least 1970, with software like MYCIN:
https://en.wikipedia.org/wiki/Mycin
MYCIN was an early expert system that used artificial intelligence to identify bacteria causing severe infections, such as bacteremia and meningitis, and to recommend antibiotics
It and its ilk (expert systems, with hand-crafted rules) (it was the '70s) where commonly shown to outperform experts:
MYCIN was never actually used in practice but research indicated that it proposed an acceptable therapy in about 69% of cases, which was better than the performance of infectious disease experts who were judged using the same criteria.
Yet, we still don't have (flying cars) AI that can help doctors make better diagnoses- and note I'm by no means advocating replacing the experts with AI. That would open a whole other can of worms (what do you train your AI on when there's no more experts, because you replaced them all with AI?).
But- just having the tech doesn't auto-solve your problems as if by magic. You gotta beat dumb politics first.
Watson employes statistical searching of large knowledge bases. You dont have to explicity ferret out all the rules and relationships. Google Translate does this too. There is no preprogrammed language dictionary.
The next frontier is deep learning which requires powerful computing to operate in real time.
All these techniques have their limitations. No magic bullet yet.
I'm a medical student, and it seems like most people in the field are rather cavalier when it comes to talking about the job outlook for physicians in most any specialty. Do you all think that some healthcare jobs will not be as vital in the next decade thanks to improvements in computing and AI?
It is also possible that many healthcare jobs are essentially AI-complete problems - in this scenario, subjective opinion is not really a reliable marker, but lots of AI specialists give around a 90% chance of human-level machine intelligence by 2070 (there's a table in Nick Bostrom's Superintelligence with the actual figures).
> (This story has not been edited by NDTV staff and is auto-generated from a syndicated feed.)
That said, I'm unsure how doing something like that would provide the right incentives for the research needed to create the next breakthrough.
Having had years of struggle to get a diagnosis on my own problems, the most frustrating part of it all is that exact process, where you are just plugged into the machinery of our health care system, and run through to get a canned answer that needed no human judgment at all, just canned knowledge and a prescribed treatment plan.
So just my own opinion, but yes, please let us automate that portion of medicine, so the doctors can do more research and improve things.
-- someone in NLP
2. I too would prefer that more open-source work (and published research) would come out of Watson project than it has been the case so far. However, there's a devil's advocate point to be made that closedness encourages diversity of approaches and implementations. When a high-profile project becomes open-source (as recently happened with TensorFlow), it exerts a lot of pull on time/attention of other developers and researchers that could have been focused on trying out entirely different approaches.
[1] http://www.scientificamerican.com/article/using-pigeons-to-d...
The Quantum Fund by Soros and others, returned upwards of 30% per year for over three decades.
Steve Cohen averaged near 30% annual returns for two decades.
Buffett's investing track record is similarly off the charts.
And lastly, a monkey couldn't do what John Paulson (or Burry and Eisman) did with the 'greatest trade ever,' producing a radical outcome from an extremely intricate concentrated investment (some of which required them goading the opportunity into existence to begin with).
If you had said monkeys with darts can sometimes outperform the bottom half of traders on Wall Street, you might have been close.
And what is so unlikely about the Watson approach anyway? Have you ever played Akinator? It's not that different.
You'd be surprised. Watson is nothing to write home about, it's an umbrella term for various disparate technologies (the chess version is not the same as the Jeopardy version, etc) and most of them are not that impressive in the first place.
Theranos hand-waved several huge problems with their approach by claiming proprietary magic. What they tried to do is probably impossible. So, big difference.