IBM halting sales of Watson AI tool for drug discovery
statnews.com
statnews.com
I have zero trust in IBM to market their ML products correctly so that proper checks are maintained.
Especially since technically, explainability is still an active area of ML research.
That's a problem, yes, but it's not a new one. Vendor management has been around for decades, centuries, millennia maybe. If I contract out part of my job, it's still my responsibility to make sure the contractors are doing their job right. "But their marketing said..." or "but their sales guys said..." is not an excuse and everyone knows that.
Doctors noticing that Watson is wrong is expected. Doctors missing the fact that Waston is wrong is a failure of that doctor and the doctor who didn't check the results is the responsible party. The checks don't come from Watson, the checks come from humans who oversee Watson.
If Watson is wrong often enough that it's hindering the doctors, then kicking it out is the right call. But there can never be an argument of "Watson got the diagnosis wrong and that's why the patient died" because ultimately IBM is still just a vendor and Watson is still just a contractor.
If I as a medical provider hire a remote vendor, who has medical teams in India look over initial results to flag issues, those humans will fail in human ways. I can anticipate that: I'm a human.
If I use a similar ML product, it's very difficult for me to anticipate (or even understand) the ways it which it might / does fail. Which makes it unlike my previous experience. Which gives it a fundamentally different risk profile.
It's the Boeing issue in a nutshell: the failure case that unfolded was unlike the scenarios the pilots were trained for. Unfortunately, in the two crashes they were unable to dynamically RCA quickly enough to solve the problem.
My point was that coupled with IBM's inept and inaccurate marketing, it seems unlikely the appropriate risk information is in the hands of those responsible for managing risk.
And honestly, if a system has unlimited failure modes, and I can't learn and limit them in practice, it's useless.
Because in that scenario I should be duplicating all the work it's done to ensure it didn't go off the rails. In practice (and guided by labor cost savings promised in the contract signed with management), that full verification doesn't happen (because the vendor is incentivized to recommend it doesn't), and people die.
I design and deploy customized automation systems for customers, and it's part of the standard process that we run the automation side-by-side with the old process for several months in order to learn the new failure methods and synchronize the process. Yes, for a few months we're duplicating the machine's work, but without the machine we'd be doing the work anyway. And no one is going to die if my automation fails, but we still do this anyway. It's crazy to think anyone would believe they didn't need to do side-by-side verification no matter what sales and marketing told them.
I don't know enough about Watson or IBM sales to say if Watson is good or bad, but I'm not trying to defend Watson or IBM. Watson may very well be a complete failure. But that aside, it's not the only failure in this story. No one should expect to implement a new tool and never verify if it's working correctly.
Well as a doctor if I’m still ultimately responsible then nothing has fundamentally changed, this is just another tool, possibly one I’ll be forced to use by someone with not one day of medical training.
And medicine in the US is in a precarious position. Software engineers are generally not licensed, and they’re not sued (it’s virtually unheard of). It’s different for doctors. There’s a complex dynamic of removing autonomy from providers to ostensibly improve outcomes (and revenue cycle) while also still holding those same personally liable.
So much of what I do is indirectly dictated by wonks in IT and billing, but they have virtually no liability. And one wonders why burnout in medicine continues climbing with no end.
If your stethoscope fails and you mistakenly pronounce the patient dead because of that, who is responsible? Do you blame the stethoscope salesman for claiming it's an accurate medical instrument that you never need to second-guess? Do you stop using stethoscopes altogether because they're "just another tool"?
If your x-ray machine fails and you say the patient's leg isn't broken because of that, who is responsible? It's "just another tool", do you stop using x-ray machines?
If your physician's assistant measures the patient's blood pressure wrong and you never double check their work, do you fire all of your PAs? And go back to seeing every patient for every procedure yourself?
Everything you have is "just another tool" and as with any tool, it's up to the human doctor to interpret the output. The idea is tools make you faster and more accurate, but everyone knows tools fail so you need to be able to double check their work. If the tool is consistently inaccurate, sure, throw it away. But if your argument is "if the tool can't completely replace me it's worthless" I think you're selling yourself a little short there.
Of course you're ultimately responsible. Your stethoscope didn't go to college for 8 years, it's just there to make your job easier.
Not just the expectation but the understanding. A doctor might very well forget which leg to amputate, so we know to Sharpie "NOT THIS LEG" on the one being kept. But a doctor is very unlikely to see a patient with a broken wrist and prescribe antipsychotics, so we don't do much to prevent that error. Human fallibility happens along fairly predictable channels, and we've spent a very long time committing resources to controlling those channels.
Watson, though, thought Toronto is a city in the USA. Anyone who's dealt with ML output knows that the errors are often quite surprising, even before dealing with adversarial inputs. Even in a system where Watson's outputs are subject to checks, the checks we have today are human-specific and developed at a significant human cost. ML answers can't just outperform individual human doctors to add value, they need to either be gracefully integrated with them or be able to outperform the entire system which keeps those doctors on track.
Unless, the leads are so bad on average that too much effort is spent on vetting them that could be put to better use elsewhere.
If the leads are worse than a random toss of a coin, then you can come up with leads on your own.
Not all leads are good. Inviting the village idiot in a brainstorming session wont be of much value -- and Watson is more like the village idiot, than a valid lead generation engine.
The MD Anderson audit is particularly bad: https://www.utsystem.edu/sites/default/files/documents/UT%20...
Sounds like every other failed $50+ million government software project ever, which IBM is apparently an expert at. Except this money donated by private citizens was redirected from other cancer research projects and siphoned into a billion dollar company’s coffers.
Really shameful stuff.
https://techcrunch.com/2009/09/02/netbase-thinks-you-can-get...
... granted this happened a decade ago:
> "Several of our readers tested out the site and found that healthBase’s semantic search engine has some major glitches (see the comments). One of the most unfortunate examples is when you type in a search for “AIDS,” one of the listed causes of the disease is “Jew.”
The engineers at IBM were probably separated from the problem by many layers of bureaucracy (meetings, project managers, technical specs).
Even working inside a company, without access to the right data and the right people you aren’t going to get anywhere.
But a person like that can very lucratively sell actual products that already exist, so why would they want to sell this mysterious X?
a) employs enough scientists who know what they're doing and lawyers who know what they're doing to file relevant and defensible patents; and
b) has enough entrenched relationships that they can pay for a sales force what knows what they're doing; but
c) doesn't employ enough engineers who know what they're doing to develop actual competitive products around those patentable ideas.
As programming and systems engineering slowly formalizes, it seems to me that's getting harder to do. I see fewer self-taught folks in their early-mid 20s these days at least, although I could be observing my personal bias from aging. Anyone else have an idea?
I have daily skirmishes at my company with our CTO and his subordinates who constantly want to believe snake oil salesmen because they’re looking for simple ways outs of hard problems. It’s really sad.
For some it seems easier to -_trust_ a dude with a power point presentation and no real experience over good engineers telling the truth and trying to help them for real.
I now think it’s madness for anyone to be in charge of anything if they don’t have relevant experience in the relevant domain. Nice people or not, they’re susceptible to bullshit.
This.
In support of that idea, here's C.A.R. Hoare, from 'The Emperor's Old Clothes', the 1980 Turing Award lecture:
'At last, there breezed into my office the most senior manager of all, a general manager of our parent company, Andrew St. Johnston. I was surprised that he had even heard of me. "You know what went wrong?" he shouted--he always shouted-- "You let your programmers do things which you yourself do not understand." I stared in astonishment. He was obviously out of touch with present day realities. How could one person ever understand the whole of a modem software product like the Elliott 503 Mark II software system? I realized later that he was absolutely right; he had diagnosed the true cause of the problem and he had planted the seed of its later solution.'
He then discusses how the problems there led to his thinking on CSP, and, eventually, formal methods.
But, go read the speech, it's much more wonderful than any summary thereof: http://zoo.cs.yale.edu/classes/cs422/2010/bib/hoare81emperor...
I hope IBM continues to fail and becomes a case study in the failure of massive bureaucracy.
You see, they are NOT tech companies. They have something you would call "tech", they sell something that looks like "tech" but they really solve problems people with access to large amounts of money have.
In other words, they produce cushy and warm beds for C-level execs to have a good night of sleep, and everyone needs some bed to sleep every night..
Technology is a risky and volatile business and IBM has more than 100 years in business.
IBM really does capitalize on the adage of nobody being fired for buying IBM. Their biggest strength has been in their marketing rather than actual technology.
Note that, being a large org, they do have teams doing excellent work too. IBM research, some of the cloud, design etc. teams seem to be doing good stuff. But most of it is still overpromising and underdelivering while charging clients a lot of cash.
Likewise, IBM has become known as a vaporware vendor. They combine the worst parts of management consulting with an offshore / outsourced development process, so I guess it’s not a surprise they ended up like all the other companies that sell business tech solutions (Infosys, Wipro, etc). They’re often mentioned in the same breath these days.
- IBM's hardware support is excellent to the point of being obsessive
- IBM's support for anything that is not a hardware problem would be better-implemented by a 5 year old eating paste
So after getting in, they don't need to be good, they just need to not be too awful.
[1] http://www.econtalk.org/robin-feldman-on-drugs-money-and-sec...
That said, if you can make clinical trial reporting data collection more efficient, there's a LOT of money to be made.
We need to prove them wrong on both these accounts.
That said, there's a lot of bunk science marketed to consumers within healthcare. Your average consumer heats microbiome and genomics and AI and thinks that the product will cure them of disease. I don't think it's the role for the FDA (until they make specific health claims), but we need to remind consumers to be smart purchases of these services. And for us in the healthcare tech space need to be careful who we want to include within our communities to ensure we share the values of positive patient.
One failure by one company, even IBM, won't stop future attempts.
https://blog.jacob.vi/an-80s-throwback-artificial-intelligen...
In response to the above blog - Watson's early small-scale success clearly hasn't scaled to bigger and broader applications.
I invite you to test "Watson Tone Analyzer" https://tone-analyzer-demo.ng.bluemix.net/ :
- "I like this product." => "this is an analytical opinion with neutral emotion."
- "I like it" => "Tentative, 50% happy answer."
- "It's not a bad product." => "analytical".
- "It's not a bad tool" => "joyful answer".
response: Fear: A response to impending danger. It is a survival mechanism that is a reaction to some negative stimulus. It may be a mild caution or an extreme phobia.
As far as I can tell there's absolutely no connection between the products being labeled as 'Watson' and often times very little machine learning taking place either.
I've interviewed several candidates from IBM over the last ~3 years and almost all of them worked on something with 'Watson' in the name. When we whiteboarded out the architecture of what they worked on it was mostly automation with _maybe_ a touch of machine learning thrown in by a module written by someone else.
Very few of them were able to pass the technical interview.
Exactly what I expected, and also flat wrong.
- "There's a monster under my bed." => "joy"
Fascinating.
They're just good enough to, with insane amount of computing resources, churn out economically viable results, but in my opinion they're the biggest setback in actually developing intelligence or cognition in a long time.
There's very little insight to be gained from what they pick up, and they function purely in a stochastic sense. Even the best ML algorithm has no ability to reason at a high level or produce counterfactuals, it works purely by correlation and still, in those 1% edge cases, will be as helpless as anything else.
That might be good enough when selling advertisements, but in automated cars and healthcare treatment, this sort of failure is not an option.
In my opinion the real hype is that ML has become so popular that new practitioners tend to forget other analytic techniques like SQL data warehouses. Interestingly these are starting to absorb ML capabilities like logistic regression, which are now accessible through SQL and can benefit from MPP and vectorwise query execute in DBMS types like ClickHouse, Vertica, and Google BigQuery.
[1] https://www.forbes.com/sites/danafeldman/2018/03/28/u-s-tv-a...
Disclaimer: I work on ClickHouse.
According to the trend, we're approaching another AI Winter
https://www.theverge.com/2019/3/5/18251326/ai-startups-europ...
I’m not sure the acquisition will create a mass exodus from the status quo there.
Lots of focus on the algorithms in the comments here, but from what I could glean generally they lacked domain experts when developing the datasets... we spend 90% just finding the best data... and even then it's tricky. I think they may have had lower standards for the input into the system... garbage in garbage out.
What do you think machine learning is, if not “quantitative math”? Deep learning is just linear algebra and calculus, and things like random forests are even simpler mathematically.
>And what were the equivalents of DL/ML for physics before calculus?
This is a good question. Before Newtonian calculus, and the laws of gravitation, people were building very complicated conic models (ie. eclipses, parabolas etc.) to get better and better prediction of planetary motion. A lot of parametric math came out of this, with many sophisticated models getting better and better, giving these astronomers an illusion of progress. However, Newton's insight was that motion is connected to mass, and this insight was the basis of how to derive the laws of motion, which gave us the laws of gravitation (F = (Gm1m2/r^2)). This insight eliminated the previous Keplarian models of motion, because you were now able to predict the motion of arbitrary rigid bodies using very simple math (we teach this in highschool). Ofcourse, Newtonian motion has its limitations that's why we have quantum physics and Einstien's relativity theory. But for practical technological applications, Newtonian physics on its own gets you incredibly far.
Where is ML/DL? It would be akin to Keplarian elliptical motion. More realistically however, it's closer to aether theory of light, and will go the way of GOFAI. This stuff isn't grounded in modelling any scientific observation. Moreover, they are mathematically useless. Back propagation doesn't converge, and why should you fit your data to an arbitrary mathematical structure? In practise, DL/ML doesn't work at all, you will be much more successful by modelling your problem mathematically. For example, consider an automobile manufacturer, which has all kinds of moving parts in their planes. They typically model each part mathematically (ie. gear x under goes exponential time decay), and imply their parameters using rigours test data. Then you use some sort of an empirical statistical model to predict the failure.
I've seen deep learning companies come and fall flat on their face trying to beat the accuracy of these deterministic systems. Those guys needed a lot of data, and GPUs. I'm not even criticizing the fact that DL is a black box. It's worse, it's inferior to everything out there on every metric imaginable. These mathematical models in contrast have been in production for decades, with yearly updates, and they run in real time with little historical data, they are fully understandable and they beat every method we know of.
This isn't the first time multi layer perceptrons gained hype. They didn't work in the 80s, or 90s or the 2000s, they don't work now. The math behind DL is the same that we had in the 80s, they just called it multi layer perceptron. None of the ideas in modern ML/DL are new, all these ideas like reinforcement learning, GANs etc.
2. Likewise Newtonian physics is also an approximation: it does not fare well near relativistic speeds or high gravity. But at least we have models which seem to be accurate to many decimal places today. Who knows what the future may hold.
3. Not all useful problems can be represented by simple equations, but they can be computed analytically (e.g. N-body problem).
4. Ultimately DL is popular because it works better than anything else in some very specific domains like speech recognition and image recognition. It is overapplied I'll admit, but if you can do better then feel free to publish a paper.
I have a neural net onboard my phone which automatically detects songs offline and tells me what they are. Is that semantically 'quantitative math' and not machine learning?
>I have a neural net onboard my phone which automatically detects songs offline and tells me what they are.
MP3 uses something called psycho acoustics, which is a quantitative model on human perception, which is used to eliminate frequencies that can't be heard based on this model.
Your neural network doesn't tell you what features make songs distinct, it's not a quantitative model at all, but a black box heuristic on what the important features are superficially. If actual mathematicians worked on this problem, I guarantee you they'd do a better job, and their models would work on a commadore64, with real time training. Moreover it would tell you things like who is singing, if it's a live performance, which concert it was.
No, this is wrong.
Some of the most brilliant people in the world have been working on image recognition, voice recognition etc. and AI is crushing all of their work.
"Your neural network doesn't tell you what features make songs distinct, it's not a quantitative model at all" - it doesn't matter at all if our objective is detecting the song. Neither does the mp3 compression algorithm.
This is very true. I take my stronger statements back, MAINSTREAM mathematicians attempting this problem are all wrong, and have been wrong for 50 years. But you do need the right theory, and the right math that realizes this theory.
"AI" is superficially beating the work in computer vision. Computer vision is complete bogus. The gabor filters, fourier transfroms etc. are all wrong conceptually. The known methods do abysmally on basic tasks like object recognition, texture segmentation etc. But they keep trying it.
I would take this one step further: computer vision, audio and NLP researchers have been stuck in a rut for the past 50 years. DL is beating THEIR math, but this is because of data and computation speed, not because of any insights. But DL is also wrong, and giving you an illusion of progress. Both of these things are doomed to go the way of GOFAI.
I can go into great detail and carefully explain why MAINSTREAM contemporary ideas in math for vision, audition and language are completely wrong, and have been wrong for 50 years. What is the right model? Like I mentioned before, the right ideas are emerging, neural networks will dominate, just not DL.
This is a No True Scotsman. Actual mathematicians did work on this problem, training the neural network to achieve it's target task of identifying songs using minimal power and storage consumption - which works.
Collecting observations aka data.
> deriving the mathematical laws that govern what you see
Fitting a model.
> Your neural network doesn't tell you what features make songs distinct
It literally learns better features that you could ever come up with by hand. This is why CNNs do better in computer vision that hand engineered filters.
> I guarantee you they'd do a better job, and their models would work on a commadore64, with real time training.
LOL if you think that a room full of people can listen to TBs of audio data, decide what mathematical functions when combined together are better descriptors of that data than a DL model learning its features.
You don't have the slightest clue what you're talking about.
It's analogous to a human being able to identify songs by remembering the chorus, just that the NN uses it's own features for both the memory and offline perception.
>In 2017 we launched Now Playing on the Pixel 2, using deep neural networks to bring low-power, always-on music recognition to mobile devices. In developing Now Playing, our goal was to create a small, efficient music recognizer which requires a very small fingerprint for each track in the database, allowing music recognition to be run entirely on-device without an internet connection.
https://ai.googleblog.com/2018/09/googles-next-generation-mu...
Why did you pick a neural network? What mathematical properties does a neural network have that makes it appealing to this problem? How were the networks trained? Back propagation? It doesn't converge, and worse learning weights for a new batch can cause you to forget previous batches. This isn't a desirable property of neural networks or back propagation. You probably had a lot of heuristics on top, fine. How do you know that the weights you ended up with will always work in practise? Given an arbitrary track, you can encode it? What about growing the database? Does the neural network get updated for new songs, or do you use the same neural network to fingerprint new songs and update the data base?
Here's how I would have done it:
A song file is just a sequence of amplitudes. I would do some kind of an interpolation of piece-wise trig function. Trig functions have very desirable properties: they are continuous everywhere, and infinitely differentiable. Moreover, a sine basis decomposition will be able to reconstruct the original signal very well. This is great, because now you can use theories from DSP and fourier analysis. So we take the entire song, do a continuous time discrete cosine transform, in a block size of 32. Now you compute the square norm of all feature vectors, sort them, eliminate the vectors that are within 1e-3 radius (they are too similar to each other, there's not point in keeping them) and only store the top 25% of feature vectors by the square norm. The 25% cut off threshold and 1e-3 radius of similarity are heuristics, and adjustable parameters.
Now you have a database. For a new song, repeat the procedure, and get a feature vector for every 32 interval. There are probably theories in DSP you can use to get a better similarity measure, but for now, we'll just use the L2 norm of the difference. Do a nearest neighbour search in your data base for all feature vectors, and rank the results based on hits. I can run all of this on a computer from 2000s which are crappier than modern phones, and have the entire backend run on equally crappy hardware too. All parts of what I'm doing are fully deterministic, updating the DB is incredibly fast, CTDCT is super fast, there are no questions of convergence, no need for training. You can probably increase the accuracy and speed by doing some DSP and doing the nearest neighbour search based on different voice, bass, instrumental etc. features.
In practise how would it compare to your neural network? No idea, but I imagine it should be very competitive. The big benefits are that you have only 3 parameters (radius of similarity, cut off threshold and block size). This seems very easy to bench mark against, it should take like a week to implement. I'm not sure about the compression of the finger print however. Not sure how much space 1000000 songs will take (probably 25% since that was our cutoff). You can probably borrow psycho acoustics to make a better data base, and get a better compressed representation. Another alternative would be to down sample the song to 64kbps before hand.
This was a paper from Shazam from 2003. This is essentially what I proposed, there is no training. Shazam works pretty well. It's not even going into the mathematical consideration I went into.
>You'll never be able to develop features with the heuristic methods you described that will work as well as the features learned by a neural net.
False.
Deep nets are here to stay. They're just not magic bullets that solve all problems equally well, especially those when training data is minimal.
What on earth are you talking about?
>Deep nets are here to stay.
Maybe in silicon valley for consumer products in things like snapchat and siri. They won't work for industrial problems.
AI is in a hype bubble right now surely, but it's a 'very real' thing that's going to infiltrate a lot of areas.
Being state of the art doesn't imply that these things will solve these problems. In ML terms, how do you know that NN/AI isn't a local maxima that we need to jump out of? All NLP systems are joke. Sure replace Watson with DL, might perform better on Jeopardy. But in real conversations? Forget it.
I wouldn't bet on these things. NN will win, but not the back propagation, ReLu, sigmoid or whatever pseudo science that is the current buzzword. There is 50 years worth of understanding in actual neuroscience and cognitive modelling that no one has paid attention to, and new design principles are emerging that will influence mathematics.
It's the best performing tool we have for NLP, image recognition, etc. Is it a local maxima? Probably. But it's out there solving real problems nonetheless. We'll capture all the gains we can and then move on after.
I suggest you are misinformed about the state of AI.
AI is currently ahead of all other approaches in many fields.
It's lead to quite a number of practical advances and breakthroughs.
The 'best examples' are those that I described, but there are many more.
Your comments indicate I think some ignorance on the issue - I think I see the point you are trying to make but I also submit that you're not aware of what AI is doing today.
'Self driving cars' would be impossible without AI today, for example. The vision systems depend on AI it's a breakthrough without which we simply wouldn't have the tech.
I just wanted to blast these other applications, because I think people get this idea that AI has to be AI for anything interesting to happen... but there are really niche applications where people don't think these tools are experimental. And what you describe may already be happening, Geoff Hinton's critique of modern deep nets seems to be a call to get more biological. (Thinking of capsules nets).
The generally available models will almost always be suboptimal due to the difference in the data that they're trained on vs the data that clients use it on. That's why most of these AI companies end up doing a ton of consulting and build specialized models for larger customers.
And its not just hype, like crypto the ai hype stepped well beyond the line to outright fraud and deception with tech folks trying to pass off backward looking pattern matching as 'intelligence' and hope no one notices. Every single commentator here knows there is nothing in computing or software engineering today that will allow one to 'create' an 'ai' as the world understands the term yet no one questioned pushing intentionally deceptive communication.
This end result of dystopian scaremongering amounting to nothing is there is now zero credibility and extreme suspicion of problematic in-built bias. At the minimum there must be some standards for machine learning solutions to be thoroughly transparent, open to verification and exhaustively tested for racist and sexist bias before any rollout, for anyone who cares about the impact of their work in the real world.
A few friends from university were hired into the Watson team as hardly technical PMs. I have never heard any of them describe what Watson is or does.
For example, for drug discovery: I'm guessing, off the cuff, that there are other drug discovery tools out there, and that, while they don't allow you to frame your searches in terms of natural language queries, that's probably not actually a problem in practice. Because the querying methods they've developed are presumably highly tuned to their real task, which is enabling a skilled and knowledgeable practitioner to specify what they're looking for with great precision.
It's sort of like programming languages: The ones that are designed to be the closest to natural language (e.g., AppleScript) do have a gentle learning curve for absolute beginners, but they also turn out to be some of the very worst languages for trying to do any sort of serious work.
Some products with Watson branding are great and industry leading. Others suck. The latter tend to appear in headlines and severely impact the brand they established with Jeopardy and their ads.
I say all this because I'm at a place now where this is happening. Hugh company with a supposedly game changing product that is mediocre (at best). Massive marketing campaign for "Product X" that is 100% buzzwords and 200% BS. After constant missed revenue targets and product disappointments internally and externally, company is clearly pivoting (but not saying so) to rolling everything under "Brand X."
This is from my (simple developer) perspective. Not sure how it looked in sales, among executives, etc. But that was very weird environment.
I had great Manager who asked me to learn react and take courses in react (mind you it was two years back!) As we "might want to do something in react in the future". So I basically spent my last six months there learning React, aka preparing to the job interviews... they even got us paid courses and all. I mean... IBM.
And once they fired me (these were lay-offs, thousands affected, many of them just hired in past year, like myself) I was paid severance pay too. I went there worked a year, last 6 months was learning for job interviews... fantastic pay too. IBM is crazy.
Spoiler: (s)he learned React.
I think big companies trying to do things like "Capture a market segment that will be a trillion (I made this number up) dollars in 2025" suffer terrible analysis paralysis. The 5 year plan has an extreme revenue ramp up and insane targets. If you combine that with politics, there is a lot of business and financial justification that has to go into every decision, and many decisions will be safe ones that look innovative (we're going to build on Insert Latest Cloud and use AI!) but have no real value to many customers.
The end result is crazy hiring (and firing a few years later) and groups with opposite experiences. Some groups have no work to do and other groups are working 80 hour weeks trying to make it seem like the marketing and growth curves are all true.
It's a comedy (if you are able to stay out of the mess and politics) or a tragedy if you have a manager who feels they want to be the shining star that supports this mad rush.
This all rolls down a few levels of mgmt to the first line guys, who are told your in a growth area, and we expect to need to hire X people over the next 24 months, who will be tasked with XYZ (frequently fancy words which when analyzed boils down to support the product we are going to sell).
This goes on for a couple years until the projected vs real revenue divergence is so large that even an CEO can't ignore the lack of growth. At which point the plug gets pulled and the next adventure starts somewhere else.
Sadly though, IMHO none of that is a problem, the real problem is that the CEO's can't actually tell or make strategic decisions about why these projects are failing to have exponential growth. (see intel & mobile chips/wireless connectivity, those are so strategically necessary for their growth that they need to keep trying until they die). So, they ax them, sometimes just as they are getting a solid product portfolio together. But they don't know that because they have been fed the same line for the past 24-36 months.
Sounds like it was working perfectly!
I'm not even at an IBM shop and the Watson sales pitches are starting to reach C-level folks.
The ads are filled with lofty buzzwords. No talk of actual technology at all, because the ads are targetted at non-technical management, the kinds of people who might be euphemistically called "decision-makers". The ads make all kinds of promises to these "decision-makers" about how their business will be utterly transformed. Actual implementation of the systems and business-critical changes is left to the IT department. The non-technical management writes a cheque and then washes their hands of the problem.
[citation needed] IBM did not do this, though funding the Deep Thought team from CMU was cheaper than a Super Bowl commercial and brought more durable effect.
Watson is not very good, and largely doesn't exist outside of slick marketing campaigns.
As soon as the patient checks in, their demographics (age, sex, etc) and vitals are fed into a mysterious program, which suggests an acuity level, and basically drives the whole course of treatment. The patient is required to be tested for a variety of disease processes based on the AI-generated differential diagnoses.
Honestly, I thought/still think medicine is headed toward this, which is why I decided to go into research.
In reality Watson took a great deal of work to carefully structure and process data, evaluation of the output, and more thoughtful of approaches to further refine the information and evaluate outputs. It also required a fair amount of involvement with the individual customer's staff. And there was always the possibility that because each use case was different... it simply wouldn't work out.
To some extent it seems like it should have been sold as a journey... not cookie cuter solution for things that Watson had never encountered before.
How do you sell that, I don't know, if I knew I'd probabbly be a pretty good salesman.