Layoffs at Watson Health Reveal IBM’s Problem with AI
spectrum.ieee.org
spectrum.ieee.org
The problem they were having is that all the various IBM lines of business added so much garbage onto the client's cookie, that eventually their pages would stop loading because they wouldn't be able to parse the cookie.
The 'solution' is to detect when that's about to happen, and redirect the client to a page that warns them that their cookie is too big (because IBM made it too big) and give them a button to delete their cookie. They then continue to start over and stuff more garbage into the fresh cookie.
That kind of problem solving pretty much made me lose faith in them successfully doing much of anything anymore.
If they had a core technology team they could handle things like this more gracefully. Or maybe not, this is IBM after all.
I mean, in the same company you have on one end of the scale lots of teams doing just outsourced IT support ("my corporate word install is broken, pls fix"). And on the other end for instance the IBM Zürich research lab, who have received multiple Nobel prizes in physics and who developed things like Token ring and Trellis code modulation.
- Their PCF solution is deprecated - They were still using a deprecated container management solution instead of kubernetes until recently - If you use Java the for you to use liberty build packs - They push XP and TDD practices from the garage yet many of their consultants don't practice what they preach.
And then they rebranded IBM Spectrum Conductor for Containers (look it up!), a Kubernetes snowflake they came up with, into IBM Cloud Private, because Kubernetes will solve all problems, the issues MUST have been leading with a PaaS and not a CaaS, right?
Other observations are spot on
Disclaimer, I compete with IBM and have to deal with their account team shenanigans, though parts of IBM are better than others and even can be good partners.
If they want to accept the positives of the IBM brand then they need to accept all the negatives that come with it. Unfortunately for them that brand is now toxic.
If I decided I wanted to visit one of our offshore dev teams, I could book a first-class ticket anytime I wanted to with no approval necessary. If I wanted $2m for some dev effort, I could have the necessary approvals in a day or so. But if I wanted to write 50 bytes to a .company.com cookie, that was weeks of meetings where I needed to justify my team's existence, the value of the feature we were building and prove there was no other way to implement it.
Admittedly it is a better practice than continuing to stuff you cookie and direct users to a strange page once in a while.
For example, after a merger in which two company websites should be integrated you might start with one simple cookie, transition quickly to two independent cookies on two domains, then use a cookie with two keys when the two sites have separate backends but the same domain, then eventually a cookie with a single key again after extensive replacements and reimplementations.
The problem is not that they wouldn't be able to parse the cookie, it's that Akamai edge node web server (Nginx or Apache, don't remember) rejects any request with a header that's above a certain size. Your origin server doesn't even get to know that there ever was such a request.
See https://maxchadwick.xyz/blog/http-request-header-size-limits for example.
what i got was:
{"elem" : "html", "children": [ ... {"elem" : "form", "attrs" : {"action": "/an_url", "method" : "post"}, children: [{"input" : {"attrs" : {"type" : "submit", "value" : "..."}}}]} ..., ]}
i'm a) pretty sure this "json interface" feature cost the customer more than i make in a year, and b) it probably broke the very instant the customer let a frontend/designer guy change the html code months later.
This is pure thedailywtf material.
I get that you do need specs but c'mon, if you ask a plumber to fit a pipe from A to B and he uses one made out of cardboard would you think it was reasonable for him to say "well you didn't specify it had to carry water, pay me 3x as much to put it right". And then you find out that this is the first time he's even seen a pipe, but his business card says "senior plumbing expert".
You expect a basic level of competence and familiarity with the problem domain and good faith from any professional whose services you hire.
Also consider that a program is a spec, and if you are going to explicitly specify every little detail then you might as well just write the program yourself, there is literally no sane reason to put an outsourcer in the middle.
A responsible developer is expected to ask insistently until the requirements make sense; whoever delivered the JSON DOM is stealing their pay, even if they did it knowingly as a joke or provocation.
strncpy(dst, src, strlen(src));
dst[strlen(dst)] = '\0';That line alone is cancerous.
The way a string is terminated in C is by a null character. ‘\0’ is a mnemonic. strlen() returns the length of a strong. It does this by doing a linear scan down the array until it finds the null, and the returns that index as the length.
That line of code does nothing useful. It means, “scan through the array until you find a null, then write a null there.” Which is exactly what you had to start.
So why would someone write this? Well, they were told to make sure that you always terminate your C strings with a null, otherwise you’ll run off the end of the array and corrupt memory. This is true, but this isn’t how you need to find the end of the array. They should have used the number from strlen(src), not strlen(dest). Even worse, strncpy() will put the null terminator in dest in this case.
The line belies any sense that the programmer had any understanding of what the code was actually doing. It’s a amateur mistake.
It has to be the size of dst, which can only be known at the site of allocation in C (except for a non-clean way of looking at heap metadata).
It never makes sense to use strlen() for an argument to strncpy(). Only the size of the destination buffer makes sense for strncpy() (usually sizeof(dst)). For other cases you would probably use memcpy().
I thought the point was that they didn't know if the dest is long enough to copy source, so they copy as much of it as they can.
The reason for the '\0' is in the case that src is shorter than dest.
A more explicit way may be to strlen dest > src then just use lenght(src) otherwise use str(dest).
But this might actually be more efficient.
Edit: And in this case Spolsky doesn't apply because it seems the developer doesn't get to allocate the dest length.
If dst is larger than src, the strcpy* family of functions will also copy over the null byte.
If dst is shorter than src, and if strlen(src) is used as arg for strncpy then the function will overflow in dst and your null byte will also be outside dst buffer. In hardened environments you wouldn't trust strings to contain null byte if it comes from network or user and use strlcpy with known size of dst then append a null byte at end of dst anyway to ensure it is null terminated.
Using strlen to compare buffer sizes is totally wrong and is the source for many bugs (hint, strlen doesn't actually return size of anything but the distance of the first null byte from the address you provide it).
[1] https://www.joelonsoftware.com/2001/12/11/back-to-basics/
Do we know that the developers were outsourced or which countries they were from?
This is the real problem. That PM of course wants the work done yesterday and has ego issues so even when the contractor does question it they just double down on what they originally said.
IBM sold a big outsourcing package to a major European insurance company: they would move ALL their stuff to the IBM Cloud (VMWare with a lot of custom BPM workflows to handle provisioning, etc.). None of this matched their other three big cloud projects, Bluemix, Softlayer, or their OpenStack project (Zenith -- my prior project), so it was all new from the ground up.
They needed to handle mass imports of DNS data from this insurance company (~1 million hosts) and other customers and they brought me in as a senior developer to do the grunt work. Since we were using ISC bind, I assumed we could use features like exporting zones with signed requests and take advantage of all the infrastructure work put into that server to scale up. Our Senior Architect poo-pooed that plan, so I started modeling the DNS records as structured data for consistent and complete import and export. Our senior architect also nixed that idea and said that we're gonna load the DNS structures with REST requests: the support people were gonna have a custom web application to load simple DNS export files (not zonefiles, mind you) into this system, and to make ad-hoc additions and changes. (When we had a new developer join, he had the same questions I had about why we weren't using all the DNS bind features since they were done and tested and perfect for our needs . . . it took a month to get him to stop asking.)
I got about 90% of the way through a solution with a Python Flask REST server at the center, high code coverage through unit tests (because it's Python and I was still finding edge cases that needed fixing to the very end), and Rabbit MQ for enqueuing the changes we've received before writing them out to DNS zone files. It works, but we certainly didn't need to blow April to August building out the architect's vision.
The reward I got was being pulled off the project when it was nearly code complete and would shortly go into production. I got a team based in India to manage in the mornings, while my afternoons and evenings were returned to developing Javascript for the BPM infrastructure jobs.
(When told I was going to leave for another company, IBM's counter-offer was to move me from my remote work-at-home location in Ohio, near family, to the Cloud Managed Services office in Rochester, MN, and then consider a raise.)
https://www.youtube.com/watch?v=C5d151lqJsA#t=2m4s
Do you happen to remember what the copy was on that page?
I'm sure it had to be something better than "Hey we made your cookie too big. Can we delete it? Ok thanks sorry about that."
I bet they went with something subtle and vague, make the problem a little unclear and complicated-sounding, maybe take a little credit for Websphere doing a good job in detecting the problem.
Like how I word all my work emails.
"We are unable to process your request due to a problem with your cookie, please click below to delete the cookie"
And then there was just a normal button input.
Their logic around having the user click the button instead of just automatically deleting the cookie and moving on (Something that still shouldn't be necessary if you are using cookies correctly) is that they thought users would be upset at IBM automatically deleting their 'data' without asking. That data just being a bunch of unreadable values for IBMs services.
- 2016. Australia decided to run a census, population wide (~22 million people), all online. If you didn't enter your stuff you risked a fine. You had one day to get a good 'snapshot'. The service, built by IBM, went down immediately.
'Census outage could have been prevented by turning router on and off again: IBM' http://www.abc.net.au/news/2016-10-25/turning-router-off-and...
'IBM to pay more than $30m in compensation for census fail, Prime Minister Malcolm Turnbull suggests' http://www.abc.net.au/news/2016-11-25/ibm-to-pay-over-$30m-i... (Have they done this? I don't think so)
They said it was an 'overseas DDOS' but there is no proof for that, personally I think it was Australians accessing the page from overseas.
Interestingly, this is not on IBM's Wikipedia page.
- In Queensland, IBM borked the rollout of a new health payroll system so extremely that the state premier banned the state from outsourcing to IBM (and that was before the census catastrophe!!)
>The commission, headed by former supreme court judge Richard Chesterman, tied a number of IBM employees working on the contract bid to serious ethical transgressions, including using leaked information about competitors to gain advantage and attempting to access tender responses by opposing bidders Logica and Accenture.
https://www.itnews.com.au/news/queenslands-ibm-ban-lives-on-...
In the end, the state had to carry the costs (1.2 billion) https://www.itnews.com.au/news/qld-health-ibm-payroll-court-...
It was supposed to cost less than 10 million.
This is also not on IBM's Wikipedia page.
This is blatantly corrupt, why the fuck would a government give a company a waiver from their contractual responsibilities on a failed project? That's utterly insane.
Of course it was fine 15 years ago but from that time we had quite a lot changes in tax system, many new conditions has been addad and in the result we have unstable deprecated software where maintainability is nowadays near to ZERO.
There are currently no people who knows how it works and who knows Informix 4GL language in 2018... After few years it will be probably replaced but cost of new project will astronomical (...of course).
I am not blaming purely IBM, quite often government officials are incompetent when it comes to software project planning.
Sounds like they drank their own kool-aid, e.g., "Products That Enhance and Amplify Human Expertise," rather than understand the actual limitations and possibilities of ML. And it seems to me that they're still doing it with this nonsense about a human-level "AI" debating stack.
The oversell seems a real shame in light of how much good can be done with EMR and machine learning / NLP.
that it will allow better and cheaper care.
This works well for IBM generally (the products are shit) but especially well for Watson because it's extremely easy to sell AI without getting bogged down in details. You want to identify brain tumors? We'll just teach Watson to do it.
Whilst IBM research might be able to pull it off, it'll never get to market because there is nobody capable of making good products at IBM anymore.
As an ex-IBMer this is so true and so frustrating at the same time. Engineers are thrashed about on a nearly sprintly basis by PM's with short attention spans and no understanding of how disruptive their continuously changing requirements are.
It doesn't help that IBM consistently puts the cart before the horse is even born and pivots multiple teams all at the same time such that nothing you build upon is stable or consistent. Working there was maddening.
The cynic in me says that every use of the term AI in any capacity is to sell experience and not functionality. When was the last time you used a product billed as 'AI' and thought 'wow, this is a huge game changer'? Siri is cool, but it's ultimately not super useful. Google translate is incredible, but it can only do what it can do because of the absolutely mind-boggling amount of training data that google can access. Most disciplines have the problem of not enough data, despite what 'big-data' folks say. In contrast, humans can extrapolate and make reliable predictions about the future based on really small sample sizes. We can pick up a new skill or recognize a new pattern with a high degree of accuracy really effing fast compared to a computer. This gives humans an enormous advantage. If IBM and anyone else in this space were really focused on delivering excellent real-world results, step 0 is building out world-class data integration and search tools (which we still actually suck at, weirdly.)
Product interfaces usually offer simple features to users and the value proposition is easy to see. Effective use of machine learning is well hidden upstream in a bunch of unsexy preprocessing or heavy lifting to get to the interface. Not something you’d ever need to emphasize in marketing, except maybe at tech meetups or in recruiting materials, but not to the end consumer.
It just makes pop references to AI-powered products more egregious.
The irony is that not one of these bills itself as AI. It's just "a product that works", and the company that produces it is happy to keep the details secret and let users enjoy the product. So you may be right that the term "AI" itself is pure salesmanship. When it starts to work it ceases to be AI.
https://en.wikipedia.org/wiki/AI_effect
Also - humans only look like we're fast at picking up new domains because we apply a helluva lot of transfer learning, and most "new" domains aren't actually that different from our previous experiences. Drop a human in an environment where their sensory input is truly novel - say, a sensory deprivation tank where all visual & auditory stimulation is random noise - and they will literally go insane. I've got a 5-month-old and a project where I'm attempting to use AI to parse webpages, and I will bet you that I can teach my computer to read the web before I can teach my kid to do so.
There was a time, not all that long ago, when SVMs, Bayesian networks, and perceptrons were considered AI. That's behind the spam filters, predictive keyboards, and most of the search signals.
There was a time, a bit longer ago, when beam search and A* were considered AI. That's behind the game opponents.
As the linked Wikipedia article says, "AI is whatever we don't know how to do yet." There will be a time (rapidly approaching) where deep learning and robotics are common knowledge among skilled software engineers, and we won't consider them AI either. We'll find something else to call AI then, maybe consciousness or creativity or something.
As much as I look into what’s being done with deep learning, I see they’re all stuck there on the level of associations. Curve fitting. That sounds like sacrilege, to say that all the impressive achievements of deep learning amount to just fitting a curve to data. From the point of view of the mathematical hierarchy, no matter how skillfully you manipulate the data and what you read into the data when you manipulate it, it’s still a curve-fitting exercise, albeit complex and nontrivial.
And
I left the arena to pursue a more challenging task: reasoning with cause and effect. Many of my AI colleagues are still occupied with uncertainty. There are circles of research that continue to work on diagnosis without worrying about the causal aspects of the problem.
I don't see how this follows from this:
the 'AI' tools we use are increasingly good function approximators
Nothing in the definition of AI says that AI has to work the same way the human brain does... and as far as that goes, we're probably not 100% sure that, in the end, the brain is anything more than a really good function approximator and some applied statistics.
I would say the canonical definition of AI, to the extent that there is one, is roughly something like "making computers do things that previously only humans could do". If people think "AI is bullshit" I'd say it's because they're applying their own definition to the term, where there definition imposes much more stringent requirements.
Yeah, maybe the marketing people said that..
Also, because something has been around for decades does not make it not AI. For ex the cheque OCR mentioned probably runs off (or can feasibly run off) of a neural network. I think the parent's comment holds well - not sure about the last line though ...
> when you say AI you don't mean AI as is practiced by most of academia and the industry but the vision of Artificial General Intelligence (AGI).
What I actually mean is people practicing what they call "AI" in academia and the industry have co-opted the name to make what they do sound more interesting. First it was called "statistics". Then it was called "pattern matching". Then it was called "machine learning". Now it's called "AI". But it hasn't changed meaningfully through any iteration of these labels.
To your point, I agree. They hype around the area has evolved much faster than the area itself.
then what is 'close to AI'?
FWIW, "AI" as a field has been around since the 1950's. So calling something "AI" in no way implies that the techniques are especially new.
I think you are on to something, put differently: If you need to use the term "AI" to enhance the marketability of the product it is probably because the product sucks.
YouTube recommendations aren't great, they have a short memory and my feed is rarely diverse, it just shows a bunch of whatever I just watched.
The most creative, intelligent and least frustrating "AI" I've ever encountered was in some games, such as Dota2 or many years ago F.E.A.R. They were frustrating but only due to unpredictability, even after hundreds of hours of playtime. YouTube and NetFlix AI after hundreds/thousands of hours invested are also very unpredictable and frustrating, but that's the opposite experience I am looking for in those situations.
Translate can be useful at times...like once a year when I want to comprehend a Japanese website, usually I close the tab after 2 minutes.
I used GMail for many years and still do to some degree but I'm moving to a different mail provider. GMail's spam filter is great!
Not sure, since 2 years it became acceptable to make no difference between ML and AI. ML appears smart because of bizillions of training samples and I feel very impressed when I hear of that. But yeah, at the end of the day it doesn't have exactly the biggest impact on me... ;)
You severely underestimate the bandwidth of your eyes and ears and other senses, and the volume of your brain's memory (despite it's uber-loosy compression). That's terabytes a day probably, if not big data than I dunno what is. Yeah, 99% of it is thrown away at passing through the first few hundreds of layers of your neural networks, but they still know what to throw away...
To get a digital computer on "equal" terms with the zillions of hacky optimizations your semi-analog brain uses you need a shitton of raw power and data volume ("if you don't know what to throw away of the input data, you need to just sift through all/more of it") to compensate for the fact that you don't have N million years of evolution to devise similar hacky optimizations.
Also, humans work as a "network of agents", that's also recurrent (aka "culture"). Current sub-human-level AI agents are far from any sort of reliable interop.
My guess is that we'll get human level performance levels at AGI tasks when we learn to build swarms of AI-agents that cooperate well and "model each other", and few people are working on this... Heck, when it happens it will probably be an "accident" of some IoT optimizations thing, like, "oops, the worldwide network of XYZ industrial monitoring agents just reached sentience and human level intelligence bc it was the only way it could solve the energy-efficiency requirement goals it was tasked to optimize for" :)
Sample size and record size are two different things.
This is so common there’s a term for it https://en.m.wikipedia.org/wiki/Moravec%27s_paradox
Evolution by natural selection is the OG genetic algorithm, and it's been "running" on billions of organisms in parallel for hundreds of millions of years. The intuition that we take for granted such as the abstract concept of a shape is all hard-coded in our brains from trial and error.
No. Siri is shit.
I’m at a large customer of theirs and they are bleeding the customer for every dollar as they get phased out. Very low caliber of services professionals too.
Watson is a $@#% amazing information retrieval system. Information retrieval is only a small part of what people think of when they think, "AI".
Which I imagine still has lots of value on it's own for large, complicated data sets.
I'd actually like to give Watson a spin for an IR problem I'm looking at, but, thanks to their hype machine being set to overdrive, they've got the thing priced in the "The Bold Leaders of the Future Creating a Bright New Tomorrow Full of People in Glasses Staring Wistfully Toward the Right Edge of the Photograph, While Blue Curvy Streaks Wave Through the Background and Random Zeroes and Ones Float Around Their Heads" tier. Sadly, I've only got a "businesses solving business problems" sized budget.
AI is not simply things that a computer can't do yet. But I think most of people who aren't currently trying to sell a piece of software would expect AI to include some things that you don't need to do to play Jeopardy. I'd want to see general-purpose pattern recognition, for example.
Here are a few of my notes (my words not the interviewee's):
- in order to use data science, you have to have creative people thinking about data on the front end
- they don't have to be data scientists, but they need to be creative and want data to support decisions and iteration via feedback loops
- that creativity and desire will lead to "doing good data science"
- management on the receiving end of data science output must be intelligent in terms of synthesizing many inputs and have a strong desire to puzzle through the implications. If management is asking the data science to actually make the decisions - the situation is broken
- data science must be done with provisions for decision support and feedback loops; this is the output that is helping drive the business.
- Lack of desire for decision support and feedback loops leads to "fancy pets" and management using data science as a means to brag about what they are doing; but the data science might not being doing anything to drive the business meaningfully.
- data science that attempts to actually make decisions vs providing decision support is likely in the category of "commodity data science". Corollary : non-commodity data science is the kind that supports decisions in executing higher-level business strategy. Strategy at that level has rather unique attributes and is embedded in unique circumstances for a particular business. This requires a good data scientist to help tackle.
(hope this is useful)
(edit typos,grammar)
whenever I'm asked to design a database for an early-stage system (I work in early stage tech ventures), I ask the following:
- what are the questions that this database should answer for you? How are those questions supporting your business goals 3,6,12 months out? (I'm trying to get to the business requirements here)
- who will be asking those questions (I'm trying to put together some user personas in my head)
- how frequently will they be asking these questions? corollary: how often will historical data be needed? (I'm thinking hot vs cold and complexity of retrieval, minimally required performance)
- how much data to we anticipate is needed to answer the questions (this is really tricky in new ventures - often the answer is more data than what will actually occur in practice in the first year)?
- finally, what systems & tools are people using to ask the questions and be notified of events? (I'm thinking about interfacing, apis)
its all an attempt to stay very focused on the questions and business drivers and the people who use the answers.
We run prediction markets inside companies and find that if we don't establish a good lifecycle of asking forecasting questions, having people respond with probabilities, then decision makers REACTING to those probabilities in some way (whether they agree with them or not, just acknowledge their existence) the likelihood of the project failing is far higher.
how many of those managers will now be able to get even a higher paying job because they have manager of Watson AI project on their resume?
> A clinic could use the system to search its patient records and find, for example, all the men over age 45 who were overdue for a colonoscopy, and then use an autocall to remind them to schedule the dreaded appointment.
Maybe this was a terrible example, and the author didn't grasp a good example of legitimately non-schematic data points?
Even if successful, a system which could "interpret" a health record (such as a freetext note) using anything other than properly codified data would set the health industry back a decade. Moving doctors away from freetexting their notes is the only way to advance the industry.
Doing so could have incredible utility for sharing data across various clinics/hospitals/pharmacies/etc.
My previous doctor (who was probably mid-50s) didn't use email or any kind of secure electronic messaging system. Everything had to be faxed to him.
My new doctor who is younger uses all kinds of digital tools including a voice recorder with a pre-trained text-to-speech engine that understands medical terminology and codifies the transcription based on keywords.
So it's not entirely getting away from freetext but at least it's extracting some structured data from it automatically.
I can imagine someone who doesn't know at IBM selling a product:
"Hey we will solve all these problems like magic!"
Then IBM comes back:
"Hey do you have all this data in a specific format and a ton of time to enter and test it and maybe we'll get back to you???"
That's a big loss of trust there with the customer if you come back with that.
It seems like these are products where a lot of caveats needs to be made clear to customers and a real careful technical partnership formed with them to succeed long term. You have to bring the customer along for the ride and exploration and keep them excited for a long time it sounds to make it work.
To see one of their articles with the common press confusion of mixing different definitions and interpretations of AI (correct or incorrect) doesn’t help build confidence.
For example, what was used to play Jeopardy vs. approaches being taken to improve cancer treatment, are just so different, it seems almost disingenuous to throw it all haphazardly into one conceptual bucket.
The article does IBM a disservice in some ways. They come off looking bad overall but some of the failed projects mentioned like MD Anderson, failed for reasons beyond any control they had, other than recognizing some obvious red flags earlier and detaching their name and participation from it.
On the other hand I believe the article lets them offf the hook to easily when they bring out the old trope they’ve been using for years, which is encapsulated here:
“IBM Watson has great AI” [one engineer said] “It’s like having great shoes, but not knowing how to walk—they have to figure out how to use it.”
It doesn’t make sense to say, xyz is great we just have to figure out how to use it, as a stand alone argument. It’s nonsensical unless you mention something about the seeming implied untapped potential, specific innovations, novel approach, or whatever makes it great.
I’m not familiar with all their IP so maybe there are some great things, you just don’t get to claim that and get off the hook so many times in the press without providing at least some detail or reference point.
Once scientists in biology and healthcare get on board like they are in linguistics and computer vision I'd expect things to pick up.
I had a contract gig at IBM Advanced Technology in Boca for about 18 months in early 2001-2002. Talk about missing the boat...
I was brought in to prop up a soon-to-be-failed project for the Japanese government...basically a "Napster for Tokyo" that would allow paid-for-play C2C song sharing for customers of "the Big 5" record companies.
I asked simple questions that no one could answer...why would people pay for content when it was so easily available via other means? you are using DRM how??? really? you need a special player to play the music?
Why would anyone do that?
I stayed for a few simple reasons...the fat consultant check I cashed every Friday. Exposure to some outstanding engineers and coders where I got to learn from true talent. The great strip club on A1A next to my rental in Lauderdale-by-the-Sea.
But reading over this article reminds me that IBM is just too big to get out of its own way, and has been for the longest time.
[edits]
Given all the expectation of its product and the importance of ML/AI, it makes sense this was impossible.
Two years ago at their vegas conference they had a coffee shop that used AI to recommend coffee types. I thought "boy, they don't understand this technology".
I worked on another one for this company called Vibration Advisor which diagnosed odd noises in GM cars.
Another interesting thing was the transition from special purpose hardware - Lisp machines - to C code on commodity platforms. A contrast from today's ML moving in the other direction.
It'll be interesting to see when specialized ML focused silicon will become readily available. Right now I find ML libraries that are able to run on blended architectures (any combination of CPU and GPU's) much more exciting/impactful than TPU's. The ability to deploy on just about any cluster a customer may have available is huge.
Are we close to it being technically feasible , leaving aside regulation and interpersonal qualities doctors bring to the table ?
The likes of INTERNIST, CADUCEUS, and MYCIN have been around and provably accurate starting in the late 70s through the mid-80s. MYCIN even arguably sparked the 1st AI boom. But there were ethical issues with computer-aided diagnosis that I'm not sure have been solved/overcome.
Perhaps the current startup generation can get past them with Zuckerberg, Kalanick and Holmes as role models. :)
Of course, there are those who make their way into the small clique of people at any tech company that get to do impactful, fulfilling work. But any given person is unlikely to be one of them, and if you want to be one of them you usually have to kill yourself working crazy hours first. (And probably afterwards too.)
Brilliantly put
Funny thing is, when it did get out of its own way, what did we get? The IBM PC
Heh... I did some work for IBM at the office of Cypress Creek Road back in that same time range. My fondest memory of the entire experience is eating at the Calypso Restaurant[1], a great Jamaican / Caribbean Islands place nearby.
I'd almost go back to Fort Lauderdale and work for IBM again (if they even still have a presence there) just for the Jamaican Jerk chicken from Calypso.
Q: What do you get when you cross Apple and IBM?
A: IBM.
While I was working at Kaleida, I gave a wild ScriptX demo to Lou Gerstner using a bouncing eyeball to navigate a map of interactive rooms. After the demo, he complained that "The eyeball was a little too right-brained for me." I was all "Dag nab it, I should have used the other eyeball!!!"
https://medium.com/@donhopkins/1995-apple-world-wide-develop...
"The last thing IBM needs right now is a vision." -Lou Gerstner
"Phytel’s contribution was analytics paired with an automated patient communication system. A clinic could use the system to search its patient records and find, for example, all the men over age 45 who were overdue for a colonoscopy, and then use an autocall to remind them to schedule the dreaded appointment"
This shit isnt AI it's literally a database query and then some 3rd party library to send a text message or a phone call.
That’s already called Musk’s Law.
For a long while now, IBM has been treating "AI" as a product that can be managed, packaged, and sold by "general" business managers -- think MBA-types with only a superficial, qualitative grasp of deep learning and AI. Doing that with rapidly evolving technology is a sure-fire recipe for failure.
Most such MBA-types today are ill-equipped to manage, package, and sell "AI." They're roughly in the same position as English or History majors who are asked, say, to manage, package, and sell a new kind of quantum-computing technology without knowing or understanding much about quantum physics. The technology is moving faster than their ability to keep up.
IBM's mismanagement is a shame, because the system they showcased nearly a decade ago -- the one that competed and won in Jeopardy -- was state-of-the-art at the time.
Stories like this are not a surprise, it's IBM's way of doing business. Maybe Watson will learn HR and just fire everybody from middle mgmt on up..
My comment mentioned specifically "MBA-types with only a superficial, qualitative grasp of deep learning and AI."
MBAs who understand what they're managing (and who know what they don't know) are not in that group. And BTW, I suspect most MBAs who read HN are not in that group either :-)
> Both Phytel engineers say the offering managers didn’t have technical backgrounds and sometimes came up with ideas for new products that were simply impossible.
The death knell of all (potentially) good products. I don't know why this is so often the case. All software companies need engineers involved in product development decisions. Period. It's not optional.
Facebook who was smart about this. They hired or retrained technical people to fill many business roles in marketing, product development, project management, etc.
I'm not sure why technical people are restricted to merely being the builders in these companies. Lots of other companies recruit internally from people familiar with the end product and train them in other business areas.
> these potential customers weren’t impressed. Instead they asked for something resembling Phytel’s old system.
So they simply imagined a new product without interviewing potential customers beforehand on what they actually want? They spent years merging databases of two big systems, pivoted multiple times, to find out there wasn't a market for it in the first place?
Why aren't the 'offering management' people getting fired?
People who know that healthcare is different try to warn them. They don't listen. Instead they charge in with people who have no experience in the field.
From the article:
After the acquisition, IBM management started the process known internally as “bluewashing,” in which an acquired company’s branding and operations are brought into alignment with IBM’s way of doing things. During this bluewashing, “everything stopped,” the first Phytel engineer says, and the workers were told not to focus on improving their existing product for current clients. “People were sitting around doing nothing for almost a year,” the second engineer says.
In the middle, there are 100 layers of middle managers that completely cock everything up, and the really sad part is that they have enough say to really cause damage. One of my first proper white collar technical jobs with them was an L2 support job for this network performance monitoring suite for huge networks... mostly large, national ISPs and the like. The job required maybe a just-post-jr-level sys-admin knowlege of networks and UNIX systems while also having smooth customer service skills. Definitely a great step up from my previous lower-mid-level IT jobs and call center work.
I had three(3) managers. Three! I had a technical manager, a non-technical manager, and my actual manager, who was the head of the department.
At the highest levels, the management was talking about switching everybody's workstation over to Linux. Everybody from admin assistants to developers to managers was supposed to be moved off of Windows at some point in the relatively near future. I was psyched— I hated windows, and the product I supported ran on Solaris, so not having to deal with the extremely primitive (at the time) tools like Cygwin to get some UNIX functionality on my machine was great. They seemed to be positioning themselves to sell the consulting for other large companies to do the same thing.
Though we got no word of this internally— I only knew from what I had read in articles— I found the internal workstation disk image on the intranet and eagerly installed it. It was pretty smooth! I was excited! As I was getting my tools set up, I noticed that it didn't have the internal bug/ticket tracking clients installed, so I cruised on over to their intranet page... hmmm, nothing listed for Linux. After hours of searching, I found some internal discussion showing that, months earlier, the department that writes that software unilaterally decided that they were discontinuing their initiative to port those applications to Linux. While there was an extremely limited CLI to these tools, critical functionality was literally impossible without the GUI app. Without the ability for anybody on their Linux workstations to interact with tickets or bug reports, the Linux initiative was pretty much dead-in-the-water for most technical people and their managers.
Perfect example of just how badly their forest of middle managers completely messes up great executive initiatives that the bottom of the food chain really wants to embrace.
(I might have gotten some of the details wrong. It was 13 or 14 years ago and I drank a lot back then.)
Look at a company like Google, which, without a doubt, has some of the best engineers in the world. How many false starts and just flat out poorly executed projects/products have they had in the last 10 years? Way more than you would expect from a company that puts such a premium on hiring the best.
The audience that gobbles up their ad campaign during the Masters that touted their "block chain" logistics probably wouldn't even notice that they had layoffs at Watson health.
It seems to me that Watson is basically just IBM's version of AWS/GCE services (at least the non infra ones). But it gets thrown around as a buzzword so often. The marketing makes it look like there's a single AI codebase that can be accessed through a bunch of APIs, but I would be very surprised if that was actually the case.
https://news.ycombinator.com/item?id=15456211
It's reasonably in depth and appears sincere.
tl;dr: it's a good search engine and a disparate set of machine learning tools. Sales is promising Hollywood AI, but the reality is that it takes a sizable project team to build anything worthwhile.
Scanning past commentary, it seems that startups are eating their lunch (more nimble, dedicated to customer space). I'll add that half the machine learning battle is getting access to data, so hyping the brand makes sense strategically.
Both, sort of. There is a "thing" called Watson, which is related to the Watson that played Jeopardy. But "Watson" is also a brand which lumps in stuff that has absolutely nothing to do with the "old" Watson.
To illustrate a bit.. "Watson Health" is (or was) made up of a ton of people and technologies who came into IBM as the result of several acquisitions: Truven, Phytel, Explorys, etc. In many cases, they repackaged stuff from those vendors, gave it a "Watson name" and shipped it. And some of this stuff was literally no more sophisticated than linear regression / logistic regression, etc.
I wonder if DeepMind at Google has a similar problem. It is certainly getting a lot of headlines, but there are plenty of other AI groups within Google that do business-relevant things like improve search or ad matching or make Google Home's voice recognition work. I would not be surprised if in the long run DeepMind becomes a group that performed a neat stunt with Go, but kind of fades in practical relevance, like Watson with Jeopardy.
It's pretty common in many industries to have products to showcase your chops while providing zero real world value and zero sales.
He called his local sales office. Somebody said, we've got a couple in a storeroom someplace. I'll bring one over. No charge.
That was customer service. That's how they built their reputation. Now they seem to be squandering it.
Sounds like they're headed now in a direction where they sell their artificial intelligence as being smarter than their customers. Sounds like they insist on disrupting their customers rather than their competitors. That Doesn't Work.(TM).
Every time somebody does that to a hospital it gets harder for other vendors to sell actually-useful stuff to health care operations. Not good.
Self-driving cars have killed pedestrians, Watson isn't doing all they wanted... is this the dawn of a new winter?
Anyways, may I interest you in cheap VR headset?
This may be possible this time round, because we’ll have a very good record of who said what and when via the web.
Without any kind of accountability, history will continue to repeat itself.
How about hyperbole.com, where you can google academic researchers and industrial leaders and pull up quotes from them, dated and fact-checked.
I’m sure you must be able to train a deep net to do this. They can do anything.
It’s not really great. They can automate the process of finding reports, but the truth is, we have people already doing that and all the reports they’ve been able to find in their POCs were either useless or some we already had.
I’m sure ML has potential, but our analytics department is run by economists and political scientists who can really utilize ML and I’m not sure they could be retrained, even if they wanted to, which they don’t.
We can’t really hire ML personal either. The only people who do it well enough are PHDs, and they have much better offers, and we’re at least 5 years away from having a candidate pool of people with the right education mix (Data + political science) and probably 10 years away from figuring out how to utilize them as well as finding enough funds for a position.
So ML is mostly left to independent companies that corporate/consult with muniplicitaties in projected owned and run by the municipalities). IBM simply doesn’t do this, and the consultant companies that doesn’t suck all work with something else.
Of course this is the perspective from the Danish public sector, it might be different else where, but I have friends who work similar positions as mine in banking, and they’re telling me the same thing.
This is really untrue. Maybe that's part of your hiring issue.
Have you tried hiring people who have documented experience doing ML commercially? Because I could understand how you get bummed out a lot if you hire recent grads to do ML.
People are faulting IBM for over-promising, under-delivering, using misleading advertising, and internally seeming to have foolish management practices.
> Once IBM has been in this technology game for a little while, I'm sure they'll get the hang of it.
I'd assume the parent is being sarcastic
> IBM Watson Health has initiated a significant RA across multiple offices.
If IBM stops milking the cow like the services, asking the customers to integrate, It will die a miserable death, and unfortunately a premature one.
It should be pluggable, like set of pick and choose building blocks of interfaces, that only should take domain expertise and custom specifications as input from the client (Which ever domain.) Until then AI only will become ambitious Sunk cost.
Undoubtedly IBM has done amazing work popularizing AI with Jeopardy and the debate bot. However, I think that halo extended to its competitors that, frankly, offer more things developers really need.
In particular, I'm not sure what sets comprise the numerator and denominator of "most technological revolutions".
PSA for anyone with a disease that man and corporate picked a name for - you are feeding yourself towards that disease. Fix your diet (and 100% rid yourself of animal products, processed sugar, wheat, and any "food" that comes in a package with a label on it - hell stay away from anything that isn't a fruit or a vegetable), go on a dry / water fast, and heal yourself. Stop relying on doctors, AI, whatever the hell they produce supposedly for you.