95% of AI Pilots Fail
selector.ai
selector.ai
95% of Companies See 'Zero Return' on $30B Generative AI Spend - https://news.ycombinator.com/item?id=44974104 - Aug 2025 (415 comments)
95% of generative AI pilots at companies are failing – MIT report - https://news.ycombinator.com/item?id=44941118 - Aug 2025 (167 comments)
The AI vibe shift is upon us - https://news.ycombinator.com/item?id=45003052 - Aug 2025 (100 comments)
95% of Companies See 'Zero Return' on $30B Generative AI Spend - https://news.ycombinator.com/item?id=44974104 - Aug 2025 (415 comments)
95 per cent of organisations are getting zero return from AI according to MIT - https://news.ycombinator.com/item?id=44956648 - Aug 2025 (14 comments)
95% of generative AI pilots at companies are failing – MIT report - https://news.ycombinator.com/item?id=44941118 - Aug 2025 (167 comments)
At my last company thankfully I was able to limit the losses to just under $450k which sounds like a lot, and it is, but it could have been about $5m/yr. I didn't sabotage anything, either. It was mostly through simply asking a lot of questions early, repeatedly suggesting we write plans down and make sure everyone on the project is up to day, and suggesting we test drive components before buying the whole car.
Spent a bunch of money, rolled out the pilot project to 70 folks, and began 30 day checkins. After 30 days we had a retention rate over 80%. By the 60 day mark, we were down to 40%. By day 90 we were at 22% and landed at 11% after 4 months and never went higher. By month 8 we cancelled the project and the 8 people still using the tool were ok letting it go, feeling it didn't help that much.
All because someone from the board said, "we need to use AI, anyone not using AI in a year will be out of business." The CEO asked what we should use it for, and the board member said, "I have no idea, that's your job. But we need AI somewhere. Look here, it can buy a plane ticket for me now!" and then wasted the next 40 minutes of the meeting talking about how amazing AI agents were. We're not a tech company, he was definitely NOT a tech person.
On the other hand though, with hindsight, imagine the board member saying the same thing about the PC, the internet, mobile computing, cloud computing…
I think both of you are right and it’s only a matter of timing. Time will tell.
It’s easy to imagine that we’re going to make the AI analog of paypal or amazon, but the vast majority of people flailed at doing anything useful with the web until patterns had been established
For the deployment of the PC, I think the vast majority of companies start from an environment where there were already terminal systems or shared workstations available, and the adoption of PCs happened as they became powerful enough to accomplish the tasks that were already being done on the larger computers. It wasn't really a "thou must use PCs, though I know not how they are useful" mandate.
A better example of such a mandate can be found in education, and most of my personal experience with such thou-must-use-technology mandates has been that they've been similarly ineffective as the AI mandate.
To be fair, if the goal is to learn and discover how a new tool works and when to use it, then a legitimate strategy is to solve a bunch of problems with this new tool
The goal is just "make all the money at all costs"
You always start with the problem that needs to be solved, not the solution. I'm concerned we'll have another bubble pop like the dotcom crash. A lot of companies are spending money like mad to have any kind of AI thing they can market about without regard to "Does it work right? Does is solve a problem?"
It took my former company a year and half a million dollars to learn that what we actually needed was a completely different tool that promised to do a lot less, but did what we needed it to do quite well. Mgmt got stars in their eyes with all the things the first tool might be able to do someday, and didn't pay attention to what it could do today.
We switched to the other tool at $70/month/user and saw a retention rate over 90% at 6 months. Exact same test group, exact same type of tool, but this tool was more focused and tangible. We went from spending $450k per year at with the first company to $59k/yr with the second company.
AI isn't magic, You can't skip all the project planning steps you'd normally do just because this is AI. Don't fast track it, don't ignore testing data or feedback, etc. It's hard to push back on excitement but it's important that we do. When our emotions carry us away that's when we make worse decisions.
But aside from that, were did you get that number? I didn't see any direct reference nor anything that would let you break it down to $/dev
Anyway, it misses one of the current biggest problems with all things AI in the workplace: It has attracted every resume-builder and ladder-climber who don't know what they're doing. They just want an AI initiative for their resume.
As long as it's not too public (like the Taco Bell AI drive through disaster) it doesn't really matter if it's successful or not. They can spin it as a success and the next company they apply to won't be able to check.
The same thing happened a few years ago when every PM and rising manager was looking for a way to put blockchain into products. Before that it was "big data", and so on.
> Before that it was "big data", and so on.
Cloud, micro services, NoSQL, serverless, …
We never learn :(
eugh, yeah that's definitely the case. I stopped reading once I got to the first repeated paragraph about data quality.
Speaking of which, anyone have a link to the study itself? Since the linked marketing post doesn't even provide a link to it, just a bunch of links to their own platform and social media accounts.
Probably better for the submission to link directly to the study, without the extra call-to-action fluff at the end?
they seem to be pushing their own take on how to implement agentic AI
frankly this study and the headlines surrounding it all seem like a bunch of marketing spin
https://www.geekwire.com/2015/amazon-makes-echos-alexa-avail...
of course it wasn't as big as this new AI push, but since the 80s AI over promised and under devlivered, it's not the fault of AI, but the greed of the business people.
I thought this system just picked the closest runway and broadcasts "get out of my way, I'm making an emergency landing!"
If they did, they might have addressed many of the issues described in the report as the cause of the failures.
I'm not the biggest fan of current AI technology, but claiming that it's a failure because it doesn't magically solves your problems, which was always more organisations/managerial in nature rather than technical, is a little disingenuous.
Sure this type of leadership can work in limited short term circumstances where you have a very clear and meaningful objective. But if you can’t quickly get the team itself to believe in the project - and make it THEIR project not YOUR project - you’re going to fail.
Just like what often happens in software documentation: a few seconds in the right spot can save hours or weeks of Chesterton Fence work