AI Doesn't Have ROI
wheresyoured.at
wheresyoured.at
6 months ago it was use AI all the time go! Now companies are putting use limitations in place, strict budget controls, and the wagons are circling around various “AI labs” teams that cost a ton and have shown little to no ROI.
It was all fun and games until the bill arrived. Now it seems there’s a mad rush for AI companies to IPO before the music truly stops.
Some people believed LLMs were that magic box for a time, and that time is coming to an end if the parent poster is correct.
The scary thing here is that I know how the parrots work, what they can't do, and how to get around them. The typical person calling will assume they've been helped by the parrot, which is just going through the motions without comprehending anything or fixing anything.
Theres an extremely ugly financial picture developing that those with full blown AI psychosis appear unable, or simply are unwilling, to see.
Ads are a zero sum game where there’s only so much ad money to go around. AI doesn’t grow the pot. Google isn’t going to lose the ad game, it would destroy them. Google got scooped early on with AI search but is roaring back now.
Also consumers won’t pay high amounts for subscriptions, that’s enterprise territory which doesn’t tolerate ads. And these are the folks now slamming the brakes on spending.
Net, “ad revenue” is not even close to a viable plan to save the present train from spectacularly flying off the tracks.
Enterprise isn't slamming the breaks on spending. At worst they've transitioned from spending like drunken sailors to spending like mildly inebriated sailors. Every single white collar worker is still going to have an AI subscription. And for people like programmers they'll still spend $1k on them.
It's no surprise that when ROI remains elusive (it's hard to measure for any knowledge work) and costs are skyrocketing that the C-suite wants to slam the brakes.
The slow part is finding those customers, syncing your deliveries with their processes, giving them time to meaningfully assess new workflows and features in the course of their business operations, collating the feedback you receive from all of them, and merging that feedback with your organization's long term growth objectives to drive new ideas into development. Well-developed organizations layer this inescapably slow flow across numerous parallel channels so engineering utilization can stay high since healthy engineering already cycled much faster than these market-engaged flows can.
Neither coding nor internal prototypes were the slow part. Market engagement and market-informed product planning were the slow part. And still are.
You may not realize it yet, and maybe you've just misrepresented it, but most of what you seem to be describing is usually considered wheel-spinning and navel-gazing. You may have made your internal process cycle faster, but you very likely just turned a wasteful busywork churn into a more efficiently wasteful busywork churn.
That is not my experience mentoring 100+ startup founders. Building a prototype, the gateway to serious customer engagement, used to take months and many startups would die before finishing their first one.
In my experience, the slow parts are around making sure you're aligning on a long-term vision, understanding the domain and customer problem well enough, balancing the technical aspects/speed today with quality down the line, etc.
This probably does depend on what kind of tech problems you work on. If you're purely doing frontend development I'm sure you'll be faster. If you work on complexer systems with e.g robotics/hardware interaction, I can't see it being significantly faster. YMMV :)
That argument always rings hollow to me. What systems were you prototyping that took that long? I don't need to build a complete MVP to present a design. Or understand an API.
In the visual art industry, there are thumbnails and storyboards that are the first iteration of any project. They are quick to produce, and can serve as the basis for brainstorming. No one wants a finished picture, because it restrain your thinking. Too much details and you start bike-shedding.
Only when you've solved higher concerns and have a concrete direction that you start to invest physical efforts. But that does require someone to have the capacity to discern higher concerns from crude sketches. If you don't and rely on "I'll know it when I see it", then you sure need finished products to clarify your thinking.
> AI really helps make that feedback loop 10X faster
Why be imprecise or exaggerate? Give the exact multiplier. Words mean something.
Iterating and prototyping can certainly help there, but at the end of the day if you launch a non-working (or non-reliable) prototype, you’re going to just have angry customers, not happy ones.
And that rarely works out well long (or even medium) term.
And most of the value from iterating and prototyping is from learning, something the AI kinda screws with.
Has this man ever heard of Jevon’s paradox?
Also all of these claims are objectively wrong today because the goal posts for what AI have been moving this whole time. The models we have today do more, are faster, smaller, and cost less than what was available 3 years ago.
But Jevons paradox explains the increase in consumption, but it does not necessarily answer the question of business profitability
"What we have done is we have tempered the pace of hiring, and we -- and this is broadly across the company, but specifically from an engineering standpoint -- the hiring ramp we have for the remainder of the year is significantly lower than what we thought it would be when we came into this year."
Uber's response looks to be cutting the number of engineers that generate tokens, not to cut the AI that is generating them. These headlines about Uber are not the victory people are portraying it to be.
- AI is a component of a larger product sold
- The product improves the metrics that customers care about, typically autonomously
- The customer is paying for the outcome, regardless of whether or not the product had AI in it
'Copilot' style AI features are much harder to measure ROI on, because they are typically further away from the base metrics that make it easy to measure ROI, and are typically used for specific tasks in a long web of other tasks within a professional job
all industries, all are willing to pay 10s of millions even with the tech being somewhat immature
I am very bullish on AI as a tool, but not as a way to completely restructure the economy overnight. Doing things is hard, and better tools don't make fundamental problems about change go away.
I read this today which really resonated and is relevant: https://deadsimpletech.com/blog/attack-on-competence
>allowed to burn thousands of dollars of tokens on a $39-a-month subscription
and whether the AI is worth it if you do pay what it costs.
I always thought the burn thousands of dollars of tokens without paying bit was unsustainable and harmful for the environment, electricity bills, investors and the like.
But I think it can do worthwhile stuff even if you pay. Like a small job I did was to get the emails of conference attendees off a website. It was tricky as they didn't want to be scrapped so I chatted to gemini and it helped figure out how to use tampermonkey and wrote a script. It was probably <$1 tokens and saved a couple of hours of mucking around. There must be a lot of things like that.
https://github.com/microsoft/vscode/issues/309947
I am considering pinning whatever the earliest version in which this setting was introduced. I can't think of a single feature VSCode has implemented in the last three years that I couldn't go without. The binary for 121 is like 50% larger than 120.
Not that I think that they _should_, that's all a farce as well, but it is something I could see others trying to use these data centers for.