* Many businesses don't need frontier level intelligence anyway.
* It's completely stateless. If your local LLM machine catches fire? Nothing was lost. Buy another.
And before that, businesses will be able to get decent results with dedicated inference hardware.
The future is not a single chat bot session of bs=1. The future is many agents performing many tasks in parallel for a single user. Large GPU clusters will always have the edge in efficiency.
Agentic AI has pluses and minuses for cloud efficiency. The plus is that usage could be very bursty, but the minus is that agents will more fully utilize a local system. The main disadvantage of local AI is that you would be paying a large amount for a system mostly doing nothing. If it's constantly working on different projects and integrating that data, you get use out of every penny that you spent. Every GPU you added would instantly make the thing smarter.
What's more, your local AI could offload an agent to the cloud if it needed to. It could do this rationally, based on your personal desire for privacy.
I could absolutely see local AI taking the place of voicemail/call screening completely. Call me and you get my AI, who will route the call to me if approved, or even choose to service the call itself. If a friend of mine calls who doesn't have $20K to spend on a great AI rig, they would certainly have permission to steal otherwise wasted cycles from mine. An answering machine isn't much different than an issue tracker, and some of those are 97% AIs having perfectly intelligible conversations with each other, and 3% humans being eagerly serviced by AI.
The old objection was "this is so hard to set up, nobody wants to run a server!" Now it can set itself up.
tldr; you don't have to rent from some data center. You can get high utilization out of a local box. This 1) puts a hard ceiling on what a data center can charge, and 2) they won't be able to compete on privacy, which locally can be complete.
The abstract benefits are quickly outstripped. The same way adding more highway lanes never improves gridlock. Its inducement.
Large orgs with significant demand might go out and buy local LLM hardware, but most businesses probably don't want to bother dropping $2k on a box with a beefy GPU and would rather just pay the lowest subscription tier so their employees can occasionally make queries.
Plus, you know, the whole economies of scale thing. Local LLM has a lot of privacy and independence benefits, but I'm not really seeing the world where it becomes more energy- or cost-efficient to buy your own hardware (and use it 1% of the time) versus sharing a giant machine, or even the same machine, in a datacenter (where it has a much higher utilization factor).
You may have better and more reproducible results using browser controls that aren't image based, or writing tools that completely sidestep browser use.
I will say one thing that frustrates me is the opaqueness of the billing model. I basically just have to pay a random amount. I guessed that it navigating the web is relatively pricey from watching my usage as it does stuff. And the whole thing is worth 10x what I pay for it anyway, it just would be nice if the pricing were somehow more transparent, even in hindsight. It can explain to me what is thinking as it does stuff, it could also explain the tokens.
Marketing definitely. My food truck side is relatively high volume and it manages my kitchen and warehouse side, basically generating all of the instructions my employees follow, managing and updating my PoSes, creating signage assets for specials, etc.
Reels/posts production and managing ad spend.
Here’s a fun one. I switched payroll providers after several years and suddenly my unemployment insurance rate went from 0.8% to 12.75% which is borderline debilitating to me. I knew something was wrong but not what and I work a lot of hours and calling the state takes forever and is usually unhelpful.
ChatGPT figured out that it was a penalty rate and dug in for me. Turns out because I’m seasonal and have no payroll for one quarter of every year, Gusto did not file a quarterly wage report, so even though I owed nothing I was delinquent. Gotta love government, it’s the only place where you can be delinquent for $0.
ChatGPT filed the report and requested a retroactive re-rate, which they granted. I’m sure I would have figured this out eventually but it would have taken hours, or $5 in tokens.
Would you say ChatGPT does everything you need today or do you still see some gaps? i.e. things you think it should be able to do but currently doesn’t, or things that still take too much effort on your part to setup chatgpt to do it.
The AI is good enough to do a lot of tasks but the tooling just isn’t caught up to it yet.
I’d say it’s freed up ten hours a week of my time. And that’ll only improve.
But, I think I’m on the black on it already after just a few months of heavy use. For instance I just tell it to book my dumpsters, bathrooms, sanitation crew, and security for X event. It goes and pulls event details, looks through my email to see who I get those from, and emails them relevant details, with no prompting.
Another good example: we launched a really fancy hot cocoa concept last fall that was a hit and I wanted to try to go to all of the local pumpkin patches in October and Christmas tree farms after Thanksgiving to serve when they have big crowds.
I asked it to contact all of the ones in my area and it sent out 70 emails and I booked several spots. It made me a nice database so I can see who followed up and who I need to reach out to again, etc. and it just gets those from my inbox.
Hours of my time saved with simple prompts. So while some things take awhile to pay off, some are instant.
Seriously, you can do years of Deepseek inference for the hardware to run just 1 or 2 requests against a slower, dumbed down model on your own hardware.
It makes no sense to buy hardware right now, when the price is completely disconnected from any material reality. It's much better to use the cloud providers VC funding by using their cheap as F offering. Either AI becomes less useful, or hardware costs come down. Either way, you'll be in a better position in 3 years than you are today.
The economies are currently out of whack because of underproduction of components and memory, so LLM providers have a few years of runway to entrench. Plus, the whole capex Vs opex thing that helped AWS will help here too, for sure.
At some point, though, things will change. More production will come online, and providers will have to end the current speculative subsidizing and jack up prices.
It's a bit like the dot-com era: the initial rush to land-grab web portals and e-commerce sites eventually died, once enough skills and infrastructure came online, and the bubble burst.
Yes though anyone with an aws bill knows that’s sales pitch lies
The bill only goes one direction and it ain’t down
- Hyperscaling “we are going to serve billions of people in our applications”, which is becoming increasing unlikely as regional tech companies become more dominant than than the global one (this one is as much about geopolitics as technology)
- Operations is hard, in which case non-frontier models should be increasingly capable. Devops for small-ish deployment is one of the few cases where it is hard to clam you need deep expertise and AI can’t do it. Previously, the claim is that you need people specialized in ops, which is expensive. Now…
My prediction is that not just cloud LLM, but cloud business general will have to change. Not yet in the next 5 years, but probably 8-20 years-ish
They will die, for some definitions of death -- I don't think they disappear, but they should be a niche, rather than the dominant doctrine.