Hacker News Reader

Comment by unmole | Hacker News Reader

> you may run some models locally if only from a cost perspective

I have a hard time believing running a model on a laptop will be cheaper than running it in a datacenter. Why wouldn't economies of scale apply here as with every other computation?

View on HN

Because economy of scale isn't really the right metric here. A machine you were you were going to buy anyway essentially has a TCO of $0.

AI models will pretty undeniably affect your electricity bill; yes you already own the computer, but it will cost more to run it if it's doing inference!

To a point, but we're talking a laptop, not a server farm. Even if you're going fullbore wide open 24/7 that's about $150/yr in electricity bills at average rates. Not quite nothing but in terms of AI costs that's pretty close to rounding to zero.

This is assuming that you'll be priced the fraction of computing that you consumed. But you are actually paying for their infrastructure, for the R&D (and also the computation that went into training the model) etc. It is not clear that, for your own small computations, this kind of costs are needed, but you will still pay your share in the investment the provider made so that they could serve everyone's computation needs.

In that analogy bigtech AI is currently investing in cleaner air for all of us? We _could_ breath it through their hose, but might as well breath it outside.

But, currently ... you're not. AI companies are operating at a loss, and are being subsidized by their investors.

Local may or may not be cheaper than remote now, depending on the details, but the factors you describe won't affect the math nearly as much as they will once that subsidization ends.

Not for API pricing. The latest models are not subsidised API wise anymore.

Qwen3.6 is practically indistinguishable to Sonnet 4.6 at least in my personal experience. And sonnet 4.6 is not that cheap.

The datacenter setting has huge economies of scale for low-latency, just-in-time inference using extremely large models, but that's not the only viable use of AI. Batched, unattended inference of possibly smaller and weaker models, while theoretically viable in a datacenter setting, is far from the best use of that hardware. This is where local AI is at its best.

A laptop is really a pretty bad form factor to run LLMs. Worst cooling, more expensive memory that you cannot replace, resell value depreciating fast. It’s fine for tinkering, small scale research, and demos but it’s definitely niche.

The vision NVIDIA is selling is pure marketing IMHO

Does it apply for every other computation? Purely for the computation part? You can host all kinds of things locally cheaper right now than in the cloud, no? (At least pre memory price hikes.) It does, of course, come with its downsides like availability/reliability, less convenience, scaling options,..., but purely the computing price - I don't see why it wouldn't be cheaper in the future - at least for some use cases.

It's cheaper for the AI provider to use your laptop instead of their datacenter.

What "every other computation"? I seem to have a lot processing power at my disposal here, between my cell phones, laptops, gaming PCs, various other hardware devices.

You're going to need to analyze the problem much more deeply because it sound like the standards you are implicitly applying would result in "economically, everything should be centrally hosted" but that is clearly not the result that obtains. Even a modern mid-grade cell phone is no slouch; you may not be running a current-gen frontier AI on it but you certainly can do a lot of other rather intense things locally that would have been laughable 10 years ago, like suprisingly high powered games.

Reply on news.ycombinator.com