Hacker News Reader

Comment by hengheng | Hacker News Reader

Or get a 3090 24 GB off eBay for 700€ and run mistral, devstral, qwen3 and others indefinitely.

I'm a big fan of two-tier systems where the 90% case is taken on by something cheap, and renting the premium tier. Works for cars, server hosting, dining ...

View on HN

I did the math, it just doesn't check out, especially if you take into account time to generate and electricity.

Even assuming you generate 100 tok/second, 24 hr, that's only 8 mil tokens. About $5 for Gemini 2.5 Flash or $50 for 2.5 Pro. But you will not be able to run it 24/7 locally - sleep, thinking between prompts, etc

Even at continuous usage (24 hr/day), it's still cheaper to just pay Google for Gemini 2.5 Flash tokens for half a year.

And this is NOT taking into account that mistral/devstral/qwen3 are vastly inferior models.

How can Google be so cheap? TPUs, batching multiple requests, economy of scale, loss-leader, ...

Running stuff locally makes sense for fun, as backup for loss of connectivity, if you want to try uncensored models, but it doesn't make sense economically.

Would be cheaper in the long run maybe but heat, noise, electricity cost and the fact that the models are not as good as sonnet 4 and opus 4 are some things to consider.

For testing always-on concepts and long-running agent experiments a local model will be superior.

For investigating concepts, can AI for this at all? Subscriptions will be a better approach.

Local models are underutilized in comparison to the promise they have. Imagine browser-agnostic universal AdBlock and that's just the surface.

I tried running those locally via Ollama on my 3080 (latency doesn't matter) and the performance is abysmal. They're almost random token generators, literally unusable. My hallucination rate was near 100% for basic tasks like data extraction from markdown.

Have you tried Qwen-32b-8bit? It's pretty solid. Anything lower, yeah, it's kind of garbage.

Slower tokens, maintenance time (installing new models/updating tools), lower SWE bench scores. For me it’s not worth it - optimize for speed.

If I worked on a codebase I couldn’t send to the hosted tools I’d go this route in a heartbeat.

How do you connect that 3090 to a MacBook? How does that work when I’m not tethered to my desk?

I think if you’re comparing it to Claude max it’s ok, but the payback period on GitHub copilot for example is almost 7 years.

> How do you connect that 3090 to a MacBook?

This is probably the easiest problem to solve, isn't it? Most tools offer a n API endpoint setup that you can use locally and also expose on a local network easily.

I haven't paid for the nicer models — now wondering if you could use less capable models to get some code up and running, then pay for better models just to "polish up" the code, get it to "final".

Reply on news.ycombinator.com