Hacker News Reader

That's 6 YEARS of their cloud inference!

And, you have to include power. Where I am, the power for a rig with 2x 3090, running for four hours a day, would be around $70/month!

It's kinda like datacenter only exist to optimize compute cost through oversubscription of hardware/time share, cheaper business energy rates, and bulk discounts, compared to self hosting. ;)

View on HN

Running local models isn't about the cost, its about owning your own data and being able to run uncensored models. If you just want to code with an assistant, local inference is by far not worth it.

You also don't need to run the models 24/7, nor is your computer gonna be doing inference all the time when you are coding.

No, you're mixing unrelated concepts here. Cloud does not mean you don't own the data, and can't do what you want. It means it's running on commoditized server hardware, powered with cheap commercial energy costs, oversubscribed so nothing sits idle.

Reply on news.ycombinator.com