I paid $1350 and threw an R9700 in an existing machine. That's a 4 month pay off or so.
Plus, I can feed it sensitive data all day and not be worried where it's going.
>> Is your comparison against a similar size model? Or shouldn't you be comparing it against the cost of a hosted model matching the one you’re using locally?
> You should be comparing the value you get
But the GP commenter specifically compared the cost of solutions such as Claude Code against a 32 GB model.
If they are going to compare cost, they should compare to the cost of a hosted ~32 GB model.
Or if privacy trumps everything for them, then just say that and don't bother comparing costs of incredibly disparate solutions, as Claude Code costing $100+ a month was a red herring if they're happy with 32 GB model output - they could have compared to a far cheaper option that matched their local model's quality.
It would be like someone saying they were able to buy a bike to get to work, saving them $x million compared to buying a Bugatti. When really, if they're going to compare cost they should compare to a cheap car, or not bring up the cost of an expensive car at all if exercise trumps everything else for them.
I still have to use GHCP at work, and I self-host at home, and aside from the fact self-hosting also forces you to tinker, optimize, etc. - there's not a huge difference in my end result in end user results. I spent quite a bit of time trying to optimize llamacpp and compare 35b to 27b, etc. I don't compare models that much at work.
I guess the other part of it is I didn't really know much about cheaper cloud models, but I was attracted to the idea of no longer renting against Claude code, etc. I figured if I could run something functionaly similar from my bedroom on a normal outlet, then all this talk about data centers needing to be built everywhere in the news cycle is obviously just plain stupidity and hype.
It appears I'm using about 20.4/7.6 million in/out tokens a month, or on open router, about $20/month.
That puts $1350 at a 5-6 year break even (thanks to cheap electricity), I guess. Beyond that, running on localhost as a nice feature of 0 no latency when doing rapid tool calling
I don't think I'm the only persona that did the math.