Hacker News Reader

Comment by _fw | Hacker News Reader

I’m not everyone else but I am SOMEBODY else and you’re absolutely right. I was paying for Claude since they first offered subscriptions, right up until this summer.

I respect Anthropic (and OpenAI to a lesser extent) but I’m not going to play these games.

View on HN

$200/mo for hiring some very capable interns, that's not very expensive.

And a local AI solution is less capable and also quite expensive, but for some worth trying.

Local ai can be about as capable as anything you get a subscription for. The benefit to local beyond a lot of other things is, no model degradation, and stationary costs. Actually, you get effectively free model upgrades.

For 2-4k you can have opus 4.8 at home running faster than anthropic. In 8-20 months you have broke more than even.

I think for almost anyone it's worth trying. Especially if you already have hardware.

It's not "about as" capable. It might be capable enough, depending on use case however.

My use case is programming. This is a viable replacement for me.

Right, but saying local models are just as capable frames it like they're equal. Which is not the case

Yes, but we are getting close to the point where the difference exists, but it is less and less significant. Tbh, Qweb27b is perfectly find for a lot of tasks; anecdotally, the difference lies less and less in quality of responses, but in speed. Local models have a hard time there for a reason ( the cheapest stuff you can get now is stuff like previously mentioned 128gb ryzen ai ).

So trade-offs are there, but do those matter to all people the same way ( are they not equal in the same way to everyone )?

I would say from experience, that yes the local models today are as good as the old frontier models. That was my original claim. I am sticking too it. It won't be long before local models match say fable 5.x as well. And people will still say "it's not as good as x". No it's not but it's as good as the thing everyone freaked out about 6 months ago. If that wasn't good enough for all y'all then then why did you all use it? Exactly.

This is the elephant in the room that OpenAI and Anthropic specifically aren't talking about -- when "good enough" is surpassed.

Because there are a large number of use cases (and large sub-portions of others) where genius-level AI isn't required.

Which means once that threshold is surpassed in people's relevant fields, available margin on that is going to collapse to commodity levels.

It's difficult to see how either pure-play AI company maintains its valuation once that happens. Their TAM is based on capturing a big chunk of all work, not just that which requires the highest intelligence.

$4k doesn’t even buy you a DGX spark these days. You can get a Ryzen AI max 128GB machine for a bit less than $4k but the models you can run on there aren’t even close to Opus 4.8.

On a single 3090 while offloading experts to RAM you can run qwen 3.8 next. Which yea is about 2.5k new and yep about opus 4.8.

Amazon recently moved $8B in GPUs to a SPV to hide away their expenses. Oracle warned their data center isn’t going to be on schedule. Worst of all, none of the big players are reporting enough deprecations to match up with GPU sales.

All this indicates to me that loads of GPUs were only purchased on paper or are sitting in warehouses unused. This almost certainly means a drop in orders followed by a drop in RAM prices (though that would indicate a demand drop to investors, so maybe it’s better for stock prices to keep paying to bills bigger warehouses and stuff them full of unused GPUs).

The second RAM prices normalize, local LLM becomes much cheaper. A machine that doesn’t make much sense at $10-15k suddenly becomes a lot more feasible at $3-5k. If those stored GPUs flood the market, we might see even bigger price drops.

that's a wild theory. You think there are really warehouses of GPUs that they are just sitting on until data centers are built, that will just all of a sudden flood the market?

First, the form factor of GPUs in data centers aren't the same as desktop GPUs, so you couldn't use them even if you wanted to in a normal rig.

Second, from a business perspective that doesn't make a lot of sense. It's much more likely that GPUs are going to the highest bidder/large contracts who are scooping them up to populate/upgrade data centers that are in operation because they are going to get more money per gpu on newer hardware. The "old" hardware might go to a warehouse to be auctioned to the highest bidder or go to a data center coming online, but I highly doubt they are sitting on market wrecking amounts of GPUs just waiting to flood the market. When they go bankrupt and they have to sell a datacenter or two, those datacenters being parted out as part of a bankruptcy deal I could see. Warehouses full of unused GPUs doesnt make any sense to me though.

"Microsoft Is Warehousing An Estimated $50 Billion to $100 Billion In GPUs" https://www.wheresyoured.at/wherere-all-the-ai-chips/

It makes sense if you consider that maybe all of these companies are colluding to forward a shared agenda. It is happening whether it makes sense or not. So there's that too.

> First, the form factor of GPUs in data centers aren't the same as desktop GPUs, so you couldn't use them even if you wanted to in a normal rig.

With eGPU PCIe 3.0 or 4.0 x4 links (or even Thunderbolt), that essentially doesn't matter for inference.

You pay the bandwidth hit on model load / unload (a few seconds), but it's irrelevant for post-loaded inference.

Anyone hosting a high wattage GPU is going to be fine hosting in an external enclosure, most of which have generous extra-spec room.

And if they don't, if a flood of cheaper DC GPUs hit the used market, you can bet Chinese manufacturers will have enclosures that fit them available the day after.

>>The second RAM prices normalize, local LLM becomes much cheaper. A machine that doesn’t make much sense at $10-15k suddenly becomes a lot more feasible at $3-5k. If those stored GPUs flood the market, we might see even bigger price drops.

Yeah but it's unlikely to happen in coming 2-3 years (at least) and then the SOTA models are going to be a completely different animals.

I would love for this to be true, but this is nonsense. The local models that you can run for those prices are not as capable as current frontier. At $200 a month, it would take well over a year to break even, and that’s if you don't include energy prices and time spent tinkering to get and keep it working.

Again not the current frontier. The frontier of 6 months ago. Like opus 4.8. which is my original claim. And it's true. Even the ones that run on a single graphics card less than 3k in cost new are that good.

Reply on news.ycombinator.com