You just stated yourself that it costs more now used than when they were new.
If everyone's running local then why are these larger companies dumping cash into data centres?
If everyone's running local then why are these larger companies dumping cash into data centres?
You need a cluster of 8-12 H100s to run the largest models locally.
It doesn't make sense to run these locally yet unless your use case also involves making it available for several dozen concurrent users.
I've seen people happily use AI that takes several minutes to generate text or edit an image because to them they already aren't using their computer when they tell it to start; they just grab their phone and walk away and come back only to check in on it.
I feel like people here and on other technology discussions -- although it's worse here -- don't seem to parse what being the minority means.
They know they're one of the few to have access to such incredible hardware -- whether it be rented or purchased for way too much cash -- but they only see their own kin; their own ilk. They only compare themselves to the best.
The reality is that nobody expects data centre speed nor power in their own home and are satisfied to just go "haha its thinking" and let their computer quietly tick in the background as opposed to paying outragious prices for subscriptions or hardware.
I'm editing photos with 32GB of DDR4 RAM at 3200mhz alongside a 3050 with a nearly decade-old mobo and ryzen 5800XT and at most? It takes 30 seconds and that's allowing the gpu to use my actual RAM as 'fallback' memory.
There are photoshop filters -now- that take longer than that before the LLM craze even began. Hell; I can train a lora in an hour or two.
This obsession with "more specs more data faster and faster" isn't going to win; it already isn't winning.
Deal with it or get wrecked.
They promise updates.
Who's running local? Image generation can make sense to run locally, but frontier LLM make no sense to run on your own hardware.