The valuations of the hyperscalars won't sustain just being more efficient than something you can run locally. There's a market there, but it's for margin on a commodity. They're priced for oligopoly on unique, premium products.
I run models on my desktop and access them from a phone app on the go. Wireguard tunnel.
Responds fast and lets me kick off tasks or workflows via text or voice.
Maybe I could use codex or whatever to set that all up nowadays, but it was such a pain doing the initial setup and getting it all running and finding sources of media yadda yadda like a decade ago when I last tried lol
Very much a personality and/or lifestyle thing, but I've also been building infra for decades. I mean it was two container startups and a phone app download (10min). Not exactly difficult. If I didn't like it I certainly wouldn't be in tech!
> pay someone
I'd do that with something like 3 story roof work or foundation work with a contractor, but tech? If I want the outcome I will be happy with, often have to do it myself.
I was into home automation and self hosting media and some other stuff for a while. It’s the kind of thing I’d only do again if i was equally or more interested in the process than the outcome.
We're looking for different outcomes.
I am also looking to actively cut my SaaS / services cost to $0 or near $0. I don't want another company touching my fucking data or prompts.
There will be some “rising star” businessperson who somehow got an amazing deal on land and energy and just likes to spend 1/2 their time in Shenzhen or something, but the datacenter is in Vietnam or Singapore or Thailand or wherever. They might even have a US branch, to make everyone feel better!
It will collapse the market, and everyone will realize that data centers are actually worth LESS than Toyota - they’ll look more like Tulips, suddenly.
In my opinion, of course.
I'm absolutely going to run this localy if I can. The only reason I'm not doing it is the fact I'm literally priced out.
The compute capacity, however, is here to stay. There will be a seemingly endless line of people queued up to buy it at pennies on the dollar if this whole thing blows up financially. And they will use it!
Just out of curiosity: Use it for what?
The demand is there and the market will find pricing that works.
Or they'd be part of the customer base for leasing compute from someone else who bought the rack.
Although there is such a thing as hobbyists with a ton of money who want to experiment with large models on enterprise grade hardware. Probably not a lot of those, though, of course.