I am becoming dependent on AI to make a living, and I need predictable spend on it. If I know I can't use a model regularly all month, my enthusiasm is limited.
I urge Anthropic to get better at this aspect of their business so I can come back to it.
IMO, if you depend on AI to make a living, I'd invest in hardware for local inference, and learn on how to effectively make a living using AI inference you control, on hardware you control. Sure, economically speaking it's way cheaper to use one of these heavily subsidised services (for now), and their models are faster and more capable, but if your livelihood depends on AI inference, and you are renting AI inference, you are a being a serf of the tokenlord. And your livelihood depends on the whims of the tokenlord. They can increase rent prices, they can decide you can no longer do whatever you are doing, and you have no recourse, because you are dependant on them to make a living.
None of us can wholly do our trades without support. Local inference is a fun idea, but you'll be out-competed by the serfs, as you call them.
It is debatable whether local inference is a competitive disadvantage. One key advantage is consistent performance. No unexpected model downgrades or yanks, no silly safeguards imposed, and no quotas is a lot of advantage.
And by the way, it is not an either-or decision. You can use local inference as your daily driver while still leaning on frontier models when you get stuck.
When privacy/compliance really starts to matter it's up to the client/business to provide you with tooling - you're not running that on your own hardware anyway.
So the local AI for individuals is just a hobby/gimmick at this point not a rational decision. Self-hosting for business is a different story.
If you run Qwen 3.8 on your own hardware, every single day, it's the exact same model running in the exact same way.
Yes, it's nowhere near as "smart" as the cloud based models. But it's consistent.
So the workflows and "ways of working" you create will work mostly similar day to day.
With Claude/OpenAI you frequently find days where the models are useless, and days when they are out of this world.
So I guess the choice comes down to:
1. Randomly the smartest thing on the planet with unpredictable rate limits that is mostly amazing, but frequently messes with your workflows
2. A really good local coding model that is consistent every day with no rate limits
I'm not sure. My gut feeling is maybe the right answer is a mix of both.
Gambling on the biggest models, hoping they are working smart that day, when planning or doing very complex work. Then doing most of the tasks/daily work using local models??
It's not closed hosted models vs open local models, it's hosted open models vs local open models where the math doesn't work for local LLMs.
The only local inference use-case I can think of is porn generation (because most providers don't want to deal with it) and illegal shit like hacking to minimize the tracing.
And if you're super paranoid - but honestly giving sensitive info to LLMs in any scenario is a gamble.
If you game and can use your GPU I guess then it works as well but models that fit into a gaming GPU suck too much to bother IMO.
That's not prepping in the sense of having a bunker full of canned beans. Its taking control of a key piece of my business, and likely saving money in the long run.
A large company might own their equipment, but an individual operator probably won't. So it might make sense for some large software companies to own their LLM hardware, but it probably won't make economic sense for individuals.
Of course the economics are different in different industries. Trucking owner operators account for ~15% of truckers, but buying a rig is six figures against 5-6 figure income. Buying a mac mini is 4 figures against a 6 figure income, so maybe lots of people will do it even if it's not economically optimal.
The first comment in this chain clearly stated:
If I know I can't use a model regularly all month, my enthusiasm is limited.
only when it's cheaper than to continue renting it in your calculations
same goes for renting vs buying a house, or anything.
the breakeven point here would come when they stop subsidizing the subscriptions, or when local ai gets to run on basically everything, making it ~free (sans electricity)
Unfortunately open models are still not as good as frontier ones and the hardware costs for a similar experience are very high.
I use Opus every day and easily have most of my weekly limit left over at the end of the week
For instance, if you had 10s of agents running all the time, it is not that unexpected that you run out of tokens quickly.
I've only rarely maxed things out and then it's t through doing extreme things.
My personal guess is that it's one of those. With effective context engineering it's hard to use all 20x quota, the limit becomes your own attention and time really.
You may argue that you're doing multiple, parallel extreme effort tasks – which may be true but then again, there will be results to actually look at sooner or later and that takes time.
What kind of things are you using it for?
I haven't tested it yet but on all the benchmarks it looks like it's 5-7x slower for agentic tasks.
Not sure what it's equivalent to, but it's super cheap and I am happy with the results
But yeah, it's very slow. I've put it to work as an LLM-as-RAG agent.
Although, RAG means search and search means latency?
I am a huge enthusiast of running local models, but when multiple quality USA vendors provide models like GLM 5.3-flash, I run locally just for the fun of it.
For the purposes of comparing to Fable 5.1, I would mention GLM 5.3 that is about 1/12 the cost.
What I'm really keen on is better auto-reasoning so I don't have to constantly have the constant inner debate on which reasoning effort to pick for each task.
I seriously hate the none-low-medium-high-xhigh-max-ultra etc that we have now, with companies frequently recommending different ones on each new model release, etc.
It's apparently called Adaptive Test-Time Compute or Dynamic Test-Time Compute and companies are apparently working on it (according to some LLM :shrug:)
if you don't trust any, then at least that's a coherent position
SuperGrok Plus is slightly better but doesn’t last me more than a few days. Even Claude Max feels leagues more generous in usage…
I haven’t tried SuperGrok Heavy because it’s too expensive
like yesterday it ran for like an hour to build a fairly basic frontend...
i like luna and sol but it feels bad lately