> We're on the precipice of something incredible.
Total dependence on a service?
Total dependence on a service?
Software engineering AI API dependence seems to have already screamed past the tobacco mark but we’re still a long ways away from oil. Though I would bet that we hit it sometime in the next few decades once the bulk of the industry has never written code in any serious capacity.
They are getting better, but that doesn't mean they're good.
We have a magical pseudo-thinking machine that we can run locally completely under our control, and instead the goal posts have moved to "but it's not as fast as the proprietary could".
It's more cost effective for someone to pay $20 to $100 month for a Claude subscription compared to buying a 512 gig Mac Studio for $10K. We won't discuss the cost of the NVidia rig.
I mess around with local AI all the time. It's a fun hobby, but the quality is still night and day.
1. You are not forced to use the AI in the first place.
2. If you want to use one, you can self host it one of the open models.
That at any moment in time the open models are not equivalent in capabilities to the SOTA paid models is beside the point.
But there’s new things like sweep [0] that you now can do locally.
And 2-3 years ago capable open models weren’t even a thing. Now we’ve made progress on that front. And I believe they’ll keep improving (both on accessibility and competency).
1. It costs 100k in hardware to run Kimi 2.5 with a single session at decent tok p/s and its still not capable for anything serious.
2. I want whatever you're smoking if you think anyone is going to spend billions training models capable of outcompeting them are affordable to run and then open source them.
How much serious work can it do versus chatgpt3 (SOTA only a few years ago)?
Those things are also regulated as utilities.
But as it stands right now, the most useful LLMs are hosted by companies that are legally obligated to hand over your data if the US gov. had decided that it wants it. It's unacceptable.
Not to mention they could just sell it to the highest bidder, or simply use it to produce competition and put you out of business. Especially if you're using their service to do the development...