What I'm really excited for is the 27B model. 3.6 is still the king of local, and if 3.8 makes the same improvements it could really legitimately make local viable as a default. I'd love to run a perpetual agent on 3.8 that's locally driven.
When I finally put $15 into Deepseek and it beat the brakes off Codex 5.5 on multiple rather complex projects without any of the obnoxious mistakes, I was sick to my stomach with buyers remorse. I couldn't believe I ever felt like I was getting my moneys worth at $200/mo. I wouldn't even use OAI's models if they were free and unlimited at this point, I'll happily pay for what I already know works. No reset bingo, no cache errors, no annoying shitposters as a primary source of info. Oh, and I still had $10 of tokens left
And yes, 3.6 is excellent locally. The rest of this year is gonna be awesome
There would be no reason to if you are in the privileged position where cost isn't an issue.
For the rest of us something that's 95% as good for 20% the price is a hell of a value proposition.
And the price difference is far greater than $2k/month once the API cost is no longer subsidized.
Would your employer be paying an extra $20k/month to Anthropic if it can save you 2 hours a month?
Self-hosting is the biggest reason.
I think that may be part of it.
LLMs can be autonomous to an extent. All of them need steering - which is why I feel they are more a superpower the more I am an expert on the subject matter.
The more you want it to be autonomous, than yeah, you may benefit from using the very best the industry has to offer, however slightly better it is.
But if you are always in the loop anyway, you may want to try DeepSeek. You will get similar results for a fraction of the price.
Stop trying to treat these things like a replacement for yourself and instead approach them as a limitless number of offshore developers. If you are willing to be endearingly literal they will make you happy
I think it's mainly due to poor education many receive and a very controlled media that suppresses information.
It's shocking considering how much money they spend on education compared to other nations.
Don't know about that but your neighbors in Iran in early january happened to be "nice people" who just followed the orders to slaughter 30 000 unarmed civilians.
We could talk about the, what 600 000 deaths, including many civilians, in the Ukraine/Russia war.
Or we could talk about the number of nice palestinians killed since the beginning of the war in Gaza. Or we could go a bit further and talk about the joy and celebration in Gaza after their heroes brought back 200 hostages after having slaughtered 1200 civilians.
You may be living in a place that you think shields you from those but I know the ideologies behind these acts.
The fallacy of gray is just that: it's not true that there's always a nice middle ground and that there's no evil ideology out there.
Something something about the price of liberty being eternal vigilance. For there are people abusing your blind trust.
I suspect that the models are genuinely close and that certain experiences get felt across providers but are inconsistent enough to convince people one is superior to the other. I for one have tried Deepseek on and off since my co-founder is fond of it and I've stopped trying now because I never have a good experience.
On reddit et al., people talk about LLM brands like their sports teams.
I think the first-party ecosystem moats they're all trying to build are exacerbating this tendency, as now people have a lot of learning time sunk in a company-specific option.
I switched to DeepSeek entirely once I decided to put 10 bucks on it and I realized that it could do whatever I was throwing at Claude or ChatGPT prior to that.
I recommended it to one of my friends, and he was surprised DeepSeek could solve task that Claude got stuck at. I was surprised at it too.
I know others that tried and were less impressed too.
Not sure what part of being charged guilty and paying a fine you see as "free".
This isn't an anti-American sentiment. It is an anti-corporate/regulatory capture/embrace and extinguish sentiment (which probably reads the same to many people these days).
But they didn't find it. The Big LLM provider accepted guilt and paid a fine.
You can argue whether it was a fair amount they paid, but there is no legal precedent that was set. It's still considered theft.
Training from copies has been ruled fair use because it's "transformative" and not simply "derivative."
This is obviously debatable, but that's where the debate is at the moment.
Because of the rulings of a couple of judges. Is that actually what the majority of people think?
> Copyright law only considers illegal ownership of a work
That's definitely not true. File sharing, for example, is illegal even if you legally own the original copy you're sharing.
Similarly, copyright has something to say if I read a legal copy of harry potter and then create a new work in that world.
There's a good reason for the law not to be based on what the majority thinks.
Sure i don’t think the majority get to dictate things like who has rights or who the law applies to. That doesn’t apply here tho.
Because that use case is actually permitted by law.
The law was written before the idea of an LLM existed, and some judges in some specific cases decided the previous law covered this usage.
So, it comes down to if you believe a couple judges ruling on a couple cases is the right way to determine a world-altering new legal framework.
It's not an either-or.
That's not how it works. You have to give it back.
Otherwise, the distiller can just pay a fine (no larger than the original did) and be okay then, right ?
Where the new generation of LLMs (Fable, Sol) shines is tasks that are much harder than typical soft eng, yet that still have a verifiable answer, think mathematical proofs or exploits. I think there's still a good amount of low-hanging fruit in those (and similar) areas.
The next frontier after that is tasks that don't have automatically-verifiable answers, and may not even have correct and incorrect ones in the strictest sense of the word.
Reasonable lawyers might disagree on the question of "which trial strategy do I use given the following set of facts." There are answers that are clearly wrong, but being able to choose between many plausibly-correct ones requires many years of lawyering and seeing many trials play out. I do suspect that most lawyers are far below the ceiling that a hypothetical immortal lawyer that has practiced for an infinite amount of time would have achieved.
So instead of relying heavily on human bottlenecks, you focus on agentic task verification since that's the low hanging fruit and verifiable at scale?
LLMs are certainly more knowledgable, but maybe not more intelligent, arguably. It's possible we're approacing a ceiling indeed.
Model capability might be on an asymptote appraching but never quite reaching parity with human intelligence.
Many extended kinds of verification can be done by LLMs, but they need to be able to follow instructions reliably and agentic task orchestration may be critical to that verification process.
There is no doubt they will surpass us as there is a lot of easy to reason about information that they can verify as incrementally proven by other knowledge. The trick is knowing what can be proven with existing knowledge and what needs human evaluation.
One of the reasons is, with good specs and design, A3B is just so fast. It isn't as smart as the 27B model, but it is close enough it can usually figure it out with the right tools.
Setting the memory to "fast timings" is good for 8-12% more tokens/second if you haven't tried yet. I miss the slightly older days of AMD when powerplay tables were unlocked and we could configure the timings and voltages manually, there's another 30% being left on the table ez