I honestly prefer this as opposed to what Anthropic has done in the past which is to continue accepting users even if they don't have the capacity to serve and behind the scenes tone down everyone's limits.
8 karma · joined November 23, 2018
You can be far more ambiguous with your tasks with the larger proprietary models as opposed to the local models. You can achieve the similar results with local models but you need to be much more detailed in your prompt.
One of the biggest things about running these local models is that the harness matters almost just as much as the model too. Codex is optimized for GPT models, CC is optimized for Claude, Cursor has a great harness that works very well across these providers. It took me a couple of iterations of the different harnesses to find one that would work well with the smaller Qwen models to do local coding.
Seems like they would've kept the invisible guardrails if it didn't hurt their bottom line.