(Speaking of both Claude Code and the desktop app, both Sonnet and Opus >=4, on the Max plan.)
As an example I’ve been using an MCP tool to provide table schemas to Claude for months.
There was a point where it stopped recognizing the tool unless mentioned in early August. Maybe that’s related to their degraded quality issue.
This morning after pulling the correct schema info Sonnet started hallucinating columns (from Shopify’s API docs) and added them to my query.
That’s a use case I’ve been doing daily for months and in the last few weeks has gone from consistent low supervision to flaky and low quality.
I don’t know what’s going on, Sonnet has definitely felt worse, and the timeline matches their status page incident, but it’s definitely not resolved.
Opus 4.1 also feels flaky, it feels like it’s less consistent about recalling earlier prompt details than 4.0.
I personally am frustrated that there’s no refund or anything after a month of degraded performance, and they’ve had a lot of downtime.
If you don't have the hardware to run it locally, let me shill my own company for a minute: Synthetic [1] has a $20/month subscription to most of the good open-weight coding LLMs, with higher rate limits than Claude's $20/month sub. And our $60/month sub has higher rate limits than the $200/month maxed-out version of the Claude Max plan.
You can still use Claude Code by using LiteLLM or similar tools that convert Anthropic-style API requests to OpenAI-style API requests; once you have one of those running locally, you override the ANTHROPIC_BASE_URL env var to point to your locally-running proxy. We'll also be shipping an Anthropic-compatible API this week to work with Claude Code directly. Some other good agentic tools you could use instead include Cline, Roo Code, KiloCode, OpenCode, or Octofriend (the last of which we maintain).
If you can find a way to secure the requests even during the 14 day period, or anonymize them while allowing the developers to do their job, you can have my money today. I think privacy/data security is the #1 concern for me, especially if the agents will be supporting me in all kinds of personal tasks.
This looks really promising since I have also been having all sorts of issues with Claude.
In terms of tenancy: we have our own dedicated VMs for our Kubernetes cluster via Azure, although I suspect a VM is not equivalent to an entire hardware node. We use Supabase for our Postgres DB, and Redis for ephemeral data; while we don't share access to that to any other company, we don't create a new DB for every user of our service, so there is user multitenancy there. Similarly, the same GPUs may serve many customers — otherwise we'd need to charge enormous amounts for inference. But, the requests themselves aren't intermingled; i.e. if you make a request, it doesn't affect someone else's.
another option could be a system prompt change to make it too long?
As a baseline from a real conversation, 270 lines of sql is ~2500 tokens. Every language will be different, this is what I have open.
When Claude edits an artifact it seems to keep the revisions in the chat context, plus it’s doing multiple changes per revision.
After 10 iterations on a 1k loc artifact (10k tokens) you’re at 100k tokens.
claude.ai has a 200k token window according to their docs (not sure if that’s accurate though).
Depending on how Claude is doing those in place edits that could be the whole budget right there.
I actually think this is psychological bias. It got a few things right early on, and that's what you remember. As time passes, the errors add up, until the memory doesn't match reality. The "new shiny" feeling goes away, and you perceive it for what it really is: a kind of shitty slot machine
> personally am frustrated that there’s no refund or anything after a month of degraded performance
lol, LMAO. A company operates a shitty slot machine at a loss and you're surprised they have "issues" that reduce your usage?
I'm not paying for any of this shit until these companies figure out how to align incentives. If they make more by applying limits, or charge me when the machine makes errors, that's good for them and bad for me! Why should I continue to pay to pull on the slot machine lever?
It's a waste of time and money. I'll be richer and more productive if I just write the code myself, and the result will be better too.
Then after using the new model for a few months you get used to it, you feel like you know what it should be able to do, and when it can’t do that, you’re annoyed. You feel like it got worse. But what happened is your expectations crept up. You’re now constantly riding it at 95% of its capabilities and hitting more edge cases where it messes up. You think you’re doing everything consistently, but you’re not, you’ve dramatically dialed up your expectations and demands relative to what you were doing months ago. I don’t mean “you,” I mean the royal “you”, this is what we all do. If you think your expectations haven’t risen, go back and look at your commits from six months ago and tell me I’m wrong.
I think you’re right. I think it’s complete bias with a little bit of “it does more tasks now” so it might behave a bit differently to the same prompt.
I also think you’re right that there’s an incentive to dumb it down so you pull the lever more. Just 2 more $1 spins and maybe you’ll hit jackpot.
Really it’s the enshitification of the SOTA for profits and glory.
People seem to turn to this with a lot when the suspicion many people have is difficult to verify. And while I don’t trust a suspicion just because it’s held by a lot of people, I also won’t allow myself to embrace the comforting certainty of “it’s surely false and it’s psychological bias”.
Sometimes we just need to not be sure what’s going on.
I've seen the cycle of claims going from "10x multiplier, like a team of junior devs" to "nerfed" for so many model/tool releases at this point it's hard for me not to believe there's an element of perceptual bias going on, but how much that contributes vs real variability on the backend is impossible to know for sure.
If you mean over the lifetime of a model being deployed, no, that's not how these models are trained.
Anyone remember GPT4 the day it launched? :)
They recently resolved two bugs affecting model quality, one of which was in production Aug 5-Sep 4. They also wrote:
Importantly, we never intentionally degrade model quality as a result of demand or other factors, and the issues mentioned above stem from unrelated bugs.
Things they could do that would not technically contradict that:
- Quantize KV cache
- Data aware model quantization where their own evals will show "equivalent perf" but the overall model quality suffers.
Simple fact is that it takes longer to deploy physical compute but somehow they are able to serve more and more inference from a slowly growing pool of hardware. Something has to give...
Is training compute interchangeable with inference compute or does training vs. inference have significantly different hardware requirements?
If training and inference hardware is pooled together, I could imagine a model where training simply fills in any unused compute at any given time (?)
Also, if you pull too manny resources from training your next model to make inference revenue today, you’ll fall behind in the larger race.
- They're reporting that only impacted Haiku 3.5 and Sonnet 4. I used neither model during the time period I'm concerned with.
- It took them a month to publicly acknowledge that issue, so now we lack confidence there isn't another underlying issue going undetected (or undisclosed, less charitably) that affects Opus.
You can be confident there is a non-zero rate of errors and defects in any complex service that's moving as fast as the frontier model providers!
> We are continuing to monitor for any ongoing quality issues, including reports of degradation for Claude Opus 4.1.
I take that as acknowledgment that there might be an issue with Opus 4.1 (granted, undetected still), but not undisclosed, and they're actively looking for it? I'd not jump to "they must be hiding things" yet. They're building, deploying and scaling their service at incredible pace, they, as we all, are bound to get some things wrong.
I'm also a realist, though, and have built a career on building/operating large systems. There's obviously capability to dynamically shed load built into the system somewhere, there's just no other responsible way to engineer it. I'd prefer they slowed response times rather than harmed response quality, personally.
"Use your web search tool to find me the go-to component for doing xyz in $language $framework. Always link the GitHub repo in your response."
Previously Sonnet 4 would return a good answer to this at least 80% of the time.
Now even Opus 4.1 with extended thinking frequently ignores my ask for it to use the search tool, which allows it to hallucinate a component in a library. Or maybe an entire repo.
It's gone backwards severely.
(If someone from Anthropic sees this, feel free to reach out for chat IDs/share links. I have dozens.)
Sonnet 3.5 did this last year a few times, it'd have days where it wasn't working properly, and sure enough, I'd jump online and see "Claude's been lobotomized again".
They also experiment with injecting hidden system prompts from time to time. Eg. if you ask for a story about some IP, it'll interrupt your prompt and remind the model not to infringe copyright. (We could see this via API with prompt engineering, adding a "!repeat" "debug prompt" that revealed it, though they seem to have patched that now.
> I started running my prompts through those, and Sonnet 3.7 comparing the results. Sonnet 3.7 is way better at everything.
Same here. And on API, the old Opus 3 is also unaffected (though that model is too old for coding).
IDK about you but I find it faster to type a few keywords and click the first result than to wait for "extended thinking" to warm up a cup of hot water only to ignore "your ask" (it's a "request," not an "ask," unless you're talking to a Product Manager with corporate brain damage) to search and then outputs bullshit.
I can only assume after you waste $0.10 asking Claude and reading the bullshit, you use normal search.
Truly revolutionary rechnology
Might be Claude optimizing for general use cases compared to code and that affecting the code side?
Feels strange, because Claude api isn’t the same as the web tool so I didn’t expect Claude code to be the same.
It might be a case of having to learn to read Claude best practice docs and keep up with them. Normally I’d have Claude read them itself and update an approach to use. Not sure that works as well anymore.
I signed up for Claude over a week ago and I totally regret it!
Previously I was using it and some ChatGPT here and there (also had a subscription in the past) and I felt like Claude added some more value.
But it's getting so unstable. It generates code, I see it doing that, and then it throws the code away and gives me the previous version of something 1:1 as a new version.
And then I have to waste CO2 to tell it to please don't do that and then sometimes it generates what I want, sometimes it just generates it again, just to throw it away immediately...
This is soooooooo annoying and the reason I canceled my subscription!
I've had the same experience. Totally unreliable.
1. Ask Claude to fix something
2. It fails to fix the issue
3. I tell it that the fix didn’t work
4. It reverts its failed fix and tells me everything is working now.
This is like finding a decapitated body, trying to bring it back to life by smooshing the severed head against the neck, realizing that didn’t bring them back to life, dropping the head back on the ground, and saying, “There; I’ve saved them now.”
I've been running ccusage to monitor and my usage in $ terms has dropped to a 1/3 of what it was few weeks ago. While some of it could be due to how I'm using it, but a drop of 60%-70% cannot be attributed to that alone and I think is partly due to the performance.
To add: frequently, as in almost every time: 1) it'll start doing something and will go silent for a long time. 2) pressing esc to interrupt will take a long time to take action since it's probably stuck doing something. Earlier, interrupting via esc used to be almost instantaneous.
So, I still like it, but at my 1/3 drop in measured usage I'm almost tempted to go back to Pro and see if that'll meet my needs.
(lol, yes, thank you.)
I picked up Claude at the beginning of the summer and have had the same experience.
Based on the discussions here it seems that every model is either about to be great or was great in the past but now is not. Sucks for those of us who are stuck in the now, though.
https://status.anthropic.com/incidents/72f99lh1cj2c
Suggesting people are "out of their mind" is not really appropriate on this forum, especially so in this circumstance.
This most definitely feels like people analyzing the output of a random process - at this point I am feeling like I'm losing my mind.
(As for the phrasing I was quoting the OP, who I believe took it in the spirit in which it was meant)
[1] https://news.ycombinator.com/item?id=45183587
[2] https://news.ycombinator.com/item?id=45182714
> New features like this feel pointless when the underlying model is becoming unusable.
I recognize I could have been clearer.
And for what it's worth, yes, your comment's phrasing didn't bother me at all.
They were wrong, but not inappropriate. They re-used the "out of their mind" phrase from the parent comment to cheekily refer to the possibility of a cognitive bias.
Yes, but I'll revisit.
On that note, I strongly recommend qwen3:4b. It is _bonkers_ how good it is, especially considering how relatively tiny it is.
FWIW, Codex-CLI w/ ChatGPT5 medium is great right now. Objectively accelerating me. Not a coding god like some posters would have it, but overall freeing up time for me. Observably.
Assuming I haven't had since-cured delusions, the same was true for Claude Code, but isn't any more.
Concrete supporting evidence: From time to time, I have coding CLIs port older projects of varying (but small-ish) sizes from JS to TS. Claude Code used to do well on that. Repeatedly. I did another test last Sunday, and it dug a momentous hole for itself that even liberal sprinkling of 'as unknown' everywhere couldn't solve. Codex managed both the ab-initio port and was able to undig from CC's massive hole abandoned mid-port.
So I'd say the evidence points somewhat against random process, given repeated testing shows clear signal both of past capability and of recent loss of capability.
The idea that it's a "random" process is misguided.
You mean like our human brains and our entire bodies? We are the result of random processes.
>Sucks for those of us who are stuck in the now, though
I don't know what you are doing- but GPT5 is incredible. I literally spent 3 hours last night going back and forth on a project where I loaded some files for a somewhat complicated and tedious conversion between two data formats. And I was able to keep going back and forth and making the improvements incrementally and have AI do 90% of the actual tedious work.
To me it's incredible people don't seem to understand the CURRENT value. It has literally replaced a junior developer for me. I am 100% better off working with AI for all these tedious tasks than passing them off to someone off. We can argue all day if that's good for the world (it's not) but in terms of the current state of AI- it's already incredible.
It might not be a junior dev tool. Senior devs are using AI quite differently to magnify themselves not help them manage juniors with developing ceilings.