Hacker News Reader

Comment by jph | Hacker News Reader

Lots of errors. Opus 5 is also giving me many more hallucinations, including things that aren't even in the right territory. It's also telling me that it's making many mistakes, and the language feels off-kilter as if it's not using typical clear phrases.

View on HN

Quality of output of both Fable and Opus seems to have gone down quite a bit over the past week. Many mistakes.

Yes, it’s way off field in many things, gets into weird minutia without seeing a way out, and it’s often seeing a clear sequence of work but then halts on a statement like “ok I’m going to start now.” Then after expiring the cache when I notice and ask why didn’t it the response is “no reason starting now!”

I see this behavior constantly in 5 - the quality of opus and fable have degraded constantly since 4.6 was such a riotous success

It is aggressively proactive in ways that make it very hard to use. I had to turn down the effort level to “low” to stop it from going off in random directions every couple of turns.

I saw this even on the weekend when it wasn't under heavy load. We agreed a problem was due to one choice of word in a prompt and then it went off and wrote 600 lines of code to "fix" the issue.

Opus 5 / Fable - instead of implementing things one-by-one in a branch - decided to branch off and do everything from the starting point - meaning that they re-did work over and over, in different ways, and that almost nothing could merge back at the end.

Looking through the logs, it kept saying, "Wait I'm confused. I already did this." But I guess it decided, oh well, I'll just do it again instead of take a second to figure out what I'm doing...

Yeah I noticed informing me of mistakes it made during sessions. It felt really off when it informed me of a mistake it almost made but caught it before it landed.

Reply on news.ycombinator.com