Hacker News Reader

Comment by giwook | Hacker News Reader

I wonder how much of this is simply needing to adapt one's workflows to models as they evolve and how much of this is actual degradation of the model, whether it's due to a version change or it's at the inference level.

Also, everyone has a different workflow. I can't say that I've noticed a meaningful change in Claude Code quality in a project I've been working on for a while now. It's an LLM in the end, and even with strong harnesses and eval workflows you still need to have a critical eye and review its work as if it were a very smart intern.

Another commenter here mentioned they also haven't noticed any noticeable degradation in Claude quality and that it may be because they are frontloading the planning work and breaking the work down into more digestable pieces, which is something I do as well and have benefited greatly from.

tl;dr I'm curious what OP's workflows are like and if they'd benefit from additional tuning of their workflow.

View on HN

I've noticed a strong degradation as its started doing more skill like things and writing more one off python scripts rather than using tools.

the agent has a set of scripts that are well tested, but instead it chooses to write a new bespoke script everytime it needs to do something, and as a result writes both the same bugs over and over again, and also unique new bugs every time as well.

I'm going absolutely insane with this. Nearly all of my "agent engineering" effort is now figuring out how to keep Opus from YOLO'ing is own implementation of everything.

I've lost track of the number of times it's started a task by building it's own tools, I remind it that it has a tool for doing that exact task, then it proceeds to build it's own tools anyways.

This wasn't happening 2 months ago.

Can you just tell it not to do that? Maybe you have to remind it every so often once context starts filling up.

It just doesn't listen. Literally a conversation that I just had:

* ME: "Have sonnet background agent do X"

* Opus: "Agent failed, I'll do it myself"

* Me: "No, have a background agent do it"

* Opus: Proceeds to do it in the foreground

* Flips keyboard

This has completely broken my workflows. I'm stuck waiting for Opus to monitor a basic task and destroy my context.

> I wonder how much of this is simply needing to adapt one's workflows to models as they evolve and how much of this is actual degradation of the model,

I also wonder how much people are willing to adapt to non-reliability for the sake of laziness instead of, at some point, do a proper take the lead and solve a problem if you have the knowledge + realiable resoources.

It seems to me, the way you phrase it, that anything a human comes up with when coding must go through an LLM. There are times it helps, there are tasks it performs, but I also found quite often tasks for which if I had done it myself in the first place I would have skipped a lot of confusion, back and forth, time wasting and would have had a better coded, simpler solution.

> It seems to me, the way you phrase it, that anything a human comes up with when coding must go through an LLM.

This seems like a creative interpretation. I never said anything of the sort.

Reply on news.ycombinator.com