Comment by state_less | Hacker News Reader

Comment by state_less | Hacker News Reader

Parent Full thread

Here's a few general observations.

Your LLM (CC) doesn't have your whole codebase in context, so it can run off and make changes without considering that some remote area of the codebase are (subtly?) depending on the part that claude just changed. This can be mitigated to some degree depending on the language and tests in place.

The LLM (CC) might identify a bug in the codebase, fix it, and then figure, "Well, my work here is done." and just leave it as is without considering ramifications or that the same sort of bug might be found elsewhere.

I could go on, but my point is to simply validate the issues people will be having, while also acknowledging those seeing the value of an LLM like CC. It does provides useful work (e.g. large tedious refactors, prototyping, tracking down a variety of bugs, and so on...).

Right, which is why having a comprehensive test suite is such an enormous unlock for this class of technology.

If your tests are good, Claude Code can run them and use them to check it hasn't broken any distant existing behavior.

Not always the case. It’ll just go and “fix” the tests to pass instead of fixing the core issue.

That used to happen a whole lot more. Recent Claudes (3.7, 4) are less likely to do that in my experience.

If they DO do that, it's on us to tell them to undo that and fix things properly.

This is why you keep CLAUDE.md updated, there it’ll write down what is where and other relevant info about the project.

Then it doesn’t need to feel (or rg) through the whole codebase.

You also use plan mode to figure out the issue, write the implementation plan in a .md file. Clear context, enter act mode and tell it to follow the plan.

Can probably give access to tools like ast-grep to Claude. Will help it see all references. I still agree some dynamic references might still be left. Only way is to prompt well enough. Since I tested this on a Ruby on Rails codebase, I dealt with this.

Reply on news.ycombinator.com