Hacker News Reader

Comment by getnormality | Hacker News Reader

I would never tell an agent to write "what does the code do" comments. Their default comments are already way too fluffy.

View on HN

I added to the memory, system prompts, and the prompt itself and every soa model still litters code with the most inane useless crap. I will then get code to review from a coworker using fable/opus. It has more lines of comments then code.

Maybe I am some god tier code reader (i am not) but i dont think i have ever found a comment in code to be useful in my day job. That isnt true, i once came across

// submit to the dark lord

Above the function that sent a payment to PayPal for processing. It made me laugh so I let it be.

Most useful code comment I have encountered read:

“”” After you give up on trying to refactor this code, increment the following line accordingly. HOURS_WASTED_HERE=26 “””

I always loved the classic _"Here be dragons"_

But then you don't know about where the load bearing seams are!

Hmm, I guess everyone here is using Claude? I find that Sol is much more restrained, to the point where I have a prompt to tell it to add short comments for things that are not obvious. Really, I find the verbosity problem to be worse in tests. I regularly prompt my review agent to remove tests used only for scaffolding to write the code in the first place. The agent is in a way following strict TDD, which reminds me why I don’t like TDD, even though some of the generated tests can be useful.

Sol adds comments when I use it, but it's not nearly as verbose as Opus 5/Fable.

For the latter I'll often include an example of a comment it wrote, along with my own rephrasing, and tell it that "future readers will understand code context; good naming is the best documentation". This works alright if I include in the actual prompt, but annoyingly it often doesn't in CLAUDE.md or memory.

I tell the agent to NEVER write comments in the system prompt and it ignore it like 90% of the time. RLHF is a helluva drug.

We trained agents on millions of pages of documentation telling them to write good comments and good code and then we tell them never to write any comments.

It’s almost the “we built a robot who loves to play Sonatas and gave it no hands” type of thing.

Being overly trained on comments in documentation could be one of the reasons why Claude models write frustrating comments (which often manifest as written justifications of how the code satisfies the prompt). A comment in a tutorial is going to be geared to explaining how the code relates back to the tutorial task or restating what the code does, rather than documenting the "why" of surprising code.

What models are you using? I've had "Never add any new code comments" in my system prompts for like a year at this point, seems every model above 14B picks this up just fine. Are you using llama 7b or similar for testing this?

Pretty much Opus 5 exclusively.

Reply on news.ycombinator.com