Of course, ensuring compilation or other checks can verify some code, which it can’t do for comments. But comments still serve the same purpose as human comments.
The problem with comments is that LLMs tend to copy their verbose chat output format and insert session/prompt specific details. It makes me think that LLMs aren’t constrained in their comment output the same way they are with their code output
Use that script as a super linter.
That’s the only way I found to strictly enforce some rules, like the no comments rule, without enforcing them against my own changes or old code.
If the model doesn't follow this, you want to start using a better model ASAP, because SOTA models for the last year or so, been able to following this without an issue.
which claims "do" rules persist much better then "don't" rules.
I agree with the general guidance, but it's a general one and not applicable for everything. Some things cannot be expressed in a "do" way rather than "don't".
I suspect rules with negations are not the same as don't rules, but unknown if really true, if so:
"when writing code do not add comments, code should not need it" may work
I use things like "Your code should be self-documenting, so as to require as few comments as possible. Add comments to explain "why" or give important context not apparent from the code itself, but keep them to necessary comments only"
But that's a much laxer rule. I don't think you can truly express "no comments, ever" without a "don't" rule.
Some of these constraints we want simply aren't possible without adding "do not" somewhere in the line, even if you prefix/suffix it with other stuff, as you noticed yourself :)
I just added Sanglard's rules to my ~/.claude/CLAUDE.md file, did another code review, and found some LLM-generated comments were really hard to understand. I think they're due to invented metaphors and flowery language instead of using standard terms, so I've added this:
- Comments must be literal. Don't invent figurative language for what a plain technical term already says — write "rows still reference it," not "rows still wear it."
Improving LLM code generation is an iterative process. I'm glad people share their efforts to improve it.
Please take the following as expressed with genuine curiosity: Do you not use an editor with syntax highlighting and collapsible comments?
At least on JetBrains you can configure the editor to collapse all comments on open and to have the comments displayed in a low-contrast color. This way, LLMs add a bunch of comments, but it doesn't affect your actual experience in trying to read the code. If you encounter code that seems inexplicable, then and only then would you expand the comment to see if that helps you understand.
Comments should be written only when there is (hidden) complexity or external context strictly required. Otherwise it is just easier to read the code. Comments then signal one of two things: a) the following code is really complex and I need to tread carefully, or b) this code is complicated, and could benefit from a refactor.
In regards to agentic coding, all these comments are extra contents, driving down quality while increasing cost. Agents also tend to be inconsistent about updating comments, I've had cases repeatedly where a comment did not match the code, at which point it is just a documentation liability.
Human comment is in another level to answer the questions mainly like "why do it like this" for the later collaborators or the forget-ed self, so the important blocks live when it is needed and can be eliminated when it does not.
The next developer doing a review will see it. The next agent iteration will see it.
If the comment is wrong (even slightly) or redundant, that will help noone.
If the human can read/understand it well enough to comment it, then it is readable by humans.
No, because the one who is writing the comment has context later reader dont. The writer knows what the requirements are, what he was trying to achieve and what he struggled to comprehend. Writer also presumably spent more time trying to understand it then the person coming later should.
Besides, it has always been like this. I sometimes can't even understand some of the things I wrote myself a couple of months ago, because I forgot the context. Good comments and documentation will help you re-acquire the context you need, not completely eliminate it.
>> This way, LLMs add a bunch of comments, but it doesn't affect your actual experience in trying to read the code
by saying that at least reading the code and generating comments is forcing some understanding. Is there a disagreement in that?
You're saying that it's not (necessarily) enough understanding, which may be true but beside the point.
I wish my coworkers would adopt this.
I’m sick of reading a fucking Charles dickens novel for every fucking tiny function