How do you stop LLMs from making comments? In my experience, LLMs treat requirements for code output as suggestions
Of course, ensuring compilation or other checks can verify some code, which it can’t do for comments. But comments still serve the same purpose as human comments.
The problem with comments is that LLMs tend to copy their verbose chat output format and insert session/prompt specific details. It makes me think that LLMs aren’t constrained in their comment output the same way they are with their code output
Use that script as a super linter.
That’s the only way I found to strictly enforce some rules, like the no comments rule, without enforcing them against my own changes or old code.
If the model doesn't follow this, you want to start using a better model ASAP, because SOTA models for the last year or so, been able to following this without an issue.
which claims "do" rules persist much better then "don't" rules.
I agree with the general guidance, but it's a general one and not applicable for everything. Some things cannot be expressed in a "do" way rather than "don't".
I suspect rules with negations are not the same as don't rules, but unknown if really true, if so:
"when writing code do not add comments, code should not need it" may work
I use things like "Your code should be self-documenting, so as to require as few comments as possible. Add comments to explain "why" or give important context not apparent from the code itself, but keep them to necessary comments only"
But that's a much laxer rule. I don't think you can truly express "no comments, ever" without a "don't" rule.
Some of these constraints we want simply aren't possible without adding "do not" somewhere in the line, even if you prefix/suffix it with other stuff, as you noticed yourself :)