Teaching people how to comment is a skill that's just as important and nothing turns me off a project faster than the "code is its own documentation" mantra.
Reading comments only tells you what the person who wrote the comment believed. It does not tell you anything in particular about how the system behaved, you have to trust the other human beings (in general a long succession of them) to have understood it correctly. And any one of you can mess that up.
Saying good comments have value is fine. But their ability to lie is unique; code doesn't have that misfeature. You can't make comments lie-free by fiat, for the same reason that you can't train your developers not to write bugs.
Given that, IMHO comments have limited value. Don't be a zealot in either direction, but when debugging hard problems, train yourself to ignore the lying comments.
Code can deceive even if, definitionally, the code states what the computer will do. So code most certainly can lie.
[By analogy, you'd still feel lied to if someone told you something that is technically correct but very misleading: "(Me) It's going to rain tomorrow." → tomorrow comes → "(You) It isn't raining today!" → "(Me) It is raining, in Japan. I didn't say it would rain here."]
I suppose it depends on whether you're considering what the code communicates to the machine or what it communicates to a person.
You can have your pipeline regression test code, but not comments. Just recently my mentee found some of my code where I had changed the code but not the comment. I'm a horrible human and wasted his time. If that comment hadn't existed the code might have taken a moment to understand but as it was, he wasn't sure which was the intent.
Code without comments only gives half the story. It gives you the what, not the why.
Sure, but it can take you on a freaking trip. Comment your code bro.
This whole argument makes me think of C coders who say C is perfectly safe as long as you don’t write bugs.
And to repeat: when debugging, train yourself to ignore the comments. They will absolutely lie to you.
[1] Which, it's important to point out since you used the term "CI", are fundamentally untestable. There's absolutely no way to ensure a comment is correct. A CI smoke test at the very least verifies code builds and doesn't break pre-existing cases. No such validation is even theoretically possible for comments.
It sounds like we have had similar experiences and have come to similar conclusions.
Learning to code, learning to comment and learning to log are all ways to learn to communicate and represent a means to risk-reduction and importantly, to cost-reduction.
immediately after composing, for each step and nested step, write a line or two of what its place in the code is for. Write it as though the code is broken and you're following the imaginary line threading through, explained as if to your rubber duck.
Then, having written out the business logic map, look at each written step and see if they're just a description of the logic "iterates through file, passes hits onto nextFunc" and you can safely delete those. They're just glue, really, holding processes together.
What you'll have left is skeletal comments that are restricted to "we did this because this stackoverflow post gave the solution" as well as those mental maps of the solution in your head, which is really what comments are for, future programmers to grok your state of mind and thus better implement their code changes.