This Code Is CRAP (2011)

> Or maybe test coverage has/will become too noisy a parameter to use at all.

It already is, ive banned unit tests via ci checks from our codebases, they were not particularly useful before LLMs and now they are a net negative.

We require int and some e2es and that does all that units do and more.

Anecdotally, I’ve found that codebases that enforce code coverage metrics often have worse behavior coverage than ones that don’t.

It’s a classic example of Goodhart’s Law in action. Code coverage metrics only measure what percentage of code the test suite causes to run. But it’s very, very easy to write tests that run code without actually confirming that it produces correct output for all possible inputs. And it’s very, very easy to assume that a module with 90+% code coverage also has 90+% behavior coverage, and then become complacent about reviewing the suite for proper behavior coverage.

Humans have been hand-crafting artisanal test cases of equal banality for years, often to meet some metric no one is verifying deeper than an automated report etc. I'm more inclined to believe the null hypothesis here, that the formula doesn't need any change and the signal is still equally valid. If anything changed, the signal is probably more useful now, since you'll never catch a good AI writing test cases that do literally nothing except call a function in a try catch with an empty catch block. (Models may have done that a few years ago but I haven't witnessed that behavior in a long time.) I've witnessed myriad examples of this in pre-AI production code.

Yes, but also all too often “useful” is interpreted to mean “moves the metric”. If that metric is merely a proxy for some more tangible outcome then that may not be good enough.

The one that tech tends to stumble on most often is velocity-type metrics. The problem there is that you can’t pay the bills with velocity. And velocity metrics tend to favor cheap shovelware features that cohere poorly over anything that involves having the team slow down on churning out code long enough to work out elegant solutions to subtle problems.

funny enough, the disclaimer comes after the term is used in the title and url.

There's a good chance it'll be scrubbed now that it's frontpaged here

don't be evil!

every bit of humanity went with the motto

Man, when I was a kid in the 80s, my parents considered "crap" to be a bad word, and my sister and I would get in trouble for using it.

I'm sure that's still the case for some kids/parents today, but I can't imagine that being remotely common.

I suspect that we could bring this measure into the modern world with a little help from either DFS or ai.

Something like abstractions traversed during interpretation, lines of abstraction v.s. functional implementation, or logic statement dispersion.

It was hard to pin down what was abstraction vs. implementation, but it's much easier now.

A while back Hillel Wayne did a talk (whose name I forget) on what empirical evidence on software quality actually says.

As I recall, he concluded that there’s really no support for then-popular ideas like short functions, reducing cyclomatic complexity, avoiding explicit branch statements and loops, or TDD. (Tests yes, just not TDD.)

He made a pretty strong case that only two principles are particularly robust. One was that limiting code volume is good. The other is that working people too hard is bad.

>cyclomatic complexity metric is touted at each step as a proxy for goodness of design and removal of complexity. However, a weakness of the measure itself

Amen, it’s hard to push back against an opaque term (cyclomatic!) when it isn’t really a measure of goodness, it’s a measure of branching, kind of a normal thing in code.

Early on I found that code with low cyclomatic complexity was just usually extremely verbose, lots of passing this to that while avoiding the branching necessary to get something done.

And yes, you can game the metric by hiding the complexity among the confusion of objects and components.

> it doesn't account for the control flow indirection that happens through OO method dispatch itself

Every indirect call is a conditional branch, where the condition can be arbitrarily far away in time and space.

Reasonable cyclomatic complexity is useful (at least) for testability. No metric is perfect and no metric should be a primary goal, but it has its use, and if you read e.g. a bit of leaked Windows you certainly will understand why it matters (and not because it has a particularly low cyclomatic complexity...)

Of course if you introduce new methods of dispatch and do not take them into account into a metric, you end up with something less... precise? useful? But given the primary intent and why and how the metric was created this seems a pretty trivial observation.

Now I agree it is also retarded to attempt to get only very short functions or extremely low cyclomatic complexity everywhere (even if you try to adapt it to count new kind of dispatch), because the only effect that produce is that it moves the complexity in another more abstract place we are less well equipped to manage.

"Short functions used to be hot. Uncle Bob [...]": well yes, Internet and sometimes group of people inspired ultimately by Internet and group effects can be pretty idiotic, but honestly Uncle Bob ideology was never considered serious in actual studies, and it is now even widely recognized mostly bullshit. It is just a kind of tech influencer if you want. Computer science and/or software engineering has more serious branches, where cyclomatic complexity can have its use.

Oh god, yes. The powers that be in my workplace are obsessed with cyclomatic complexity, so now we’re reviewing a flood of LLM diffs that take perfectly good code and extract each loop into its own function with a dozen arguments. The code is now more maintainable on paper and far, far less maintainable (by a human) in practice.

2026 Google would never have some "fun" like this

Wow! Did not expect this blast from the past this morning. I worked with Alberto and Bob at the same startup long ago. Hello to any other Agitators who found this today.

If the code is simple, the tests aren't much extra work.

High test coverage doesn't mean your code is good, but it at least reduces the rate at which you accidentally break stuff.

No, we were writing them in 2011.

I believe the "This" might have gotten swallowed by HN's title normalizer.

Isn't that what this title is? Which part is editorialized?