Getting 100% code coverage doesn't eliminate bugs | Hacker News Reader

Getting 100% code coverage doesn't eliminate bugs

blog.codepipes.com

14 pointskkapelon13 comments

Open articleView on HN

Mutation testing with Java pitest measures three things: line coverage, mutation coverage, and test strength.

Jacoco measures both line and branch coverage, same as Jest.

Preferred thing to do is establish a baseline, and then

* Write tests for new features going forward.

* Write tests for fixes to production incidents.

Don't let things get worse than baseline. (The number will vary from repo to repo.)

So how do you decide what should be the baseline on a brand new project? 70%? 80% 90% ? :-)

Good question. Depends on the reference template, if there is one. Otherwise, Unit Testing: Principles, Practices, and Patterns (Khorikov 2020) suggests

---

Let me repeat myself: coverage metrics are a good negative indicator, but a bad positive one. Low coverage numbers--say, below 60%--are a certain sign of trouble. They mean there's a lot of untested code in your code base. But high numbers don't mean anything. Thus, measuring the code coverage should be only a first step on the way to a quality test suite.

Getting <100% coverage means you have no idea what some of the code does when run in dumb, non-parallel mode. Thar be bugs!

And those bugs are easier to find than the ones that might still exist if you have 100% coverage.

No one ever anywhere (except for numerous strawmen) has claimed 100% coverage "will eliminate all bugs and produce the perfect software."

> This article is useless. Everybody knows that 100% code coverage doesn’t eliminate bugs.

A double-secret strawman is still a strawman, dude.

> Junior developers certainly don’t know this. Misguided team managers don’t know it as well.

Great. Do they read your clickbait? If so, do they believe it (on authority, I guess)? Because unless they do, you're certainly a problem and not part of a solution.

> No one ever anywhere (except for numerous strawmen) has claimed 100% coverage "will eliminate all bugs and produce the perfect software."

So don't we need some articles to defend against the existing articles by these strawmen?

Code coverage only tells if a line or statement was hit. I’ve been working with offshore development teams which get sufficient code coverage, but don’t have any asserts. They call the code, but the test doesn’t test (except maybe a “smoke test” - it doesn’t throw an exception).

No, but it helps.

Like money doesn't give happiness... but it helps.

When reading the title, the first thing that struck me too was division by zero.

I doubt this is a common misunderstanding among any but the most junior of devs.

I chose divide-by-zero because it is super easy for everybody to understand.

But that is not important. In a real application it might be a unicode error, a date error, a wrong-format error or something else.

In a language with errors as values, 100% code coverage (and no `unwrap()`) would force you to handle all the types of errors you mention.

Tomorrow I'm going to ask my scrum master if this is really true.

News at 11!

What is the news doing at 39916800?

Reply on news.ycombinator.com