Hacker News Reader

Comment by KingOfCoders | Hacker News Reader

The benefit of 100% test coverage is that there are no more discussions on what to test and what not. When in doubt, test it. In larger groups of developers there are otherwise ongoing discussions if A needs to be tested or not. I have seen culture wars around this, from people who don't want to test and are in the eyes of others always testing not enough and vice versa. Especially with a diverse development force with different ages, seniority and cultural background.

It's often easier to just aim for 100% test coverage instead (with excluding some categories of files).

EDIT: I would not and did not start with 100% unit testing. But if there are ongoing culture wars and discussions didn't lead to a workable compromise, 100% test coverage worked for me and after some days test coverage was a non issue.

View on HN

> just aim for 100% test coverage instead (with excluding some categories of files).

That's where the 'gaming' comes in.

The tests start just going through lines without hitting a single expect statement.

The ignore files start becoming battlegrounds in the PRs because people just exclude half the damn project.

We just have a simple rule... if you wrote code, you have to write coverage for it. If it breaks and your test doesn't catch the breakage, the bug fix goes back to you. Some people will ask "but what about what I'm working on now", you'll have to communicate that you feel your previous work was far more important.

> the bug fix goes back to you. Some people will ask "but what about what I'm working on now", you'll have to communicate that you feel your previous work was far more important.

this feels punitive, especially in the eyes of management. unless you're in a safety critical area where fully testing every code path is a hard requirement, people will eventually write bugs.

i'd rather work somewhere that recognizes defects occur and has a fast iterative process to push out new changes rather than one based on shame for having written a bug.

> has a fast iterative process to push out new changes rather than one based on shame for having written a bug.

That is the fastest most iterative process we have found so far... as the expert on the original code, you are able to deliver the best outcome.

You're not being shamed for writing a bug, you're being shamed for not testing your code.

Even with 100% coverage, that doesn't mean you've found every bug that can possibly be found with unit tests. Your tests could always cover more inputs, more situations, etc.

No, you can't find bugs that you can't think of.

Property based testing and random values in unit tests find lots of bugs you didn't think of though.

> you can't find bugs that you can't think of

Rather, you won't find bugs that you choose not to think of because you've let the "100%" number lull you into complacency, even though you know it's 100% of lines/branches, not 100% of inputs.

Yes.

My problem in 40y of programming is still making bugs and those I make come from not thinking about edge cases or from wrong assumptions and not from being lulled into writing tests to meet a 100% number.

But personalities differ and if being lulled into security by writing towards a 100% number is a problem for you I would be careful, I totally agree here.

I agree that fuzz testing+linting can help you. However from my persective lower level tests help you to build trust about the software you're releasing.

100% test coverage is pretty easy to game as long as the metric is known. Just exercise all paths and write as few assertions as you can.

I'd prefer <100% coverage plus discussions about what to test (and how) much more than working with a test suite built on the wrong incentives.

If you are someone who games metrics for his benefit or hire people who are gaming metrics for their benefit, I assume yes, this metric is very easy to game as are most metrics. Metric systems are not cheater proof.

This argument, that it avoids discussions, does make sense but only if you're in a team where discussions tend to be a waste of time.

When my colleagues are knowledgable and open minded I would embrace every opportunity to have a good discussion.

I like thinking about the trade off between a simpler rule that is mostly right vs. decisions require judgement and consensus. I think the simpler rule is usually the better side of the trade-off.

But in this case I think the cure might be worse than the disease. Tests for plumbing code often end up being brittle tests of methods getting called on mocks in the right order. People will notice that these require a lot of toil to keep them running as code changes while providing very little benefit in avoiding mistakes. People will rankle at being told that they must write these tests, which they can see are a waste of time.

I've done it both ways. I'm much happier with my work when I'm not trying to write tests that are tedious and don't seem to provide any value, in order to hit an arbitrary coverage metric. I suspect my teammates feel the same way, so on teams where I have input into the decision on this, I do not advocate for 100% coverage. It does make it harder to have the discussion of which tests should and shouldn't be written, but I think it's worth that cost.

Yes I have seen bad unit tests very often.

Writing good testing code is harder than writing business code. Especially junior developers struggle with this, most often because many companies write not enough tests to learn writing good tests.

And if you're in an environment, where this is a non-issue I think thats great. Don't fix something that doesn't need to be fixed.

Yup. The debate over unit testing, being political, is a far bigger impediment to progress than the actual tests, which are a technical hurdle. It's essentially the same reason for linters and auto-formatters.

It has a side benefit that it forces devs to write testable code, which inclines them to reasonbly factored code.

> It's often easier to just aim for 100% test coverage instead (with excluding some categories of files).

Congratulations, now you have a war over which categories of files are excluded from the "100% test coverage" rule. ;)

Thank you!

"larger group of developers" is the phrase that caught my attention. Humans don't scale well. This is where microservices do become attractive. This service is owned by a small team, and that team makes these types of judgements. It may be very different from how other service is owned and maintained, and that's ok.

From my limited experience you get cross team discussions about unit testing, especially if one microservice has too many bugs in the eyes of other teams giving development a bad reputation or making working with a microservice hard. Especially if it breaks with releases and other teams get paged.

Yes. you move the people don't scale well issues to multiple teams don't scale well

Rule by diktat is a recipe for demotivated staff.

Ongoing culture wars are a recipe for demotivated staff.

Perhaps I am wrong, and I would not start with a 'diktat' for obvious reasons.

As a manager, you didn't have discussions about the level of necessary code coverage? Would be interested on how you managed unit testing without 'dictat'. How it would fit into integration testing and explorative testing. What level did developers in your deparment usually find "adequat" ? If you considered it too low, how did you raise test coverage as a manager without defining a coverage level?

This is dangerously close to not understaning what you are doing. _Thinking_ what to test has a benefit of making you think, not just cargo-culting.

Obviously you need to think about how to test. Usually this leads to finding omissions in your code adding edge cases.

Reply on news.ycombinator.com