Hacker News Reader

Comment by tvst | Hacker News Reader

I'm not convinced of the "always rewrite" thing, but what you're talking about would be solved by writing tests when you fix a bug.

View on HN

It's not the first time someone proposes to write a test when fixing a bug, but you have to question the actual ROI.

At my previous job, they checked how likely it is for a fixed bug to reappear again. Odds are really low. So we decided it wasn't worth the investment to write a test specific for the bugs use-case, because it's unlikely to appear again. We focused our efforts to other, more effective things.

We use the test as a way to prove we know what the bug actually is.

Our debugging "path" normally is:

1. Find out bug exists

2. Write a test that can reproduce the bug

3. Distill that test down to the minimum needed

4. Fix bug until test passes

5. Test bug is actually fixed by trying to reproduce it the same way the user/bugreport did.

When done like that, the test isn't "extra work" but is just part of narrowing down your actual problem most of the time.

And that test (assuming it's not extremely slow to run), takes very little to maintain, so leaving it in the suite of tests is basically a positive only.

Whist I agree with this approach, sometimes it just isn't possible if the code was written in a non-testable way to begin with. Such as code that hits the database a lot or relies on network data. In this case, weighing up the ROI is a worthwhile endeavour, with a note to refactor more thoroughly at a later date.

That's a really good reason to write business logic in a way that doesn't have those dependencies hard wired in. I often end up in those sorts of nightmares in codebases that use the active record pattern (hiding database access everywhere), hooks, critical logic in the HTTP layer, and so on.

> Such as code that hits the database a lot or relies on network data.

well the first problem is not a problem since you can cheat it. if you rely on the native database you can actually fake your driver and insert transactions (subtransactions or in postgresql savepoints) so for the whole test suite you just rollback to the latest savepoint, this saves a lot of time. (it's actually not that hard in java+di or in languages where you can monkey patc code.

80% of all tests should not be written, or if already written: deleted. A test is only valuable when it fails alerting you to a change you made that introduced a bug. There is a catch though: I don't know which of those tests are valuable.

I find the ROI for tests is high despite the large number of worthless tests because of the ones that fail a small number alert me to something I wouldn't have found otherwise.

The 80% number was of course made up. In my experience it is realistic, but I have never done a formal study.

If you rewrite a method from scratch rather than editing it, then those odds go way up.

Furthermore to Klathmon, there is an opportunity cost. Because you don't have the test, changing that method in the future becomes problematic. Is not the bug reappearing, is the refactoring that cannot longer take place.

In practice, the rewrites that I have seen, are not limited to just function bodies, without changing interfaces and class structures.

What I see in refactoring is that whole blocks of code get an overhaul. And in that case, all the test you wrote on those interfaces need to be adapted, rewritten or removed.

So actually, in a proper refactoring, some tests might help you out, because the interface of the classes didn't change. But other tests cost you time.

It's all a pretty complex balancing act. Some tests save you time, some cost you time. They all make the code more stable, but you have to put your effort where it really makes sense.

Were you writing tests before? It could be that writing the test was causing the original bug to be fixed properly and prevented it from showing up again.

Reply on news.ycombinator.com