Hacker News Reader

Comment by dmm | Hacker News Reader

> But some tests, I have no idea what changed that made them break.

Michael Feathers in "Working Effectively with Legacy Code" defines a unit test as a test with two qualities:

1. It runs fast (1ms is slow for a unit test) 2. Has excellent locality, so when a test fails it's obvious what code broke it

From my experience fixing sparc64 package builds for Debian the second quality is hard to get right. Many times I would encounter a test failure that was the result of a complicated chain of function calls and it wasn't obvious what was broken.

When I was fixing the gtk build I discovered their codebase is sprinkled with ASSERTs which turned out to be SUPER useful, particularly because they communicated exactly where a precondition wasn't holding up. Previously I was never that interested in asserts but that experience makes me want to look into them again.

View on HN

On the flipside the reason excellent locality usually doesn't happen is that the narrower the scope of your tests are (think on the spectrum of integration tests to unit tests), the likelier that they are highly coupled with implementation details that might change, making tests brittle and refactoring harder. Or you go the full java style strategy-pattern-dependency-injection-abstractsingletonproxyfactorybean route to get both locality and ease of changing behavior, but then you're now building these complex class hierarchies and doing anything takes a lot more effort and boilerplate. Now changing behavior is easy but changing architecture is hard. It's a tough tradeoff, really. I'm not saying we don't need one or the other, but it's more of a case by case judgement call that folks would like to admit IMHO.

Assuming the tests ever passed, git-bisect should make it possible to find the commits that broke them. This is tedious and time-consuming (particularly if the build and test cycle is slow) but it can be automated.

https://lwn.net/Articles/317154/

With continious integration (CI) one gives excellent locality in time/changesets. So I think the locality argument might not be as important today. Though I am biased, since I generally prefer to have tests that exercise more code, normally only against external APIs.

Strong agreement on that pre/post-condition/invariants are highly useful though. I prefer that they are enabled also in production code, except for in the hottest of codepaths.

To be fair, the original article does not contain the word "unit" - it just talks about tests, which is arguably more general.

In my personal opinion, tests that are "non-unit" tests are still useful - testing the validity and correct functioning of larger pieces of software is a good thing, even if it does not run fast, or if the broken code is not immediately obvious.

Reply on news.ycombinator.com