Good code needs few unit tests | Hacker News Reader

Good code needs few unit tests

andreyf.tumblr.com

59 pointsandreyf39 comments

This argument is not sound at all. The statefulness of your code has nothing to do with why we write unit tests. Even if you write a purely stateless function, it would still be useful to write a unit test to check for correctness. If you are doing TDD, you wrote the test before you wrote the function and that way you know exactly when you are done (because the test now passes). If you wrote your test afterwards, it is still useful if you ever change the behavior of the function, because any unit tests that were written that use this function will now let you know where and in what manner your change has affected the entire code base.

If you are writing bad stateful code that has a lot of bugs and you are not smart enough to figure out how all the states interact, unit tests will help you a lot, but there are many other scenarios where unit testing is useful.

I would not say good code has few unit tests. I say arrogant code has few unit tests. If you write few unit tests, as the size of the codebase grows, you have increasingly less certainty about how any change to the existing codebase will affect other parts. When the codebase is small, this is not a problem. If you are very intelligent, the size of the codebase that you can manage in your head might be quite large. But you are just wasting mental capacity that might have been doing so many more interesting things if you didn't have to remember everything and could trust your unit tests to give you the information you need when you need it.

I'm not completely against TDD. However, one thing I've found is bugs are caused by things you didn't think to write a test for (or write your code to handle sans tests). The mundane stuff I see in the vast majority of unit tests are things an experienced programmer almost never screws up. So how do you write a test for a case you couldn't foresee?

My biggest argument in favor of writing tests is to ensure my code doesn't break relative to other people's code. There are many times I've encountered an update to a ruby gem with no major or minor version change that has broken a, usually implied, contract with my code. I say implied, because duck typers, at least what I've seen in the ruby world, tend to be slack about formalizing interfaces. I write tests to protect against that sort of thing.

So how do you write a test for a case you couldn't foresee?

You don't, but once you discover it, you write a regression test to make sure it doesn't come back.

However, IANATDD, I am not a test-driven developer - I don't see any reason to be dogmatic, but I do think unit tests can be extremely valuable. So this answer might not be TDD-approved.

"So how do you write a test for a case you couldn't foresee?"

"You don't, but once you discover it, you write a regression test to make sure it doesn't come back."

Sure, that's SOP in any good dev. house that does testing. But you've failed to address a key weakness in TDD. The real question is how do you discover these sorts of defects given that a single developer writing in a TDD style is unlikely to do so? TDD is clearly not a cure all, and this is a major weakness of it. Other development techniques, many of which can complement TDD, such as formal code reviews and beta testing can do a better job of getting you to higher product quality than TDD alone.

"TDD is clearly not a cure all"

Who said it was?

Exactly!! There is no such thing as a cure all. Anybody angry over a technique that does not solve all problems is clearly suffering from the Silver Bullet Syndrome.

What? Why does something have to be perfect for somebody to get angry over it?

Not exactly sure what you are asking. I mainly meant any tool you use will not solve all of your problems. Pretty basic common sense, IMO.

> So how do you write a test for a case you couldn't foresee?

TDD is a design technique, not a testing technique.

For me, I don't always know what the edge cases could be until I write them down. Writing tests allows me to get a clearer picture of what kind of coverage I have.

Another nice thing is you can read what the edge cases are going to be, rather than having to remember them all of the time.

TDD does not claim to fix all of your bugs. You often do not need to theoretically prove your software is correct, so you don't need to have you unit tests test all possible contexts (e.g. handling disk failure vs. any other error when rendering a web page). That is what abstraction is for.

TDD does force you to think about the interface of your abstraction, however. If it is painful to write (or maintain) the test, then that is a good indication that the abstraction is wrong or not being properly tested. Don't be afraid of deleting tests when they are not useful.

I often find that these unanticipated bugs are often due to incorrect abstractions. To fix it, create the correct abstraction and test it.

It is also simply faster to develop in larger projects than creating the state that would test your unit. For example, you could make a change, manually refresh the page, and debug or write the test and make it pass. You also get the benefit of regression testing.

Note that the narrower your unit, the less likely it will be useful for regression purposes.

I often find that when beginning a project, my units are smaller. As abstractions are built, the nonessential tests are deleted and future tests are built upon abstractions that I care about.

I think I'm blithely misreading the article here, but I'm inclined to take it as advice about how to do TDD, rather than an argument against it. Specifically: if a class has lots of tests, this is a sign that the class is too complicated. Refactor until it is simple enough not to require as many tests (possibly by narrowing its interface, possibly by breaking it up into multiple classes, possibly by rethinking how it's implemented).

From what I know, Andrey (the poster) is pretty neutral on TD, maybe even a proponent. He's saying you should have an interface in mind when you start to write unit tests. Sometimes you can do that before writing code. Other times, exploring with a REPL first is better.

> Even if you write a purely stateless function, it would still be useful to write a unit test to check for correctness.

Indeed. Tests are a good idea in, say, Haskell, too.

The point doesn't follow from the headline. The argument isn't that unit tests are bad per se, it's that good code has fewer testable abstractions, and thus requires fewer unit tests.

Which is basically isomorphic to "good code is tight code". Duh.

The argument is that good code [...] requires fewer unit tests

Great point, my mistake. Title and headline updated (was: Good code has few unit tests). I really wish it were "duh", but measuring code quality by "unit tests per line" [1] is frighteningly common, and seductive in its simplicity and intuition.

1. Not necessarily directly. It might be in the form of "I'll add more unit tests to improve this code" (unit tests either cover an interface or they don't; going in to 'add more unit tests' to 'improve' code means you didn't define a clear enough interface to begin with). Or "our open source project has more unit tests than another" (but I won't point fingers).

But "unit tests per line" is a good metric. Why? Because programmers (well, at least I) hate writing unit tests. If you have a high unit-tests-per-line ratio, then writing one fewer line of code will let you avoid writing several lines of unit tests.

The easiest way to bump your unit-tests-per-line metric is to delete lines of code. That's a positive thing, in my book.

Which isn't necessarily right, the whole "good code is tight code" thing.

If you're writing something simple, abstractions are probably just extra typing.

If you're building something that actually has multiple parts, those parts should be abstracted from each other.

If an abstraction requires extra typing, then it isn't really abstracting anything is it? Ie. you wouldn't use it if it wasn't going to benefit you.

Using an abstraction which increases the total amount of code could be worthwhile if it makes the difficult parts of your code simpler.

Pulling an example out of thin air, shorter total code:

    var counter = 0;

    function foo() {
        log("Incremented");
        counter++;
    }

    function bar() {
        log("Incremented");
        counter++;
    }

Code with an abstraction which makes foo and bar shorter:

    var counter = 0;
    
    function increment() {
        log("Incremented");
        counter++;
    }
    
    function foo() {
        increment();
    }

    function bar() {
        increment();
    }

Even longer code, but which encapsulates the counter variable, possibly making it easier to reason about:

    var Counter = (function ()
    {
        var counter = 0;

        return {
            increment : function() {
                log("Incremented");
                counter++;
            },
            current : function() {
                return counter;
            }
        };
    });

    function foo() {
        Counter.increment();
    }

    function bar() {
        Counter.increment();
    }

Well, if you abstract properly, you often wind up with fewer lines of code at the end, and certainly more predictable and easy-to-modify code. Am I really defending this? What's the opposing view, spaghetti code? :)

No, the opposing viewpoint to "more abstractions" is "better abstractions". ;-)

There are a surprisingly large number of programmers who learn "Oh yay, I can use abstractions to simplify this bit of code" and then turn that into "I think I'll make abstractions out of every bit of code". I was once one of them. In the process, they turn something that could've been a simple program, one that you could fit into your head, into an Enterprise Behemoth that you can't modify without implementing a half dozen interfaces and touching 30 files.

There's a cost to abstraction - it's (usually) extra code, it makes it harder to follow the logic of the program, and it makes your program less flexible in directions other than the intended use. So don't do it unless it actually gains you something in simpler code. Programs can become spaghetti-like through IFactoryFactories just as easily as they can become spaghetti-like through switch statements.

Also on Hacker News at the moment and fairly relevant:

http://coderoom.wordpress.com/2010/03/19/5-stages-of-program...

I said "properly", check out my other comments on this thread :)

But if you do test driven development then aren't you forced into testable abstractions?

I'm not forced to program at all. I do it because its most always held my interest. I've passed my 10,000 hour mark years ago. So for those that program well and do so by choice, why can't we spend our time focusing on defining good clean interfaces on the "real" code instead of the tests?

Well, don't get me wrong, I've seen my share of codebases with "Iface" and "Impl" classes (ew), and stupid, counterproductive unit tests that were done to check off boxes on a PM's checklist rather than improve stability. My favorite was a test to ensure that some static method somewhere returned the same hard coded string every time. The method was one line long, 'return "foo"; '.

But. If you've defined nice clean interfaces, and a particular module has complex behavior that will have to evolve over time, then adding tests to certify that that behavior is correct shouldn't be too much of a chore -- and they save your bacon when you need to change it and track any regressions afterwards.

The title should have been "A lot of unit tests does not mean you've written good code." I think the implications of the post are correct, but I think it would be incorrect to say that if you have a lot of unit tests it means poor code.

There are engineers who have a difficult time seeing things architecturally (for reasons ranging from ability to priorities). Razor sharp focus can produce a lot of unit tests while missing some refactoring that might have helped reduce the code base, and therefore the number of unit tests.

[Edited]

The thing I rarely see brought up in discussions of automated testing is longevity. If you have a team of developers and the codebase is going to be around for a while, you need to grow tests over time so that the someone doesn't accidentally break something they weren't aware of and so that you can refactor your code with confidence that you're not breaking existing functionality.

Sure, in an ideal world, everything is isolated by clean interfaces and encapsulation, but even in that ideal world, you still have a complex system that produces sometimes unexpected side effects (the emergent behavior of a software ecosystem).

  - One quality measure of an abstraction is the complexity
    ofits interface (ie API size).
  - Another measure is the amount of state the abstraction
    encapsulates.

The simplest abstraction of many interfaces is two methods:

  setParameterForNextCall(paramName, paramValue)
  call(functionName)

The API is small, not complex and the amount of state is small, given that parameters are only set right before a call, all parameters are cleared after every call and no other state is retained. Yet it's clear to anyone that this isn't a good abstraction and that it doesn't result in an implementation that requires few unit tests. It requires an immense amount of documentation and allows a great many variations that yield bad/unexpected results.

  Hence, good abstractions require few unit tests.

Abstractions don't have unit tests: implementations have unit tests. It's the size and complexity of the implementation made possible by the abstraction that matters.

That interface is certainly inconvenient to use, and potentially quite dangerous without a clearParameters().

It's not at all a rebuttal of the article, though. The article points out that the number of unit tests a design requires is larger for more complicated designs. An interface like yours would require no fewer tests than another interface exposing the same functionality using conventional function calls.

Good arguments need few straw men.

Hm. I think, one could condense the argumentation as follows: Number of Unit tests = c * (Complexity * Use Cases + State-Tests) for some constant c, if Complexity is a measure of the interface complexity, use cases is the number of use cases of the API and state-tests is the number of tests required due to statefulness of the API.

I think, this approximation is at least not wrong. I don't know if the characterization is complete, but it looks right enough to fool me :)

However, I think the problem in the argumentation is that the author assumes that "use cases" is small. While I agree, that usually, the number of use cases for an API should be small, there certainly are abstractions with a lot of use cases which are good even though they require a lot of unit tests.

An example of such a big abstraction is something I developed in the recent university project. Basically, it is a domain specific language which is compiled into state machines on partial bit sequences (which are used to describe data deserializers). The major job of this abstraction can be described in a few formulas and the abstraction hides enough complexity to be worth it, but it requires quite a large number of unit tests due to the number of possible edge cases in the specification of the DSL. Thus, at least I'd consider it a good abstraction in spite of needing many unit tests.

BDD encourages the developer to define a clear behavioral specification for the unit of functionality before you implement it. By first focusing on the usage I find that the abstractions I produce are simpler and in turn have fewer unit tests. I think the author is saying that TDD done well results in fewer tests for simpler code.

All good code has the property of clear specification. The place where BDD really shines is that it forces you to use your own abstraction in practice, which often uncovers where it is unwieldy to use.

Another way to get lean code is to use a proper, powerful, static type system. It enforces you to think about the code "skeleton" and the "tests" are done automatically by the type checker at each compile/load of the code. It also reduces the amount of testing needed: There is no way something different from an integer can be passed as a parameter, so you do not need to account for that.

My code does not use many Unit tests per se. But it does autogenerate a lot of test cases to check certain properties. Autogeneration is excellent at producing corner-cases, so if that is what you are after... I generally find this is a better use of my time as a programmer: Better spend some time developing the autogenerator than writing boring tests which my code for most part will pass in first attempt.

All he is really saying is that simple architectures are better than complex architectures, if they fulfill the same functions.

I don't argue that underlying point, but the way he got there sure was a lot more complex than, "KISS."

Any line of code metric is fundamentally flawed.

Including yours? ;-)

Exactly!! I can take your response as meaning two different things, both of which I agree with.

- My assertion is fundamentally flawed. - Even the line of code metric YOU come up with is fundamentally flawed.

Reply on news.ycombinator.com