Comment by pmiller2 | Hacker News Reader

Comment by pmiller2 | Hacker News Reader

No. It's a great idea.

The author mentions "characterization tests," which document what the code actually does as opposed to what it "should" do, or what the documentation says it does. These kinds of tests are gold, especially if you go back through the bug tracker and create tests for the major bugs that have been fixed. Doing this gives you a good framework for constructing a regression test suite, which is what you really want when working with legacy code. It's like the opposite of TDD, because you want a green light to start, rather than RED -> GREEN -> REFACTOR.

OTOH, some code just isn't all that testable by its very nature. I'm thinking of stuff that requires expensive, custom hardware that would be difficult to mock out, for instance. Or GUI code. Or, if you're unlucky, like I was in a previous job... GUI code that requires specialized hardware.

Also keep in mind the amount of work it takes to make the code testable. You can easily end up breaking things just doing this if you're not careful or if the code is just not written with testability in mind.

Seconding the "you want a GREEN light to start." When you're dealing with legacy, untested code like this, you're sometimes going to write a test that by all accounts should be RED, but it'll come up GREEN.

And the problem then becomes "okay, now, which of the other 16,000 untested files in this codebase are relying on that broken behavior." It's not a fun place to be in (welcome to my day job) but you have to remember--You're writing characterization tests, you're trying to capture what it currently does. Not what it should--every difference between those two is, unfortunately, potentially an important one.

It's not worth fixing one minor thing if it breaks 14 other things down stream (that you then won't catch, since they also don't have tests).

Red -> Green -> Refactor is important because it indicates that the test genuinely behaves as required.

(RED shows you that the test fails when it should: there is good reason why this step should not normally be skipped; then GREEN shows you that the test passes when it should)

Parent was suggesting that an existing system's tests should already pass (green) before you monkey with it (red).

Although if you're fixing bugs, red-green-refactor can work well.

It's very possible that without a red test, your green test would be green with inputs for which you intended it to be red.

It's not enough to know that the test pass, you need to make sure the test would fail. Otherwise it's like having a cybernetic canary that doesn't need air to survive (great example, isn't it?).

If you were expecting the test to fail for some inputs you might as well just add another test case for that scenario.

Reply on news.ycombinator.com