Hacker News Reader

Comment by richcollins | Hacker News Reader

90% of the code you write at a startup is exploratory. It's written to learn something about your customers. If you break it, you typically don't incur a much of a penalty in terms of cost / time.

Testing every bit of code does cause overhead. I find it takes more than twice as long to write code when you test it. This overheard puts a big drag on the speed at which you can move, which can kill a startup.

For these reasons, I only test code that's proven its long term value.

View on HN

As much as I've tried, I really can't get into "test first" development. I find that I make so many tiny changes when I am first working on a new project, and refactor so much, that the tests would really slow me down.

But once things get more solid for me (and, like you said, I know the code has some long term value), I'm a big fan of building a solid test suite before a full production launch.

I also think it is worth the time to use code coverage tools to find sections that are not being covered. Odds are your test suite only covers 60-80% of the total logic in your app.

For me it was a discipline thing. Then a mentor of mine convinced me that not writing tests was actually okay when you're exploring something. He suggested I only write tests for "production" code - something I would deploy.

So I routinely would whip something up, see that it works, then start a new project with tests and rewrite my experimental code.

You can imagine what ended up happening - I realized how much time this was wasting, so I started writing tests to test my experimental code so I didn't have to throw it away.

I got my brain hacked. Maybe that story will help someone else. :)

How can you refactor if you don't have a test suite? Do you manually check everything in your application after every change? That's the main value of having a test suite you can trust. It speeds up refactoring massively.

If it's taking you twice as long, you're doing it wrong(TM).

Or more likely, you're not measuring the time properly, using memory to estimate the difference. It's easy to discount the time re-running the program every time to check the result, and think you spent more time than you did writing tests because that seems painful to you.

So: measure it rigorously, and pair-program with someone that knows how to do it properly.

you're not measuring the time properly, using memory to estimate the difference. It's easy to discount the time re-running the program every time to check the result, and think you spent more time than you did writing tests because that seems painful to you.

Given my (admittedly small) sample of other co-workers I have seen, it is most likely this. I see people constantly code, compile, run, type in a command, go "yeah, that's right" or "no, that's wrong" and then go back to step 1. I ask them why they aren't writing a test suite instead. "Because that takes way longer!" is the reply. They never actually bookkeep the time they are spending doing it. Their mind says "running it this way takes me 10 seconds" not "running it this way 50 times a day takes me 500 seconds". Selenium is amazing for web dev.

If you can reduce your problem state down to a sentence that sounds testable "When I press the login button, I go to the login screen", it probably is. If you can break a complex sentence into multiple smaller ones, you have a test suite. If your sentences are customer focused black box, you have acceptance tests. That's sort of it. If you have a problem where that doesn't work, then that's OK. There are a small class of hard problems that you can't test that way, and good developers know what they are. Great developers know that doesn't mean they can't test the other bits ;)

One day, I'm going to write a research paper where I watch people who refuse to write test cases and see if they actually are working faster, or whether they've just rationalized what they're doing in their minds.

Or maybe -- just maybe -- his problem domain might be harder to write tests for than yours is.

Look, as one of the Rakudo Perl 6 developers, I love our extensive test suite. It's an absolutely terrific development tool. Writing tests for it is usually quite easy, helps make sure we're getting the subtle details and interactions correct, and benefits not just Rakudo but every Perl 6 project. It's a huge win.

But I'm also working on a ABC music notation to sheet music program, using Perl 6 and Lilypond as tools. And I don't see how I could fully test the results of that without using some sort of optical sheet music recognition software -- and even if I could get that up and running, it would still only be testing that the musical symbols were correct, and not that the sheet music actually looked good. If you can tell me how to set up an automatic test for those things that will take me less time to setup than the 40 or so hours that writing the actual program took, I'm all ears.

Hmmm... I dunno. A lot of times I like to bang out the design for something fairly significant without running the entire thing once. I might have an interpreter open and I'll play with a bunch of values trying to get the right form of everything up and going. Y'know, working on the details.

Then I start slapping it together, writing some tests on it, adding some fixtures for my data. Etc. etc. At the end, I've got all my tests done and my code working.

TDD or BDD is fine when you know what you're trying to build. If you've only got a few vague, high level ideas about your design and you're just iterating on the problem space, it's a drag man.

He is talking as a business owner and not as a programmer/coder. You should also consider his background in Lean Startup Circle. Fundamental rule of MVP (minimum viable product) is to get your product out there in the market as soon as you can. Writing tests goes in the opposite direction. The goal of an MVP is to validate your idea with real users. To see if it works or not in the first place. There is often very little time to write test cases or even look at edge cases. In that light writing tests doesn't make much sense.

Why can't they write a test that makes sure the basic case works?

Interestingly, the same thing could be said about using a powerful type system or not.

It's easy to assume that the compiler errors you are getting are just wasting your time, forcing you to put arcane type declarations, annotations, and constructors around your code. But that time might reduce the testing burden, because the compiler can make more guarantees for you; and therefore it might actually save time.

I guess I'm not a true Scotsman.

I'll give an example . Imagine that you're writing an append method for a list that's implemented as insert(x, size). TDD requires that you test the interface, not the implementation, so you'd end up with lots of test cases. How would the test code compare with the implementation?

While I disagree with your hypothesis that "it takes more than twice as long to write code when you test it" (because you discount the bug fixes that are you deferring until later rather than discovering them with TDD), I do agree with one of your points:

"Testing every bit of code" is unwise.

I am a test-obsessed TDDer. That said, I do not test everything. I test proportionately to the cost of failure for a given particular feature.

Zero cost of failure -> zero to very few tests High cost of failure -> many tests sometimes with full path testing

This warrants discriminating degrees of test coverage:

* Partial coverage: not all of the code under test is executed

* Full coverage: all code under test is executed

* Path coverage: all code under test is tested for every possible path that it can be executed

Without path coverage, any suite of tests is potentially imperfect. Therefore, as Stu Halloway sometimes presents about, it is extremely possible to fail yet still have full test coverage.

But that's the point of this article; starting your test suite is as simple as spending a couple of hours actually learning a test library, then instead of throwing away your hack tests, keep them. There's no way that's a 2x speed penalty, no way, and in practice it's faster to develop that way.

If you spend five minutes constructing up some test cases for your code, you can either throw them away immediately after that, or continue to get leverage on your code with them for the next weeks, months, or years. Which do you think actually produces a better velocity, vs. which might make you feel like you're going faster?

Why does it take you twice as long to include unit tests?

Here is one possibility: when you don't test it, you don't know it's broken, so you don't spend time fixing it. When you do test, you do know it's broken, so you spend time fixing it. Hence, writing tests takes longer.

Edit: But, if you only use 10% of your code, it is probably fine if 50% of it is broken.

Reply on news.ycombinator.com