Testing is Overrated | Hacker News Reader

Testing is Overrated

infoq.com

14 pointsnreece23 comments

Testing zealots never seem to understand that some errors are expensive and some are not. Testing should be a cost benefit analysis.

I never waste my time writing tests when the consequences of errors are cheap.

Conversely, I always write tests when the consequences of errors are expensive.

I agree that with you that "some errors are expensive and some are not," but you don't know which category a bug falls under (cheap/expensive) until after the fact. So making that decision of what (or what not) to test isn't always so black and white.

Also, a bug might be cheap now, but when it breaks a year down the road and no one knows what happened, it's going to be a big problem.

It depends on the context. I've had some very simple fixes/updates I've wanted to put in production that I knew were incredibly low risk, and anything that could go wrong could not possibly be catastrophic, only to be vetoed and told to go through a very expensive testing process.

Yes you do. When you are making a prototype errors aren't expensive.

When you are modifying code that customers are already paying you lots of money for errors are expensive.

I always check to make sure a method roughly works, before moving on to the next one. Most of the time, this means either writing a unit test, or firing up the code and all of its dependencies, and clicking through it looking for bugs.

If you are good at testing (like any skill, it requires experience), writing a unit test will be quicker than manually checking the code.

So, perhaps, the cost isn't in writing the tests, but in acquiring the experience to test effectively.

Also, and YMMV, I find writing code in a tests-first manner cuts my stress/frustration of coding by an order of magnitude. It greatly reduces the amount of context I have to keep in my head, so I'm not in the state where I can remember 90% of the information I need to effectively write this class.

> I always check to make sure a method roughly works, before moving on to the next one. Most of the time, this means either writing a unit test, or firing up the code and all of its dependencies, and clicking through it looking for bugs.

In my experience, well factored methods are correct 95% of the time. When there is a bug you notice when you test things that exercise an entire host of methods at once.

> Also, and YMMV, I find writing code in a tests-first manner cuts my stress/frustration of coding by an order of magnitude. It greatly reduces the amount of context I have to keep in my head, so I'm not in the state where I can remember 90% of the information I need to effectively write this class.

Testing is not a good way to help with context. A good debugger and well written code is.

Also, it's important to remember that tests are code too. A test needs to stand the test of time just as code does--if the tests are too brittle then you're just creating busy work.

However people who haven't tried methodologies like TDD or BDD shouldn't parrot this as justification for not writing tests. First you need to learn how to write tests, and how to write more testable code. Test-first is invaluable training for developing these skills.

Once you have that experience you are better qualified to decide what constitutes useful testing. Frankly, I find the most value comes from basic sanity tests. Testing obscure edge cases of algorithms is often times not worth it because the mistake is made once and corrected. Having the test case is good (especially if written up front as an excercise in specification), but in many cases the chance that these things will break once fixed approaches 0.

So many unit tests just rephrase the function you're testing -- all they check is whether or not you edited the code and test at the same time.

There are some tests that are unassailable -- like checking to see if the output of a sort function is sorted and whether it has pathological complexity. On the other hand, duplicating all the conditionals in the code as tests is bullshit. Mocks are even worse!

And this is where the line between testing and documentation becomes blurry: I trust a running test more than (outdated?) documentation. So 'just rephrasing' does make sense in some cases.

The reason rephrasing is valuable is that it stop people from taking out or removing needed cases. If there is a test and someone changes that logic, it forces them to at least realize someone else tested and wanted that logic. Then if you search through the code and no something is unnecessary you can change the logic and the test.

Please watch the video first, testing is useful, you must write tests, and perform tests.

But! tests will not increase your quality, they will just keep it.

My dirty little programmer secret is that I've never written a unit test. Thanks for making me feel bad :(

I watched the video and, indeed, it did not absolve me of my sins.

lots of people have this dirty secret. lots of well-seasoned programmers to be certain.

I find when writing in scripting languages such as ruby against frameworks that are in constant flux, I have little choice but to write tests against key things like user registration, activation, authentication.

I wrote a webapp in erlang 4 years ago. I felt the code "was" the tests since my erlang code was concrete and well guarded. I never had any surprises between test and production.

This is not to bash the value of scripting languages. Its a "pay now or pay later" problem.

So what about Java? Its static typed so I should get the same benefits as I did with erlang right (btw, erlang is not typed)? My experience tells me:

1 - ruby - "pay later"

2 - erlang - "pay now"

3 - java - "pay now and pay later"

I never wrote tests until recently, when I started working on a functional JS library, and realized that if you're not writing them, you're doing them in your head (which is slower and more error prone). This only applies to langs/libs/APIs though; for apps, I generally find the UI itself to be the best test.

Yeah, the last thing I worked on like that was a networking engine and I could really see where tests would have helped me a lot (all existing code was covered, but not the new stuff I was writing). Hopefully I will get to work on that sort of stuff more in the future and can get into the habit of writing tests in the process.

I disagree. Writing tests forces you to comb your code enabling you to find obscure bugs and slowdowns.

Testing can help find bugs. Testing makes refactoring a lot easier. Testing protects you against other people breaking your code.

I don't know if testing is overrated, because I haven't carefully measured how much hype it gets, but I do know testing is useful.

There was a slim volume called the "The Art of Software Testing" that I borrowed once from my mentor.

In it, the author posits the following perspective: a successful test is not one that finds zero errors, a successful test is one that finds errors.

Zero errors is also OK. As is a proof of correctness.

Are all the Ruby Fringe videos available someplace for downloading (such as happens with the assorted RubyConf vids)?

I find the following sequence of testing is productive.

1. Smoke Testing

2. Performance Testing

3. Other tests

infoq is overrated.

Reply on news.ycombinator.com