Hacker News Reader

Comment by mercurial | Hacker News Reader

What I wish people had told me before about this article:

- this article only deals with unit testing

- it does not help you write more maintainable tests

For the record, I don't think (unit) testing is simple. Tests are, in my experience, the least maintainable and readable part of a codebase. You often end up with many lines of setting up complex business objects. And usually, the only help wrt to what the hell is being tested is the method name. And that's just unit tests. Integration tests are a major pain, and often even harder to understand and maintain.

View on HN

Not to mention the fact that unit tests can only help with verification - i.e. that you have built the software to expectations and to catch regressions.

I would say that 90% of the grief I have experienced building applications is the validation part - that you are actually building the right thing.

I work in web consulting where we frequently take on a project from scratch (greenfield or otherwise) and ramp it up quickly in 6-18 months then hand off to the client who will likely maintain it for a year or two or three then redesign (nature of the projects, not a result of quality). My company is pushing for more and more sophisticated automated testing at the time that I'm starting to think that automated testing is costing us more effort than it's actually saving. Our bugs are, as you say, predominantly misunderstood requirements. The tests never catch them. Even with correctly understood requirements, unit tests frequently miss edge cases. Something that gets missed in so many of these process debates is that different kinds of work require different processes. If your job is to maintain a massive software product, then automated tests are probably your bread and butter. But for my line of work, they just ain't.

You folks are in an odd situation: you always see the costs of testing, but you never experience much of the benefits. Your intuition of the right amount of coverage is guaranteed to be off.

For me the right amount of automated testing is all about exploring the cost vs benefit boundary, which shifts with tooling and circumstance. But my firm rule now is: code must either have a firm expiration date or full test coverage. Nothing between is allowed.

So if you're going to cut back on testing, make sure your customers have agreed in writing on a date after which they should throw the code away.

When you say "misunderstood requirements" my read on that is a failure of functional testing. Unit testing is good at testing small chunks of code for things like valid inputs/outputs, but not for validating whether the software performs as specified. Functional testing is hard to automate initially, but can be automated for regression testing. Performance testing, on the other hand, is easy to automate. Sounds like your organization is pushing automation as a way to control costs, without perhaps fully understanding what can be automated and when.

Software is not just development, there is also a maintenance perspective. If I have a bug report in a complex piece of code it's a lot easier for me to write a test that covers the use case and work from there then building, fireing up a test device, etc.

Or when refactoring a piece of code that renders UI, write some tests to verify a couple of common cases and then refactor. Run the tests and I'll know whether the output is the same.

I'm saying there's a curve measuring the time spent writing tests and the time the lifespan of the application. Think of like the rent vs buy debate. Tests are a the big downpayment you make hoping that it will pay off in the future. The kind of project my company does are rentals. Same reason we rarely do major refactors.

It sounds to me like the benefit is to the customer, easing their maintenance. Maybe you should raise your prices?

As well has helping with verification, I find unit tests help with design. I write less code, and that code is simpler and with looser coupling when I write tests first.

This has been my experience as well. And I actually like the rspec style of testing that the author dismisses because it allows you to group tests by functionality covered. This goes a long ways towards making your tests more approachable and maintainable. When I contribute to a Ruby gem with good rspec-style tests it's generally trivial for me to see how to format and where to put my tests covering the new functionality.

I agree.

I've always said that RSpec is about documentation, not testing (hence the name - it's a specification for your code).

However, recently, I've been using Minitest-spec; not quite as readable but less magic and it feels faster.

This is mostly what unittesting is about. Not actual "testing".

And then every time that stuff changes, all the tests are suddenly out of date, which means people get used to the idea that failing tests are fine and fixed by updating the tests.

This is a people problem as much as a test problem.

1. Write your code so that independent units of functionality (algorithms) can be tested. 2. Make sure people understand, when changing code, either a) change the tests first, or b) recognize what tests should be failing because of the change and which shouldn't.

For good, testable code combined with good tests, it should be rare that the tests start failing when you change the implementation. Admittedly, that's true in theory and can vary widely in practice.

That sounds like a problem of too-tightly coupled tests.

Sure, that's not a problem unit tests are meant to solve.

The appropriate practices there are short iterations, frequent releases, acceptance tests, and having product people sitting next to developers.

For those wondering how unit testing relates to other practices, go read Kent Beck's Extreme Programming Explained. He's one of the people who led the current wave of unit testing adoption via jUnit. Years before the term "Agile" existed, he was on a team that found a highly productive way of working, one for which unit testing was an important foundational element. But it was just one of 12 practices they saw as necessary.

Once you have that working well and you're building what the businesspeople expect, the next step is to make the businesspeople start testing their assumptions. That's basically what the whole Lean Startup thing, minus the fad, is about: let's not just go build whatever the guy with the tie says. Let's all get evidence about what's really worth building.

Plus the article only shows the most basic kind of unit test possible. Yes, unit testing an 'add' function is easy

I think that was kind of the point.

The point being that testing simple things is simple?

I imagine the point is to overcome inertia --- to get people to take a first step. Often that's the hardest part when someone is facing something they expect to be painful, dreary, and complicated.

It's sort of like "the first dose is free", but applied for good.

Yup. "testing" is sometimes mythologized as this huge, complicated, complex thing. And it is. But it's also really, really easy. As the post says, fundamentally, you do something, then see if it did it right. That's it. Everything else builds from there.

I absolutely agreed that writing high quality, maintainable tests is hard (in fact, I say so in the article). My point is really that the way testing is discussed, particularly in the Ruby community, makes testing seem more complicated than it is, and I think puts some people off.

I think that's more to do with the Ruby community than unit testing. They seem obsessed with new DSLs for everything.

I'm not sure this point really gets through. The way the article read to me was "Testing is simple and easy (1)"

1: small print - writing good tests is hard

I could likewise argue that "writing code is simple and easy, but writing good code is hard", but this statement has little value in itself. It seems more an argument against using complex testing frameworks and going for simpler syntax. As a fan of Nose, I absolutely agree.

If you write your tests first (TDD), your "complex business object setup" (aka state setup) invariably gets simpler.

The more complex your code, the more coupling there is, the more sources of bugs there are potentially... and the harder it is to write tests for that code.

Break down your code into smaller methods and more focused objects and just watch your unit tests get simpler.

People tend to forget that TDD isn't a 3-step process. Even this article makes that mistake. Write a test, pass that test, refactor. People generally forget the refactoring step.

Basically though, if you find your test difficult to write, there is a problem, and it's not with the test. And that should be your sign that you either don't understand the problem, or you are trying to do too much.

Now, this is hard. It's hard to accept that way of thinking. But the end result has always been cleaner, better code in my experience. It's not always the obvious way, but it's the best way.

I'm not convinced, for a couple of reasons. I haven't found unit testing to help in any way with the complexity of business objects, which are often more-or-less dumb beans. When you are handling complex problems, you are going to end up with complex data structures. Unit testing can help with modularizing services, though.

The second problem is that whenever you're faced with a large, older codebase, it's often difficult or impossible to refactor away complexity in core components.

TDD is really enjoiable in Coffee Script. Since Coffee Script supports literate code you can write your specs in markdown and embed litte code snippets which verify that your claims are true.

Currebtly I am only toying with it in one of my pet projects but it seems promising.

This is why I write unit tests in the exact framework the op dislikes: rspec. It makes testing far more readable.

If your unit tests are hard, that's a non-SOLID smell.

That's a dynamic language problem. In a static language they are trivial to maintain.

Reply on news.ycombinator.com