Hacker News Reader

Comment by crdoconnor | Hacker News Reader

I used to think this too, but then I realized that this is actually backward thinking. It's quicker in the short and the long term to write tests.

Every time you code (whether requirements are clear or not), you go through the following steps (often executed in the space of 2-3 minutes):

1) Think 2) Write code 3) Build/run code 4) Test and see the outcome 5) Go to step 1

Even if you are not writing tests, you are doing step 3 and 4. Moreover, if you are not writing tests, you are doing steps 3 and 4 manually.

Manual is only quicker if there are a few of these iterations. If you are doing this iteration every 2-3 minutes, you will be doing it hundreds of times a day and thousands of times in total over the course of a project.

Automating that process will save you a lot of time in the long run, and will still save you time in the short run, as well as leading to higher quality code.

View on HN

I think a lot of disagreement about TDD comes from the fact some things are a lot easier/quicker to test than others.

If I'm writing a library to add two numbers, it's easy to create fast tests with no false alarms. Assert.equals(10, new BigDecimal(5).add(new BigDecimal(5)) - simple, fast, no false alarms, easy-to-maintain.

Now imagine you're making a website and you want to test with multiple browsers, and it uses caching and connection pools and sessions and whatnot, and the website has several dozen pages, and there are a bunch of configuration options and feature flags and you need to test with all of them.

Now all of a sudden you're administrating a pool of VMs with different OSes, writing web-scrapers, launching web servers every test cycle, waiting for tests of pages you haven't changed, running n tests * m pages * o server configurations * p clients, and all of a sudden you're spending more time keeping your tests working, or changing your software so tests run faster, and you're hardly spending any time working on features users want.

Naturally, to someone writing the first type of software, unit testing seems perfectly logical. But I can also see how someone working on the second type of software might get frustrated!

I wouldn't call your second case unit testing, but rather integration testing.

Part of the point of unit testing is that you keep the 'unit' under test to be as small as possible so you don't have these sorts of problems and can run tests as frequently as is useful. It doesn't remove the need for other types of testing suites, but they can be isolated and run at separate times.

I think the real point is that "testing in production," more commonly known as "monitoring," has to happen anyway. So early on, if you have a choice between adding better, more complete monitoring and adding tests, even integration ones, the monitoring will prove more helpful once you deploy than the tests were. Don't get me wrong, it sucks having to manually perform integration tests but when a product resembles a website and is semi-automatically deployed (e.g. has staging servers with automatic deployment, rapid rollback deploys or a canary-server deployment model, a few minutes of caching and few to no expectations that the app operates perfectly as long as it can be fixed within half a day, well, that's a very different project. In that environment, the piece of mind monitoring provides (no client errors in production) largely outweighs having no monitoring, just tests (did I test everything? What about (obscure government agency) using (outdated version of IE) who is reporting that bug?). Of course, as code is more complex than a website, or where you begin to have significant JavaScript, having at minimum tests for success-paths will help prevent regressions for application-wide changes, e.g. adding PJAX to a site and missing an onload. It's just harder to justify tests until you really need them on some assignments.

Your second case there is still relatively easy to test!

Consider the problem of intersecting two arbitrary 3D surfaces to get a curve. There are an infinite number of possible inputs. Except for rare simple cases, for each pair of inputs there are an infinite number of valid approximate solution curves AND an infinite number of ways to express each of those curves. Also, you want solutions which are well-behaved -- no kinks, no singularities, etc -- and as simple as reasonably possible (but no simpler).

And even that's still better than something like MP3 encoding or digital musical instrument UI testing, where the quality of the result is arguably subjective...

>Now all of a sudden you're administrating a pool of VMs with different OSes, writing web-scrapers, launching web servers every test cycle, waiting for tests of pages you haven't changed, running n tests * m pages * o server configurations * p clients, and all of a sudden you're spending more time keeping your tests working, or changing your software so tests run faster, and you're hardly spending any time working on features users want.

You're right - it eats a lot of time and it shouldn't. It's not as easy as unit testing and it should be.

I actually wrote an open source testing framework to handle this type of thing precisely because I was getting so frustrated with having to rebuild the boilerplate code to do this everywhere I went.

This is a simple example test I made with it - it uses a browser (w. selenium), libfaketime, and a mock SMTP server to test a django/celery/postgres/redis app:

https://github.com/crdoconnor/django-remindme/blob/tests/tes...

And this is the harness - it runs 5 services and a browser in 50 lines of code:

https://github.com/crdoconnor/django-remindme/blob/tests/tes...

The framework is currently in pre-alpha now, but it should be a bit more stable in a few weeks. It should be usable to test virtually any kind of software that runs on UNIX, also (not just python apps).

Easily handling the "m pages * o server configurations * p clients" problem is on the roadmap. Different browsers and server configurations will be treated as "just another kind of fixture", and it will be easy to feed fixture groups so that you can run the same automated test under multiple scenarios.

Reply on news.ycombinator.com