Hacker News Reader

Comment by bulte-rs | Hacker News Reader

> If you have a test suite that you trust so much that you are willing to deploy the system based solely on those tests passing; and if that test suite can be executed in seconds, or minutes, then you can quickly and easily clean the code without fear.

My spider sense always starts tingling when someone says something like this. Blind trust in an automated test-suite is a nice thing to strive for. But even for projects in which I have every confidence that the test suite is complete enough to actually trust it I'm still hestitant to "insta-deploy-on-pass".

Call me cautious, call me chicken, whatever... I'll not drink this flavour of kool-aid

View on HN

I think it's dangerous to rely solely on a unit test suite to guarantee the correctness of your program. Each unit-tested component may pass its own tests, but that doesn't guarantee that by interacting with other (present and future) components it won't exhibit any bug. I guess immutability would go a long way forward in realizing this guarantee, but I've never had the pleasure of working on a software where each component was truly immutable and independent of each other.

Very true. Here's an example of a bug that has bitten me more than once, has everything to do with immutability, and isn't reasonably within the scope of traditional testing tools:

A crucial class writes files to a certain folder. The test sandbox provides such a folder, and the class knows to use that folder when it's running in test mode. Obviously, the production environment is also supposed to provide that folder. But oops, it doesn't!

The tests exercised writing a file to disk, the tests were all green, and the system crashed anyway.

I always progressively test at different levels before insta-deploy, and run a set of "is this still OK" tests against production post-deploy.

Unit tests are at the heart of the test gauntlet, but absolutely can't be trusted by themselves.

And this is - IMO - where the magic happens. It's somewhat doable to "waterproof" your unit tests; but getting every (possible? Required? Anticopated?) interaction tested?

Magic.

I'm not saying tests are evil, I actually love my test suites. But as always; as soon as someone uses ALWAYS and/or NEVER, those tiny hairs in the back of my neck do a sort of wave and I try to ignore their message, which is a shame since (in this saga) both Uncle Bob and Nephew David have good advice to offer.

As for immutability; add in referential transparency, static ish typing and you've got yourself a nice development setup ;)

I have done two major projects with insta-deploy-on-pass and I would be hesitant to do it any other way. You still do manual and exploratory testing, you just automate as much of the "does thing X still work" that you don't have to manually test everything in order to ship anything.

Its one of those practices where the end goal may be unattainable, but the process of trying to achieve it has a huge benefit imo.

Completely agree. Every time I read something like that I always ask myself if I'm really the only person in the world who's had a bug in his test suite.

Maybe you're the only person in the world who's never made a mistake when manually testing new code? My test suite isn't perfect but it's more reliable than I would be.

(if you're thinking "do both": if I thought the test suite wasn't reliable enough, I'd get a better return on time spent improving the test suite than I do on doing manual testing)

I'll call you uncomfortable with the idea that the critical element of test driven development is making the programmer's life easier rather than producing software user's can trust. The correlation may be the reason for using TDD, but the goal is software that doesn't harm users. Making the programmer's life easier is only a potential side effect.

Reply on news.ycombinator.com