Comment by nickjj | Hacker News Reader

Comment by nickjj | Hacker News Reader

Parent Full thread

> Has anyone ever actually found this to be true?

It depends.

I've written a number of apps where I'm fearless with making changes because I trust the tests I wrote. One of the apps has been running for 7 years and it's had a lot of big updates, refactors, etc.. I'm breaking every rule there is on jinxing things but there hasn't been a single bug introduced due to those updates and there's ~85% coverage. It's only a ~3k line Flask app though, but it does get deployed straight to production with no staging environment and gets hundreds of requests per day. It writes to a DB, Redis, interacts with multiple 3rd party APIs and sends out webhooks so it does quite a few "external" things.

I've never been a fan of TDD and personally I think tests that really hit your DB, Redis, etc. help a lot more than mocked out unit tests or a billion unit tests and nothing else. I tend to write more tests that really test things together. Not full blown Selenium style tests, but I do really write to the DB and other data stores in tests and often use a framework's built-in test client for making HTTP calls. Everything can still be really fast too (~100 tests in 2 seconds is my usual rough guide for having an assortment of "real" tests with Flask).

> I've never been a fan of TDD and personally I think tests that really hit your DB, Redis, etc. help a lot more than mocked out unit tests or a billion unit tests and nothing else... ...but I do really write to the DB and other data stores in tests

I've come to the conclusion that the idea that "unit tests" should test functions/objects in isolation with completely mocked dependencies is based more on the slow speed of those dependencies in the past than what actually makes for good tests. Now that we have faster computers and storage devices, and easy/fast store creation, we should move past this.

Obviously, this is very dependant (no pun intended) on the dependency in question, but as a minimum, anything with SQL should have a test that hits a real SQL DB (PG in docker for example) at some point.

It's not even about faster computers and storage, but just about better test techniques, and better written tests. You don't actually need very fast hardware for fast tests; you just need to spend some time on it. And roughly know what you're doing.

External dependencies (such as PostgreSQL, Redis, what-have-you) are a pain though. I feel pretty strongly that just a single command ("go test", "cargo test", "npm test", etc.) should run all the tests on all platforms, reliably, with a minimal of fuss and messing about. Things like docker-compose or whatnot quality as "a fuss and messing about".

They can be a bit of a pain, but if you have a lot of logic tied up in your external dependencies (SQL usually bring main offender here) it's not possible to properly test without using said dependency.

That said, in the current job, we have a dedicated Postgres container that comes up with with a just file command. The setup, test, and teardown of schema all happen within the standard test system of the tested platform (pytest or go test specifically).

Reply on news.ycombinator.com