Hacker News Reader

Comment by throwaway0asd | Hacker News Reader

My learning from test automation:

* The primary goal of test automation is to prevent regression. A secondary goal can be performance tuning your product.

* Tests are tech debt, so don’t waste time with any kind of testing that doesn’t immediately save you time in the near term.

* Don’t waste your energy testing code unless you have an extremely good reason. Test the product and let me the product prove the quality of your code. Code is better tested with various forms of static analysis.

* The speed with which an entire test campaign executes determines, more than all other factors combined, when and who executes the tests. If the test campaign takes hours nobody will touch it. Too painful. If it takes 10-30 minutes only your QA will touch it. When it takes less than 30 seconds to execute against a major percentage of all your business cases everybody will execute it several times a day.

View on HN

Tests are not tech debt. You could have bad, brittle tests that you could consider debt but just having tests isn’t debt. Debt implies there is something you could do about it in the future to pay it down, which isn’t the case for a good test suite.

It’s debt. When you can’t add new features quickly because you have nightmarish tests to fix and you spend more time on the tests than the product, I’d say it’s debt. Especially with the insane mocking setups.

Yes, those are the bad tests I was referring to. NOT having tests greatly increases the debt burden of your production code because you cannot refactor with any confidence and so you simply won’t.

> Yes, those are the bad tests I was referring to

This is the No True Scotsman issue with testing. When it fails, you just disregard the failure as "bad tests". But any company that has anything that resembles a testing culture will have a good amount of those "bad tests". And this amount is way higher than people are willing to admit.

> you cannot refactor with any confidence

Anecdotally, I've had way more cases where I wouldn't refactor because too many "bad tests" were breaking, not because I lacked confidence due to lack of tests.

There are many things beyond tests that allow you refactor with confidence: simple interfaces, clear dependency hierarchy, modular design, etc. They are way more important than tests.

Tests are often a last resort when all of the above is a disaster. When you're at a place where you need tests to keep your software stable you are probably already fucked, you're just not willing to recognize it.

You shouldn't have zero tests, but tests should be treated as debt. The fewer tests you need to keep your software stable, the better your architecture is. Huge number of tests in a codebase is typically a signal of shitty architecture that crumbles without those crutches.

Tests can carry tech debit, just like any code. They certainly are not identified by it.

Tests are one of the ways you have to ensure your code is correct. Consequently, they are business-oriented code that exist to support your program usage, and subject to its requirements. How much assurance you need is completely defined by those requirements. (But how you achieve that assurance isn't, and tests are only one of the possible tools for that.)

They are still debt since they don't directly contribute to product value: you can delete all your tests and your software will keep functioning.

It doesn't mean it's a debt worth taking, though IME most companies are either taking way too much or way to little. Not treating tests as debt typically leads to over-testing, and it is way worse than under-testing.

Also, what you're talking about (business-oriented requirements) is more akin to higher level tests (integration/e2e), not unit tests.

If you can simply delete it that’s not debt, it’s just cost. My bank doesn’t let me delete my loan obligation and still live in my house.

You can simply burn the bank, no?

I've never seen a team where bad tests were simply deleted. They were always "fixed" instead.

You can also delete the source code after compiling it and your software will keep functioning. Does it mean that code don't directly contribute to product value ?

I think that's projection.

Your tests being a nightmare doesn't imply my tests are a nightmare.

NOT having tests is dept. When you can’t fearlessly add features quickly because you introduce regressions in another end of the product that you didn’t think of, and later have to spend all your time on firefighting because it got deployed to prod.

If your mocks and tests are in the way when introducing features or refactoring, they are likely not on the right level. Too much unit testing of moving internals, rather than public apis, usually being one of the culprits.

* tests are a handy way to run one single function in isolation without having to spin up an instance and navigate the UI, too.

Reply on news.ycombinator.com