Comment by lucasyvas | Hacker News Reader

Comment by lucasyvas | Hacker News Reader

As a predominantly fullstack web developer, I will only ever voluntarily write unit tests and end to end tests, but avoid in-code integration tests in most circumstances.

This is frankly the easiest and best bang for buck.

Unit tests force you to make your code more flexible, they're fast to write, fast to run. Maintenance depends on how overboard you go with your verifications - try to verify the most important parts.

End to end tests require no elaborate scaffolding of internals and allow you to test it as a real user interacts with it. Generally fast to make, slow to run, and maintenance depends on how disciplined you are with stable identifiers/interaction points.

As for in-code integration tests - I love the idea of them, but they're absolutely miserable 9 times out of 10 due to extremely convoluted processes to "bring up" parts of your application. If you use DI it shouldn't be as bad, but almost nobody does and it becomes a total clusterfuck not worth the maintenance burden.

As an idealist, I want all 3. But most codebases frankly cannot support all 3, so the most value will come from whatever is easiest. I've found that's most often unit and e2e.

The article actually argues the opposite. Developers should move their focus to integration / "real-world" tests. The major summary bullet point being:

"Aim for the highest level of integration while maintaining reasonable speed and cost"

My experience mirrors the author's. In any "real" business application, the unit tests end up mocking so many dependencies that changes become a chore, in many cases causing colleagues to skip certain obvious refactors because the thought of updating 300 unit tests is out of the question. I've found much better success testing at the integration level. And to be clear this means writing a tests inside the same project that run against a database. They should run as part of your build, both locally and in CI. The holy grail is probably writing all your business logic inside pure functions, and then unit testing those, while integration testing the outer layers for happy and error paths. But good luck trying to get your coworkers to think in pure functions.

> in many cases causing colleagues to skip certain obvious refactors because the thought of updating 300 unit tests is out of the question.

Good! They shouldn't do the refactor.

Because "obvious" refactors often introduce bugs (e.g. copy/paste errors), and if developers can't be bothered to write tests to catch them, they're going to screw over the other team members and users who will be forced to deal with their bugs in production.

> The holy grail is probably writing all your business logic inside pure functions, and then unit testing those, while integration testing the outer layers for happy and error paths.

So settle for half a loaf.

Write all the easy unit tests first. The coverage will be very incomplete, but something is better than nothing.

Write all the easy integration tests next.

Never write the hard tests if you can help it.

> Good! They shouldn't do the refactor.

> Because "obvious" refactors often introduce bugs (e.g. copy/paste errors), and if developers can't be bothered to write tests to catch them, they're going to screw over the other team members and users who will be forced to deal with their bugs in production.

In my opinion, useful tests should be able to survive a refactor. That is the only sane way I've ever done refactoring.

If I'm doing a large refactor on a project and there are no tests, or if the tests will not pass after the refactor, the first thing I do is write tests at a level that will pass both before and after refactoring.

Rewriting tests during refactoring doesn't protect from regression on my experience.

Yeah a refactor that changes how you expect the interfaces to work sounds like much more than a refactor.

Unit tests which can survive a refactor are a nice-to-have.

I would not rule out a refactor merely because I'd have to refactor some unit tests too. That's just part of the cost benefit analysis.

> Rewriting tests during refactoring doesn't protect from regression on my experience.

Your experience is completely at odds with mine. Every time I change code, there is the possibility for simple errors such as copy/paste mistakes. Trivial, cheap-to-write unit tests have saved me time and again from having to debug something down the line.

Overconfident devs who act as though they're above making such simple mistakes make for bad team members.

I couldn't disagree more.

Tests that will survive a refactor are the most important tests to have.

The other tests are, at best, a false sense of security and often an active detriment that slows down future development. They might sometimes catch actual mistakes, but just as often they fail when nothing is broken, leading to the tests not being trusted and broken tests being updated even when something was actually broken.

> I couldn't disagree more.

*sigh*

I know where you're coming from. This is the classic argument for limiting unit tests to black-box testing of public APIs exclusively, avoiding clear-box testing altogether.

I agree that it's possible to write absolutely wretched fragile clear-box tests. And I agree that if you have a black-box test and a clear-box test which provide equal validation of functionality, the black-box test is superior because it will survive a refactor.

I generally dislike absolutist rules of any kind when it comes to unit testing and prefer to think of things in terms of ROI. Sometimes you can add a lot of value with a clear-box test because the functionality is impossible to write a black-box test for without a ton of extra work and time.

But sometimes you may be in an environment where absolutist rules are the only way to go.

>Good! They shouldn't do the refactor. Because "obvious" refactors often introduce bugs.

& if your tests arent catching those bugs and require extra maintenance to go green again you are doing them wrong.

I've had the same experience. Suboptimal code isn't refactored because of the test code overhead, or, much worse, the tests on that same subpar code somehow morph into a perceived "gold standard" for how that code should work.

I avoid tests (aside from hands-on end user testing) as much as possible, actually, since they rarely seem to tell you anything you'd didn't already know.

I understand what the article is arguing. I agree with it, but think it's idealistic. If swaths of your code are a mess, integration testing is super painful. You can't easily add it until you clean up the mess, so the other forms of testing are more practical more often in my experience. If you get to a point where your code isn't a mess, I'd agree that you should start introducing meaningful integration tests.

I think this is just one of those cases where there is a context-sensitive strategy to testing. It depends completely on the cleanliness of your code and experience working with it.

> The holy grail is probably writing all your business logic inside pure functions, and then unit testing those, while integration testing the outer layers for happy and error paths. But good luck trying to get your coworkers to think in pure functions.

I've come to a similar conclusion. Functions don't necessarily have to be pure in the academical sense, though - but I feel like the more the business logic is decoupled from dependency injection and the less it is relying on some framework, the better.

It makes testing a lot easier, but also code reuse. I've just been writing a one-off migration script where I could simply plug in parts of the core business logic. It would have been very annoying if that was relying on Angular, NestJS or whatever.

Trying to write your code such that it can work with arbitrary data and arbitrary amounts of it is a step towards the holy grail I think.

Then you get to use fuzzer & Arbitrary for basically what is a coverage guided property based test.

But it's hard to maintain that idea at all times when you are writing.

> As for in-code integration tests - I love the idea of them, but they're absolutely miserable 9 times out of 10 due to extremely convoluted processes to "bring up" parts of your application.

I've definitely worked on projects like this (actually, I'm working on one now at $dayjob), but I found that with a little bit of effort you can get this right, and usually isn't that much effort.

What I often see people do is "oh I need a thingy for this test, and a thaty for this other test" and create it ad-hoc when needed, instead of creating a good convenient API to do all of this, which is then also used in the application itself.

The big advantage of these tests is that they tend to be a lot faster than E2E tests, at least in my experience, and often also easier to reason about once you get it right.

"Overrated" is the right word here. It's not like unit tests are useless, instead, the problem is that everybody uses them much more than it makes sense. And yes, I'm a critic of them since the fashion started at the turn of the century and I also use them more than they make sense.

Unit tests are very cheap, and people love cheap things. The problem is that they also provide almost no value, so their cost and benefit are on similar levels and they can easily get into negative net value.

I disagree. I also feel UTs are a waste of time. I think in some cases UTs can work such as helper classes, but for business logic they are pretty useless and a waste of time. I guess it gets at that verification level DEPENDS ON what is being verified.

Reply on news.ycombinator.com