Hacker News Reader

Comment by TrianguloY | Hacker News Reader

The one assertion per test doesn't mean you need to use only one assertion call but rather that you only need to do one assertion block. Checking everything after a response is considered 1 assertion, no matter how many assert calls you need.

The issue is when you use multiple assertions for multiple logic statements: do > assert > do > assert... In that example imagine that you were also checking that the reservation was successful. That would be considered bad, you should create a different test that checks for that (testCreate + testDelete) and just have the precondition that the delete test has a valid thing to delete (usually added to the database on the setup).

View on HN

Sometimes, it takes a lot less code to test a particular code by doing it in a long test with multiple assertions.

Migrate up -> assert ok -> rollback 1 -> assert ok -> rollback 2 -> assert ok

I don’t see much benefit to breaking it up, and you’re testing state changes between each transition, so the entire test is useful and simpler, shorter, and clearer than the alternative.

Sometimes it's also more meaningful to the reader this way.

Imagine an object which sees numbers, tries to pair up identical numbers, and reports the set of unpaired numbers. A good test would be:

Create object

Assert it has no unpaired numbers

Show it 1, 2, and 3

Assert it has 1, 2, and 3 as the unpaired numbers

Show it 1 and 3

Assert it has 2 as the unpaired number

This test directly illustrates how the state changes over time. You could split it into three tests, but then someone reading the tests would have to read all three and infer what is going on. I consider that strictly worse.

That test is asserting a flow, not features. How can you be sure that the second rollback fails because that rollback code is wrong and not because the previous two functions made some unexpected changes? Or rather, how can you be sure that the second rollback is ok if you dont know the state it was run from? Maybe the migration set an unexpected flag that made the rollbacks pass that test without working properly.

This is also the reason why tests should be run in arbitrary order, to avoid unexpected interactions due to order.

Flow tests can be useful in some situations, but they should never replace individual feature tests.

This can be a path where things do go bad. Let's say thus test pattern is a success and then is replicated for many tests. Now, the schema or migration changes. A small change there now breaks the entire test suite. At this point the number of failing tests only indicates how many hours you will be fixing assertions.

Another failure mode is when test scaffolding builds up. Imagine that migrate up part becoming multiple schemas, or services. It then fails, now finding exactly where to fix the test scaffolding becomes a multi-hit exercise.

I'm not saying the example is bad, but it can put you on a path where if you constantly build on top of it, it can bad (eg, developers that don't care for tests nor test code quality, or just want to go home, and they just add a few assertions, add some scaffolding, copy-paste it all and mutate some assertions for a different table & rinse-wash-repeat across 4 people, 40 hours a week for 3 years...)

Sorry the example was vague. It was a test for the migration library itself— something I wrote recently when playing around with building a 0-dependency web framework for Bun.

I wouldn’t actually write tests for migrations themselves.

I think the issue is that you’ll always have one of those teammates who see this as an excuse to test the entire happy flow and all its effects in a single test case. I think what you want is reasonable, but how do you agree when it is no longer reasonable?

If you logic depends on that happy path, make a test for it. But as I explained in another comment that test should not justify the lack of individual feature tests, which should not only test the happy path but other corner cases too.

On my company we developers usually create white-box unitary/feature tests (we know how it was implemented, so we check components knowing that). But then we have an independent QA team that creates and run black-box flow tests (they don't know how it was implemented, only what it should do and interact)

Sounds like a fine approach, and I wasn’t criticizing. Mostly I was pondering out loud why people come up with blanket statements what good tests should look like.

Reply on news.ycombinator.com