I'm done listening to smart people proposing fixes for my dumb code.
Either I'm not smart or disciplined enough to make it work, or my colleagues are not. Mostly both.
Either I'm not smart or disciplined enough to make it work, or my colleagues are not. Mostly both.
I can very much recommend his latest book ”Tidy first?”. It’s extremely short and concise, and is perfect for a very light book club within any tech team.
One term I can use is "defensive programming" (not sure if the term has been used before). People are going to do awful stuff with their code, it's up to you to draw boundary of correctness to stop it from spilling on the part that you're responsible for. It should be automated (with tests and static analysis) as well as documented. I think of those boundary as hazmat suits and I take extra care of maintaining their integrity.
It's not as much composition as compounding, and can work quite well.
Let's say something takes 20 steps, and you want to test all 20.
Instead of this:
test 1:
do step 1, assert step 1
test 2:
do step 1, assert step 1
do step 2, assert step 2
test 3:
do step 1, assert step 1
do step 2, assert step 2
do step 3, assert step 3
...
test 1:
do step 1, assert step 1
test 2:
do step 1
do step 2, assert step 2
test 3:
do step 1
do step 2
do step 3, assert step 3
...
As most things, it depends on what those steps are. Perhaps you only need that one final (integration) test instead of 20 intermediate unit ones.
test 1:
do step 1, assert step 1
test 2:
requires: test 1
do step 2, assert step 2
test 3:
requires: test 2
do step 3, assert step 3
test 4:
requires: test 2
do step 3, assert step 3
But it could also do things like skip test 3 if tests 2 or 1 failed - because it knows about the relationship.
- How to reconcile this with tests that execute many times with varying input data. You’d need some way to express requirements with specific inputs or shared inputs.
- Passing state between test dependencies.
- When, if ever, it’s fine to share step results between tests. If tests B and C require A, can you run A just once? Not always, but you should be able to when it’s safe.
I don’t think I’ve ever used a test framework that gets these things right.
Actually, I wasn't even thinking about passing state. I was thinking about shared setup steps. I'm perfectly happy for the same steps to run for each test - as long as each test doesn't need to list the steps (when they're setup steps not directly related to the thing being tested)
e.g. If I am running a long-running UI-test scenario, I absolutely don't want test-5 to walk through 80% of the UI that was already exercised in tests 1-4. I am creating test coupling, but I'm saving cost/time by doing so.
But, you'll also hear why not to do this, because it creates test coupling / breaks atomic tests, which is generally seen as bad.
If that is a local integration test and those early steps run is millis? Then maybe we keep things uncoupled to allow the system to exercise the pathways without explicit expectations.
I wouldn't have a problem with something like
test-1:
setup:
do-the-thing
verification
assert-the-thing-happened
test-2
setup:
depends-on: test-1 // tells it to run test-1's setup
do-the-next-thing
verification
assert-the-next-thing-happened
GIVEN
Some initial setup
WHEN
I run command
THEN
The result of that command is what is expected
I actually use the GIVEN/WHEN/THEN keywords in my tests, to make them easier to read
In special when testing against a DB.
Forgive an old man some ruby:
it 'relies on mobile setup defined elsewhere', :mobile => true do
# test that relies on mobile setup here
endThose are two separate ideas:
1. Execution dependency: if a cheap contract test fails, skip expensive downstream tests because their results won’t be informative. 2. State dependency: test 2 consumes state left behind by test 1.
The first can be useful. The second tends to create order dependence, awkward retries, and failures that are hard to reproduce when a CI runner shards or parallelizes the suite.
A safer model is a DAG of independently reproducible tests: each node declares prerequisites for scheduling/reporting, but creates or restores its own input state. Then a downstream test can be marked “blocked by X” rather than failed, while still being runnable by itself when debugging.
That distinction also keeps the dependency metadata from becoming a hidden setup mechanism.
You're not going to remember what tests "test 3" relies on. They aren't actually linear progressions 123. They will be "test this", "test that". If 3 fails you're going to want to immediately go and add all those asserts back to help you debug your assumptions.
The tests _will_ drift and that should be fine. Implicitly depending on other tests doesn't really get me anything.
The article mentions not to do this because "Deleting test1 loses us another property from the Test Desiderata—tests should be specific. That’s the property of tests where, when one fails, you know exactly where the problem is." but you'll know what line it failed on. And some test runners let you break a test into steps, where groups of lines are given a description.
Or put each step + assert in a helper function (e.g. `doStep1AndAssert()`), and each test only calls these helper functions?
Nothing is perfect, but copy/pasting chunks between tests like this isn't great when you want to refactor and it's repetitive to read.
That depends.
Sometimes test 1 tests a combination of ways (e.g., property testing, or just going through a bunch of various inputs), and only a few of those are needed for test 2.
Sometimes you don't want your test 2 to be more complicated than it already is. Or the same things are needed checked in other tests. So you extract them into test 1.
And sometimes (and in some of code bases most of the time) test 1 is redundant and unnecessary. That's why I always advocate investing in integration tests (test 20) and skip all the intermediate tests.
My approach is to have acceptance tests documented for any feature. Like how it would be from the user point of view to actually use the software. Then do Integration tests for each part of that workflows. That's usually the most ROI you will get for testing. Then I invest into unit tests for particular elements that are very important. I start from the middle of the pyramid because an actual e2e is expensive to setup (easy to maintain afterwards) and having lots of unit tests (easy to setup) is expensive to maintain.
There's nothing wrong with hitting the same assertion multiple times, even if it doesn't sit nicely in your gut.
From a purely philosophical point of view: If I have testFoo(), testBar(), and testFooAndBar(), and my Foo is plain wrong, then both testFoo() and testFooAndBar() must fail. Anything less is misleading/dishonest.
From a practical side: Changes happen. Someone will remove testBar(), and then you're down to 0 assertions on Bar, even though you have a test claiming to testFooAndBar(). It's not even a crazy hypothetical. Someone with a different test philosophy will think (to quote TFA) "They are redundant! Something must be wrong." and delete testBar() because obviously testFooAndBar() already covers it.
Anyway, we all know how to deal with repetition. That's what programming is!
testFoo()
_ = validateFoo(foo())
testBar()
_ = validateBar(bar())
testFooAndBar()
foo = validateFoo(foo())
bar = validateBar(bar())
_ = validateFooAndBar(fooAndBar(foo, bar))I say leave both tests as is.
Kent Beck says:
From a purely aesthetic standpoint (& don’t discount aesthetics), leaving both tests as is offends my sensibilities. They are redundant! Something must be wrong.