> The worst sort of tests are integration tests which secretly depend on another integration test having run first
That's an example of bad integration tests. Well engineered integrations tests don't do that.
That's an example of bad integration tests. Well engineered integrations tests don't do that.
If the system is crappy/stateful/implicit, and you somehow manage to write nice/clean/stateless integration tests against it, I'd argue that the tests won't be close enough to the expected running of the system to tell you anything useful about it.
If I set up an integration test that sets up a database from scratch and tears it down and tests only the behavior of the app in that rigidly defined context, then yes, it will be useful. It will tell you how the code behaves in the scenario you've created.
Bad integration tests will share state with each other - e.g. by using the staging DB.
> Bad integration tests will share state with each other
The real-world system shares state.
> If I set up an integration test that sets up a database from scratch and tears it down and tests only the behavior of the app in that rigidly defined context
... constructing a particular set of circumstances which will never occur in the real-world system.
What do you find unlikely about a scenario where a test uses an app in a realistic way (e.g. with a browser) set up in a realistic context (e.g. with some fixed sample data) to reproduce a realistic scenario (e.g. a bug that already happened)?
I wouldn't say that isolation and realism are completely orthogonal but I find that well engineered integration tests are usually able to reproduce 90% of bugs sourced from production while unit tests can often manage only 10 or 15%. Bug in the SQL? Browser is involved? No can do.
Neither would I. I'm arguing that when you write a test method, you deliberately make the choice to include some kind of 'before-all' method, or not.
The reasons you would choose to include a 'before-all' method will vary from case to case. Let's say you're testing an addUser method. If you choose to isolate its state to avoid 'test flakiness', it is you making the call that addUser is flaky when run against shared state.
What is it about your application code that would make you think that addUser is flaky enough to need a clean slate to run against? Why not change the application code instead?
Not really. I would always purge anything that would cause tests to share state. I wouldn't do it on a case by case basis.
>What is it about your application code that would make you think that addUser is flaky enough to need a clean slate to run against?
The user already existing in the database? The behavior of the app would change in that case. Something has to wipe the db clean to test that scenario.
Thats why tests shouldn't share databases.