90% of the time, needing to use a mock is one of the clearest code warning smells you have of there being an issue in the design your code.
It took a while, but the industry seems to be finally (although slowly) coming to this realization. And hopefully with it almost all of this can go away.
Even funnier, this was all hypothetical and yet taken as gospel. We hadn't even written the tests yet, so it was impossible to say whether they were slow or not. Nothing had been measured, no performance budget had been defined, no prototype of the supposedly slow tests had been written to demonstrate the point.
We ended up writing - no joke - less than 100 tests total, almost all of which hit the database, including some full integration tests, and the entire test suite finished in a few seconds.
I'm all for building in a way that respects performance as an engineering value, but we got lost somewhere along the way.
Hopefully one day you'll back at that, and realise what an immature attitude that was.
Unless that whole paragraph was a figure of speech, in which case what I said doesn't apply, and we can both go about our days.
Why fake it when an integration test tests the real thing.
I’ve seen what you clearly have. Mocked ResultSets, mocked JDBC templates. “When you get SQL, it should be this string. These parameters should be set. Blah blah.”
It’s so much work. And it’s useless. Where does that SQL to check come from? Copy and paste, so it won’t catch a syntax error.
Test data is faked in each result. You can’t test foreign keys.
Just a bad idea. You’re so right. I find it odd some people are so anti-mock. Yeah it gets abused but that’s not the tool’s fault.
But DB calls are not a good spot to mock out.
What you're describing is a very limited subset of testing, which presumably is fine for the projects you work on, but that experience does not generalise well.
Integration testing is of course useful, but generally one would want to create unit tests for every part of the code, and by definition it's not a unit test if hits multiple parts of the code simultaneously.
Apart from that, databases and file access may be fast but they still take resources and time to spin up; beyond a certain project and team size, it's far cheaper to mock those things. With a mock you can also easily simulate failure cases, bad data, etc. - how do you test for file access issues, or the database server being offline?
Using mocks properly is a sign of a well-factored codebase.
You seem to say it's not worth writing a lot of tests, but then you talk about tests breaking due to bad changes - if you don't write those tests in the first place, then how do you get into that situation?
I didn't word my earlier comment very well - I don't mean to advocate for 100% coverage, which I personally think is a waste of time at best, and a false comfort at worst. Is this what you're talking about? What I wanted to say is that unit tests should be written for every part of the code that you're testing, i.e. break it into bits rather than test the whole thing in one lump, or better, do both - unit tests and integration tests.
This is what I mean about a well-factored code base; separate things are separate, and can be tested independently.
> Unit tests have proven more harmful than helpful
Why? I find them very useful. If I change a method and inadvertently break it somehow, I have a battery of tests testing that method, and some of them fail. Nothing else fails because I didn't change anything else, so I don't need to dive into working out why those other tests are failing. It's separation of concerns.
i have concluded a unit needs to be large. Not a single class or function but a large collection of them. When 'archtecture astronaughts' draw their boxes they ar drawing units. Often thousands of functions belong to a unit. even then though often it is easier to use the real other unit than a test double.
If your unit is thousands of functions wide then you have a monolith, and there are widely discussed reasons why we try to avoid those.
The common pitfall with this style of testing is that you end up testing implementation details and couple your tests to your code and not the interfaces at the boundaries of your code.
I prefer the boundary between unit and integration tests to be the process itself. Meaning, if I have a dependency outside the main process (eg database, HTTP API etc) then it warrants an integration test where i mock this dependency somehow. Otherwise, unit tests test the interfaces with as much coverage of actual code execution as possible. In unit tests, out of process dependencies are swapped with a fake implementation like an in-memory store instead of a full of fledged one that covers only part of interface that i use. This results in much more robust tests that I can rely on during refactoring as opposed to “every method or function is a unit, so unit tests should test these individual methods”.
Well-factored codebase doesn’t need mocks.
now I do take care to avoid writing tests that depend on other code's results that would be likely to change. This hasn't proven to be the problem I've so often been warned about though.
...or, you tested each thing individually, so only related tests break and you can quickly zero in on the problem. Isn't that easier?
you could extend this to say 85% of the tome just write the code directly to prod and dont have any tests. if you broke something, an alarm will go off
More broadly, suppose foo() has an implementation that depends on Bar, but Bar is complicated to instantiate because it needs to know about 5 external services. Fortunately foo() only depends on a narrow sliver of Bar's functionality. Why not wrap Bar in a narrow interface—only the bits foo() depends on—and fake it?
I'm not a maximalist about test doubles. I prefer to factor out my I/O until it's high-level enough that it doesn't need unit tests. But that's not always an option, and I'd rather be flexible and use a test double than burden all my unit tests with the full weight of their production dependencies.
To easily simulate failure cases, a range of possible inputs, bad data etc.
To make the testing process faster when you have hundreds or thousands of tests, running on multiple builds simultaneously across an organisation.
Off the top of my head :-)
You can do all that without mocks as well.
Making the tests run faster at the expense of better tests seems counterproductive.
Now you should think of reasons why you should not isolate.
OK; it's your choice to do what you think is right.
> and comparing it to scientific experiments doesn’t really apply.
Why not? I think it's a fairly apt comparison; you have a theory ("this piece of code does the following things"), and write tests to prove it.
> You can do all that without mocks as well.
OK, but mocks make it easier and cleaner - so why wouldn't I do that?
> Making the tests run faster at the expense of better tests seems counterproductive.
Smaller, more focused, cleaner tests are better in my opinion; speed is a beneficial side effect.
> Now you should think of reasons why you should not isolate.
Why? That's your argument - it's not on me to prove it for you. If you can give me some good reason why mocking out the interfaces you are not testing is a bad idea, and some better alternative, then we can have a discussion about it.
Your position is reasonable and I do think isolation can be beneficial, but I still wouldn’t use mocking to do it.
>Smaller, more focused, cleaner tests are better in my opinion.
Cleaner is subjective. I can write “small” and “focused” functional tests that are also quick to run.
I am of the opinion that functional tests provide more value. They are testing more of the actual code than an approximation, which in turn gives a better indicator that it works. Functional tests are less likely to change unless the input/output changes.
Now let’s say you mock something in your function. Let’s say you make a change to that but the input and output are the exact same. Now you have to update your test.
Not to labour the point here, but no, the primary reason you isolate variables in a scientific experiment is that you want to ensure you're only testing the thing you intend to test. A medical study is a good example - you want to be sure that the effect you observed was due to the drug you're testing, and not some unrelated lifestyle factor.
Thanks for sharing your views on the rest; there was just one thing I wanted to expand on:
> Now let’s say you mock something in your function. Let’s say you make a change to that but the input and output are the exact same. Now you have to update your test.
I think the scenario you're describing here is: a function's dependencies have changed, but the inputs and outputs of that function have not; therefore even though the behaviour is the same, the tests still need to be updated. Is that right? In which case I would say: of course you need to update the tests - the dependencies have changed and therefore the behaviour of the function depends on different things and you need to model the behaviour of those new things in order to properly test the original function. To me this objection only holds if you are mainly focussed on code coverage; however, to me, good testing exercises the same code paths in multiple different ways to stress the code and ensure that the results are correct given all possible inputs. The dependencies of a function are also inputs of a kind.
>Is that right? In which case I would say: of course you need to update the tests.
That is right. I think it is bad for you to need to update a test where the input and output are the same. Your mock is there for you to essentially ignore, but now you need to update the test. You now do not know if you introduced a bug.
You are losing out on encapsulation, the test should not know about the internals, generally speaking.
>The dependencies of a function are also inputs of a kind.
Typically that should not be a concern to the caller of the function.