Do you consider test mocks to be a second implementation that warrants the interface?
Interfaces are one of the more powerful abstractions available and should be well thought out points of flexibility you specifically designed into your solution. If you use interfaces for other reasons your codebase is full of false flexibility and not-actually-abstracted abstractions which makes it difficult for other people to figure out what they can actually do with it.
Writing testable code is not "damage". An interface is a small concession in exchange for the well known benefits of automated testing.
I see where you're coming from and many programmers use too many interfaces as a reflex. But I absolutely think interfaces should be used to create seams for automated tests in front of components that do things like interact with databases, file systems, networks etc.
It's one of the reasons I tend to minimise my use of mocks as much as possible.
A "unit test purist" would likely shun this, but I take more of a realist approach; in many real-world cases, mock-heavy tests make for brittle tests that don't give confidence of correctness.
I tend to focus on integration tests, as I find these provide the most value while still executing very quickly (depending on the stack, obv).
That doesn't mean I don't use unit tests, or that I never use mocks - I just use mocks very sparingly.
Correct, but you can go a long way in writing testable code without requiring mock implementations. While I don’t deny the usefulness of mock testing, it comes at a steep cost (= vastly more complex code base), and I increasingly question whether that’s worth it. In practice I find it more useful to unit test what can be tested in isolation, and to perform integration testing for the test, and to limit mock testing (and thus unit tests which would require mocking) to a few well-defined interfaces. This cuts down drastically on boilerplate, with little (if any) loss of testability. And these few remaining mock object dependencies can be modelled via simple parameters and higher-order functions, further cutting down on extraneous interface classes.
For some languages (e.g. C++), if you are using classes, it can actually be fairly challenging to do unit tests without mocks. And almost every solution to this problem out there is to use interfaces/dependency injection/inversion.
Yes, but you can write integration tests instead. There’s no rule that says that unit testing must cover every aspect of the software. Unit test those units that can be used in isolation, integration test the rest. Case in point, our main software at work, written in C++, has close to 100% test coverage, and uses almost no mock tests. Instead, we have extensive integration tests where unit testing without mocking doesn’t work.
And, again: I’m not saying that there are no tradeoffs involved in that. But the benefit of having a drastically simpler code architecture outweigh the cost of having to move those tests into integration.
Incidentally this is vastly easier when the architecture cleanly separates business logic from general logic and moves as much of it as possible into side-effect free general-purpose functions.
In my last C++ job, perhaps less than 5% of our code was unit testable. We had very good test coverage using integrated tests, but those were very expensive tests. Running a full test suite took 10 hours. Our SW heavily relied on certain equipment. Fortunately, the vendor had provided a very good simulator for it - so almost all our integration tests needed that simulator. And the simulator was such that there was a fair amount of overhead in loading and setting it up and unloading it. We didn't do this for every test - that would have taken days/weeks to run - but for logically grouped tests. And unfortunately many of the integration tests we wrote were much more suited to unit tests, but because we didn't have interfaces, we were forced to test them via an integration test, which was very expensive.
Oh, and the simulator would not allow parallelism - you could have only one simulator running on a given machine at a time. So all our tests had to be run in sequence.
And of course, the simulator had bugs, so whenever we had a failure, we had to determine if the simulator was at fault. Especially for intermittent failures.
So yes, we had very thorough tests and were proud of it, but unit testing for perhaps 30% of our tests would have been better. But to unit test those we really needed interfaces. I know because I convinced management to give me some weeks to convert one of our simplest portions of code into something we could unit test. I thought it would be trivial, but I ended up needing interfaces.
There is a whole other approach which our sister team (very similar SW, but different HW) used. They did not have a reliable simulator. So they wrote a "fake" simulator. They simply reproduced the APIs and wrote code to return certain values under certain conditions - a poor man's mock. On the plus side their test suite ran very quickly. The down side was they needed to write many mocks for the same function (for when they needed to return different values). Writing tests was very expensive for them.
Why did they write fakes instead of using a mocking framework? Because every mocking framework they found in C++ required interfaces, and they didn't want to change their SW architecture.
In many cases integration tests are OK. But in our case, they were extremely expensive.
> Incidentally this is vastly easier when the architecture cleanly separates business logic from general logic and moves as much of it as possible into side-effect free general-purpose functions.
This is precisely where interfaces come from! It's not trivial to separate business logic from general logic without them. When I took the time to show how we could write unit tests for our code base, I ended up with interfaces. And no, I didn't read any books/sites telling me I should do it. I didn't even know they were called interfaces. I just took some of the simplest code in our codebase, and asked "How can I write unit tests for this?" and essentially reinvented interfaces. I separated the business logic from the logic requiring the simulator, and then wrote unit tests for the business logic, and had an interface class to interface with the simulator.
To give you an idea, the code I converted to unit tests was trivial: Given this input string with these flags set, return this converted string, etc. It should be one of the easiest things to unit test! However, the input string and flags were entities understood by the HW and so we had simulator dependencies. I wrote a class that dealt only with strings and booleans to implement the business logic, and then had the interface class translate back and forth between strings/bools to entities the simulator expected/understood.
I appreciate your comment, and trust me, I'm not trying to be difficult. But while I've heard many claims, every time I've gone into the details I end up with interface classes. I work at a large company and went around asking several teams in different C++ projects, and they all independently had come to the same conclusion: Either use interfaces or live without unit tests. I presented my work to convert portions of our code base to unit tests at internal SW conferences, and would say "This is a highly nontrivial and crappy way to do unit tests in C++. I'd really like to hear from people in the audience if they found a better way." And whoever approached me afterwards said they had gone down the same path as me and ended up with interfaces.
Testing in C++ sucks, to be frank.
As an aside, I'm not pro-mocks, BTW. I recently did a small personal Python project where I insisted doing TDD. I would not write code till I had a test. And this involved several mocks. And while all my tests involving mocks passed, in the majority of cases involving the mocked code, my program had bugs and would fail when I tested with real data.
(Maybe that's more a criticism of TDD than of mocks, though).
I just wanted to pick up one of your points:
> I've gone into the details I end up with interface classes
My reply to this is that functional programming languages tend to manage entirely without, just by doing “dependency injection” via higher-order functions (in fact, outside Java I’ve never [needed to] use a DI framework). This (often? always? I honestly don’t know!) makes mocking possible without impacting the overall architecture. And like you, I practice strict TDD for all my current own hobby projects, and I manage without interfaces. But I don’t want to make a claim of generality here: it’s entirely possible that this approach doesn’t work everywhere, or even in the majority of cases.
You should be writing pure functions which are easily testable, and using integration code to wrap the pure and testable parts.
Interfaces are typically a hack to get around the challenge of writing well designed, pure, and testable code.
This often ends up with interfaces...
> and using integration code to wrap the pure and testable parts.
Once you know there’s a bug, you will definitely want to write more tests, in order to find the cause. But always writing a lot of tests, which are only usable in case you find a bug, seems like wasted effort to me.
If you need to triangulate the origin, do that when it’s necessary (i.e. when your broad test fails) and not all the time.
There's also a tradeoff around testing error states, but that can be a wash. Some errors are easier to create with a mock, some are easier to create with the actual driver.
But yes - Java/python, etc. you shouldn't need to do this.
I’m generally hesitant to add test libraries if there’s an easy design choice alternative that keeps the code simpler.
edit: Not "Moquitto".
The fact that dependency is versioned separately from your code base is one source of complexity that it adds. The fact that it has many lines of code is another complexity.
It is however often easier, even although i'm advocating very small self-written test doubles, they are still less easy than using mockito.
You're possibly confusing easy with simple but adding a versioned library of some thousands of lines of code to a project is in no objective way simpler than adding a small test double which you wrote.
(What people don't care about is the size of libraries that are scoped for the test context only, and don't even impact artifact size at all.)
Also, a Mockito mock is simpler in usage than a hand-written and normally instantiated test double.
Simple - composed of a single element; not compound
Complex - consisting of many different and connected parts
Easy - achieved without great effort; presenting few difficulties.
public void allocationOnlyPossibleWhenOpen() {
LocalDateTime expected = LocalDateTime.now();
TimeAllocator sut = new TimeAllocator(new Clock() {
public LocalDateTime getTime() {
return expected;
}
});
sut.setOpen(false);
assertThat(sut.allocate()).isNull();
sut.setOpen(true);
assertThat(sut.allocate()).isEqualTo(expected);
}
public void allocationOnlyPossibleWhenOpen() {
LocalDateTime expected = LocalDateTime.now();
Clock mockClock = mock(Clock.class);
when(clock.getTime()).thenReturn(expected);
TimeAllocator sut = new TimeAllocator(mockClock);
sut.setOpen(false);
assertThat(sut.allocate()).isNull();
sut.setOpen(true);
assertThat(sut.allocate()).isEqualTo(expected);
}
A versioned external dependency, externally versioned == opportunity for future conflict. Remember the java logging library wars? Today it's trivial to encounter conflicting spring dependencies on your classpath
A complex (but convenient) reflection based implementation, the object has non-obvious behaviour with a clever implementation
An entire api surface distinct from the components of your software system
Tens of thousands of lines of opportunity for bugs, security risks or just plain old $$ cost to hide in
Once you're able to see the world in complexity, it opens up a whole new dimension of cost & quality for your code.
All this said, in practice complexity is just a consideration. Often the complexity is worth paying. I don't want to risk the impression that i'm saying never use mockito. If my argument is anything then it's consider the impact of complexity.
I've yet to find a library for C++ that generates mocks without requiring interfaces, but I'll admit I'm not an expert on it. Can you recommend any?
As an example, Google Mock/Test will require interfaces in a C++ code base.
Of course, we may be disagreeing on what is meant by the term "interface".
This avoids having to introduce the extra interface, having to code the mock, sometimes having to change the mock or tests when changing the code and also lets you catch bugs in the dependencies that the tests on the dependency itself missed and that would otherwise show up in production.
One might have to do some work to partially replicate the production setup in testing and making it fast to start and run, but that's usually less work and gives a much better result.