I really like the way they keep repeating this as the answer to so many questions. To me it reads as a softly softly approach to weaning people off what appears to be a mania for micro level unit testing, driven by people like Uncle Bob.
"The structure of your tests should not be a mirror of the structure of your code. The fact that you have a class named X should not automatically imply that you have a test class named XTest."
"The structure of the tests must not reflect the structure of the production code, because that much coupling makes the system fragile and obstructs refactoring. Rather, the structure of the tests must be independently designed so as to minimize the coupling to the production code."
http://blog.cleancoder.com/uncle-bob/2017/10/03/TestContrava...
Does anyone know of explations of this with a more hands-on approach, or is this simply a collection of ideas that can’t really be shown?
So far, I've not been able to find any examples of test architecture that follows those principles online or in my day job.
I've also been too lazy to do a side project and explore a more decoupled and scalable test suite. Maybe I should get off my arse and finally do it.
Now you are testing the behaviour of your application through its public surface. This reduces the brittleness of your tests because you can change the internal implementation without rewriting your tests. You have higher assurance. It'll also force your hand to enforce invariants and place guard clauses in the right places rather than "everywhere".
If you follow Cockburn's Ports and Adapters approach too, then you can substitute adapters like persistent state, buses, HTTP clients for appropriate in-memory equivalents.
The jump indicated by the "therefore" confuses me. I essentially couple my test to production behavior, insofar as it is possible. I do this to test program behavior realistically and because I believe tests should, in some sense, form a specification for the program they test. I don't really care about "testing code"; I care about testing behavior.
I don't have the problem of small changes to code necessitating large changes to tests. It makes me wonder what he does that is causing that.
Don't they use the wrong term for them?
I believe these tests are more commonly called "integration tests", not "unit tests".
Or, did I misunderstand something?
The more likely it is that a piece of code will break, and the more business damage it will do if it does break, the more tests I wrap around it.
For self-contained algorithms that have a lot of branches or complex cases, I use more unit tests. When the complexity is in the interaction with other code, I write more high-level tests. When the system is simple but critical, I write more smoke tests.
If I’ve got simple code that’s unlikely to break and it doesn’t matter if it does break, I might have no tests at all.
https://fsharpforfunandprofit.com/posts/property-based-testi...
This is a valid observation but the problem clears up when you start integrating better debugging tools into your testing infrastructure: having the ability to intercept and inspect API calls made by your app, launch a debugger in in the middle of the test, etc.
It is also minimized by writing the test at the same time (or just before) the code that makes it pass.
In this same vein, I propose another question:
How similar does the test look to the production code?
I think this comes from "unit tests" where they have assertions such as
assertThisOtherSpecificMethodNameWasCalledWithTheseSpecificallyNamedParameters
For testing dependencies, an environment where a test instance of the database, API, or other external service is needed. Then the code could be tested against an actual implementation rather than a mock of it.
I am not really following. You can write a seperate function that handles database query, then call this function in your code under test, or one of your argument in the code under test is the databse object handler (this allows you to abstract from a specific db handler). But if you don’t mock out the handler with a “dummy handler”, you will be making real databse call. That is not UT.
You will be doing functional testing.
So you can only abstract so much to get your code testable.
If you think this sounds like mocking, you’re right (it is like mocking, it isn’t mocking). It has the bonus of making it easier to inspect the logic & expose it to the user ("hey I’m about to do these things, does that sound ok to you?").
Well, not really. Because now I have the DB stuff all contained in a separate function (or handler which can be proxy to the correct DB engine/type of DB), so if with careful design I should be okay.
def create_account(user_manager, user_details):
user_manager.create_account(user_details)
Of course this is ideal, but that was the motive. I do agree on mocking == testing implementation in general, and functional testing is almost always the way to go... since mocking returns dummy data / fixture, might as well return from a real database. The downside is speed :/ (there are various of tricks like setUpClass, run every test or group of tests in docker containers in parallel, but takes much longer than UTs). Ugh trade-offs.
You can, but that still requires that you mock that call in order to test the method that calls your database query method. But, if you make it such that your method takes the result of the database query as a parameter, then you can test your method without having to use a mock. In other words, you can change this (which requires mocking get_db_result to test):
def a_method_to_test(param1, param2):
db_result = get_db_result(param1)
# do something with db_result
def a_method_to_test(param1, param2, db_result)
# do something with db_result
Basically, doing this will separate code that manipulates data from code that either writes or reads data. You can unit test methods that manipulate data, but you will need to do functional/integration testing of code that writes or reads data from other sources.
Secondly it decreases your confidence in the refactor. In a classic refactor, nothing should break. This is because a good set of tests are a way of documenting and verifying the design/behaviour of the system.
Finally, often in a codebase with micro tests, there will be an expectation that you replace one set of micro tests with another.
I'm still not sure how to get the good outcome every time. Maybe it's just well designed interfaces ️
That "feedback" is the pain you feel when you have to maintain one form of tight coupling (ordinary bad code) combined with another (micro-level unit tests).
Higher level tests won't cause you that 'guiding pain' because they aren't as tightly coupled to your implementation.
IMO you don't need to write tens of thousands of lines of unit test code to spot that you're building tight coupling in. It's something you can spot simply by knowing what to look for.
Moreover, if you make a design mistake or introduce tech debt lower down the stack, having a panoply of low level unit tests means that refactoring will break those tests even if you haven't introduced any bugs. IME that causes a dangerous cementing effect on bad code.
I think if you've got a module which is self contained, reusable and tightly scoped, it's worth surrounding with a lower level test. But, if you don't, you're doing more harm than good by surrounding a module with tests.
I don't think you do. I think a testing pyramid is a sign of an unhealthy approach and an unhealthy codebase.
I think what you need is a testhenge. For those who aren't familiar with henges, here's a typical (although poorly maintained) one:
http://www.english-heritage.org.uk/remote/www.english-herita...
Rather than being massive at the bottom and then getting thinner as it goes up, there is a continuous band around the top, covering everything, supported by uprights that dive deeper down, just where needed.
That is, you need a layer of high-level tests for everything (for a web application, some mix of browser tests or controller-level integration tests), along with component-level integration tests and isolated unit tests for the bits of the system that are particularly risky (complex, error-prone, critical, new, old, etc).
Because most of the time, it reduces "test precision". Which means when such a test detects an issue, there's now a bigger area of the code where the issue could be located.
> Doesn't efficiency outweigh respecting feature boundaries in tests?
In the end, what we're trying to minimize here is feature development time.
This of course depends on the feedback loop duration (build + run tests), which is why test efficiency is important ; but there are other factors.
Feature development depends on the time it takes to locate the error when a test fails ; a test signalling that "something is wrong somewhere" is not very usefull (in some cases, it can be, if it runs extremly fast - because you can generally see the error with "git diff").
However, infinitely precise tests are often undesirable, because they tend to have extremely rigid expectations about the behavior/API of the code under test, discouraging refactoring, and thus slowing down the development process.
Let's keep in mind that "testing = freezing". More precisely, you're freezing what your tests depend upon (by adding friction to future modifications). So be careful what you freeze: the initial intent of testing is to increase code flexibility. If you can't modify anything without breaking a test, you're probably missing their benefit.
Tests are not a debug tool. Tests are here to tell you when you broke something.
When this happen you can get your debugging toolbox out: stack traces, profilers and things like GDB. And then follow the steps your test script did.
But "when you click on this after that this happen while it shouldn't" is usually enough info to start debugging: you have reproducible steps. Which you can do with a debugger running so you see exactly were and how things break.
And it is a lot less brittle than "well we tried to refactor some minor thing and now everything is red; but we don't know if the software behavior changed or just because our tests were just checking the implementation".