The Difference Between Unit Testing and Integration Testing | Hacker News Reader

The Difference Between Unit Testing and Integration Testing

blog.typemock.com

32 pointstelma123414 comments

The language about software testing always seems to centre on the taxonomy of the tests rather than the software that is being tested. The concepts of controllability and observability (taken from hardware testing and control engineering) would seem better placed to further the discussion on testing and give developers a better vocabulary for describing what the tests do, at what level of system abstraction and how effective those tests are.

When developing web apps the best testing that works for me is a combination of unit and integration testing. Leaning mostly on the integration side.

Thing is, in most of what I do every day, what I care about is the end result. It doesn't matter so much to me that calling a specific function with a specific set of parameters produces a specific result, what I'm more interested in is sending a signal into the application and getting an expected output somewhere down the line.

Of course when I'm working on something more algorithmical, unit tests are preferred and tend to produce much better results. But this kind of work is unfortunately quite rare.

Yup, I'd agree with that. These days I only use Unit Tests in a few very specific scenarios:

1) I like to write a 'nominal case' unit test because it helps me think about the sort of API that I want, and it let's me see the fun stuff up on the screen very quickly.

2) I've found a bug, so I write a unit test because apparently this bug was in code that I wasn't able to reason about correctly when I first wrote it, which means it is probably tricky - the unit test will stop regression.

3) When investigating a bug, and I want to assure myself that a subsystem is reacting the way that I expect it to, I sometimes write unit tests as a way of proving to myself that I have understood what is going on.

I've seen other people write loads of unit tests for every single little thing, but it actually makes their code more rigid, as if you want to change an interface, you not only have to change the code in the product, but you have to change all of the unit tests too. The result is that people don't like changing what is written, and prefer to overlay a level of abstraction to avoid the rewrite, leading to less clear architecture overall.

Unit tests are not that useful IMO if you are alone on a project, but it is quite useful when multiple people have to work on the same codebase. They are a coding tool firemost: they help (but cannot substitute to !) documenting intent, they help isolating regression, etc...

As you mention, unit tests work well for algorithmic stuff.

There's a school of thought which says that unit tests should never touch a database. This has always struck me as a strange way to write tests: Mocking out a database is quite painful, and it means that you're testing an artificial system very different from what you run in practice. It's much easier just to unit test your model classes against SQLite, which is blazingly fast.

Similarly, some Rails developers try to completely isolate their models, controllers and views from each other when unit testing. A controller test, for example, won't render any views, and it will mock out the model layer. Similarly, a view test will ignore the controller, render the view manually, and mock out the model layer.

In my experience, these kinds of fully-isolated unit tests take a fair bit of work to set up, and they result in overspecified module boundaries. Plus, there's always the risk that your view objects will work fine using mocks, but fail using real model objects.

Personally, I prefer to test my Rails model objects using SQLite. And increasingly, I sharply minimize the amount of code in my controllers (and views, of course), and test it almost entirely using integration tests in a simulated browser. This makes it easy to get 100% code coverage with very little work, and it's still quite fast for a medium-sized app.

I mock web services using the 'vcr' gem, which can record a series of HTTP responses and play them back later. And of course, I mock out credit card gateways, AWS API calls, and anything else which requires a network to run. But I don't use mocks just for the sake of using mocks.

Actually, Unit tests should only test 1 thing. (Hence the name 'unit' tests.) So they shouldn't be testing both your models and your logic at the same time.

Plus, they are -way- faster when the database stuff is mocked out.

Of course, that's not always possible and efficient.

My philosophy is slightly different: Unit tests should test one API. But whenever possible, they should respect the encapsulation of that API, and only test the public access points.

In particular, I dislike unit tests which dig around in the private implementation details of a module, and specify how it connects to other modules "behind" it. Invasive unit tests make it much harder to change those implementation details.

You're not really talking about unit tests, then: you're talking about progressive integration tests, since the failure of a module "behind" your module can cause extra spurious failures.

Now, it may in fact be the case that you may argue that you can replace unit tests with all of these sorts of integration tests. I'm not so sure: digging around the private implementation details ways usually the easiest way to test strange error conditions—but it’s by no means easy. Though these days, I try to write as much pure code as possible, nullifying the benefits of an approach like that.

That's a good point. It's generally considered to be bad form to lock down the internals too much. It makes refactoring harder.

In general, when testing a class I obey the spirit of 'testing 1 thing' by writing tests that gradually increase in scope, so the first tests are testing 1 thing, the next tests are testing 2 (1 of which was already tested) etc.

Yes. And this way a controller test sometimes catches an unspeced model bug. Happens often enough to me to not mock everything.

A quick way to distinguish these in web apps is that integration tests typically are going to cross IO-barriers like calling a web service or accessing a database. If you're writing a layered architecture, this will be across those layers.

Unit tests should always be really, really easy to write. If they are not, then that is a 'smell' in your code that indicates one or more of the following things:

* If you cannot inject fakes of dangerous/unpredictable dependencies, you are probably creating these within the class. Pull that up! Or at least make it override-able. [1]

* Your class/function under test is doing way too much. Break out responsibilities into other methods or classes.

* Your test is too big! Think smaller. For example, if you're writing a test case for a recursive function, maybe you can start with the terminal condition instead of the whole thing together.

* You're actually writing an integration test. That's okay. Those are useful too :) They just take a bit more elbow grease.

[1] A great example of this is the DateTime object that this author mentioned. If you make heavy use of the DateTime.Now/Today in your code, you can and should make a 'freezable' wrapper class. Basically, you can freeze the clock at any time you wish. You can then thaw the clock and it goes back to using DateTime.

I see it this way - Unit Testing is to ensure that components & features are coded correctly. Integration Testing is about ensuring multiple components can work nicely together as an Application.

Yes I totally agree. I don't compare them, I consider them both critical.

For the record I tend to avoid controller tests for non-API controllers; if the app only got HTML views I don't see the point of testing controllers really. Then I think integration tests using browser simulation covers it enough.

great article, well put gil.

Reply on news.ycombinator.com