Hacker News Reader

Comment by bluejekyll | Hacker News Reader

> Unit tests are good, integration tests are gooder

I would personally rephrase this as unit tests are necessary, as are integration tests, as are end-to-end tests.

The testing pyramid: write most of your tests as unit tests, then write a bunch of integration tests, and finally throw in a few end-to-end tests.

> Tests make better APIs

Yes! This is such an important point. A perfect example I love is that globals and even thread-local values make testing hard, and that’s reason enough to avoid using them.

> Make tests that you know how to run on the command line

Again great advice. I would also add, get your tests running as early as feasible. This saves you rewriting APIs and other decisions that might make testing, even deployment (which you need for end-to-end tests), significantly harder.

Anyway, this entire set is a great read, and I agree with much, but not all.

View on HN

I'm with Julio on this one. The triangle ought to be a diamond, with the integration tests being the fattest part. This is a level where tests can be at the component level, where they can correspond to specifications which are comprehensible to stakeholders (which helps keep developers accountable to focus test effort on things that really matter), and where refactoring of component internals do not require corresponding refactors of test code. This creates challenges, 90% of which are resolved by applying the same good development practices (re-use, abstraction, performance) to your test code that one is accustomed to applying to application code.

The model i like, keeping to the monumental theme of the test pyramid, is a testhenge:

http://www.ox.ac.uk/sites/files/oxford/styles/ow_medium_feat...

You need a thin layer of high-level tests (controller and/or browser) covering the entire application, and then columns of lower-level (integration and unit) diving deeper into it where there are areas of particular complexity or risk.

You seem to imply that I think integration tests aren’t valuable. But I’m not saying that. What I’m saying is that it’s easier to produce unit tests that cover sections of code and test their contract across edg-cases easier than it’s to do the same with integration tests.

An example: Let’s say you’re writing a DNS resolver.

You need to parse all of the various RDATA portions of records. It’s far easier to write an exhaustive suite of unit tests that verify the code is parsing those data elements properly and throw in many tests for edge-cases that might mean insecure software if done incorrectly.

But it’s absolutely invaluable that we also have integration tests that verify an entire Message parses, and even moves through the entire stack from parsing to response. But why construct an entire Message when all you need to test is the RDATA? It’s easier and less complex test code to localize the RDATA test, and then to do a smaller set of Message based tests that verify the message moves through all the stacks properly.

Then popping all the way up the stack, you actually need to generate network traffic and send real DNS queries and verify that those work. That means multiple processes in your end-to-end test. Trying to produce a full set of tests that would validate individual edges cases in all RDATAs would be an extreme amount of redundant code, and also would fail due to many other reasons not related to an RDATA change, thus making it more time consuming to track down a failure.

Each area of tests is absolutely important and all are necessary for different reasons.

Many people consider module-level tests to be unit tests.

Integration tests would mix multiple modules.

Depending upon the specifics of your application, unit tests at a lower level than the module level may be suitable. E.g. complicated algorithms a module might internally rely upon.

Our language around testing is missing a slot. We have unit, integration, and end-to-end. Should be unit, module, multi-module, and end-to-end. Depending upon the specifics of the application, you might have more unit or module level tests.

> unit, module, multi-module, and end-to-end.

A great semantic insight.

I still prefer to test complicated algorithms via the external interface. It does end up requiring some scaffolding code for generating inputs based on test cases, but code is for automating repetitive tasks, and they're pretty fast these days, so it's rarely a problem, even for many thousands of test variations. It enables you to test the interactions between your various unit-level variations, which is more likely to catch errors. Again, it also helps make your tests visible, along with the edge cases you're testing, and many times developers working at the unit test level are way more thorough than they actually need to be. Couple this with efficiencies around refactoring and it can actually save time.

Reply on news.ycombinator.com