Google has a pretty mature TCP testing framework, called packetdrill:
http://static.googleusercontent.com/media/research.google.co...
It supports all the features you need, from packet structure to timing: https://code.google.com/p/packetdrill/
It supports all the features you need, from packet structure to timing: https://code.google.com/p/packetdrill/
Most issues seem to be catchable with this, and way too many unit tests end up becoming 'change detector' tests, where you test tautologies about implementation details.
One has to consider the performance cost of black-box tests, though. It will almost always be higher than that of unit tests, simply because more code will get run per test (this is especially true of network stuff, as even going through the loopback device is comparatively insanely expensive). That said, in some cases, it may well still be the right choice.
But that sounds more like integration testing, not unit testing, doesn't it?
The tests are run in real time - running 650 tests take almost half hour, which would normally be considered completely unacceptable for any kind of TDD. The timing can also vary from one run to another, so all timestamps have a few milliseconds of wiggle room automatically, and a small proportion of runs (1/2500) still fail randomly due to timer inaccuracies. And unit tests that fail non-deterministically are really unpalatable. Is the talk outdated re: these issues?
Great hack despite that, and I would certainly have loved to write a scripting language for expressing tests rather than use C++. And the packet syntax they've chosen is probably superior to what I ended up with. But what I describe in the post dates back to 2011, so packetdrill wasn't yet around for us to take inspiration from :-/