Operating systems? No. Sorry. Operating systems, as a rule, predate modern practices by some 1-4 decades. Making a kernel like Linux, Darwin, or FreeBSD testable would be a task somewhere between "massive headache" and "really massive headache". I've done some in-kernel unit testing, but only on our own driver subsystems that could be abstracted from the difficult-to-test underlying monolithic code base.
Outside of kernels/operating systems, just Google. Unit/automated testing is prevalent in just about all modern OSS software ecosystems.
A few examples, off the top of my head.
Apache APR: http://svn.apache.org/viewvc/apr/apr/trunk/test/README?revis...
Clang: http://llvm.org/svn/llvm-project/cfe/trunk/test/
Go: https://code.google.com/p/go/source/browse/src/pkg/math/all_... (see all *_test.go files)