bazhenov/tango does something like this for performance tests, basically to counter system behaviour you run the old and new implementation at the same time.