I've used a similar technique to test, generating an expected output, actual output and then diff them.

One trick I found helpful was using JSON to serialize test results instead of unstructured plain text.

Test results stored as JSON are much easier parse and therefore process. You can quickly whip up programs that verify the tests satisfy invariants, diff the tests and filter out expected test changes from unexpected test changes.