We already have something that serves the purpose. It is called a test suite. Unit and functional tests usually. Keep them updated and run them for every build.