What you’re describing sounds very similar to “mutation” testing. https://en.m.wikipedia.org/wiki/Mutation_testing