Sounds like AI behaves similar to TDD where you blindly refactor until green, except AI have like a zillion more test cases(training data) to pass?

Is that really a better way of writing code? (for example compared to being able to reason about the code to create something provably correct)