What he’s doing is building the new version in the same branch as the old, with a simple code switch to use one or the other

This is very similar to A/B testing, but not for user engagement but for accurate replication of the original results.

When you decide to do this up front, it can be fairly easily done and is very valuable for debugging the new version