For a unit of the same identity to suddenly start doing something different is plain nonsensical, never mind the technical challenges that come with breaking behaviour that should scare anyone away from trying. Logically, a unit is additive until the unit are no longer used, at which point it can be eliminated.
> But the old tests disappear with the old code (when the feature toggle goes away).
Absolutely, but static analysis can easily determine that the tests being removed correspond with units being removed. If (TDD) tests are removed and the unit code isn't, something has gone wrong and your infrastructure should make this known.