This is a current limitation, and an artifact of the data+method but not something that should be relied upon.
If we do some adversary modelling, we can find two ways to work around this:
1) actively generate and search for such data; perhaps expensive for small actors but not well equipped malicious ones.
2) wait for deep learning to catch up, e.g. by extending NERFs (neural radiance fields) to faces; matter of time.
Now, if your company/government is on the bleeding edge of ML-based deception, they can have such policy, and they will update it 12-18-24 months (or whenever (1) or (2) materialises). However, I don't know one organisation that doesn't have some outdated security guideline that they cling to, e.g. old school password rules and rotations.
Will "turning sideways to spot a deepfake" be a valid test in 5 years? Prolly no, so don't base your secops around this.