In trading all threads are pinned to a core, so that scenario doesn't happen.
Also - all modern systems in active use have invariant tsc. So even for migrating threads - one is ok.
But googling suggests that if the CPU has the invariant_tsc flag then it’s not a problem, as foobar mentions.