System Preferences > Accessibility > Display > enable "Reduce motion"

Though I think this reduces motion on everything, not just the virtual desktop animations. I don't mind that but YMMV.