Being able to efficiently allocate labor with a more mobile workforce does increase productivity, this is well studied.
Here's a source, although it isn't that detailed and is more of a descriptive literature review: https://www.urban.org/sites/default/files/publication/103581...
This too: https://www.oecd.org/en/publications/2025/07/oecd-employment... - but I didn't have time to read it in detail.
I don't think anyone believes labor mobility reduces productivity.