We're talking about freedom of movement between countries. A state is not a country. Why shouldn't all countries be like states? We can all hum John Lennon's Imagine there's no countries but I'm not sure how that actually works. I'm sure you can at least see some difficulty there and definitely there is a strong sentiment in parts of the US against this vision.
Why shouldn't Canada and the USA be the same? Why shouldn't Mexico, and Nicaragua, and Venezuela, and Brazil, and the USA, and Canada be the same?
If you want to move to Montreal or Mexico City or Beijing you can't just pack your bags and go. You can still sell something to all these places, manufacture them in all these places, etc. At least generally speaking.