And no, virtual numbers like Google Voice are often (but not always) blocked.
Also, inertia
you end up paying ridiculous roaming fees to keep your number active in the other country, or you lose any ability for people to contact you by phone. it's incredibly frustrating when voip is so close, but not the 100% solution. couple that with providers still charging ridiculous fees to call numbers in other countries and it gets even worse.
What places other than the EU does the "easy to cross borders as a part of life" apply to?
Good question - e.g. why isn't a phone number like an email address.
My guess is that it functions as an anchor - not to help motion, but to guarantee immobility. Like a physical address, the inconvenience of changing makes for fewer changes.
A low-pass filter, for system stability, if you want to look at it that way.