I'm not even sure immigration laws have much to do with it. I would hazard a guess that the vast majority of people never get as far as finding out what the other country's immigration laws are.
I imagine people are vaguely aware of the degree of difficulty for different countries, but your point stands that most people with a reasonable standard of living prefer to avoid the upheaval of emigrating.