First, I think we should seek bilateral treaties such that we can enter their country as much as they can enter ours and that both parties have reciprocal rights. This should allow all parties entering an agreement to slowly stabilize, or reach some kind of equilibrium.
One of the downsides is that this adds to globalization and homogeneity of culture, language, etc. Although personally, I don't think that's a big problem --it's mostly an emotional issue rather than actual, but it's still an issue given how people cry 'globalization', 'imperialism', 'monoculture', 'mainstream', etc.
From Europe, where people share some culture, they never the less still experience integration issues where locals resent foreign populations, example France and polish workers, or Germany and Serbs or Turks, or Spaniards.
Maybe grant larger quotas based on population would be fairer and then harden the borders, else the closer neighbor has an unfair advantage. So say Italy booms, quotas would make living in Croatia no better than new Zealand or south Africa in terms of odds of getting into Italy (or Canada the US or Nigeria.) But politics would never allow a fair quota system to work.