However, in order to achieve this, certain prerequisites have to be met first. Opening borders has to be accompanied and indeed preceded by economic development.
The Schengen agreement for example only works because of the relatively similar economic situations of the member countries and even then it unfortunately has been a considerable cause for discord in recent years.
Let's say opening borders does have to to be accompanied by economic development: fine. How does that then not work when migration occurs from areas with different economic situations?
You seem to be proposing that migrants from economically areas necessarily depress a developed economy?
Or, to put it another way: what gives you the right to exist in a public place in the country you live in, but not a poorer person?
However, an argument can be made for the self-determination of groups as well, particularly their freedom of association (or lack thereof).
Then there are more pragmatic matters. You can't simply let people starve if they can't provide for themselves. Providing for everyone who chooses to stay at a place but can't actually afford to do so might just not be economically feasible.
Sure, over time these differentials will be evened out because people won't come to a place if it doesn't make economic sense to do so but simply opening all borders without taking possible consequences into account could lead to quite harsh outcomes.
If you don't apply the social safety net to the incoming immigrants, I don't see how you hold any higher moral ground than people who want controlled migration.