The moral issue is whether it is ok to take skilled people that have been trained using the limited resources of poorer areas of the world without supplying an equally skilled person in return.
Consider it within a country. The cities take the skills from the rural areas. What does that do to the rural areas? It is no different between countries.
Fiscal transfer payments don't correct the problem. A pile of dollars will not treat your sick grandmother. You need a doctor in the local area. If they have been poached by a richer country, then you're in trouble. And no country on earth has a surplus of medical staff, and not everybody can realistically become a doctor.
The framing with immigration is always down to the choice of the individual, and never really takes into account the desires of the community the individual has moved into, nor the desires of the community the individual has moved from - particularly if the source area has expended effort training the individual. What do you owe beyond money to the people that helped you be who you are?
As with companies if the drain continues, then nobody bothers training anybody - or doing the training - and we all eat our seed corn.
There's no simple answer, but net zero migration seems the best prospect to satisfy all sides - for somebody to move in, somebody else has to move out. A swap of equals. After all fair exchange is no robbery.