There is a difference between that and the systemic concerted efforts by wealthy countries to poach and take the most valuable, skilled, educated of the human capital from poorer countries who are in positions of weakness and far less bargaining power.
I understand that the narrative is that it's about "helping the unfortunates", but really it's not. If a country wanted to help people you wouldn't "help" mainly the ones who can give them something (they are actually helping you), leaving the many more less fortunate ones in an even worse state.
In Germany(already a wealthy country) doctors move to Switzerland for better pay and better working conditions too.
It’s also very expensive to move internationally. Visas can cost up to $10,000 alone, so I doubt that so many people actually leave developing countries for developed ones.
There are also tons of people who would never move countries because their family and social circle is in that particular place or they want to make their country a better place.
I also think many people move back home from overseas to their home country, bringing their improved skillset with them.
I think you're rationalizing or painting yourself a reality that you wish was true. Brain drain from disadvantaged countries on an industrial scale is a serious problem. Not for the wealthy privileged countries who benefit of course, which is why nobody says much about it and those who do get attacked.