None of the discussions on sending migrants out or being stricter on immigration is about color or race in principle. The issues are that legal processes aren't followed for immigration, or that the other cultures bring a disproportionate amount of new forms of crime.
Since enforcing laws and fighting crime is what states are doing anyway, then doing them more on immigrants achieves that goal more. (The situation is of course also unfortunate for any decent immigrants, as it will be more difficult for them to enter.)
But it's you who for some reason turns the merit argument into a race one. They do conflate, yes, perhaps that is why you can't tell the difference?