People are still people, no matter what country they're from. To restrict immigration for work means you're saying some people, through no fault of their own, deserve to earn less money by working in their own country.
Any time you say "I'd hire a local over a foreigner", you're effectively being racist. It might not be their race specifically, but their nationality, the identity of their parents, their wealth (to do their own immigration), etc. This is still discriminating based on something the person has no control over. It's still saying "people similar to me deserve more good things in life than people I can't relate to" or "people in my in-group deserve to be treated better than members of out-groups".