The world does not end at trade. There's a huge difference between a company from my country setting up a division somewhere else and people from another country moving into my neighbourhood.
You may be fine with either one, both, neither, and even some combination depending on the particular good or person. Conflating them is disingenuous.