It's kind of like the difference between helping your own family and a stranger.
If I take a stranger in, I will throw him/her out if they do unacceptable things.
With family, I may not throw them out.
Refugees etc. are not born in the U.S. = strangers.
U.S. citizens born here = family.