>> I just don't accept the idea that "this person lives on this side of the border" is a meaningful way to decide if they get to live in a place or not.)

I curious where do you draw the line then? Can a random person move to your backyard or your house?