Tougher border control is restrictions on immigration. Why do you think various billionaire nutjobs want to build a great wall of mexico?
Moving towards an open borders would be changing restrictions on immigration.
But I'm guessing you're ok with the folks who migrated to North America. Just not ok when those same people move around their own continent.
Wait, who is preventing the entire population of California from moving to Rhode island?
I'm not aware of any regulations that forbid anyone from moving to Rhode island from California, except for things like house arrest or other things requiring someone to stay in one place.
That said, CA and RI have similar quality of life, but Africa and North American probably do not. So CA/RI is not a good example.