We do, with the Republic of Ireland... and we have a borderless travel treaty with them called the Common Travel Area which has existed in one form or another for nearly 100 years.
And if we leave it open? All the migrants will just flood through NI. The whole idea that we can control migration by leaving is absurd nonsense.