There's precedent of sorts for this though. Currently a non-EU citizen with a visa to Ireland still needs a visa to travel to the UK. This could be policed in the same way as that case.