I suspect that what will eventually happen with Brexit is Britain will negotiate enough free trade deals that it will remain part of the EU in all but name.
Modern visa granting can be done electronically like the U.S. ESTA and could easily be achieved with little disruption. It could also prove a handy earner for a small fee.
I think it is misrepresenting the majority of brexit supporters to say they don't want any immigration. All the ones I talked to just wanted to have a say, and the EU did not give them that.