That's not to say there aren't adverse effects of internationalisation, or to belittle people who suffer from them. But this whole "taking back control" agenda was steeped with British exceptionalism and exclusivity.
Immigration was literally the sticking point of what triggered the whole shitshow. When David Cameron did his EU roadtrip to negotiate a better deal for the UK ("give us what we want or we'll leave"), there were 2 sticking point requests: lower the EU fee, and curb EU immigration. EU said no, Cameron called the talks over, and organised the sorry referendum.