> I could hop on the next plane to London and legally start working today.
That's exactly the point, and something I don't think many British people realise. (I don't think Americans etc realise it either, but I don't expect them to.)
Moving to work in a different EU country requires what I'd guess is a similar amount of paperwork as moving between US states.
- Register your address with the local government (which may lead to an id card, tax registration etc)
- Update your driving license, if applicable
- Tell the old country you'll stop paying tax there (may lead to a tax refund)
These are the same kinds of processes as are used when a child is born, a teenager gets a first job, or an adult moves house. Nothing requires permission, nothing can be refused.
Additionally, there are agreements for things like state healthcare, pensions and out-of-work benefits.
If the UK raises barriers, then London will be less appealing for other European citizens.