I'm pretty sure people will tell you that Brexit was bad for a laundry list of other reasons too that do very much directly affect the average person
downvotes ahoy
The vast majority of people arriving from Nigeria and India do so on visas, and would have near zero chance of getting asylum claims approved.
Also a lot of regular Brits have moved abroad. Dyson who famously advocated for brexit to help Britain moved to Singapore, my friends have moved to France, Portugal, Spain and Dubai.