The amount of housing actually taken by immigrants is minimal and releasing those wouldn't fix anything. Most of them live in squalor and doing jobs regular English guys wouldn't touch, in particular for the exploitative pay. You want your 2am burger delivered for a quid. Or the puke cleaned up in the metro. Who's going to do that these days?
It's this age-old problem: https://realfiction.net/assets/careful-mate.jpg
But now it's being weaponised for political gain. They're making you mad at the wrong people. The people who are causing it all are the ones you're voting for.