You dismiss the possibility that they are right. Many people do support Brexit for wholly racist reasons (e.g. my father who wants to 'get the poles out').
An example of what I mean:
'well you can imagine poor Joe Racist living across the river is now worrying about whether his boss can afford the harvest. That's what happens when you're stupid'
They will invent false characters with flaws that support their argument. They never claim they are real, they aren't basing it off some real observance or any real account.
It's not a useful discussion point and serves to further isolate that community.
As with so many things about the referendum, the problem could just be that a very vocal but actually quite small minority is getting all the attention. I know for a fact which way quite a lot of my friends and family voted, and in private with calm discussion I'd say the majority on either side are willing to debate the issues and often do find a fair bit of common ground if they do. However, many of the Leavers won't say anything about it publicly now, for fear of sustained attacks or even losing friends over it. That does mean that if, say, you look at the Facebook news feeds, they are heavily slanted towards strident Remain advocacy and topics like ignoring the result in Parliament and staying in or throwing Boris in prison, but that is certainly not representative of the overall sentiment among people I know, just the ones who are willing to state their position and live with the consequences.
If this is as bad as it needs to get for large swathes of people to realise the behaviour of parliament needs to change then it spells a bright future.
If it gets any worse though, or this isn't enough, then I guess we are pretty much doomed.
Unfortunately, it has become dangerous to hold conservative opinions publicly in certain places. There have been too many highly public witch-hunts, and not toeing the line on the established orthodoxy will result in being painted with the brush of racism, sexism, xenophobia, homophobia, idiocy, bigotry, or whatever the latest outrage of the week is.
It's definitely protectionist and perhaps xenophobic but I don't think it's racist. (There is a clear distinction between xenophobia and racism - xenophobia is purely about an unjustified dislike for those from other countries while racism is an unjustified dislike for those who look different to you)