Do you know what stokes division? Demonstrably false narratives and accusing vast swathes of people who hold myriad views on an immensely complicated subject of racism. Whatever makes you feel better, I suppose.
So maybe not racists, but xenophobes for sure.
EDIT longer answer:
You live in a society of people, the people that you interact with daily are people just like you. Some of them might be immigrants and some of them might have a foreign citizenship. That does not make them any lesser. They pay taxes to the same government as you, they abide by the same rules as you. If they brake the law, they have to answer for it just like you. The only reason I can think of why they shouldn’t accept the same benefits from your government as you is xenophobia.
From my experience:
1. I have a lot of friends from all over the world, that live in the UK. They have been there since their late 20's, make far more money than the median, and hence contributed a lot of money to NHS
2. I was living in UK during the Polish influx, and what I observed was that British people would prefer to use Polish builders as they were doing better work and got things done faster. My landlord was complaining that she was unable to book her favorite Polish builder for some work in the house.
Not saying that this is always the case and there weren't people who arrived to UK to take advantage of the better social services, but that's what my experience was, and maybe it's because of the specific bubble I live in.
https://www.theguardian.com/politics/2016/jun/16/nigel-farag... for example.
With the EU the racism on display is usually against eastern European migrants rather than black people.
That is not to say that even the most hardline Brexiteers are necessarily racists - they're naive and stupid but not racist.
On the other hand, most of the people who I know who voted 'Leave' had poorly disguised (and sometimes completely insane) xenophobic rationale for doing so.
Also - and significantly - racially motivated aggression and hostility toward my (mixed race) family suddenly increased to intolerable levels in the weeks and months following the Brexit vote.
The whole affair has been an undeniable victory for the nastiest sort of hatred, and has demolished the pride I felt in my country as a bastion of reasonableness and tolerance.
However, in the context of Brexit, Farage proudly stood before “breaking point” posters that showed the same scenes as a literal Nazi propaganda film picturing large streams of Jewish refugees as a threat:
https://www.researchgate.net/figure/Comparison-of-the-poster...
And also the whole narrative that everyone in Romania and Turkey was going to move to the UK, and so on. Race-baiting was a big, important component of the Brexit campaign.
Obviously not everyone who voted for Brexit was a racist by a long shot, but it’s naive to think that it wasn’t a factor.