Anti-Brexit is the status quo. It's by definition not controversial since they can't do anything other than advocate for how things currently are.
Pro-Brexit is by definition controversial since they were advocating for a complete upheaval in the relationship with the EU constructed over decades. And it was always going to be tinged with xenophobia and nationalism since that was the entire premise for the Brexit vote to begin with. And it doesn't get any more divisive than xenophobia.