You're right, you can't because the anti-vaxx movement is as old as vaccines themselves. Wakefield's license was revoked for actually legitimate reasons. For his actions in falsifying data, not for expressing the idea that vaccines cause autism.
> I also think the ideas that Jones and Yiannopolous promote are easily seen to be harmful. Yiannopolous has promoted violence against journalists and associated himself with white supremacist movements.
Incitement to violence is a crime. We have processes for reporting crimes and seeing justice is done.
> I suppose it's up to you if you feel those ideas could cause harm - I feel they (and other ideas both have promoted) do.
"Ideas causing harm" is exactly the wrong way to look at it. Actions cause harm.
Sometimes ideas lead to action, but we should always police the action, not the thoughts. We start down a dangerous path to thoughtcrime as soon as someone is no longer permitted to think or express an idea.