It is not necessarily fascism just because they use violence. They view it as Nazis work towards a a violent society where certain minorities will be oppressed or even killed. So using violence to stop that is warranted just like it is for example warranted for the state to use violence to capture and hold a person that is preparing a terrorist attack.

I disagree of course as even though I really dislike Nazis political violence is something that is really dangerous for our democracy. For example in in town in Sweden one Antifa group jumped and beat up a libertarian politician just because he was seen as promoting politics that was hostile towards workers. Generally that is not something Antifa does but when you start using political violence is easy to use those methods more and more. Or inspire other groups that they should do it as well.