"X is a fascist" is not just a simple insult.
Pretending that's all it is is ignorance at best
“You are fascist” actually isn’t just an insult. If you display fascist tendencies then you’re a fascist, and he displays many of those typical tendencies.
There are actual fascists (and not as few as I would like) and they need to be called out, but using the term inaccurately and provocatively on a broad group makes it easier to oppose the usage outright. Optics are important to politics, like it or not.