Yes, it's good to be careful to not label as people as fascists unless they do things like, for example, literally do the Nazi salute while publicly addressing a crowd.
Of course it's not only fascists who can be totalitarian, but totalitarianism is a bad smell in any form of government and should be avoided.
You have some known historians who openly say that Trump fits the criteria for fascism. Republican party? Not really. Trump? Yes.