When people espouse positions that are friendly toward white supremacist ideology, and adopt policies that are demonstrably racist, I think it's absolutely reasonable to call them white supremacists and racists. When people hold positions that are quite clearly far down the right side of our traditional political spectrum, I think it's correct to call them far right.
It's not an ad hominem attack or partisan name-calling to say someone is <insert negative description here> if they actually, factually fit that description.