This isn't "mob rule," this is the reasoned opinions of skilled scientists saying that Hsu's political opinions are 1) not backed by the science, and 2) establish strong ground to believe that he will not use his position of power fairly.
People have a right to be judged fairly in their careers, and those in leadership positions should not be unfairly prejudiced by bad beliefs. It might be fine for a tenured physics professor, but it's not OK for somebody making hiring and promotion decisions.