Questioning novel science is one thing but questioning if the Earth is flat or Germ Theory is another thing all together. The problem with skeptics is that they sometimes hang around conspiracists.
It's hard to not discount these people when the person next to them thinks black people are biologically inferior. Then when those skeptics don't distance themselves or don't explicitly condemn those bad actors, it brings to question if their positions are born of skepticism or some strange prejudice, and that they merely constructed the cover of skepticism to hide their strange prejudices.
For example, during the Covid pandemic there was a lot of questioning around masks. In hindsight, the answer is obvious: it doesn't really matter if masks were or were not effective, because they're essentially free to wear. Even in the worst case, nobody is actually hurt.
But there were many, maybe millions, of mask deniers who would simply refuse to wear them. They were doing this because of institutional distrust and political motivations, not because they truly believed the masks were dangerous. And this is the trouble: these people are skeptics, but they're skeptics with an end-goal of political destabilization, i.e. they're dangerous.
When you mix it all together, which people often do to themselves, it's discredits the very thought process.