This is far from true. "Holocaust deniers" very often turn out to be "The holocaust was more than just killing Jews, so stop making it just about them. A lot of the figures about those killed are inflated. The Holocaust needs to be stop being used a trump card for getting everything from Hate Speech Laws codified to a free pass for Israel to wall in the Palestinians and bomb densely populated areas."
Now I tacitly accept the figures for those killed in the Holocaust (6 million Jews, 5 million gypsies, gays and others) because it isn't an important issue for me.
But trying to call holocaust deniers closeted genocidal maniacs is pretty absurd.