The sulfuric smell in the hot water takes a little getting used to, but it's pretty amazing that it's naturally heated in most places throughout the country (though, a bit easier in such a tiny country).
Reykjavík artificially adds sulfur to its hot water to reduce the corrosion of their hot water pipes.
I can't find a source for that in English but there's one in Icelandic, it's the "Vinnslutæknileg vandamál við vinnslu jarðhita" section.
1. http://web.archive.org/web/20160813045634/http://www.samorka...
Seems like Iceland could really use some sort of myth/fact information as they continue look to grow their tourism.
They have very little incentive to "correct" people
I also know some Iceland natives who actually like it, so I wouldn't completely dismiss it as a tourist trap.