That might have been true a few years ago but it just isnt these days. Malware is being injected into ad networks on a semi-regular basis (I see Adblock as a defensive mechanism morese than something to get rid of ads). Sites also get hacked and can infect you that way.

Its like saying people only care about NSA spying because they have something to hide -- its faulty logic.