I'm completely with you on this one. I pay the bills doing computer forensics and defensive security, and the amount of times that I've seen clients compromised due to advertising networks serving malicious content is non-trivial. I'd recommend ad-blocking purely from this perspective.

When you have a single user the attack surface isn't so bad, but across an organisation of several thousand people it quickly escalates.

For me, distraction is the #1 reason. I started blocking ads as soon as it was possible (almost 15 years ago I guess), having been fed up with animated gifs. Next on my list was Flashblock, again because of distraction first. By then I had already permanently muted my browser.

I started blocking javascript because my pc was getting old and sites just ran much smoother without it. First using the global "block javascript" flag, and when too many sites became unusable I switched to NoScript and now RequestPolicy.

The tracking and security concerns are secondary to me; usability was my motivator. But by now, I'm so used to it, I never browse without protection. I literally feel dirty when I have to use IE or any other bare browser.

For me, #1 is tracking, #2 is malware.