In 20 years on the Internet, most without an ad blocker, I haven't suffered from any lapses in browser security (that I know of, sure, but I don't much care about those I don't know about)
The tale of frequent compromises of browsers via ads is told merely to legitimize the practice of blocking even entirely plain and benign ads.
You were never clickjacked?
Never had pop-up or pop-under ads/windows created without your consent? Recursively? Crashing your browser?
Never visited a page that was hijacked with an iframe?
I've experienced a lot of malicious ads in my time on the internet, it baffles me that someone has not.
Nowadays, the first thing I do when "cleaning up" a non-tech friend/relative's computer is to install uBlock Origin. Since I started doing that, the number of repeat calls dropped precipitously. (To be honest, it's probably good for their fake news intake, too...)
The web is a lot scarier for most people than you might realize if you've been successfully navigating away from sketchy sites for 30 years.
I think the actual malware infections themselves legitimize the practice of blocking even entirely plain and benign ads.
Ads do still infect people with malware. https://blog.confiant.com/tag-barnakle-the-malvertiser-that-... Hell, yahoo was hit not that long ago and users were at risk of infection just by loading yahoo.com.
JS exploits and malware written in JS are common, but so far I haven't seen CSS used to infect systems, just steal data/log keystrokes and add a lot of privacy concerns which is bad enough. I'm already keeping an eye out for a CSS blocker that will only allow a sane subset of CSS and block or limit externally hosted resources.