Malware will also slow down your computer, and could result in your personal information being stolen. If the choice were between waiting an extra few seconds for my computer to boot or having all of my files accessible to people I neither know nor trust, I would choose the former.
If you're working without at least common-sense anti-virus protection, you're going to deserve what you'll eventually get.
I'd like to know what your definition of "common-sense anti-virus protection" is, though. Apart from "don't download/install anything from untrusted places".