I agree. It does really depend on what type of user you are targeting. For some sites IE usage barely registers.
Unfortunately though, the people who use IE6 are the same people who click on "You won $10,000!! Click on the monkey to claim" adverts, so you can't really ignore them if you're going after advertising revenue.