Most of the time I don't upvote because the link isn't good enough. It's very rare that I see a link I think is really worth people's time - maybe once or twice a week.

I could lower my bar, but the real problem is the binary nature of the vote. There's content that I really want more of, and then there's content that's better than average. If I vote both with my single vote, I'm not delivering the right level of information. It's almost like I want a supervote that can only be applied once a day, or once a week, that's worth more than one normal vote.