I saw this a lot on slashdot when I used to frequent that site. Comments that were at 3 or 4 got pushed to 5, but comments at 2 or less got ignored. But on that site, there was a hard upper limit of 5 points.
As far as my voting practices: I upvote select comments I think are particularly lucid and have not been voted up "significantly"
I downvote what I think are innapropriate comments such as hateful, vulgar, insulting (see this site's guidelines) "excessively" whiny or demanding or just plain babble (rarely, usually I just ignore babble).
I do NOT downvote opinions I disagree with. I either ignore them or reply.
Overall, though, I think the lack of hard and fast rules to voting adds more than detracts. If we all vote in different ways, then that makes the site more dynamic.