The opposite position gets downvoted too. (By me, for a start). But it's also more likely to be deleted as being out-and-out offensive, so if you're a casual reader maybe you don't see it as much?

I've experienced the same and have always been of the mind that voting systems on aggregate sites are anti-discussion and promote brigading. When people see a highly upvoted or downvoted comment, the tendency seems to be to follow the actions of those before you. Things quickly turn into an echo-chamber, although Reddit is admittedly more prone to that than HN, since it's divided into topic-specific subreddits.

Still, I'd be fine if the voting system were eliminated and threads were managed chronologically, keeping flagging for obvious rule violations, of course.

This is all, of course, tangential to the post at hand, but that's part of the beauty of it, in my opinion. Start on one topic, end up on something different.