For a contrary (but perhaps former) view of the site founder, see
http://news.ycombinator.com/item?id=117171
"I think it's ok to use the up and down arrows to express agreement. Obviously the uparrows aren't only for applauding politeness, so it seems reasonable that the downarrows aren't only for booing rudeness.
"It only becomes abuse when people resort to karma bombing: downvoting a lot of comments by one user without reading them in order to subtract maximum karma. Fortunately we now have several levels of software to protect against that."
A few months ago, the site founder, pg, asked about whether implementing more flagging options for marking comments would help make more clear what the community thinks of various comments.
http://news.ycombinator.com/item?id=1006589
There was some interesting discussion in that thread. On my part, I will just try to follow the local cultural rules and do what I can to contribute interesting content to the site, and to discourage postings that don't contribute to the site. I am just one participant among thousands here, and I don't get to set any of the rules.
I will ask an informational question here. You wrote, "I always thought that you should upvote posts you agree with," but that is not parallel to your idea "only downvote content that actually doesn't contribute to the discussion." In other words, is it possible that some kinds of content deserve upvotes even if you don't agree with it? And, on the other hand, is it possible that we all should downvote certain comments whether or not we agree with them? But let me emphasize the positive, as I have attempted to do before,
http://news.ycombinator.com/item?id=1065084
and ask everyone reading this thread, "What Kinds of Comments Should Be Upvoted?" What comments are good comments, and contribute to the discussion even if you don't agree with them? I like comments that add new verifiable information to the discussion, even if they force me to change my mind.