But, the mixing of the "quality" axis with an "agree/disagree" axis risks coarsening discussion. A disagree-downvote is mildly censorious: it lowers a comment's placement and docks the commenter's karma. It stings.
I don't want to downvote comments simply because I disagree, but if such comments are highly-placed because of some superficial popular appeal, and comments I more agree with are lowly-placed, I want to restore some balance of attention. I then tend to cast a few votes based on agreement/disagreement.
One possible answer is an idea I've plugged whenever this issue comes up: start a second orthogonal rating axis that is explicitly for agree/disagree [2][3]. It could be little left/right arrows, at the end of the comment. Then you might 'disvote' a comment you disagree with, but still 'upvote' it because it was well-stated... etc.
[1] http://news.ycombinator.com/item?id=117171