Slashdot has a basic solution to this: moderation votes with a category. You can vote up or down, in a specific category: Insightful, Interesting, Informative, Irrelevant, Flamebait, Funny, etc. If your comment has a -2, you usually know why.

I've suggested it about 50 times here. Not the same categories, but some form of metadata associated with the vote. As soon as you click the arrow, it can prompt a list of tags to click. You could very quickly express yourself along with your vote and move along.

My suggestions for categories would be emotional (angry, happy, funny, confused, indifferent, stupid, crazy, awesome) and non-emotional (citation needed, seems right, false, morality, ethics, expert, novice, etc). This way you get a good picture of why somebody voted: what their emotional state was, and what their (perceived) intellectual position was.

Perhaps one reason it doesn't happen is it would fundamentally change the dynamics of the ingroups here. You'll notice that on some posts, it seems like everyone is parroting the same opinion as if it's an established fact. Or they upvote someone's comment because they have a high reputation. If you had to add a qualifier to your vote, it may quickly become transparent why these people are voting, and then the algorithm would have to change so as not to expose these defects in group social dynamics. Or the algorithm would stay the same, and people could see that the structure of the forum is based on a lot of crappy human heuristics and not strictly adhering to "the rules".