Your comment highlights exactly why comment scores should not be shown.

If you use comment scores as a proxy for agree/disagree you are much more likely to downvote comments you don't agree with even if they add to the discussion.