You should be upvoting comments because you think they add value to the topic/discussion, not because you happen to agree with the poster. That is how echo chambers are created.
Sometimes you upvote a comment that you don't agree with but see it as genuinely informative or at least insightful.
What you don't is upvoting a comment which is not informative, or even misleading, but still conforming to your views.