There's nothing wrong with using the arrows to express agreement or disagreement. I personally prefer the discussion to be a little polarized, because it seems like we put in a lot more effort into our words in that case.
If you think you're going to lose a bunch of karma for just expressing your opinion, you'll follow the group opinion and it leads to groupthink.