The next time you are about to post something online, or say something to friends, or patronize a business, do you first ask yourself "What would it do to my public image if other people I knew saw me doing this?", and let that affect your behavior? Not me.
PS - I can also say that I have never up/downvoted a comment anywhere simply because I disagreed with it. I have upvoted many comments critical of me, or something I believe in/agree with, because of their eloquence or wit. I downvote very, very, very few comments, and only because they are completely useless (say, off topic), or ridiculously foul-mouthed in an inappropriate context (by the same token, I have upvoted such if funny enough). Even if few others seemingly believe in the original rationale for up/downvoting, I still do.