How about a limited amount of down-votes allocated to each member to dish out? Possibly tiered based on karma level? Perhaps a member that down-votes a comment loses karma if the comment is then up-voted by other members (ie deemed an unfair down-vote by others)?