The only way to fix this I can see is either some form of meta-moderation where you hold people accountable for their votes (ala slashdot), or the system I'd personally rather see, where rather than a full-democracy the users pick whose votes they see on comments. I'd have my pool of 20-30 users whom I trust and I could see their scores vs the throng's.
Just as you hand-picked some users as moderators who you feel represent what you want HN to be, let us pick who we want to serve as our content filters.