The same problem is happening on every site where voting is allowed. There is usually a loud, highly emotional and political minority with a lot of time on their hands that invariably takes over and impose a fascist dictatorship on the rest of the community. Digg, Slashdot and Wikipedia are cases in point. I wish there was a way to motivate a greater percentage of readers to take part in the voting process.
Another problem is that a lot of people receive updates via RSS news readers and I suspect that most go directly to the link and rarely check the comments or attempt to vote. YC needs a way to entice people to visit the site and incorporate the number of times a link is clicked to the formula that computes the popularity value of the story. Just one man's opinion.