Comment by bermanoid | Hacker News Reader

Comment by bermanoid | Hacker News Reader

Parent Full thread

...which is a problem with a crystal clear cause: if you allow purely democratic voting, then you'll end up with really popular crap reaching the top of the pile fairly often. Imagine how useless Pagerank would be as an algorithm if it considered links from every drooler with a domain name to count for the same amount as those coming from the frontpage of Yahoo - that's essentially what you're doing when you tally up votes on a site The only reason it's not worse than it currently is is that there's no real incentive for people to game the system, so we don't end up with massive comment spammers. But we still have douchebags, fools, and all other sorts of sheeple, and in large enough numbers they end up diluting the votes that should count the most.

Several solutions are probably obvious to everyone here and I won't go into them, but the point is, you'd have to be willing to throw fairness aside in order to fix the problem - if a brand new account's vote is worth the same amount as mine, which is worth the same as tptacek's, then something is wrong.

The site can generate a lot of traffic to a submitted URL, so there is certainly a temptation to game getting a URL onto the front page.

I think that an informal oligarchy was established, certainly with respect to downvotes and flagging. The question is why is it breaking down. More troubling are the number of quality commenters who seem to have departed.

One hypothesis I have is that many on the leaderboard (and may of contributors I value who aren't on the leaderboard) don't seem to have any plans to apply to YC, they just want to take part in a community of hacker entrepreneurs. If more effort is not given to establishing a richer community model that is not managed as an adjunct to the accelerator they will probably continue to drift off.

But a more fundamental question is what we mean by 'post quality'. Right now, the vote total is the only objective metric of post quality; if we're trying to rethink the voting system itself, we need a lower-level set of objective quality criteria if we want evaluate how well a proposed ranking system works.

What are 'crap', 'douchebags', 'fools', and 'sheeple'? Different people have different thresholds for assigning these labels. Until a discussion board establishes quasi-official definitions of these terms, whether by consensus or by fiat, you can't use them to measure anything.

Your last suggestion seems to make things less democratic understandably, but how would it be implemented?

On a hunch: Run Pagerank (the original thing, i.e. just the stable distribution of the random walk) on the graph of who-upvotes-whom. Then tally upvotes weighted with the upvoters pagerank. It's probably safe to assume that the pagerank of a given user changes rather slowly, so this shouldn't be a too big problem to actually compute (and needs to be update at most every few days). Also, to favour recent history a bit one could decay the wheight of older upvotes in the who-upvotes-whom graph.

Now that upvotes are non-public information, people can't know their own rank (at least not easily), so this looks at least halfway resistant to gaming.

Advogato has used an attack-resistant trust metric with similar features; Raph Levien called such systems attack-resistent trust metrics. I'm sure they would do the task you describe successfully. Check out:

1. http://www.levien.com/free/tmetric-HOWTO.html

2. http://www.advogato.org/person/chalst/diary/189.html

Whoops, missed a lot of responses here, sorry about that!

Yes, vanilla Pagerank (where the "sites" are accounts and "links" are upvotes) is the first thing that came to my mind, mainly because it's been pretty well battle tested; running it on the comment graph is actually a much simpler problem than running it behind a search engine because you don't need the additional refinements based on search keyword. A comment's Pagerank is, by itself, enough to set an ordering.

If that's too much work, though, even a simple vote weighting based on some function of the upvoter's karma would be a decent approximation. Say every user's votes had an impact based on some sigmoid function of their karma (maybe a Gompertz function?), tuned so that the plateau is hit for the top 10% of users or something like that.

Your last suggestion seems to make things less democratic understandably, but how would it be implemented?

Weighted votes based on karma. Or restrict the number of votes a user can cast based on karma or account age.

A problem with these kinds of things is determining what does or does not work and picking up on unintended side-effects, and doing so in a timely manner.

For example, if "young" accounts have very limited votes available will new users be less likely to stick around long enough to become more active voters? How long would it take to see this? What if by the time you recognize an undesired side effect you've already sent the site down the road of ruin (or something)?

For example, how long will pg keep comment vote hidden? How many people have found this to be sufficiently detrimental that they have, or soon will, leave HN and not come back?

How do you craft meaningful site experiments like this while keeping risk to a minimum?

I suggest similar to Reddit's anti-spam mechanism, fuzz the voting but give the impression that karma is somewhat important.

Maybe all we need is 'bad, average, good, great' to be shown alongside the comment. Or something along the lines of what slashdot do with allowing you to vote for comments as 'insightful' or 'funny' etc.

>> if "young" accounts have very limited votes available will new users be less likely to stick around long enough to become more active voters?

I suppose that depends on whether or not they are on the site to read submitted news articles and comments or to play the voting and karma game. I rarely vote (or comment for that matter), but I get value from the content. For that reason, I have continued to consistently visit the site for a couple of years.

I like the idea of upvotes and downvotes being multiplied by your avg score. tptacek's votes would be worth 8.85 at the moment, bermanoid's 3.5, and mine 2.92. Newbies would come in at 1.

Wouldn't you want newbies at 0.01 (or some other value below 1) until they have a reliable score?

That would be a violation of the democratic principle behind social news and would incentivize people to game their scores. Members have different interests and expertise, so having a high avg score doesn't mean they are experts in everything (Abuse of power - OMG this is getting more intricate than i thought)

You could take the log of any score > 1 to reduce the incentive to game the system.. that way someone who relentlessly karma whores won't have that much more influence than a mildly uprated commenter, but both will have more influence than a noob.

but then a mob of noobs could outrank two wise elders.

Well, with the situation we're in now, only three noobs (or one noob with three accounts) can outrank two wise elders. As to your gp comment, yes, incentivizing karma gains could be a problem, but that's the tradeoff that's got to be considered.

I'm not sure what the best tradeoff is; personally, I'd probably go with something like Pagerank for a while and see if the comment quality (above the fold, at least) improves, but perhaps pg has a desire to keep things more neutral and fair, I'm not sure.

I've actually implemented a system like this on isdaily (a news site I'm building.)

I came to the conclusion that using Karma as a score is a rather bad idea as it assumes that people who get voted up have better judgement: "If you post well you must vote well."

But I'm sceptical that this is the case, it's easy to see someone like patio11 has both excellent judgement and a high karma score. But vote power = karma probably means you're giving a casting vote to patio11 if he turns up but mainly you're giving vote power to the people who post the most.

The implementation I have is basically: "If you made hard decisions in the past and voted well, you get more vote power."

So in your implementation, you gain voting weight by voting on the posts that have themselves accumulated the highest vote counts?

In order for this to work, you'd have to hide vote totals on all posts and comments, which, as jamesbritt pointed out above, eliminates a feedback mechanism that users may find desirable.

I do hide the totals. However you gain vote power for difficult decisions.

So the most vote power is assigned to someone who votes early on a contentious story which later becomes popular.

Also, in case it was unclear to any still reading, Democracy is for suckers.

Reply on news.ycombinator.com