HN voting ring improvement | Hacker News Reader

HN voting ring improvement

twitter.com

50 pointsdman49 comments

A technical solution to a social problem. I am not confident that this will solve the root cause of the problem. Not that I understand the root cause.

Coming from a game development background ("real" games not "social" games), there are feedback issues with the voting system. Frankly, I don't vote a lot because I don't understand the system. I usually only up-vote one story a day.

Sometimes, if there is a "crap" story in the top 5 I will up-vote the other 4.

Look up the term "meaningful choice" in the book Rules of Play. http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&... http://en.wikipedia.org/wiki/Rules_of_Play http://www.amazon.com/Rules-Play-Game-Design-Fundamentals/dp...

The problem is that I don't have to choose between story A and B. I can up vote them both because there is no scarcity built into the system.

I don't even think there is a scarcity model with voting. How many up votes do I get in a day? If there is a model then I don't understand it. That implies the second issue with the system is that there is a lack of feedback.

This isn't only an HN problem. I see the same problems with other social sites that try to implement voting or other game mechanics.

Focus on improving the system for good players. Don't focus on squashing behaviour by bad players trying to game the system. The good players will police the bad players if the system is fairly balanced. No one reads the instruction manual, therefore the good behaviour has to be designed into the system. And feedback, feedback, feedback. There needs to be better feedback when there is good behaviour.

I wonder if providing the ability to downvote submissions would fix the problem of voting rings. Surely there will always be more legitimate voters than ring voters, and if legitimate voters can downvote submissions that were artificially inflated by ring voters it would balance things out.

Of course there is a reason why we don't have downvoting of submissions. The negative side-effects would probably outweigh the positive.

I am not sure I support downvoting of links. Though I am intrigued whether allowing downvoting for a set of users with some higher criteria than upvoting (say account date > 1 year + 500/1000 karma) will have some positive results.

Adding down votes would add another input to the system. In general, elegant systems aren't improved by adding more inputs.

My plan of attack would be to think about the internals of the system and decide which inputs need to be simplified and require better feedback for the user.

Sometimes, if there is a "crap" story in the top 5 I will up-vote the other 4.

Maybe the better idea there is just flag the crap one. I wouldn't ever up-vote anything unless I actually opened it and read it.

I flag stories that are spam or flamebait.

I'm not going to flag a story where the authors conclusion does not follow from their premises. But clearly these stories shouldn't hit the top 5.

story where the authors conclusion does not follow from their premises

If something like this is in the top five, it won't be there long from what I have seen.

I would flag spam, but also anything that you don't think is fit for HN, which is what I would call crap. From the guidelines: If you think something is spam or offtopic, flag it

how do you know there is no scarcity built in? maybe it ignores all but your first vote each day.

I don't. That's part of my point. The system needs better feedback.

On Twitter I follow and am followed by a lot of HN users and I know a lot of HN users independently and.. I can recall fewer than 10 times anyone has ever said "check this out and vote it up?" Sometimes I do, sometimes I don't - it entirely depends on if I'd vote it up anyway.

It saddens me, then, that there are "voting rings" which, I'm assuming, rely on people voting up each other's stories in a mutually beneficial way. I hope this can be curbed but, I fear, innocent groups of friends or popular HN users could fall foul of it merely for consistently submitting or voting up each other's stuff in an unorganized fashion.

The nice thing about dampening collusive votes in code is that we don't have to sit around in threads like this arguing about the "ethics" of voting and searching our souls. The nice thing about having a benevolent dictator behind the site is that we can just have things like this fixed automatically.

I suggest: stop worrying, vote however you want.

The ethics of voting are not my focus. My concern is the potential collateral damage of the "punishment." Google has had to take similar actions with its algorithms and even their best efforts yield collateral damage - I suspect this is why it takes them so many people and so much time to make improvements.

The voting system on Hacker News is small beans compared to Google rankings (which can make or break entire businesses), but it doesn't negate the validity of showing concern for a site from which I get much of my daily reading material.

This must be difficult. I presume that some of the more popular personalities on HN receive quick and multiple upvotes per blog post by a number of "fans" rather than a ring. Should these also be ignored?

How does one differentiate between a voting ring and genuine fans who like a particular blog article by a popular author (say having read an RSS feed and independently submitted the same article to HN)?

For the San Diego Hacker News meetup announcements, we generally post the HN link on the SDHN mailing list and invite people to upvote so there's a realistic chance new potential attendees will read about it. Would this constitute a voting ring?

"If that is wrong, I don't want to be right."

Would love to know more details about this.

What does he mean by "voting ring"?

A group of accounts used to upvote a submission.

At the time of this comment this submission has 5 upvotes in its first 22 minutes and it is #5 on the front page.

This means if you can convince just 5 of your friends to quickly upvote your submissions, they will all get promoted to the top of the front page. You can do the same for their submissions. That's a six-person voting ring.

Ideally pg would like to detect these rings and assign your friends' votes a lower weight on your submissions.

I think user engagement with /new needs to be substantially improved to reduce the effect of this. I'm a heavy HN user but only remember to check out /new once or twice a day.

It only takes a handful of people to get a story into a front page position where the votes can rocket upwards This means that the more people checking out /new and voting fairly (on average), the larger these voting rings would need to be to have any effect. (A war of numbers, sure, but "bad" stories would get flagged a lot more quickly too.)

A second problem is that the seemingly low engagement for /new can mean that reasonable items submitted at "bad" times don't make it past 2 or 3 votes whereas similar content submitted at busy times does well. In a good, fair system, that shouldn't be happening, but it's a reason I time my submissions to HN to strike when, I'm assuming, /new is at its busiest (4pm-10pm Pacific seems to work best for me). Without growing the international audience of HN, though, this might be tough to crack!

I think user engagement with /new needs to be substantially improved to reduce the effect of this. I'm a heavy HN user but only remember to check out /new once or twice a day.

Hmm. I probably go to /new 10 times more than I go to the front page. I suspect the problem with /new is that there is so much of it (especially at those peak PST times) that stories don't have time to garner enough eyeballs before they are pushed off the page.

PS. Wish I could un-downvote - I just clicked the wrong arrow due to clumsiness, sorry... Upvoted another one of your agreeable comments to compensate :-)

Sounds like a NP-hard problem, am I correct?

Not so much. Trust metrics (underpinning Google's PageRank) are pretty efficient at this sort of thing. If people A-Q all vote each other up, but nobody votes them up, their weighted effect is pretty close to nil.

Try: http://en.wikipedia.org/wiki/Trust_metric http://en.wikipedia.org/wiki/PageRank and http://www.advogato.org/trust-metric.html for a bit of an overview.

I think the bigger problem is developing an algorithm that reliably solves this problem (and doesn't introduce collateral damage), rather than the computational complexity of the algorithm.

A voting ring is a group of people who vote for each others' submissions in an organized manner.

Same. Suspicion of voting ring activity on Reddit is why I stopped submitting links there years ago.

Though if pg prefers to keep the details obscure to prevent rings from gaming and evading them, I'm cool with that.

Check the reddit source code and check out the blog, there is a lot of work done to cut down on this type of spam. Obviously as an end user, without a lot of data mining it would be hard to determine, but I don't often think to myself "Ah, this must have been vote-ringed-up".

What?!? I challenge you to cheat something up successfully on reddit.

Whoever downvoted you must not recognize your handle. /upvote to compensate.

This happened years ago, around 2006 or so, so hopefully you guys have solved the problem now. When I first discovered Reddit I became a contributor, most of my links got upvotes, one or two even hit the front page.

But after a couple months, suddenly every single one my submissions was getting downvoted within the first minute or so of submitting, no exceptions. Even many of my comments were getting pointlessly downmodded.

I figured at best I had pissed off some script kiddie with a downmod bot, at worst a voting ring had begun systematically suppressing competing links.

I just read the site for a while after that without submitting or commenting, eventually created a new handle and retired the original, and then discovered subreddits which imho is the best idea ever. It's not a problem anymore, at least among the subreddits I subscribe to.

I <3 reddit, have learned more from people there, esp Proggit and the like, than even my undergrad education, and was one of your recent contributors. I'm sure the anti-gaming code is significantly more evolved now than it was then. Though as this article and this one at Reddit[1] show, voting rings and the like are still a major problem for social link sites.

[1] http://www.reddit.com/r/IAmA/comments/d7e24/my_job_was_to_ga...

Source is open, yes?

Yes and no; news.arc is open but the exact code running the site isn't public.

Would it be possible to apply this research to the new page in order to reduce or eliminate information cascades?

http://www.overcomingbias.com/2007/04/popularity_is_r.html

If we split the new page into three or four parallel universes then it seems like that could go a long way to improving the quality of what hits the front page.

That would likely backfire, already a lot of stuff that is good or even great does not get enough votes to make the homepage. Splitting the new page in to multiple segments would make that worse.

I can see the case for randomizing the entries on the newpage, and maybe making it a bit longer to offset the variation in traffic on the site at different times.

Happy to see PG work on more voting ring busting, there is definitely still quite a bit of that happening, the new page sometimes shows pretty weird things.

"Already a lot of stuff that is good or even great does not get enough votes to make the homepage."

The reason this happens is because stuff scrolls by very quickly, and people are most likely to click on the stories that already have votes. And the first stories to get votes are always the ones from outlets that people are familiar with, like Mashable and TechCrunch. If you split the new page into three universes then all of the stories that would normally get free votes have their vote count divided by three, whereas the other stories do about the same, which is comparatively better than they'd do otherwise. At least this is my thinking. And then you just add up all the votes from all the new pages to determine the front page.

If we can get a PG in here, I have a question:

Trust metric based, or something else?

I'm not asking for code, as I suppose exposing that could make it more attackable, but I am curious about the technique.

I would like one story taken from the new section to be randomly inserted in my top 10. This would encourage readers to vote and make it easier for good news to go top.

Does HN give less Gravity to a vote if they don't actually view the story? (i.e. click through to read it)

I'm not sure if HN tracks this, currently; it's pretty obvious that clicking on a story doesn't send you through an intermediate redirection page, which is the simplest way to monitor that sort of thing, but there could be some fancy javascript stuff going on to register click-throughs. It's all in the source code, of course; if you want to go hunting for it, then by all means do so.

I suppose this explains why my rep suddenly dropped by over 12 points today. Perhaps I'm part of a voting ring that I wasn't aware of?

It looks like you were downvoted a lot for this post: http://news.ycombinator.com/item?id=1646903

The karma for that post is actually less than -4, as explained elsewhere, the karma for a post can now be lower than -4.

Could you point me towards where that was explained? And why would it be less than -4 and only show -4?

PG mentioned it today actually:

http://news.ycombinator.com/item?id=1647322

Cool :) And thanks for keeping it a level playing field.

Once voting rings get the upper hand it would be a real problem.

It'll be interesting to see some statistics on this.

Quora does this by not identifying the poster, so that self-validation does not occur.

Call me naive, but I thought stuff like this only happened on digg and reddit.

Not worth a hacker news post.

A post concerning how the site functions isn't relevant? Dang, picky crowd.

I didn't say not relevant, I said not worth a post. The main problem with twitter is that it turns content which would normally be too light into something that seems like it is substantial.

But your quip got your 1.5 month old "drive by account" a dozen up-mods, so good on you.

Reply on news.ycombinator.com