Online Reputation Systems: how to design one that does what you need | Hacker News Reader

Online Reputation Systems: how to design one that does what you need

sloanreview.mit.edu

67 pointsRod16 comments

So many sites get their reputation system slightly wrong, and end up incentivizing behaviors that they claim to not want. On discussion forums with reputation systems, the "echo chamber" effect is encouraged - it's been a running joke on slashdot since before the term "online reputation system" was coined. Also, early replies get viewed and upvoted more often, encouraging quick responses over more thoughtful (but later!) ones.

Reddit has worked on this, here is a guest blog post by Randall Munroe explaining: http://blog.reddit.com/2009/10/reddits-new-comment-sorting-s...

I independently observed this on slashdot and made a few bucks exploiting the psychology of readers. I wrote up my thoughts a while back: http://squareone.pheared.net/2009/10/make-money-posting-to-s...

That's awesome. I love that the secret sauce turned out to be Simpson's quotes.

Would a time-delayed showing of the comments help?

I don't think so. Or at least not at the cost of inhibiting discussion. I don't have any answers, and as the article discussed, you can't design a reputation scheme in a vacuum - you need to define your goals and align the incentives with them.

That said, it seems to me that sites like HN vastly over-reward early posters to popular threads. On the other hand, this is only a bug in so far that it leads to problematic behavior. I don't see people taking advantage of it (yet). I'm more concerned with the problem of under-rewarding late, thoughtful replies. I think this is a critical problem with the Stack Exchange (StackOverflow) platform.

The question is how can u get people to re-read the comments page for new comments? Maybe something like a "(5 new comments)" next to the comments link might work??

Metafilter does that, and I think that helps people continue a discussion that they are interested in, regardless of its effect on "reputation". HN does a similar thing with the "threads" link, but it only helps for conversations the user is active in (to encourage involvement?).

My thinking on late (and hence low-exposure) contributions is more along the lines of weighting. If 10 people read a comment and 5 upvote it, it should be rewarded similarly to a comment with 100 reads and 50 upvotes.

Just yesterday I noticed that OReilly recently published Building Web Reputation Systems (Randy Farmer, Bryce Glass)

http://oreilly.com/catalog/9780596159801

http://buildingreputation.com/

Randy was consulting with a company I was at a couple of years ago and while the two of us certainly didn't see eye-to-eye on a lot of things, it encouraged a lot of thought and debate.

I'll be ordering the book today.

It's not clear whether this book really grapples with the problem of pseudonymity on the web. What happens when a single user can start multiple accounts--one for good behavior and the other for trolling? When the trolling account gets banned, can they just start another one?

There can be no 'reputation' without an answer to the problem of pseudonymity.

The guys who created the 4chan web board code seem to think that the comment is what should have the reputation, not the user. This is one reason why they have a default-to-anon posting style (With options to force anon only posting).

People then focus on the content of the comment, not the posters aura/name/backing.

One thing I initially hated about HN but have quickly grown to like: usernames are displayed in tiny, faint text. There are no avatar images, signatures, or other visual identifiers. This has a similar, though limited, effect -- it puts the focus on the content, not on the user. (My wife regularly reads my comments without realizing I'm the author; this would not happen on a board with avatars.) Yet it also allows users to build a reputation over time, encouraging us to consistently produce quality content.

For anyone who might want to try the opposite - boldy highlight some users - try my Chrome extension: https://chrome.google.com/extensions/detail/mkdhfabjcebcgnpg...

When it comes to opinions though it does seem that who says something should play a part. Think of reviews - someone who says something is great and has a history of thinking the same things are great that I do is more relevant to me than an anonymous person thinking something is great. Reputation is a level of relevance that a user can build within a community.

There is a detailed description of the mechanism used in Yahoo! Answers in chapter 10 of the book. You can read the draft text here: http://buildingreputation.com/doku.php?id=chapter_10

BTW, everyone keeps assuming that reputation is all about karma. It's not. We cover that in chapter 1 (and over and over again in the rest...)

Cheers,

Randy Farmer, co-author Building Web Reputation Systems

Here @ HN the simplicity of Karma allows for the most inciteful and productive community discussions on the net. Would any of these suggestions actually improve the way things work here?

Also, as much as i hate blippy and swipely - if companies know who actually purchased a good - then their feedback becomes hugely valuable.

Reply on news.ycombinator.com