Ask HN: How to design a karma reward system | Hacker News Reader

Ask HN: How to design a karma reward system

16 pointsyannis22 comments

I am building what is actually a large portal for a 'tutoring site'. Visitors will be able to join groups, create groups and post comments, lessons etc. They will also be rewarded for completing on-line exercises.

I would like to introduce a 'karma reward system' in order to a) motivate people to contribute and b) to crowdsource ratings both for people as well as content.

I am leaning to something that is in-between what http//thesixtyone.com offers and HN's or stackoverflow's floating method of ratings.

I am looking for suggestions, formulae and good links to resources, books etc as well as other sites than the above three that you favour.

View on HN

What I've wanted to try but I have never had the opportunity to was having karma be accumulated on a per-topic basis. I think any time you've got a wide set of topics this is necessary to avoid the issues of people being treated as credible outside of their domain of expertise, and it could help mitigate the issue on Stack Overflow where people who answer lots of easy questions can get much more karma than those who answer a few much harder ones.

Jeff Thanks! What you saying is very true of stack-overflow and although I visit occasionally their whole karma formula does not appeal for the reason you mentioned. I was thinking in similar lines and have introduced the groups concept for example when you belong to the 'lisp circle' you will be able to earn additional points when you answer or solve lisp problems. My concern is not to let the users focus too narrowly, I was thinking that joining groups should be limited i.e, when you accumulate 100 points you can join a group and as you earn open up more groups... To link the whole thing in a sort of 'PageRank' algorithm to stop people from gaming the system is what is puzzling me.

This is a good idea. It's be cool to see a user was an "expert in...X, Y, Z" or something like that. You'd need real critical mass for something like that. Most new sites are happy just to get users with any reputation, much less divide it amongst topics. Maybe it would be a good feature to add after a site matures.

I've posted this on HN before, but I wrote a long essay about problems with account-karma-moderation systems: http://www.kuro5hin.org/story/2009/3/12/33338/3000

Probably too involved for your project, but there are some problems you should be aware of from the get go.

Perceval thanks and by the way an excellent essay. You hit the nail on the head with ... 'our hypothetical comment-driven forum would push stories up based on quality of conversation....'. Do you think that part of the formula can be improved by including automatically a 'readability formula', where the childish comments and language would automatically be penalized by the system?

Some folks are already working on that: http://stupidfilter.org/main/

It's supposed to identify formally stupid comments, i.e. those that have bad spelling and txt abbreviations, etc. It doesn't do anything about substantive stupidity, unfortunately ;)

I was thinking something like the Gunning fog index http://en.wikipedia.org/wiki/Gunning_fog_index

> I would like to introduce a 'karma reward system' in order to a) motivate people to contribute

What evidence do you have that any karma system motivates people to contribute the kind of content that you want?

Seriously - look at the top karma people on HN. What fraction of them post for karma? Of the ones who do, how many of them do you want?

> and b) to crowdsource ratings both for people as well as content.

Again - what evidence do you have that karma helps with this?

A karma system guarantees that you'll have people posting for karma and discussing it. Both are bad. What benefits that you can reasonably expect that are more valuable?

It depends on your choice of language and some specific implementation details....and whether you are more interested in the user interface side, the data modeling, or both.

There's a range of approaches to motivational systems, and I think in 2010 we'll see some real innovation in this area. For instance, I think Gowalla is doing a particularly great job.

Thanks! I added Gowalla on the list! I am more interested in the data modeling and algo and good examples like Gowalla.

Here's a blog that might be relevant: http://buildingreputation.com/

Some great articles on there...

I'm planning on working on something potentially very similar. Here's an outline post I made a while back: http://karanbhangui.posterous.com/course-learning-aid-propos...

Lemme know if you're interested in collaboration.

Thanks I will send you an email! Is that UWaterloo like in the UK?

The University of Waterloo ( http://uwaterloo.ca/ ) is in Ontario, Canada.

Hey, as selven pointed out, the one in Canada. My email is karan.bhangui@gmail.com

Cheers.

Related question: does anyone have any info on using a super simple voting system (thumbs up/down, or just up) that does NOT suffer from time bias. In other words, an item might have more up votes just because it's been around longer.

Bias your comment ordering system in favor of new comments so that every comment gets its 5 minutes of fame before staying around with a good score or floating to the bottom with a bad score. it's not perfect, but it should mitigate the problem.

stackoverflow has a few interesting incentives:

- activity is renamed to 'rep' which implies that your activity ranking reflects on you personally, even though everyone knows its really a measure of how long you've been active and whether you answer broad beginner questions that get a lot of google hits

- it makes sure you know when you are downvoted

- though the numeric penalty of a downvote is negligible, but since it is a personal criticism of your reputation, you have ego incentive to improve your contribution

http://public.research.att.com/~volinsky/netflix/

For all your rating/karma/rewarding needs :-)

Read/watch all the Amy Jo Kim presentations you can find online.

Thanks lots of videos - will need three days to view them! :)

Reply on news.ycombinator.com