Ask PG: More transparency on average karma? | Hacker News Reader

Ask PG: More transparency on average karma?

11 pointsgjm116 comments

[Note: Yes yes yes, obsessing over karma is unhealthy, we should focus on actually making a valuable contribution, etc. Consider all that said.]

The "average" karma displayed on a user's page is computed as follows. (1) Take the user's 50 most recent comments. (2) Delete the latest 5 (because their scores may still be increasing, making their current values unrepresentative), and the highest-scoring of the others. (3) Take the average of the remaining comments' scores. This is done asynchronously; empirically it seems like it's every few days.

This seems like a sensible system -- it's not too sensitive to outliers, ignores comments whose scores may still be changing rapidly, etc. -- but because it ignores recent events and has an unknown, variable delay in it (because it's evaluated asynchronously) and isn't clearly documented anywhere, it's not great for user feedback.

I suggest (with the understanding that there might be one or two other calls on PG's time) that

1. there should be some documentation (e.g., a link saying "(what?)" after the average score, going to a short page describing the algorithm and mentioning that the number may be a few days out of date;

2. visiting a user's user-page should trigger an update of their karma if it's more than an hour out of date -- unless user-pages are visited so frequently that that would be too expensive;

3. when the average of the most recent 5 scores is >= the average of the others, they should not be excluded from the calculation.

To further expand on the update interval, I believe that every 45 seconds or so a user is selected at random and their average karma calculated. I don't know if any measures are taken to prevent starvation or a user - there are (of course) simple techniques to prevent that. Whether they're necessary is an interesting question.

[IANPG] Just out of curiosity, what would be the benefit of the proposed changes?

Benefit of change 1: It would be easier for users to understand the otherwise-mysterious fluctuations in their "average" karma.

Benefit of change 2: Average karma estimates would never be far out of date. (This would also make it easier for users to see what's going on.)

Benefit of change 3: Average karma would be more representative. The point of ignoring the 5 most recent changes is to avoid giving a misleadingly low average when recent comments are still gaining karma -- but if a user's most recent comments happen to be high-karma ones, the result may actually be to lower the estimated average, taking it further away from what the average would be if the scores all had time to settle down. Again, this is liable to be confusing to users.

(The specific situation that prompted this: My "average" karma is currently substantially lower than (1) the values it's generally had in the longer term and (2) the scores of recent comments I've made. I found this puzzling -- I knew about the ignore-highest-score trick but not about the ignore-five-latest.)

[IANPG] Since it doesn't result in better posts or comments, it probably doesn't improve HN.

It's worth distinguishing immediate from indirect benefits.

The purpose of the karma system is to encourage better posts and comments. To do this, users need to be able to see and, preferably, feel the link between karma and post and comment quality. Their ability to do this is impaired if the link is subject to delay and confusion.

Of course this is only true in so far as users are being guided by the "average karma" figure. But if they aren't, then either it shouldn't be there or the principle you're appealing to is incorrect.

Just out of curiosity, do you think that the black bar displayed at the top of HN pages when certain people die results in better posts or comments? Or would you consider the possibility that, important as those things are, they aren't all PG cares about? I think they aren't; I think he probably cares to some extent about how pleasant HN is to use.

Of course, I'm probably as much of a karma whore as the next guy...but I take my karma average with more salt than the score of individual posts because it offers less direct editorial feedback.

The link between numeric score and quality of post is tenuous. Good comments in low interest threads may still wind up as one point, and well timed snark can score in the double digits (though less likely now that scores are hidden).

The difference I see between the average karma mechanics and the black band is that while the former is of individual concern, the latter fosters a sense of community.

Reply on news.ycombinator.com