Hacker News Reader

Comment by tansey | Hacker News Reader

>Ideally, you should have some other behavioral metric in mind (say, mean comment quality, top comment quality, bottom comment quality, engagement, etc)

That's precisely what my metric is intended to measure. The community derives quality of individual comments. The article explains why measuring mean, top, or bottom comment karma is a flawed approach. By measuring what we might call an "enchanced Sharpe" [1], we encourage consistent, high-quality engagement.

I suppose I'm not really understanding what your issue is with the formula. It's certainly not trying to "define away" some social problem-- how users vote, which articles make it to the front page, etc., is an exercise left to the reader. The only thing this metric is intended to do is replace the total score shown in the top right with one that better reflects your contribution to the community.

[1] Or just the Tansey Ratio, if it's not too presumptuous.

View on HN

There are, however, potential problems with this formula, which I think is what the thread parent was getting at.

For example, it is discouraging to log in to HN and see that your karma has fallen since your last visit. Using your formula, this would be a very regular occurrence for all but the most active users. Discouraged users are less likely to continue attempting to engage and many would eventually give up their attempts to maintain a decent karma. Then, while your measure may be "more accurate" in some sense, it would easily be less effective for goals that have more to do with engagement & participation than with notional accuracy.

Not trying to poo-poo the spirit of your post though, because as someone without much of a math background, this type of discussion is very enlightening. Just trying to clarify that inaccuracy may very well be a feature, not a bug.

Alternatively (and equally bad), upon seeing the karma has fallen, they might strive to post anything that won't be down-voted, but doesn't really add anything either, in the hopes of at least getting 1 vote to stem the tide.

I can see this punishing infrequent, but quality posters and decreasing the signal-to-noise ratio.

Sadly, somethings that work out just fine in math, don't mesh that neatly with human behavior.

That's an interesting insight.

I'm not sure that's true. Lots of people play games where their rankings shift down if they don't constantly play. In a lot of cases, this actually increases engagement. I think we would need to see some evidence here, but the null hypothesis should be that user engagement does not change.

If one wanted to assume that it does negatively effect engagement, however, then maybe an extended approach then is to show the ranking of the user's ratio score? This is less of a judgement of their score and more of a pleasant reminder that they are not contributing as much as others. Alternatively, you could also set the risk-free-rate to 0 for both the comment and day, then only update the scores periodically.

I suppose one could argue that it's not a competition and you shouldn't be vying for a higher score, but then why show us our karma at all? Similarly, if one believes that consistent contribution is not important to the community, then it's a philosophical difference and we'll have to agree to disagree there.

Part of my point is that even engagement might not be the best metric. For example, here in hn, I'm far happier if people don't post than if they post something trivial or uninteresting (your post and comments are interesting as I think this is a discussion worth having).

> For example, it is discouraging to log in to HN and see that your karma has fallen since your last visit.

To counter this, all we would need to do is define what rate of karma inflation is acceptable, then adjust all displayed karma ratings to compensate.

Reply on news.ycombinator.com