Hacker News Reader

Comment by amix | Hacker News Reader

I have actually done quite some research into HN's and Reddit's algorithms (i.e. http://amix.dk/blog/post/19574 and http://amix.dk/blog/post/19588 ) and I would say that using The Wilson score confidence interval would improve HN's comment sections a lot.

The real meat is here: http://www.evanmiller.org/how-not-to-sort-by-average-rating....

Even if it looks hairy it's fairly simple to implement in e.g. Python (from Reddit's code, rewritten from PyRex): https://gist.github.com/amix/5230165

Maybe pg could try this out for a few days and we could see what the results are!

View on HN

While this is more sophisticated, I doubt it's the "correct solution". The underlying assumption of independent draws from a Bernoulli distribution is vitiated, as the order in which the siblings are read usually affects their proportion of upvotes to downvotes. It still significantly penalizes late comments, because the 95% confidence interval is much wider for them (and its inferior limit consequently lower). Last but not least, it brings down comments that are at all controversial; whereas an opinion with 1000 upvotes and 500 downvotes is probably more interesting to read than one that only gathered 50 upvotes within the same interval. In short, this sorting algorithm seems ideal for quickly placing on top bland comments, that nobody really disagrees with but will upvote anyway because they haven't yet seen the much more incisive discussion below.

> the 95% confidence interval is much wider for them

If you change this to an 80% confidence interval, it'd become narrower and might actually somewhat favor new comments, with only a few votes (upvotes)? So this might be configurable?

> whereas an opinion with 1000 upvotes and 500 downvotes is probably more interesting to read than one that only gathered 50 upvotes within the same interval

Isn't it more likely that the 1000 upvotes and 500 downvotes is a cute kitten photo? Or something similar (a very short but strong and popular comment)? :-) And that the one that gathered 50 upvotes and no downvotes is the truly interesting read?

However! If you're able to estimate how many people actually read the comment that got 50 upvotes — then you'll know if it's truly interesting, but too long to read — only 50 people have read it. Or if it's boring — 1500 people have read it.

Perhaps a good sorting algorithm would be: Lower bound of confidence interval for:

(upvotes - downvotes) / num-people-who-read-the-comment

Letting different comments reach the top wouldn't be such a bad thing most of the time, as long as there is more than 1 solid comment.

Only time this would be bad is if there is something important, such as a correction or a key reply by the author or the person they were calling out etc where it makes sense that everyone should read first.

Knowing how many read a comment would help with comment assessment but how do you get that information. I suppose we could track scroll positions, I'm not sure how reliable it would be.

It'd be cool if websites could use eye tracking, then we could easily tell what got read. Maybe in the future.

The problem is that kittens are quite popular. They are more popular than a well formed discussion. What algorithm separates kitten posts from quality content? How could you tell the difference between a great photo and a lolcat? I think the only way to do this is to empower moderators. Any other scheme is just a race to the middle for bland, unoffensive and easily digestible content to maximize points.

Wilson score confidence interval is easy to implement and already proven to work greatly on Reddit, where it is the default sorting algorithm. Do note that Reddit also has down votes and Reddit also has threaded comments. While I agree, it's not the "perfect" solution, but for a perfect solution HN admins would need to invest a lot of time (and it would probably be a lot more buggy and in the end they probably won't come up with something that's better). Going with Wilson they can do a quick fix that would improve the sorting a lot.

I can't speak for anyone else, but I've been consistently more happy with the comment rankings on HN as opposed to Reddit. This could be something to do with voting behavior, but I think it's at least partly thanks to HN's ranking algorithm as opposed to Reddit's.

I'm actually using the algorithm that Evan Miller writes about (i.e. the lower bound of a binomial proportion confidence interval), in the discussion system that powers the demo in the linked article. I'm using the Agresti-Coull interval, http://en.wikipedia.org/wiki/Binomial_proportion_confidence_..., however, because it seemed easier to implement. (I didn't know about Evan Miller's article (or any other).)

Reply on news.ycombinator.com