Comment Ranking Algorithms: Hacker News vs. YouTube vs. Reddit

When looking at numerical experiments like this, it's critically important to carefully think about the exact details of the simulation rather than take the result at face value. It's fairly easy to get any result you want by cherry-picking model characteristics.

Just taking a quick look at it, the utility function strikes me as suspicious. No value of freshness? Of course this will penalize youtube who does value freshness.

Another thing:

> they stop reading or keep going based on a probability that depends on the vote they just gave (0% for upvotes, 50% for downvotes, 15% for non-votes)

how is this justified?

> So YouTube is marginally better than random, Reddit is worse than the simple difference, and Hacker News is the only one of the three better than Bayesian average. Disappointing but also plausible.

There used to be a lot of gold among the trash in YT comments back in the day, but for some reason YT will show you spam comments from two days ago over comments with hundreds of likes from a year ago, so it's become literally impossible to find those.

They don't allow you to sort by votes and their "best comments" sorting is complete BS.

Nowadays I just block YouTube comments out with uBlock Origin, reading through them is rarely worth the time and is usually just a pointless distraction.

For anybody scratching their head as to why you don't just subtract downvotes from upvotes, or similar obvious techniques, a discussion here (also linked in the article in the OP):

https://www.evanmiller.org/how-not-to-sort-by-average-rating...

Just a suggestion, instead of using the average upvotes per visitor, how about something objective and not directly tied into the model itself? For example, we could look at the average Pearson correlation of the rankings over all individual trials.

That still wouldn't factor in freshness, though.

Comments are universally shit. My system is to rate them all -inf.