Collaborative filtering ... has issues. Most simple or single-factor fixes don't address the larger problems.
Much of the time, it tends to amplify interesting, and diminish nonproductive contributions. Spam, trolling, clickbait, and vacuous responses seem to get quickly downvoted and flagged on HN, which is good. Comment voting has numerous other connotations, however.
- Simple and short expressions tend to be disproportionately upvoted. They're easier to assess. Sometimes they are insightful, most often, only tapping a common sentiment.
- Controversial views tend to be downvoted. Sometimes controversy is flamebait, sometimes it isn't. There's not much distinction made between the two.
- Truth isn't a popularity contest. HN often, but not always, avoids this trap. True expertise in complex topics is rare, but at times necessary. An epistemic system should reflect this to a greater extent (though also be aware of misuses of such claims of expertise or authority).
- Topics strongly shaped by remunerative interests, ideological alignments, and tribal allegiances tend to perform most poorly under popular voting systems. There is a List of Things HN Cannot Discuss, and most of these seem to fall under this umbrella. Dang has voiced frustration with this on occasion, and I've had numerous email discussions with him about this ... more in the past than recently, I think. HN is guided to an extent by its moderator team. It is not directly steered by them. And the overall philosophy is broader than critics of the principle current political tribes seem to admit.
Collaborative filtering / the wisdom of crowds was all the rage in the late 1990s / early 2000s. Whilst it has some uses, there's been an overreliance on it.
I'm not entirely sure how best to patch the system, though I have some thoughts.
- Bounding ratings to a range rather than a simple sum seems more useful.
- Weighting individual's ratings by their own assigned mean similarly. If someone always hands out 5s (on a 1--5 scale), then those are re-scaled to a median score. Rescaling ranges similarly: someone offering only 1s and 5s might see that rescaled to 2 & 4 repectively.
- Factoring in expertise, or rescaling based on other factors such as truth, tone, conciseness, accuracy, references, etc., might help. Goodhart's law applies. But sometimes, what you want isn't the average opinion of the passengers, but a competent pilot.
Another point is that HN doesn't aim simply to be a place where truth is sought, but a conversational platform. Excessive strictness would probably diminish this.