Hacker News Ranking Algorithm
vigneshwarar.substack.com
vigneshwarar.substack.com
If there's more of a ranking algorithm, identifying who is most likely to have actually read the article could be neat.
Wouldn't that encourage people to "actively engage" even when they have nothing to say just to keep their score?
Goodhart's law comes into mind here. People who want to stay relevant on this site will take your measure as their target. There's no system that can't be gamed if you know the rules. But the more "rock solid" you try to make those rules, the more you embed intrinsic discrimination between participants. You put "Proof of Work" in the rules and you alienate newcomers while all the rest do "busywork".
Your own comments and submissions would sink like a stone because you didn't engage enough, regardless of how interesting your points are.
> Will I ever publicly write about how HN ranks posts if I am Dang (HN moderator)? No, because Pagerank can be manipulated by people despite its reputation. In fact, Pagerank is being exploited for years. Moreover, there are financial incentives for companies to get on the first page in HN.
Lack of transparency is not a solution, and certainly not one that will sell well in a place where "security by obscurity" is disdained and open source cherished. It's particularly bad when you're dealing with a high profile site filled with intelligent people who would most likely make short work of reverse engineering just enough of the algorithm to understand how it works.
> Reading time for the article.
Thinking about understanding the content of an article and seeing who's most likely to be referencing that content in their post is more of what I had in mind.
I know I'll be out of step with most who comment on HN but I question the necessity of having a ranking/voting system at all. All it seems to do is to act as an ego booter for commentators and it messes up orderly discussion when people insist on posting replies to the first commentator.
As a news aggregator, no doubt HN needs some method of ranking stories to set their listing lifetimes and their importance (page ranking) but that could be accomplished by the number of clicks a storey receives and or how long a visitor/commentator stays on a story (or the time between link clicks).
I also question aspects of the voting system as currently implemented. It's not unusual for me to receive down-votes on comments to controversial stores but not receive a comment from the down-voter. Why bother with this system if the voter hasn't the gumption to say why he/she disagrees with one's views? (One could discuss the psychology and social philosophy behind this but here is not the place).
Another annoyingly aspect of the voting system is the lack of stats. The time I take from posting to checking to see if someone has replied varies wildly—from minutes to days to never. This wide variation is significant when I post to controversial stores. Days later I can check back to see if there's been any votes or replies only to find there has been none.
At other times, I've monitored the incoming votes almost in realtime and watched them oscillate around 0 or 1, that is I've received many up and down votes but the long-term average is zero. Come back and check days later and one hasn't a clue from the stats if anyone at all has actually read one's comment let alone the fact that it was equally controversial to both sides of a polarized audience. HN should allow users access to these long-term stats.
I also question the concept of 'hiding' comments. Are my eyes so delicate that I can't see certain comments by default? It seems like just a feature that exists to bias subsequent viewers against comments.
It'd be really interesting to see a system where voters have both one upvote/downvote for the submission and the ability to upvote one comment and downvote one comment.
There is only a single up/down vote. You can't upvote a thoughtful comment for its quality, while also downvoting its conclusion. Sometimes I have strong opinions about something; but I also try to educate myself with the other side of the issue by reading thoughtful articles and/or comments from people with different points of view.
It would be nice if you could choose between two different 'modes' in the interface. One where the highest quality comments are near the top. The other where the most popular viewpoints are prioritized.
@dang, as a Arc expert, let me assure it's only 4 lines of code, 1 field on db, 52 bytes of js, css, and html, and it won't cause performance, security, accessibility, privacy, or usability issues /s
I was going to say that the HN guidelines advise doing this, but I cannot actually find anything about what an upvote 'means' on HN. Plenty about what makes a good submission, or a good comment, but nothing on voting. Maybe they just gave up on trying to do that!
The above notwithstanding, I would like to see something like your proposal implemented, just to see how it works out empirically.
1. As an "agree" button - you upvote to get the perspective you share to be displayed more prominently to other people. It is a way for you to influence other readers.
2. As "it was useful to me and a good use of my time" button. You use it because you think other users will find it informative as well. In this case you are doing purely for the benefit of others.
3. As "it made me laugh or evoked some other emotions" button.
All three could be viewed as different dimensions of content and one item/comment can have different combinations along these dimensions.
It is complicated by the fact that content creators are actively optimizing their content along these dimensions to gain more visibility.
When I come to HN, as a reader, I want to see mostly informative content - which is only dimension 2. I don't want to be influenced or persuaded. And while I enjoy a good laugh - that's not my goal coming to HN. This makes the other two content dimensions contribute to noise.
Your suggestion to split out the upvote into separate labels like "agree", "informative" and "funny" [1] and to let users sort by the dimension of their choice may not solve the problem of separating "agree" from "informative" because there is no incentive for the voters to vote truthfully. People who upvote content they agree with want other people to see that content. If most users are sorting by informative then those voters will upvote content they agree with using the "informative" button.
I think you need to change the system of incentives in order for voters to upvote only informative content and not 1 and 3.
One way to do this is to make what you upvote have more consequences for the type of content you see in the future and less directly affect the ranking of that content for all users [2]. With a feedback loop like this the upvote button is no longer "what content do I want other people to see", but "what type of content do I want my future self to see". It makes you think before voting - "Was it worth my time?", "Is this kind of content aligned with my long term goals?"
[1] - Slashdot has something like this: "funny", "interesting", "insigthful" https://en.wikipedia.org/wiki/Slashdot#Peer_moderation but I don't think you can sort by it. Also, only a subset of users are given these moderator points.
[2] - I'm building https://linklonk.com that works this way.
In practice, it doesn't really seem to affect outcomes that much, but it's a cute feature.
Where I have more trouble with is with comment-ranking within an article. The interface makes it hard to understand conversational flow, and of course that flow changes over time according to whatever the algorithm does. This makes it very hard to catch up later on an article’s comments and get a view of what has changed since my last view. Maybe there is a way to do this via the interface, but if so it’s not obvious.
I don’t know the status of it, but it seemed like an interesting idea.
My biggest complaint related to this is that all threads are dead within 24 hours, which kills a lot of interesting discussion. And commenters have no incentive to reply to dead threads.
This discourages in depth back and forth conversation which I usually find is the most interesting.
TBH, I am not sure if there is a good non-invite-only forum that facilitates that.
In other words, looking at karma won’t tell you whether someone replied to your comments or not. There is nothing on HN to indicate replies unless you dig into your comments history. There nay be other sites or browser extensions that could help on getting notified.
Absolutely not, think about the incentives this creates! Imagine yourself to be an adversary, how would you game this system? Easy: spam low-effort highly agreeable comments across the site. The more the better, they're basically guaranteed to get some upvotes and boost your karma. Now, with your newfound power, you can shape the HN front page to your will.
Instead of the front page being a signal of quality, it would be correlated to the posters' karma. And because the front page is a limited resource, you basically get a crappier version of Twitter where relatively low-karma accounts have practically no chance to get on the front page even with high-quality posts, and high-karma accounts will be able to roll their front-page posts into even more karma. This is pretty much one of the biggest flaws of major social media websites today: they concentrate power in a few users with massive influence.
Is there an official reason? I’ve wondered this as well and sort of assumed it was because hackers like having some community aspect and are novelty-seeking so YC created this due to the largesse of PG for the YC group and someone said “why not just make it public so applicants and future YCers can participate.” And job postings were added on to generate some revenue so it wasn’t a pure cost center.
This headcanon reenforces my belief that the best community sites are accessories to something else and that you can’t make money from a community without diminishing its value (since the incentives for monetizing conversation are not the same and I’d guess 90% at odds with good conversation- mainly because bad things draw attention).
> Initially called Startup News or occasionally News.YC.
HN exists to serve the interests of YC. It’s a marketing/advertising tool. It’s great to believe that it’s a community, a place where interesting conversations happen, etc. (and those things may also be true) but remember what it is at its core.
But what is interesting? To you and I, perhaps, it is an intellectually well thought out comment.
To YouTube, perhaps, it is what makes people engage emotion.
What's more: if you can't stop bots from dominating your signal, then directly predicting things like quality may be one of the only good options left.
Meaning give more posts a chance, a certain number of views to determine the upvote rate.
Conventional wisdom would apply spam filter, boost new users (e.g. first post), detect blatant manipulation (or more sophisticated manipulation using clustering).
ML to make the recommendations more dynamic per user means people are not seeing the same things and engagement tanks.
ML to make the recommendations more global means a lot of people are going to see stuff they don't care about and they're going to have no idea why. It's usually this family of problems that ML adds an extra layer of opacity to the most.
Then you add extra tabs to give the user more control, and now you've added a fresh complexity trade off.
The sanest aggregators have simplistic recommendation algorithms and a bunch of janitorial work on top in the form of moderation/censorship. There must be something to this formula.
Link: https://www.righto.com/2013/11/how-hacker-news-ranking-reall...
The metric in action: https://news.social-protocols.org/
HN Discussion: https://news.ycombinator.com/item?id=35183317
It's a tool where you can create your own HackerNews clone. Obviously, while working on this, I had to emulate the HN ranking algorithm. But what I found is that the algorithm actually doesn't seem to work well for a "young" forums.
I had to tweak it so that on the frontpage, the posts that are more relatively recent rises to the top over older posts of higher votes. I know these are considered in the original algorithm, but I had to tweak it.
Which made me think that the algorithm should be considering the age of the community and also how active it is.
The entire site is officially moderated by ~2 people, and autonomously by the community. Whatever secret sauce they're using, that should be proof positive that it works. Keeping it a secret deprives countless communities of best-in-class tools and knowledge.
Would it be detrimental to HN? In the short term, possibly (gaming the voting ring detector comes to mind), but it's hard to say what kind of impact there would be without knowing what goes on under the surface.
its cohorts ranked 270 to 282 (by my screenshoot) are of age 1day+ and with ten or more comments
it's weird that post which is new, popular and engaging then sunk to the bottom 275 very fast within hours
here's the link: https://hnrankings.info/35078392/
it shows "update to kagi search pricing" started and stayed in the front of hn for 1.5 hours
it then left the front page and raced to rank 300 within the next 4 hours
the disturbing thing is that this hn post with id 35078392 generates 116 points with 132 comments
poof, just like that
""" Since it's likely that one user may upvote multiple comments from the same user, we check whether a user has already upvoted a comment from that specific user before considering their upvote. In other words, we treat user profiles as nodes and upvotes for comments as edges. """
This is a very lossy conversion of the actual data:
a. It does not distinguish if user C and D upvoted the same comment of user B or not. Maybe one comment was good and the other was bad. But when you convert it to the above graph you only get that C and D upvoted some comment of user B.
b. It does not account for the number of comments user B left - 1000 or 5? This incentivizes spam because there is no upside to not posting.
c. It ignores users that do not comment but upvote valuable comments themselves. Then the PageRank is used to weigh upvotes of users. This is backwards. The PageRank values should capture the value of each user's past upvotes in order to use it as a prediction of how valuable their future upvotes will be. But the suggested algorithm uses the value of the user's past comments as a weight of their future upvotes.
To fix this I think the graph needs to be changed to a bipartite graph of users and comments as nodes and upvotes and flags as directed edges (when the author posts a comment - this should be represented as an implicit upvote of the comment). Then you can calculate how valuable each user's upvotes (and flags) are.
2. The "gameability" of PageRank stems from the fact that the random walk algorithm treats each users equally as a starting point. It means that you can create a ton of fake users and upvote the comments of a target user you want to artificially boost in upvote-power. Each time the random walk starts at one of those fake users the walk will end up in the target user - increasing their PageRank score.
My proposal to solve the "gameability" problem is to start each walk from you - the user that views HackerNews. It means that your past upvotes become the starting step of the random walk and so the resulting PageRank will be personalized for you. Instead of a single PageRank reputation score (which captures how user A's contributions to HN have been to all users), there is a set of personalized scores that capture how useful other users have been to you.
I'm building https://linklonk.com which uses this kind of algorithm to rank both links and comments. The details of the ranking algorithm are here: https://linklonk.com/item/3292763817660940288
Edit: I suppose there is some insinuation that HN doesn't really use the original algorithm, and this is the author's attempt to reconstruct what's going on. Is that what I was missing?
In my personal experience, YouTube and Windows 11 are some recent examples of usability degradation due to unnecessary changes.
Post ranking works pretty well, but as far as discussions go I often see insightful responses sent into low-contrast oblivion whilst a glib comment or quip gets accolades.
In my experience, HN seems okay for reading and participating in meaningful discussion. Comments that don't contribute or are inappropriate don't stay in sight for long (possibly due to the community upholding the guidelines), whereas useful discussions and references to projects tend to stay towards the top. I don't always agree with what sits at the top, but if nothing else it's civilized and reasoned.
In some respects HN has the typical issue of most attention focusing on the first page posts and most discussion being in one or two top level comments, but that's just how people engage with the content, short of randomization there's no way around that. There will always be a top comment and everything else will receive slightly more lukewarm reception and activity.
1. There seems to be a group of people that abuse the flag button to get stories downvoted. You notice this when highly upvoted stories with a lot of points suddenly drop from the top and fall quickly beyond older stories with many fewer votes. For instance, there seems to be a group that was mass flagging submissions about Rob Lee's recent death until one was made that didn't mention his cause of death and explicitly forbid discussion of it in the comments.
2. Partly because of #1, you can have a topic with a lot of votes at the top of the page suddenly drop to page 5 and disappear within a very short amount of time. This significantly lowers the incentive to spend any time writing a thoughtful response or researching useful information, because even spending 20-30 minutes on a response might mean that no one is going to see what you wrote.
I’m not sure that I’d characterize this as abuse. HN stays interesting by avoiding flame bait topics and part of the realization of the community guidelines happens via this flag feature.
One way to reframe #1 is that the community’s self-moderation features are working as designed, and that any post about Bob Lee’s death that somehow editorializes it or positions it in a way that is likely to lead to political debates is against the site guidelines and thus were removed appropriately.
Regarding #2, this has happened to me more than once, and while frustrating, it made me think more about posts I encounter before commenting. When a post fits the criteria for “potential flame war” or “clearly outside of site guidelines”, I flag it instead of commenting, even if I know that at the core, the topic could be an interesting one. Most things get posted more than once, and usually at least one of the posts is within guidelines.
When I do lose a comment to a flagged post, I’ll reuse it in a more level headed discussion about the topic.
A lot of discussions on HN lead to discussions of government policies (both U.S. and foreign) when there's a connection to the story, including saying that certain policies lead to the death of individuals. There's only a subset that gets mass flagged. If he was shot to death by police at a protest, killed by a U.S. military strike in another country, died as a result of a mass Covid surge, stories mentioning the reason for his death wouldn't be mass flagged. People wouldn't be forbidden from mentioning the causes of death on the only one that isn't, and it wouldn't be dismissed as "political flamebait."
If Hacker News decides that discussion about government policies is off limits, that's fine. It might even make discussions better. But allowing it in some situations and having it "removed appropriately" in others is only reinforcing bias. I think characterizing that as abuse is appropriate.
Edit: If you want to see an example, look at the Hacker News discussion about the death of Aaron Swartz. The cause of death isn't hidden, and there's a lot of criticism of the government, none of which is getting dismissed as editorializing or political flamebait[1].
The initial post about Bob Lee was split off into two, so one could remain focused on remembering Bob for who he was [0], and the other could explore the surrounding circumstances [1].
With that said, the site guidelines are pretty clear:
> Off-Topic: Most stories about politics, or crime, or sports, or celebrities, unless they're evidence of some interesting new phenomenon ... If they'd cover it on TV news, it's probably off-topic.
> Please don't use Hacker News for political or ideological battle. That tramples curiosity.
So this will often boil down to the framing. If the title is tinged with political implications, that's quite a bit different than mentioning the cause of death.
The circumstances surrounding Bob Lee's death are relevant to quite a few readers here, and so there was a discussion about it. But a default assessment of any given submission if weighed against the guidelines should assume that such content is off-topic, unless there's a reason to believe otherwise (as there was in this and cases like the one you cited that are close to the tech community).
Even with that separation, there was quite a bit of discussion that was clearly far outside of the guidelines in that thread.
> Given that discussions are my favorite part, why don't we apply PageRank for every user based on the upvotes they receive for the comments they leave on any post, and replace the P (points) value in the current version of the HN algorithm?
Because that would lead to absolutely terrible longterm outcomes due to obvious reasons.
Well, the article is something of an ad. The author discusses current ranking and effectively proves that it could be gamed to an extent ( author links his non-SEO search engine, which obviously you cannot do unless you are deep into SEO stuff, which includes HN ranking algorithm ). The complaint is mildly self-serving, but only mildly so. FWIW, it worked on me.
Still, I will echo what others said. Despite my grievances with HN at times, don't change what ain't broken.