Hacker News Reader

I don't think you can fix these problems without totally changing the way HN works. The "News" in "Hacker News" means that current topics will always be prioritized over a long term discussion, pushing relatively healthy topics off the front page. The threaded Reddit-esque conversation model makes it incredibly painful to follow any modest conversation or even just read them after the fact. That comments are ranked in addition to this, unlike a mailing list, just makes it even harder to follow, because everything is constantly jumping around the page. The upvote system gets into your subconscious; as much as I tell myself that the points don't mean anything, it's hard not to feel bad when your highest rated comments are snarky one liners and the ones you actually put some time into rarely get any. Sometimes I find myself spending 15+ minutes writing a comment just to delete it because it "isn't good enough," despite being relevant to the thread and possibly helpful. We pay lip service to the benefits of long form writing and thoughtful conversation on a site that is designed to bury them.

The Reddit discussion model is completely oriented towards providing the fast food of news commentary, where your enjoyment of the first look you take at a comment page is prioritized over active participation. That's great if you're trying to attach ads to pictures of cats, but if you're trying to have a real conversation, it just gets in the way. I can read, I don't need an algorithm to tell me which comments are notable and which ones are junk. If I see a troll post on any other site, I don't seize up and have an aneurism, I just scroll past it.

I've said it before, and I'll say it again: I can easily follow 50+ page PHPbb threads that drag on for months or even years, or 4chan threads with hundreds of posts and more on the way every time you refresh, but a small conversation (in reality; I considered it huge by my HN standards) I had on here the other day with about 10 comments or so made my head spin. I wrote half the posts and it was still hard to follow! Oh, and of course, no one but the people I responded to will ever read the things I wrote hours after the thread dropped off the front page.

I think that the community, not the particulars of the software, is the reason for this site's success, and things would have turned out the same or better if it were a barebones BBS with a flat discussion model. You don't need complex ranking systems or update notifiers to browse those, you just need to be able to scroll down.

There are indeed many proponents of flat discussion models. The most famous of them is probably Jeff Atwood, cofounder of Stack Overflow:

http://blog.codinghorror.com/web-discussions-flat-by-design/

But one major disadvantage of a flat BBS is that after a while, it becomes difficult to keep track of who's replying to whom. This is especially dreadful when two or more discussions are going on at the same time and they get interleaved; it's catastrophic when the moderators merge several loosely related threads. That's why most forum users make extensive use of quotes. Without quotes, you quickly lose context. So after a few pages, half the text on the page is made up of quotes nested inside one another. A forum archive will probably compress very well ;)

Of course, this can probably be fixed with a better UI. But if so, the threaded discussion model could also be greatly improved by a better UI. So I think it all ultimately boils down to how good the UI is. The ability to highlight new comments and collapse threads, for example, will probably help a lot. I also like the way Reddit notifies you of replies. It helps you avoid missing replies in the mess of a large thread.

> I also like the way Reddit notifies you of replies.

This is one thing I wish HN did. This is a good post, and I care far more about the replies I get here (when I have generally at least tried to provide thoughtful input) than the ones I get on reddit, but those confront me more directly as a result of the UI.

I use http://hnnotify.com/, it seems to work pretty well.

Me to. Same thing with mobile. There are services/apps/sites for it, but you need to know the possibility exist. I see requests for 'notify on comment' and 'mobile-view' pop up every once in a while. I understand HN doesn't feel a need to provide extra services beyond this site, but it would help a lot of (new) people if there was a way to show them that, yes, you can receive a notification and you can have a good mobile experience on HN.

I just use the "Threads" link at the top of the page.

Yeah, I feel that pain when browsing PHPbb-style forums and reading mailing lists, but the 4chan/2chan/futaba UI solved this problem over a decade ago.[1] Each post has an associated number, and clicking it inserts a >>[post number] link into your comment. You can then hover over the >>links in any post to see a floating preview of the post in question, or click the link to be taken to the post directly. Any links to your posts have (You) next to them to make it easier to know when someone is replying to you. At the top of every post is a link to every post that linked to it. In addition, if you're replying to a large post, or one that contained multiple side conversations in it, it's common courtesy to include a small literal quote under the quote link.

Put it all together and a typical thread might look something like this:

  Anonymous No. 1   Replies: >>2, >>3
    I like posting here a lot better than posting on Hacker news.
  
  Anonymous No. 2
    For what reason, >>1_(You)?

  Bob Saget No. 3   Replies: >>4
    >>1_(You)
    I don't really like multi-threaded discussion models either. Way too hard for me to follow.

  Anonymous No. 4
    >>3
    >I don't really like multi-threaded discussion models either. Way too hard for me to follow.
    I strongly disagree, you are stupid.

With this system, I find it very easy to navigate anything from month-long threads with medium to large sized posts and very linear conversations, to threads that last a few hours and accumulate hundreds of small to medium sized posts that span dozens of simultaneous conversations.

>I also like the way Reddit notifies you of replies. It helps you avoid missing replies in the mess of a large thread.

4chan has a few similar systems. You can set a thread to automatically update every few seconds, or you can "watch" a thread so that even if you close the tab, you can see when new posts arrive. Also, in addition to the (You) next to links to your posts, when a thread updates, any posts that link to your posts are colored differently until you scroll past them. So these systems make it easy to follow entire threads in progress, not just the people directly talking to you.

I totally get that 4chan and company are not everyone's thing, but I don't understand why very few sites that aren't already 2chan or futaba clones borrow elements from its design. They're very usable while being minimalistic and low-tech; even without javascript and most of the goodies I mentioned disabled, its still very easy to use. Meanwhile, web forums are huge and clunky and filled to the brim with verbatim block quotes and obnoxious avatars, the average mailing list email is an enormous email quote followed by a single line of commentary, etc...

On the note of the rich getting richer, it isn't only when someone arrives in a thread. Average comment score is also a big factor. The way the algorithm works is that late arriving commenters with high average comments scores can often jump ahead of older comments from users with lower comment scores.

Once example, I created a new HN account a little over a year ago to switch from a slightly unprofessional name to my current one. Some of my early comments got upvoted a decent amount giving me a higher comment score. That higher comment score led to appearing closer to the top of comment chains. That in turn resulted in more visibility and again more upvotes. I don't think the quality of comments is any different between the two accounts, but the comment score on this one is 2.5x higher simply because I started out with a lucky advantage.

I am not sure if people will view this as a problem, but I certainly wasn't aware of how big advantage having a high average comment score was until I had one.

Having a high average comment score helps a lot (maybe a bit too much as you've said) but AFAICT the average is only among relatively recent posts.

I'm pretty sure it is something like the average of your last $X posts that matters rather than average score of every post you've ever made. So while it may still result in a "rich get richer" situation, it isn't quite as dire as it would be if your entire post history worked into the average.

The last mover advantage of waiting for other people to do the heavy lifting for them and then swoop in for the win.

I have strong recent evidence of this in my own score, and It was an interesting experience for me. Lets just say my average was less than one for a while, but then climbed back quickly over the last 72 hours. Its an interesting system that really punishes the type of comments that get downvoted, and while I was upset about it for a few days I can see that it is a good system.

>...to switch from a slightly unprofessional name to my current one

I suppose it's not technically possible, but I for one would not mind a possibility to update your user name (even if only once).

I think that is a feature of limited utility. In most cases where I would want a new username, especially in the case of wanting to appear more professional, I would want a new comment history as well as a new username.

I really wish, instead (or in addition), that there was a "new" indicator so that I could quickly see recent comments. This won't do a late comer any good as all posts appear as new, but for those engaged in the thread, it would help them be able to scan and read any new comments since they last visited.

And/or indicate posts made in the past X minutes.

Maybe these could be toggle options to quickly show/hide or just shrink/lighten to the background older/read posts.

This is something I've been wanting since day one, but may impact performance too much, unless they handled the indicators/toggle client-side based on a last read timestamp cookie (or similar implementation).

Some of this (especially the new post indicators) can be provided by a Chrom{e,ium} extension I've been using called Hacker News Enhancement Suite [0].

[0] https://chrome.google.com/webstore/detail/bappiabcodbpphnojd...

I just installed this. How do I know when a comment is new? I just looked at a few threads i hadn't opened before and I was expecting all of them to have the "new" UI element but they all looked normal (other than the new UI that the plugin brings for all of HN)

edit: and as I post this, my own post is marked as new. For those interested, new posts seem to be denoted by an border on the left

One new experiment we're currently working on is something like that for stories. If it works for stories, we may try a variation for comments.

Maybe this is what you're thinking of, but instead of randomly altering the actual score, how about randomizing just the position on a per page view per user basis?

I.e. a particular comment/story has it's normal score + X% chance of being promoted to somewhere near the top. You can vary X depending on age & actual score.

This seems much easier to implement, and many more stories/comments would get exposure (each to fewer users).

So there is something that is different with the ranking/presentation of stories?

Recently I have noticed that the front page will look dramatically different (story wise) than it did five minutes ago. By dramatically different I do not mean story #3 is #15 five minutes later, but that stories #2-12 will be completely different some of which I remember reading 8 hours before.

No, the experiment I referred to hasn't been rolled out yet. We'll post about it when it does. In the meantime it's mostly moderation as usual.

So the method for listing stories on the front page has not changed recently?

Correct.

Correct...dfc, you are going crazy.

While you are here I should take the opportunity to thank you for making my favorite corner of the internet even better. Thanks dang...

You're right, the problem is especially acute for story submission.

Once an article is on the frontpage, it is likely to stay there. People are much more likely to upvote a frontpage article than something in new, simply because they are far more likely to look at the frontpage.

What you really want is to estimate the probably that a reader will click and/or upvote a link, not the total number of clicks/upvotes. You might also want to discount this by time, but this discounting might not be necessary---there might be a natural decay in the probability, particularly if you compute a moving average.

If you'd like my feedback about how to model this, feel free to email me. I'm happy to provide you with a more concrete methodology.

Have you considered, instead of basing rank on upvotes/time, basing it on upvotes/views? Views could be either views of the actual link, or just views of the listing page containing the story, or a combination of the two. (Of course absolute number of votes would still have to factor in somewhat so brand new stories with one vote don't jump to the top.)

Perhaps this is already happening, but my (probably simplified) understanding of the system was that it only took into account time since the story was posted, not the number of views it's had, compared to votes.

That's an interesting approach; it would cause comments that are really good to float to the top no matter when they are posted. In terms of Bayes' theorem:

    P(upvote | good post) = P(good post | upvote) * P(upvote) / P(good post)

Let's neglect that opinions vary, and assume P(good post | upvote) = 1. P(upvote) should be proportional to P(post is read); but the readership at the bottom of the page (or wherever latecomers are dropped) is quite small.

But this begs the question: what does it mean to "float to the top" if highly-rated comments are being placed in a random location?

See also: Reddit's "best" ranking for comments: http://www.redditblog.com/2009/10/reddits-new-comment-sortin...

Yes, every commenting system which has an 'upvotes' system should learn from reddit's sorting, because it's the only major one I've seen so far which takes care of this 'problem' of early comments get more votes, later ones get buried.

Would it be possible to use that algorithm and get the same sort of 95% confidence here given that many people can't downvote?

Is there a resource on how much karma you need to do what on HN and what the username colors mean? I assume green is new, blue is a couple months old and grey is even older?

Not that I'm aware of. Also, I've never actually seen blue, somehow. I believe that downvoting comes in around 500 at this time.

But green is not reliably "new". It tends to be strongly correlated to newly created accounts, but I've seen weird behavior (like a child comment is marked green... whereas the comment thread was started by the same commenter in gray in the first place), which leads me to believe there's something subtler going on. I thus have no idea what the difference in color is supposed to indicate at all.

It would be cool if HN opened up a read API so that folks could build alternate frontends / scoring algos.

This is planned.

Cool. Looking forward to it.

Reddit's algorithm gets around much of the brain-damage of simple vote counts or capped moderation (e.g., the Slashdot method -- utterly broken).

What it doesn't address is the quality or relevance of the upvotes themselves. Spend much time on reddit and it's clear that there are a great many people who are voting on stories and content with little regard to its significance or truth.

As a measure of popularity or agreement, the metric is ... better than most. As an indicator of true quality it continues to inherit the faults of virtually all online moderation systems. Absent either creating a class of users who are considered "trusted moderators", or a means of finding agreement between a given reader's preferences and those of the raters indicated, that's pretty much a persistent issue. The better subreddits, in my opinion, address this through guidelines, posting policies, and very strict subreddit moderator policing: removing nonconformant or low-quality posts, banning individuals who troll or don't follow posting / commenting guidelines.

Even subreddit mods cannot change the voting on submissions or comments themselves (nor can they see who's voted on what), but they can influence what is visible within the sub -- editorial prerogative.

> The better subreddits, in my opinion, address this through guidelines, posting policies, and very strict subreddit moderator policing: removing nonconformant or low-quality posts

A moderator deleting someone's comment should be the last resort. It's irritating to spend 5 minutes writing a comment in a subreddit, press post, then see "the comment you're replying to no longer exists" because the moderator decided deleting a thread of 10 comments was more appropriate than taking some time to write a comment with their own opinion. It's also irritating when one's own comment with 4 upvotes is deleted while someone else's comment with 6 downvotes remains untouched although collapsed. This recently happened to me twice, a week apart, on a subreddit I'd been visiting intermittently for the last 4 yrs, and very occasionally commenting in, without any trouble.

> Even subreddit mods cannot change the voting on submissions

Perhaps this is the real reason a subreddit moderator deletes upvoted comments they don't agree with, while leaving downvoted ones. Downvoting and replying should be the means to control quality, with a moderator deleting a comment as the last resort.

Some sub reddits make it clear that they delete a lot of comments. (Askscience is one example). This is great. A well crafted answer to a deleted post is still useful - someone has a mistaken idea about science and that is deleted while your well crafted post corrects the myth.

And in some subs it is iust a lot easier to delete posts that are almost trolling than to allow a large angry thread to spawn. Often those are the same tedious meta topics, or very simple beginner topics, that have been covered very many times and which are not interesting but which are flamebaity.

AskScience and AskHistorians regularly delete comments. Both are intended for the serious discussion of the topic at hand. Hell, they'll even delete comments which are serious but don't reference their sources. I've had this done to a few of my own comments. The mods are fair-handed about this, and point to the rules. The result is a good experience and high level of discourse.

The tune's a bit different in /r/economics, where while posts aren't deleted, there's a very strong bias toward economic orthodoxy, despite the many, many, many flaws it contains (full disclosure: my degree is in economics).

For my own subreddit, the focus is on intelligent discussion. I make no bones about deleting trolls, or even just vapid comments. I'm not looking to win any popularity contests.

I thought HN already did this. At least new comments usually appear at the top of the thread and then gradually fall if they don't get any upvotes.

Someone made a post about your suggestion here: http://danluu.com/randomize-hn/

And another person suggested an even better algorithm here: http://www.bayesianwitch.com/blog/2013/why_hn_shouldnt_use_r...

Yes. That is what they do. Or used to do.

"It should be easier for a late-arriver on a post to add a useful comment, and have it be promoted"

That'd be nice, but not important enough to the discussion for me to recommend a feature. It'd just encourage threads to last even longer and with even more bickering.