The Hacker News New Page Scroll of Death and Some Possible Solutions | Hacker News Reader

The Hacker News New Page Scroll of Death and Some Possible Solutions

almostinfinite.com

58 pointscurtis46 comments

I feel that a lot of HN visitors do not at all check out the "new" page. How about another solution, where the front page shows a box on top with last 3 submissions from the new page, so that every new post get's at least some exposure, and get's a chance to get up voted once or twice. Each new post would be shown 2-5 times in the new box on the front page (depending on the volume of traffic to the front page vs. volume of new submissions).

So, for instance, let's say there are 9 new posts in the "queue" and there are 3 concurrent visitors to the front page. Visitor A will see links 1-3 from the new queue, visitor B will see links 4-6, etc. After being shown at least 3 times to front page visitors the link get's dropped from the new queue if it did not gather a single up vote. However, if it does get an up vote, that also resets it's show counter, and it has 3 more chances to gather another up vote.

The number of showings (lives) can be adjusted based on the ratio of new submissions to front page visitors. But I suspect that that ratio is more than high enough to accommodate 3-5 showings.

I feel like this would be a relatively easy system to implement, and could result in better overall content. Thoughts?

That would work great for people like me. I often forget that the "new" page even exists. Having the "new" stories in their own box separated from the rest of the front page would be helpful in considering these stories (e.g., one might be more skeptical of these stories).

I would make one suggestion: On each page load, randomly select three stories to display from those submitted within the last 60 minutes, instead of selecting the stories using the algorithm outlined above. If there is a problem with too many people submitting too many garbage/spammy stories, perhaps that problem can be solved by considering downvotes or flags (i.e., an affirmative action that the story isn't good) rather than a simple lack of upvotes (which could happen from people simply not taking a look at the story).

I personally think this is probably a good solution for visibility. If the issue is visibility of new submissions, random promotion of them to the front page seems like a valid solution.

Though it might also mean that scores will be more variable for new submissions, and that may have an impact on front-page/2nd-page variability and bounciness, which doesn't seem like it would be friendly to casual users.

Hmm, well the way I was imagining it, every new post will get the same amount of promotion, but it's true that it is random in that you never know who you are going to show it to. Some people might pay attention to the New Posts box, some may not.

We ran experiments with placing random items from /newest on the front page. It worked poorly. The median story is far too low in quality for this to work—you're basically dumping a bunch of junk on the front page. This doesn't encourage users to tastefully consider the stories, it just provokes annoyance: what is this junk doing on the HN front page? In a best-case scenario, people tune out the bad stories, but that defeats the purpose of having them sift for the good ones, and meanwhile you've partly spoiled the front page.

We didn't roll this out to everybody, but we did enough to conclude that it isn't a good way for HN to solve this problem.

Understandably, people would be upset if they had random articles from the new page inserted into the main feed on the front page. However, if the articles are in a separate box on top, and there are just 3-5 of them, they would have correct expectations. This way new articles get some exposure, but that exposure does not come with unreasonable expectations.

We did this on reddit (and they still do). It works really well.

I just use the comments page to find interesting discussions.

> I feel that a lot of HN visitors do not at all check out the "new" page.

My simple reason: It doesn't have an RSS/Atom feed.

I would just fold it it without putting it into a new box. It'll be more organic that way and you'd be able to tell if it's new by upvotes anyway.

Of course I don't look at the "new" page. It is full of crap.

I have it open right now and I count: 5 [dead] stories; 2 stories about Microsoft following Netlix's lead on leave; 2 stories from linkedin.com; Captain America appears to be joining Twitter as head of security (I didn't know that Hydra had threatened them); and no headlines that interest me.

That's a pretty selfish attitude, no? You get a lot, or at least some, value out of the HN community, and yet you do not contribute to it in the way that really matters. Sure, commenting on articles is important, but it's also important to at least some times donate your time and sift through the garbage on the "new" page and up vote the jams. How else do you think quality articles end up on the front page for you to read? Magical pixies?

With the system outlined in my original post everyone could very easily contribute to HNs ranking system without spending too much time on it. Worst case, you have to look at 3 crappy article titles, that's not that bad.

I assume the front page gets assembled from vote rings, favours from friends, and people stumbling onto the same stuff and submitting it.

I had one article up on the front page for a while, and I am pretty sure no one else cared to submitted because it's on a really obscure subject, and I didn't bother getting my "vote ring" involved in it :) Though I am pretty sure that having lot's of friends in SF with HN accounts would help get up on the first page. I wonder if HN monitors for this and ignores up votes if they notice a distinct pattern.

If that were true, we'd never get the best stories, which are the out-of-the-way but fascinating things that most of us haven't heard of before.

When I switched from lurker to a registered user and submitted my first link it hit the front page. Took me by surprise. The same happened again with my third submission a few months later. There were no voterings or favours from friends involved (I don't know anyone in real life who reads HN, let alone has an account), it just happened.

It may be worthwhile to reference my "If your HN submission doesn't get any upvotes, don't worry about it" note (https://news.ycombinator.com/item?id=9864254) The probability of hitting the front page is low at all times of day and all days of week. (see: http://i.imgur.com/MdUvMB9.png)

A separate /new list won't make a difference because it won't improve the average content quality. Not everything submitted to HN "deserves" to receive upvotes, to be perfectly honest.

Personally, I've had a number of submissions get upvotes after it's fallen off the first page of /new. I admit it's counterintuitive, though.

> Not everything submitted to HN "deserves" to receive upvotes, to be perfectly honest.

This is certainly true. It is also the case that some things submitted to HN deserve upvotes and don't get them. A case in point, this very article:

  Try #1: https://news.ycombinator.com/item?id=10007658
          ==> 4 points by curtis 17 hours ago | discuss

  Try #2: https://news.ycombinator.com/item?id=10011219 (this submission)
          ==> 42 points by curtis 2 hours ago | 18 comments

The obvious conclusion is that the first submission didn't get upvotes just because it wasn't very good -- it simply didn't deserve them. And the second submission got upvotes because it was better. It was quality content. But of course in this example, they're the same article!

Now let me be clear. I understand that it is the nature of this particular beast that there is simply going to be a lot of randomness. However, I believe we can make it noticeably less random with some fairly modest effort. And if we can make it less random, then I think that would be a good thing. Would the proposed "preview" page actually do that? I don't know. But I certainly think it's worth talking about.

> Personally, I've had a number of submissions get upvotes after it's fallen off the first page of /new. I admit it's counterintuitive, though.

It might be because of the experimental story-rescuing program I've written about at https://news.ycombinator.com/item?id=9866140 and https://news.ycombinator.com/item?id=8790134.

> Personally, I've had a number of submissions get upvotes after it's fallen off the first page of /new. I admit it's counterintuitive, though.

It's especially true on 'Show HN' and 'Ask HN' posts, since they appear for a longer period of time in the other sections.

> The probability of hitting the front page is low at all times of day and all days of week. (see: http://i.imgur.com/MdUvMB9.png)

Not entirely convinced that you can draw that conclusion from that plot. 10+ votes articles percentage might increase with the number of readers online, but so will the percentage of votes on other articles, hence 10+ voted articles != popular front page article. I think that 10+ votes may be a wrong way to look at it.

This is a problem we had to solve at reddit a long time ago. It's called the "rising" sort:

https://github.com/reddit/reddit/blob/c6f959504466333c0d7d51...

Let's be realistic, even when the number of articles coming in is really high, you're talking about 100-150 posts for an hour.

The HN algorithm has temporal-gravity to help pull down old content that wasn't hugely popular. Due to that algorithm if the content doesn't take off in the first hour, it likely won't take off.

Those 2 points point to a far simpler solution. Just show most of the items from the last hour on the new page. So instead of 30 items, you show 100 items on the new page, which should accommodate most of the recent posts without requiring any clicks.

(minimaxir's point is also very valid.)

> Just show most of the items from the last hour on the new page.

Someone else suggested this recently, and we've heard it before, too. We're game to try it as an experiment.

What's the best way to measure whether it works?

Well, the first step is identifying what the problem is. To me it's not obvious and a concrete statement of what we want to solve would help.

That is, we know it's that "articles that should be on the front page get overlooked"—but votes are the main metric we have for what should be on the front page. How can we tell an article that fell off /new because it was bad from one that fell off because nobody saw it?

One way of evaluating this would be to exercise editorial discretion, pick out articles you (or some set of people) believe are good and compare how they progress with different versions of the page. However, this is rather labor intensive and risks biasing the experiment based on who picks the articles out.

I can't think of other approaches off-hand. Coming up with a concrete statement of the problem would probably help here.

I agree. That's why I asked how to measure whether it works—it requires defining "works".

Upvotes alone don't determine HN's front page. If they did, HN would be dominated by controversy, gossip, and fashion—those get the most reflexive upvotes, and there are more reflexive upvotes than reflective ones. Coutervailing mechanisms include flagging and moderation.

The fundamental difficulty is that HN cares most about quality and one can't easily measure that or automate it. Human curation is needed, plus humans don't always agree about what counts as high-quality.

Anyone who wants to know about our current approach to this problem should read https://news.ycombinator.com/item?id=9866140 and https://news.ycombinator.com/item?id=8790134.

It might be interesting to be able to follow other users (who you know will tend to like the kinds of stories you want to see) and you can share saved stories or something. Or have related content or per-account filters that change the frontpage.

If, as it seems, there's no objective standard for quality (since it can't be measured or defined) then maybe subjective filters are the way to go.

I think there's value in HN having a shared single view of the front page and the threads. It means we all can't do the standard internet thing of self-selecting into subworlds that don't interact. This leads to friction in the short term—we're all exposed to things we don't like, some of which is uncivil and shouldn't be here, but some of which is merely disagreeable because it contradicts what we believe or like, and that's unpleasant. In the long run, though, my bet is that having a single whole community provides something deeply valuable.

On the question of how objective vs. subjective HN's notion of quality is, I don't think it's completely subjective. Everyone has their personal tastes, but HN's standards are reasonably well-established too. So I still think we may be able to come up with a story curating mechanism to supplement upvoting that is open to any user who wants to put in the effort. The big question, again, is how to score the effort. It can't be just "vote for whatever you like," because we already have that.

Determine if a nontrivial percentage of upvotes for a story are by users who are reading /new and the ranking of the story on /new at the time of the upvote is 30 or greater.

How about making the front page more sophisticated? Three buckets: popular, upcoming, new. The popular bucket is what you have now, and gets 25 slots. Upcoming gets 3 slots, and new gets 2. "New" is the two most recently posted links. If they receive an upvote or two, they get into upcoming. I think that'd help worthy links make it to the front page.

Look for a decrease in resubmissions, per this comment above:

https://news.ycombinator.com/item?id=10012199

I think many important entries get to the first page not because people upvote them on the "new" page, but because people keep on re-submitting them, where resubmition actually functions as an (unintended) upvote.

That said, it seems that HN has grown so big, many "top stories" don't get enough time for discussion before completely disappearing from the front page. It might be time (shudder the thought) to introduce channels...

I have long suspected that this is true for certain types of articles. If a new article appears on some really popular website like CNN or Slate (for example) and it is about some HN-worthy topic, there's a good chance that multiple HN users will encounter the article browsing the original source site and submit it from there. Now you've got a submission with 3 or 4 upvotes, none of them from the /new page.

Especially 'Ask HN' and 'Show HN' posts.

Allowing submissions only from users with a minimum karma (50, 100, 200, x) would be a better solution for preventing spam and flooding. Something similar to down-voting permissions that currently requires 500 points.

It would be also much easier to implement.

A few problems with that: missing a good story costs HN far more than tolerating a bad one (kind of like startup investing that way); users with higher karma by no means reliably submit high-quality stories; by closing newcomers out, karma thresholds heighten the risk of the community becoming insular, something we definitely don't want.

c'mon now, they aren't even about to fix the simple 'comment layout on mobile' problem. You really think they're going to go through the trouble of adding a whole new ranking system? :-)

> they aren't even about to fix the simple 'comment layout on mobile' problem

Why do you say that? We've put a ton of work into that problem, which is not quite as simple as you'd think.

Hey dang!

Just wanted to first say that I absolutely have enjoyed your work and time on HN, but from my experience as a user (important distinction here), the mobile interface hasn't changed much in the last few years.

The only thing I see is a slightly bigger triangle button and a whole lot of tiny tiny text. I completely understand that my phone isn't the target audience (S3), and so I use an Android app to make up the difference, but when I saw your comment I reopened HN on my phone to see if had improved since I had last checked a year ago.

Here's what I saw: http://i.imgur.com/CRonP5c.png I am NOT a UX expert (or even novitiate) but if you would, compare the size of the refresh button to the size of the upvote or comments buttons and notice the vast difference in area.

I took the liberty of coloring in the comment link in red, and comparing it to the refresh button: http://i.imgur.com/GZEs7cN.png and that it is less than 1/10th of the same clickable area.

Now, I am 100% not saying you should be targeting this button size or that specifically, but that would be a reason I would make this type of comment.

Hopefully that isnt too frustrating of a response, I assume there are a lot of factors going on that I am not aware of.

Thanks.

Sorry! I see now that my comment was far from clear. You're right that there haven't been major changes to the interface, but that's not because we haven't been working hard on it. It's because we haven't released that work yet.

We did make a couple attempts at quick fixes for HN on mobile that provoked howls of protest, so we rolled those back and have been working instead on rewriting HN's markup entirely. This is not a small project, and everyone working on it has other responsibilities, so it has taken a long time. It is, however, getting closer to being ready. That's why I objected to the statement that we "aren't even about to fix" it (and also that it's a "simple" problem)!

When we do have something ready, we'll release it for a while as a different set of URLs and start taking everyone's feedback about what still doesn't work right. I imagine there will be a long process of tweaking it, and we'll be happy to do that over time.

(Also, that was very nice of you to be so friendly even though it sounded like I was saying something silly.)

That sounds great dang, thanks for continuing to work on it even if there was a backlash. I will be interested to see what shows up in the next few months (hopefully) :)

Why do comments change font size when I tap their Reply link?

How about say... collapsable threads? It is the year 2015 after all.

One of the possible advantages of the "preview" page from the proposal is that it might draw more eyeballs to new submissions than what they're receiving now. This is because the design interleaves new submissions with submissions that have received some degree of filtering. I really spend very little time checking the new page, and I suspect I'm not alone in that. I don't know if HN would be better if regular users spent more time looking at the new page, but I suspect that it would.

It's also worth noting that HN could interleave new submissions into the front page to get them more visibility, forgoing a preview type page altogether.

Perhaps the New feed could be tweaked, so that if a New submission gets an upvote it goes back to the top of the New feed. If the submission reaches the front page then its chronological position on the New feed could be restored.

Put in a random recently posted story on the front page, different for each visitor.

Reply on news.ycombinator.com