Comment by btilly | Hacker News Reader

Comment by btilly | Hacker News Reader

I like this solution, and would like to point out another way to look at it.

It can be viewed as simulated annealing. A brand new article starts at a "high temperature", it has a chance of showing up randomly for a selection of people. As time goes on the temperature lowers. And it either made it or didn't.

That is suppose we have a function called rand_hash that takes an article id and a user id and produces a random number between 0 and 1. Suppose that T is the time in minutes that a new article has been up. For position we treat a new article as having a vote of:

    votes + C * rand_hash(article, user) * e^(-k * T)

Where C and k are factors that affect how much we want a random boost to be able to raise a new article, and for how long we want it boosted.

Then what happens is that every person will see, mixed with the popular articles, a random selection of new ones. The new ones that are particularly interesting will organically pick up a few votes and then snowball. The new ones that aren't will disappear from the random mix.

When we tested a variant of this idea, we found that it didn't work for what I guess you could call psychological reasons: readers reacted negatively, even violently, to seeing junk on HN's front page. By "junk" I mean stories that were placed there randomly and that weren't of high-enough quality to otherwise be credible candidates for the front page. Basically, it was an "ow my eyes", "get that shit away from me" effect. (I don't mean to demean the submissions people posted that got randomly placed in this way; it's just that words like junk and shit reflect how users reacted to the random placement thing.)

HN users have an intense emotional relationship with the front page. This shows up in the inevitable "How is this on HN?" comments when a story somehow violates the commenter's personal contract or mental model of HN. Randomly placing candidate stories on the front page created a version of this that was a lot worse. It violated people's model of what HN is supposed to be, in ways that the randomness exacerbated. Two lessons I drew from that experience: (1) people's relationship with HN is surprisingly personal; and (2) never jeopardize that bond. Runner-up lesson: don't introduce noise into an emotional relationship.

If we had software that could measure how credible an article is as a candidate for HN's front page, I'd be willing to try this again. But I suspect that would amount to software that could measure how good an article is, and that would be the P=NP of internet news sites.

I'm still convinced that a lot of the best content—i.e. submissions that would gratify intellectual curiosity and that the community would have a great time reading and discussing—languishes in obscurity in the lower echelons of /newest, so HN's front page is still far from optimal. The second-chance process has helped (https://news.ycombinator.com/item?id=11662380) but I'm convinced that there are more things that we can do, mainly by building ways for the community to do them. It's important that we not build new mechanisms that just reproduce the upvote system, though.

Interesting. I am glad that you tested it.

I would suggest as a followup that you allow people to opt in to getting a mix of new stories. My suggestion would be that "people are automatically opted in, and every new story comes with a link to opt back out". I do understand if that version scares you though.

I suspect that only a vocal minority of people will have a strong negative reaction. And if objectors have a painless way to solve the problem, you shouldn't get many complaints.

Seconding the opt-in. Even marking it as a random entry such would be a decent way for people to work within this system without much complaining.

I mean ultimately this is the “new” page but a bit more streamlined

Isn't 'opting in' just going to /newest? You can already do that.

No, it is getting a couple from /newest when you're on the main page.

One thing that might make /newest a little more user-friendly would be an option to have a showdead just for it. I have global showdead on and often it means staring at a /newest full of autokilled spam. showdead in /newest just keeps me out of /newest.

This is a good tweak. In fact: there's an argument that showdead doesn't fit /newest at all; aren't the people reading it deliberately not discriminating that way?

There are a few things that are flagged for no apparent reason and that need to be vouched for, and a lot that needs to be flagged manually. The [dead] flag is useful. Were vouching not a thing, you'd be right, though.

The response you saw probably wasn't very surprising to anyone who has spent time looking at the /newest page. It's fun to find the good submissions early, but there's a lot of low-quality submissions to sift through to find the good ones.

So this ends selecting for content that appeals to the people that are willing to sift through the low-quality submissions.

It feels like using the homepage traffic to validate /new has a lot of promise, and maybe some tweaks could solve the problems you had. One idea I had from reading your comments and dan's article is to have a permanent slot on the homepage with something from /new in it.

I think you could address the issue that "readers reacted negatively, even violently, to seeing junk on HN's front page" by making it clear with a special visualization, that this slot is special. You could also add a "report low quality" button which would make that story less likely to appear in the random slot in the future.

You could even add a "show me another" which fetches another random story instead.

So roughly:

- somewhere near slot 5 we add a special "new" slot, visually identified as such

- on each page load, the slot is given to something randomly posted in the last 8 hours

- the slot is different for each request/user, so everything on new gets seen a little bit

What if they only feel that way because they see he vote count is 1 (no upvotes)?

What if you did this and also hid the vote count for all the stories?

Can you try this again, but this time add around 4 of these types of stories the bottom of the home page in a separate section? (i.e. not mixed into the "good" stories)

I don't always remember to check /new and vote up good stories, but I feel if I saw random stories while anyway browsing the home page I might vote up any good ones.

Seconded. This would probably remind me to check /new more often, too. I also like the suggestion elsewhere in the thread of hiding the vote counts for this section.

I don't always like looking at "/newest" also for the reason it can take time to sift through to find the good ones.

What if there was a "/random" or "/custom". First would maybe weigh "newness" into the formula, but "/custom" could be based on user preferences. Might take a min to find right mix of "settings / allowable combinations" that would allow the site to remain performant. That way you all would be able to monitor trends in how users set up their preferences.

Would it be possible to mark the "candidate" stories in some way, to explain to the user why they were seeing this possibly lame post?

I used to be bothered by seeing off topic stuff and memes in a subreddit I visit but the sub started adding flare for different kinds of posts. I found this completely alleviated my psychological revulsion to the things I didn't like once they were categorized correctly. So, a meme on the front page would have annoyed me, but that same thing labeled "meme" didn't bother me at all.

I know HN sticks to a very simple UI, but I wonder if giving readers a way to control the mix for their personal front page would work. Maybe a little slider to adjust the threshold somewhere discreet (but on the front page, not in settings).

Why not have 2 columns on the front page with top stories and random?

Perhaps half way down the list inject, advert style, "Three from /newest:" add in three rows from newest, styled slightly differently so viewers can continue to scroll past. You could even try putting these at the bottom instead of half way down. If it was at the bottom, you could increase it to 5 or more. That would be the least intrusive and get the most eyeballs.

Hmmm.

A good thing about psychological "problems" is that symbolic solutions can sometimes solve it.

..some way for users to see why the post is on the front page, without triggering our "why is this on the front page!" nerve. Tag, colour, or such.

What about tacking a seriously "new boosted" and possibly randomized 10-30 submissions, clearly marked with different background to the end of frontpage, maybe after "next page" buttons? It would hardly make a dent to the load time, and people would in theory not feel as cheated since they know what to expect. It would still give "new" section a lot more eyeballs to find the golden nuggets.

I second this one. Having one-maybe-two new random links at the bottom, marked with a different color (maybe brown or golden, given the associations? ;)) would probably solve this.

Perhaps there could be a fold with one or two random ‘candidate’ stories underneath, so it’s clear they are not part of the ‘proper’ front page.

I'm curious, do you have any theories as to why the job ads don't elicit this negative reaction? Is it just because there's no comments, so people have no place to complain?

IIRC, the job ads are a perk of getting into Y Combinator. So even if they are seen as annoying by some, it's expected that they will just suck it up and deal. After all, Y Combinator is why the forum exists.

The job ads have been around forever (or almost forever, a few diehards will insist), so they stand out less than something being introduced for the first time. Also, the noise level is lower because they are sort of all the same and easier to tune out—whereas random posts from /newest bring new forms of annoyance on a regular basis.

It isn't just that, though. It's also that the community has a sense of fairness about HN giving some things back to YC in exchange for funding this place. I say this because when we introduced Launch HNs for YC startups about 3 years ago, there were few if any complaints.

https://news.ycombinator.com/launches

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

As always, thanks Dang for your presence. HN is the most civil place I’ve found on the internet in the better part of 30 years.

I’m wondering if the mistake was letting things make the front page, when maybe page 2 is acceptable even for people who feel vehemently about trash submissions. If we as a community can expect some pain in order to find better content it might be worth it (especially if we can train people not to abuse mods at the same time). I dunno. The idea of adding noise to smooth out results is a very common technique in signal processing, I’d like to think it’s useful in this context as well.

I will say, as an anecdote, I find the variety of interesting content isn’t as high as it was 6 years ago (let alone 10). And weekends tend to be better IMO. I assume the community has shifted, but even just being bigger is going to tend to lower the variance of topics. So I really appreciate any effort made towards helping quality interesting topics surface.

i had thought of using simulated annealing for a meme-based social sharing platform around 2014.the idea was after "cooling" to a point it would be removed from the stack. honestly i never developed it past that line of thought.

Reply on news.ycombinator.com