Ask HN: What's stopping HN going the way of Reddit/Digg/Slashdot?

I don't think it's as hard to keep a site from sliding as one might think from the examples of previous sites where things went downhill as they got more popular. Slashdot, Digg, and Reddit were all companies. They wanted to grow. Whereas News.YC is a side project. We don't care about growth. It's much easier to do things to keep up the quality when you're willing to sacrifice growth.

As long as the desire for excellent content outweighs the desire for growth and the users vote down irrelevant stories, then we are good to go.

We can't downvote stories here...

You can when you get a high enough karma.

Really? What's the number?

I'm not sure. But I have enough karma at 85. I think the number has changed over time, but I'm not sure.

Maybe PG will share it with us or someone has recently hit the magic number?

You sure you don't mean comments?

Yeah my bad, I'm talking about comments.

My karma's 4700 and I can't downvote stories.

I have 867 karma and can't downvote stories. At some point there were talks of letting the top X downvote stories but I don't think it's been done.

Yeah it's calculated. I thought you were talking about comments. Or it was last time news.arc was released by PG

  (= gravity* 1.4 timebase* 120 front-threshold* 1)
 
  (def frontpage-rank (s (o gravity gravity*))
    (/ (- (realscore s) 1)
       (expt (/ (+ (item-age s) timebase*) 60) gravity)))

http://github.com/nex3/arc/tree/master/news.arc

As a relative newcomer here, I sincerely hope lolcats are a bannable offense.

Eventually you will grow and you will need to start making things bannable.

What you also need to have is a probation system where you can't post for anywhere from 4 hours to a month depending on what you do. That acts as a warning. Get put on probation too many times and you're banned.

That's how it works at Something Awful, and they still have pretty good quality after almost 10 years. And they are for-profit.

Also, it might be desirable to extra strict during certain times when people are likely to go berzerk and act really stupid (the SA moderators call this the "banhammer" and make a sticky thread notifying users when it is in effect -- what would normally be a warning becomes a ban).

No, S.A. is a pay site, it's some trouble to get yourself a new credit card, but on free sites you just make a new alias.

Although that's true, the new account won't have any karma, which actually matters on Hacker News, since it affects whether you can downvote, etc.

So while banning certainly won't have the same effect as on S.A., it can be useful despite how easy it is to create new accounts.

Is it sufficient to simply not target growth as a goal?

HN, if I'm not mistaken, has kept a general upward trend in number of users; presumably, this is accompanied by an ever-increasing variance in quality of those users and what they consider to be interesting news and good etiquette.

Don't get me wrong; I certainly agree that it is easier to make changes with an eye towards quality when you're not concerned about growth. However, I do wonder if there it a point where the dark side of network effects, which are primarily sociological in nature, will simply overwhelm any technological aspects or other moves on the part of a community's stewards / leaders.

"Is it sufficient to simply not target growth as a goal?"

Growth is a big part of the problem. It's much the same for movies. You can either appeal to a more discriminating audience, which is inherently smaller, or go lowest common denominator. Actually, the pattern is to start out appealing to the discriminating crowd to get a "cool" cachet, then expand and go mass-market.

Another big problem is the monkey-see-monkey-do nature of Homo Sapiens. Over time, you will see more noise in the form of attempts at imitating signal instead of true signal.

It is not sufficient simply not to target growth. But not targeting growth removes what is otherwise one helluva tough design constraint for a filter system. Imagine writing a spam filter when you're being paid by beancounters whose bottom line is maximizing the number of messages. Even, sometimes, beancounters who face strong temptation to maximize the number of bytes of email this quarter and worry about the longer-than-a-month-term consequences later.

Also, some kinds of filtering (and related things, like informal community norms) don't scale, so if the community hasn't grown too much, the problem might tend to be easier. Hacker-oriented mailing lists with 100 posters don't seem to be too hard to moderate, but I'm not eager to try 10,000 posters.

Metafilter simply turned off new user registration for a long time when they didn't like where things were going.

> ever-increasing variance in quality of those users

I'm not sure it's necessarily the quality of the users. Call me an optimist, but I think most people in general are decent and most people that bother to come here are pretty smart.

I think the problem with size is the same reason everyone else is a crappy driver but you. In a normal day of driving, you are surrounded by mostly good drivers, but a couple of them are bound to make really stupid mistakes (just by random chance). Since this is your only experience with them, you label them a "bad driver" and it overshadows all the good driving around you. If you only drive on lightly traveled roads, you are less likely to see a stupid mistake, but get on a major interstate highway in a major city and there are enough people that something is bound to happen.

Likewise with large social news sites. Every user is a 'good user' 99% of the time, but has that random moment when they do something trollish or get carried away with an argument and say something mean. (I know it has happened to me before.) When a site gets large, the probability of this happening on any given thread rises accordingly, and since everyone focuses on these instances, it seems like every thread is full of trolls and angry people.

I think anonymity also plays into it. Large communities are by default anonymous, and small ones are not. People in small towns don't cut each other off or tailgate each other, because you are likely to know the person in the other car, or at the very least be headed to the same place. (I know, I'm from a small town.) In a major city, you are never going to see that other car again, so no one cares if they act like an asshole.

In one word: focus.

That excites me. I am also a newcomer from Reddit. I believe I'll stick around now, since, being a student majoring in computer engineering, this interests me. Feel free to ban me if my comments become too low quality. :)

I think this has always been the key. Its like the mom/pop store versus the Walmart super stores. They need (digg/reddit) to be popular to be a viable acquisition target or business whereas hacker news is just a place where we try to find like minded people trying to share information and grow learn together.

Actually, slashdot manages pretty well. Considering the number of users they have. Their moderation system has matured over time. And the typical WTF, 'funny pics' and youtube crap that ruins other sites is isolated on http://idle.slashdot.org/ where it can harm no-one. I think it starts with comment moderation ( http://slashdot.org/moderation.shtml ). It's hard to get karma on /. by just fooling around, and you have to be a dedicated user to even get to moderate other comments. Contrary to digg where a sensible comment about X can be downmodded into oblivion by an army of 13 year old fanboys.

Here's my thought on the matter... The "Killer feature" here is the lack of killer features. There are no buddy lists, stupid picture icons, special groups, or karma-whoring incentives. It's just a simple place to exchange news that is generally relevant to a hacker community.

The lack of needless bells and whistles to constantly hold peoples attention will generally make this site un-fun for the unwashed masses of Digg and Reddit refugees.

BTW, welcome to hacker news...

What? there's no karma-whoring incentives? Damn! I totally thought I was going to get to change the color of the toolbar...

And polls! Why else would I post but to reach the HN Nirvana of being able to post polls?

for when that day comes...

http://news.ycombinator.com/newpoll

PG http://news.ycombinator.com/item?id=196547 :

"1. The kind of stories that are most popular on reddit and digg (pictures and politics) are banned here.

2. Because that kind of stuff is banned, the average digg/reddit reader, if he comes across News.YC, finds the content boring and leaves.

3. The custom of the site is to be civil in comments.

4. Votes on comments affect karma.

5. Trolls are fairly rapidly banned. The only reason Giles hasn't already been banned is that I thought perhaps he was joking."

I'm hopeful - but there was a time when #3 was true of Reddit, too. Now if your Reddit comment includes the word "fap", you're about guaranteed to be upmodded.

As a fellow Digg->Reddit->here refugee, I think we're the problem. Judging by the fact that you didn't defect from Reddit sooner, it can be reasoned that you enjoy a dash of offbeat humor, politics, and funny pictures with your programming (and in this case entrepreneurial) articles and news. In other words, you're like me, and the rest of the early majority.

We go from social news site to social news site, dragging down article quality with our penchant for less-than-total fact saturation and off-topic posts.

The only responsible thing is to get your hacker news here, and your funny pictures elsewhere. Refrain from contributing posts or articles and let the people that make this site great continue to make it great.

Wait, crap. Except for this post and that post.

The most important thing, besides the general desire to keep the community positive, is this: http://ycombinator.com/newsguidelines.html.

On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity.

Reddit, on the other hand, has the following guidelines:

Submit a link to anything interesting: news article, blog entry, video, picture...

The HN community is built around a particular standard. Reddit's official standard is the lowest common denominator. If the lowest common denominator is "Impeach Bush" and "Funny pic [nsfw]", then that's going to dominate.

Unfortunately, I've already seen Hacker News change over the last year (less programming-related content; more general business news). I'd love a gentle nudge back in the hacker direction. But it's still the best social bookmarking site at the moment.

The only hope is our elitist comment trolls. Also the fact that HN has far fewer users makes it much easier to deal with the problem. http://siteanalytics.compete.com/digg.com+reddit.com+news.yc...

I'm mildly pleased that HN shrank, not grew, this month. Not by a lot, but still. Hopefully that won't reverse. If the community doesn't grow, it can't turn into another digg.

Note that the statistic isn't for HN only, but also includes the rest of ycombinator.com. I suspect HN didn't shrink; instead YC got press hits in April + May after registrations and kicking off the summer cycle. Also, for that level of traffic, I'd be wary of compete.com stats. More so than usual in any case.

Traffic actually grew last month. It went down at the beginning of the summer though.

I can't believe nobody has mentioned collaborative filtering.

Simple collective voting and new/hot/controversial are very unsophisticated measures of how much any particular user will like something.

For an example of how powerful collaborative filtering is, check out movielens.org

The key is to take each user's votes and use them to figure out based on other peoples' votes what stories and comments to display. Then the community doesn't really "grow". Sure you may get some niche communities that grow, such as people who all tend to upvote cat pictures, but the "thinking person" doesn't have to worry, b/c as long as there is a weak (or negative) correlation between cat pictures and your upvotes, the system will never recommend any to you.

Imagine rather than one massive community, an infinite number of beautifully overlapping ones. I think the ideal system would consider votes for stories and comments, and show each person mostly content that had a high probability of being enjoyed.

The "new" tab might show things with less of a strict filter to prevent false negatives.

The research group that built Movielens has some papers on the web and the math isn't that tough. With a properly design collaborative filtering system there simply will not be the digg/reddit degradation where the site gets so big and noisy and low quality. Sure there may be stories that are hugely popular, but as long as the algorithm gets enough quality input (votes) on a variety of things, no user would get too many duds.

Ironically Reddit's redesign only increases noise by tending people away from the subreddit niche communities that had existed -- I would more likely click on a weak title if in a quality subreddit than if it's the #4 item on the screen.

At the rate it's going, news.yc will surely become the next reddit, and eventually the next Digg. As others have pointed out, we'll all have probably moved on by then to the next site (or hopefully one that implements CF!).

It seems odd to me that nobody has implemented a site like this with CF... As with most of my posts, if anyone wants to collaborate to do an experimental site like this just let me know. There are some cool ruby libraries out there and of course gsl.

Don't make your posts too entertaining. Don't make your posts too short. A community of people who post large chunks of dense text about esoteric subjects will keep the horde out.

I read all the answers, and I'm now convinced that there's nothing keeping this site from eventually going down that same path.

Death. Taxes. Gentrification.

Yes, the majority of people here just support banning, harsher punishments, and locking out all outsiders. Those are very poor solutions!

#1 and #2 cause strong negative backlash. As you would learn from marketing theory, a user that you have personally screwed with will likely cause 20x more damage to your brand than your best users improve it. #3 Locks out a lot of useful expertise. For sure, HN could use a lot more knowledge about programming. So far, this site has attracted lots of people who want to learn but much less so people who already know a lot and have wide experience. We still very much need to be inviting to the latter.

Another point that very knowledgeable people are very opinionated, stubborn, and generally assholes. Linus Torvalds is the stereotypical example. This is not to excusing the behavior, but the correlation exists and we will all have to try not to get so offended and ban people like immature children.

If you try to hold onto the past forever instead of trying to improve then you will fail.

I hope this was useful.

"As you would learn from marketing theory, a user that you have personally screwed with will likely cause 20x more damage to your brand than your best users improve it."

I'd say that a jackass active on the site is more damaging than one banned. If you ban someone, you have only upset one user. If you allow a spammer/troll/flamer/etc to continue unchallenged, you have upset all of them.

The thing with online communities is, this is Web 2.0, your users are your product. The better the users, the better the product. If you have a good community, smart, helpful people will be attracted and will thus improve the community. If you have a bad one, they will be warded off. Scum breeds scum.

Think broken window theory.

If that could be achieved then, yes you would be quite correct. However, it is very hard to completely cut people off on the Internet. Furthermore, they can start disparaging you through other mediums (eg blogs) and influence people they still know to be using your site.

So? Thats free publicity. If they something bad about you, all they're doing is informing people of your existence. People will judge for themselves whether you are worth using, usually by checking it out themselves. At least, the kind of people you want on HN will.

The problem isn't with them badmouthing you on your site, it with them polluting your product. If the product sucks, people won't use it, no matter how much you suck up to them.

Once your site is big enough, it won't really be a problem.

In fact, it's unlikely to ever be a problem: if your site is small, nobody on a bigger site is going to care about a someone bitching about it, so the story will get buried. If it's large, any site worth its salt will delete a griefing post to avoid the inter-site grief (see: reddit deleting the posts that lead here, mefi deleting pretty much anything leading to LGF)

The key to banning is to not notify people who have been banned. You just let them post stuff, but nobody sees it except the poster. Another approach is to make the site seem broken to the banned user. For example, when Arrington's sock puppets try to log in, just throw up a fail whale. Should also make for amusing commentary on Tech Crunch. "IS PG THE NEW BLAINE COOK???" etc.

Those techniques work to protect against bots and similar, but do little to help in the case of a real person.

I think there are a few things going on here that differentiate it from Digg/Reddit (I'll leave out Slashdot for the moment, because I think they have different concerns).

On Digg and Reddit, there is no "focus". Digg started out as a tech news site and degraded slowly until they opened up the doors to include all news, which brought what you have refered to as posts on Obama's latest news and pictures of their cats.

On reddit, the day I stopped reading was the day I saw a post on the front page encouraging folks to not post political stories on the politics sub-site because they don't get enough attention that way. That would imply that both sites have a problem with "activists".

To me, that sufficiently explains why the front page of HN "works". What I can't figure out is why the comments on this site tend to be so civil. The HN community seems to lack the hostility that plagues virtually every other site of its kind. Is it fewer angry people? Older, or more mature audience? Maybe it's migrants from the three sites you mention who were fed up with the poor behavior exhibited there?

The main difference is very simple, yet effective: HN has a specific goal. It is not a place which will just flow with whatever is posted. Some things are allowed, and some things are not.

Hopefully, the choice of content will keep people interested in that kind of content, and remove those that don't. They already got reddit/digg/foobar for their stories.

Comments would take more time to moderate, if we think of users up/downvoting as a differential equation that slowly is moving towards the "intelligence of the masses". So in time, the level of comments will degrade. The way of keeping that from happening is, of course, to mercilessly kill bad comments and keep them out with overpowering ;)

PG has said he won't let it, and will do what it takes to keep it high quality.

This is a very noble aspiration, but many benevolent stewards of online communities have tried and failed; it is a historically intractable problem.

I think the OP is asking what is the "what it takes"?

There is the difference between HN and reddit/digg etc. that pg and co. do HN for love, whereas, whatever other motives they may have, the other sites are trying to make money.

Classic counterexample: kuro5hin.

i wonder if performing a restart... something like what The Architect from The Matrix talks about...

i wonder if simply restarting the entire software would help a bit. in other words, deleting the entire database and starting from scratch every n years. meaning the front page would be blank, people would have to 'register' again with their karma at 1

if it takes making a new community, a restart might be a good approximation, among having other effects (eg the effect on karma whores)

Amen to that! It's the only refuge from all the trash out there. Hopefully fanboys will never invade it...

A good first step is, when you catch yourself making a comment that contributes nothing except maybe some sentiment, to not make it.

I also have faith in this even with threads about economics.

It probably will. I followed a similar path of another person on here: /. -> k5 -> reddit -> HN.

At the end of the day, nothing can stop 50 users up-voting cat pictures except to shut the site down.

Except a Bayesian cat-picture filter written in Lisp!

lisp? you mean arc good sir. I'm not sure how effective a bayesian filter would be at classifying news stories as interesting/non-interesting though. Has anyone tried this?

Arc is a Lisp.

Yes, it looks like pg built something to recognize good/bad stories:

http://news.ycombinator.com/item?id=213951

You say that jokingly, but spam filters would probably work if someone was willing to slog through the digg front page long enough to build up a decent set of training data.

This is why hacker news has a collection of editors with the power to kill cat pictures with a click.

Don't underestimate the power of mass stupidity. It's killed many sites in the past. Even with a "collection of editors with the power to kill cat pictures with a click."

I don't understand; are you implying that there will simply be too many stories for editors to kill them all? The particular defense HN has against people voting up mind-candy is that editors can (and do) kill those stories. What makes you think this strategy won't work?

If growth isn't a concern why not force people to wait a certain amount of time after signing up before they're allowed to post?

That way new users can get a feel for the community before they start posting and you only get users with a real interest in the stories posted here.

I think just not compromising is good enough to keep the quality of the site. I don't see HN doing any deals with NY Times or getting a ad deal with Google. They have kept it simple, just pure news and it is surrounded by smart bunch of people who want to keep it that way.

the quick and dirty answer is the userbase. it doesn't, at least right now, turn into digg/reddit/slashdot because the people who want to see a digg/reddit/slashdot are already at digg/reddit/slashdot.

the community here likes how things are and has a vested interest in keeping HN like it is. so thats what they do, try and push out the bad where they can and promote the good.

I don't think anything... the algorithm that decides what's on the front page is currently just so so, and no one wants to improve it.

Limit the number of news submission per user in 24 hours. I suggest max 5 news/24 hours. That way, we can guard HN from spammers.

If this were actually a problem, they could circumvent this restriction by simply registering many accounts. A better solution -- as others have pointed out in the past -- would be a karma threshold for posting articles, but since it's not a problem yet, why bother?

If you think your website is "worth enough", you can make it a paysite and raise the barrier very much: see Metafilter (which has a 1 post/day hard limit, except for the owner)

That would be a bad idea since submissions follow a powerlaw , meaning that the majority come from a relatively low number of users that submit a lot. Check http://news.ycombinator.com/user?id=nickb for an example.

Are you saying that those posts wouldn't be made by someone else? I doubt it.