Solving The Hacker News Problem | Hacker News Reader

Solving The Hacker News Problem

al3x.net

407 pointstianyicui270 comments

                 Quality of HN Comments Over Time
   |                   . .
   |                  .   . 
  q| . .             .     .
  u|    .           .       .               . . .
  a|     .         .          .           .       .
  l|      .       .              .      .           .
  i|       .     .                  . .               .    
  t|        . . .                       you are here -->. .
  y|                                      (that's all)
   |________________________________________________________
    N D J F M A M J J A S O N D J F M A M J J A S O N D J F
       '09                     '10                     '11

(It must be that time of year again...)

http://news.ycombinator.com/item?id=926604

http://news.ycombinator.com/item?id=1646871

The question is, of course, are we entering the Eternal February?

As of now (9:10GMT) I've read every post on this page, and I think as short as it is, the parent (Eternal February) perhaps is the one that is closest to what I think has happened.

I think this is a people problem, not a technology one.

For those who haven't heard this phrase before, the "Eternal February" apes the "Eternal September". Pre-1993, each September would herald the start of a new university year and an influx of new students who would suddenly gain access to the internet. Whilst their presence was being integrated by existing on-line communities, the standard and quality of debate in Usenet was perceived to decline as the same questions got asked again and the conversation standard was forced again. The same problems came forth until such a point that the existing communities had assimilated the newbies and by October or November things had returned to where they were, just a bit bigger.

Then along came AOL, giving out free/cheap internet access to all and lowering the technical barrier to getting on-line. A good thing for the world, but a bad thing for those Usenet communities who suddenly had to assimilate an influx greater than a normal September, and also an influx that didn't subside. At some point the ability of those in Usenet to assimilate was surpassed by the people getting on-line and finding Usenet. The term "Eternal September" was coined to describe this problem and the general failure of Usenet to keep up with the flow and the lowering standards that are the result of this.

I think HackerNews has no technical problem.

The discussion around karma and mod points is a distraction, the same goes for invite only systems and "classic" view.

I'm going to speculate that the problem is purely the size of the population and the growth rate, and that the community gravity is now such that this is not going to stop.

Anyone who has been here for years, and anyone who has a shed load of karma points... those who give in time, effort... there's no questioning that we/they all want to have a place such as this, and few really want to leave it. But it's hard to see how staying is going to help, and here's why: Time is valuable to us, and our expectations have risen as our experience and knowledge increased, and we feel defeated by that influx and we sense the gravity (that if anything increases the rate of influx).

Due to the growth, the noise level is so much higher (even if only 0.1% of the population post new things or respond, 0.1% of hundreds of thousands rather than thousands is still a far higher noise level). How can we cut through the noise to find the good stuff (subjective to the individual) given this noise? I read the firehose (/new) in an attempt to not miss the good, but realise that I've amplified the volume of everything to try and not miss the gems.

The influx is such that it feels non-correctable to some degree. We can look back at the side-splitting Erlang day as a humorous attempt at a correction (which it was, but we probably didn't perceive the bigger picture then... the "Eternal February").

The thing is, most old-timers are aware there is still a lot of value in learning from new people, of passing on knowledge to new people... it's a two-way street and some good comes from growth and closing the gate isn't the desired thing. As the noise levels get higher closing the gate seems tempting, but it's silly. Shutting the gate is not going to help.

There is a natural contribution rate at which the noise levels are manageable, and there's a natural growth rate at which people can be assimilated such that the noise doesn't go above manageable levels. When either rate is crossed it can be difficult, when both are crossed we end up here.

My suggestion would be along Alex's lines... to not divide by date registered (just those with early registration dates), to not exclude by network (an invite only boys club), nor to kill HN (which still serves as a really important hub by bringing us all together, and I'm sure some would feel that anything new is a conscious threat to the old).

Instead we should create some thing (or multiple things) that is so much more narrowly focused that it doesn't have much appeal beyond a smallish number. That focus needs to be narrow enough that growth is naturally limited too.

I think that defining that 'thing' is where it gets interesting. What that narrowly focused topic is, well that is going to be different from person to person.

It's likely that many small communities would spring up; maybe one focused on devops + computer science (a bit like the ACM SIGACTS-SIGOPS), maybe one focused on purely the business side of startups, etc.

HN is rare because it encompasses all of these, but I'd suggest it may be better to not be so far encompassing and instead to have a set of communities (subreddits vs stack overflows vs unconnected sites) more narrowly focused and people would join subsets that interested them rather than have them all exist in a global scope (where the noise level is that of the whole set).

The hardest part of doing anything comes down to the people.

Ultimately it's the mix of people here that is so awesome, it's what acts most as gravity to those outside of the community. Having this calibre of people choose to invest in smaller communities is going to be the hardest thing when staying on just HN, or just opting out altogether, is so easy to do.

I too think it's time for something more focused and smaller to emerge to tackle the noise level. That this is likely to be many smaller things and that I'd miss part of the debate is kinda fine, I'm fine with missing some stuff if the standard on the bits I'm more strongly interested in can be raised and I don't miss that stuff.

No snark intended, but I feel like you are basically describing reddit, with its subreddits and customizable front page. If you unsubscribe from main, funny, pics, politics, askreddit, and add programming, computer science, mathematics, etc to your front page, you will get a pretty high quality experience, IMO (at least no 4chan garbage). The main difference that I perceive between the two sites is that reddit seems to have a slightly younger demographic, skewing towards college students, while HN seems to be mostly older, working professionals, and silicon valley heavy-hitters. Remember when reddit was the upstream digg? Now digg is dead and HN is the upstream reddit. If there's an upstream HN, by the time I find out about it, people will be complaining about how badly the community has declined lately. :)

EDIT: Never mind, I just unsubscribed from all the default reddits and it's still mostly crap. Carry on...

What does it matter? These complaints of decline in community quality is all superficial. In the end, all that matters is how you use a site.

What is it that you want out of these sites? Expert Opinion? Helpful articles? The Zeitgeist among your peers? The analysis of the community might help you decide how to use a site to your liking, but because it is superficial, it will be of little help. Seek out the experts yourself or trim through the article listings yourself instead of basing it completely on a democratic vote system. The quality of the community will only make this process more convenient, but it is the quality of individual posters that define what is most helpful to you.

Now, I would like to compile a list of posters whom I can trust for insightful comments on a particular matter instead of posting assumptions.

So far, I know of pg, joshu...crap, I wish there were avatars here. Why was I stuck with just visual memory like a common human? Unicode avatars would have been helpful: ❤

In simplest terms the HN community & culture fosters open minded discussion while reddit fosters group think.

Even if HN has lost its edge over time that fundamental difference is worth fighting for and worth coming back to HN.

The mere fact that you took the time to compile this lengthy response and have received meaningful responses is evidence enough that HN has not "jumped the shark."

It is a people problem, but I feel the biggest problem is that people are pointing out that there's a problem. Communities evolve; that is, in fact, practically all they do. Perhaps HN isn't the same as it was 2 or 3 years ago, saying that it's worse is a bit of a stretch.

I've yet to find a community where people have meaningful discussions in the comments without some snarky response like, "lol wut"." On HN these comments are self-moderated; those with the karma downvote the meaningless comments and they disappear into the aether.

Simply put, I don't think HN is dying. Rumors of its demise are highly overblown.

I agree, to an extent. This navel-gazing tends to precede a noticeable and rapid decline in quality, as the community becomes too focussed on itself, and less upon content and discussion. I've only been on reddit for just over a year but have noticed a marked decline in quality just in that time. Around this time last year, there were many discussions about reddit's declining quality and what's to be done about it. Now, about half the posts on even my heavily-culled front page are talking about reddit itself, half of the rest are using absurdly editorialized headlines, and there's a smattering of good content here and there. Heavy moderation seems to be key, as the mods quite simply don't seem to care anymore.

When HN first started it was kind of like r/truereddit but with an emphasis on startups. HN today has almost nothing in common with this. That's not necessarily a bad thing, but I don't think it's correct to say that the quality level is just an oscillation.

I guess I don't really agree; HN of several years ago typically had much briefer and less "weighty" articles and discussion than r/truereddit, and a lot more offhand stuff, like the typical techcrunch stories, and "here's how you do a thing in vim" type quick-tip articles. Certainly some good, in-depth technical articles and discussions, but that's the case now too. I've been digging through archives to try to remind myself of whether I had missed some sort of golden age (I was a lurker then, and considered it an interesting but not amazing site), and I still don't really see the golden-ageness, even in retrospect. Nothing like the Kuro5hin golden age, anyway. ;-)

"Nothing like the Kuro5hin golden age, anyway."

Well I think we can all agree on that. It seems like the entire web really peaked in 2005-2006, and it's been all downhill since there.

I sort of agree, but I meant it a bit tongue-in-cheek (hence the smiley). I'm sure plenty of people would argue that the Kuro5hin golden age was nothing like the WELL golden age, or the Usenet golden age, or various other possibilities.

One thing I did like about the Kuro5hin model was discussion organized around original, substantive stories, written by the same userbase who were the commenters. That, and stories posted at a rate of one or two a day. Sites like Slashdot/HN/reddit are instead mainly based around discussing offsite links, which K5 allowed but sort of discouraged as "mindless link propagation" (there was a section for it, but people didn't want too many submissions to it, and you typically had to add some substantive commentary to the link to get the submission accepted). The articles were also directly written for / targeted to the K5 community. People do sometimes write blog posts specifically to target HN, but they're often more multitargeted--- stuff submitted here might not be written with an HN audience in mind at all, or HN might only be one of several intended audiences.

Some classic Usenet groups were sort-of in that style as well. You could start a new thread with a one-line post or "what do you think of this?" query, but some groups had a culture of starting new threads only when you had a significant, well-thought out post to make; otherwise you were supposed to stick to existing threads.

I'm afraid it's harder to attract good content to a community these days, though. The people who used to consistently write good stuff for Kuro5hin now have their own blogs, and post their content there instead. In the early 2000s, it was a win-win situation: Kuro5hin provided an outlet for people who had things to say, and people who had things to say provided interesting content to read and discuss. These days if you have something compelling to say, you can just start a blog, and then try to get readers by submitting links across the web.

Having never heard of Kuro5hin, I decided to google it. If it ever had a golden age, it must have fallen far. The current front page reads like a mix of 4chan, freerepublic, and the occasional tidbit of interesting content.

Seriously, a selection of headlines:

"OFFICIAL NOTICE: SITE CLOSED DUE TO AIDS"

"Judiciary Abdicated. Republic Dead. All Hail Emperor Obama"

"The New York Times Editorial Board Should Be Executed For Treason"

"Our lunatic Military"

What kind of content did it have originally?

It's sort of a dive bar these days; a surprising proportion of the good posters are still there, but they just post different things, while they post "serious" stuff on their blogs or somewhere else. Most discussion goes on in the diaries (which are also dive-bar-ish, mixture of serious and unserious chat), while the front-page is a bit of an active joke. Hasn't had much serious content in probably 3-4 years, with the occasional exception.

(Actually that's a strangely common thing on the internet; the offensive-joker and troll types sometimes are the same people as the good-technical-content types. For example, the Rands of http://www.randsinrepose.com is also the Rands of http://www.jerkcity.com.)

As a half joke, I collected a bibliography of k5 articles that have been cited in academic papers, which includes some of the better ones: http://www.kuro5hin.org/story/2007/12/8/23637/9295 , http://www.kuro5hin.org/story/2010/10/17/5512/5360

I've posted this on HN before, but I wrote a long article about the decline of online community, which includes some retrospective about k5's decline as well: http://www.kuro5hin.org/story/2009/3/12/33338/3000

I don't think this answer holds true anymore, i.e. the newbies who are a year old would claim comment quality is going low. I think it now has become the norm. Why not just aim for a standard near the top quality submissions?

Nice chart. How did you make it?

It'd be nice to put in another curve for 'value of HN comments in shaping startups.'

Do those peaks feed periods of frenzied success (inspiring people to do better stuff faster)?

Or do they lag periods of success (pearls of wisdom come out when people are past the sweat-soaked white knuckle phase)?

Or is there some other correlation?

I think if we could see random frontpages from days a few years ago, we'd find that the top stories weren't that different, and that there was the same "jack of all trades, master of none" aspect to the site that Alex complains about. It may be that a site whose design spec is to satisfy hackers' intellectual curiosity would necessarily feel that way.

Maybe I'll write something to regenerate past front pages, so we can check if things are different now. That should be possible, because news.arc has always logged vote times.

It may be that a site whose design spec is to satisfy hackers' intellectual curiosity would necessarily feel that way.

As the infamous "How to Become A Hacker" states "The world is full of fascinating problems waiting to be solved"[1]. There is no mention of those problems having to be "deeply technical".

I certainly see the tremendous variety of topics that make the homepage of HN as an asset in keeping with the true Hacker Mentality.

[1] http://www.catb.org/~esr/faqs/hacker-howto.html#believe1

The non-technical questions on HN have poor associated discussions though. As OPs go further from tech/startups (and towards econ and policy), commenters display a poorer understanding of basic concepts.

Bottom line: sure, hackers care about many problems. But that doesn't make up for the poor discussion quality surrounding many non-tech articles here.

The non-technical questions on HN have poor associated discussions though

I've seen what looks like fantastic advice and discussion on a range of non-technical topics (particularly things like venture capital/financing and laws around starting a company and that kind of thing).

Of course, not being an expert in any of that stuff I'm at a loss to make a judgement as to it's veracity ;)

Hence my next point:

"As OPs go further from tech/startups..."

I'm definitely no funding savant, either!

A diff of past front pages would be interesting. Some historical front page entries:

2007AUG15 ~ http://www.flickr.com/photos/bootload/1119991071/

2008FEB08 ~ http://www.flickr.com/photos/bootload/2250089864/

2009JAN18 ~ http://www.flickr.com/photos/bootload/3205568708/

2009MAR12 ~ http://www.flickr.com/photos/bootload/3347109733/

2010JUN24 ~ http://www.flickr.com/photos/bootload/4728476663

Of that sampling, it hit the apex for me with Jan 2009 page. The Mar 2009 page is when there was an influx, and we started posting Erlang stories as a joke to stem the flow.

That Erlamg screenshot is hilarious. Only one story on the front page didn't have Erlang in the title, why HN had been slow...was it due to the amount of Erlang submissions?

The slowdown and Erlangfest was because of a big new influx of users, see pg's post here: http://news.ycombinator.com/item?id=512145

And pg's appeal for sanity: http://news.ycombinator.com/item?id=512280

They were a joke? I thought finally people were interested in something cool. :(

What brings a tear to my eye is that, back in those days, it was hard for me to tell at first that it was a deliberate plot. It felt quite natural to have multiple discussions of Erlang on the front page.

http://news.ycombinator.com//item?id=512145

I don't know what you tried to do there, but it didn't work. Link is broken.

http://news.ycombinator.com/item?id=512145 He had an extra slash in his URL

I was part of that influx (I registered 5 days before pg's Erlang post). I apologize for making the page worse for the rest of you ;-). Honestly, the Erlang posts contributed to keeping my interest (though I didn't pay much attention during the following summer as I was busy coding).

Political and other front-page-news posts with little "hacker" content, blogs by irrelevant magazine columnists, posts about how X product/market/phenomenon is dooooomed, random questions about how the site works because the FAQ is a tiny link in the footer...and lots of posts by people horribly concerned that those damn noobs will stink the place up.

I really haven't noticed HN get any worse in the two years I've been around.

I don't see any difference between the 2007 page and today's.

From August 2007: "Am I the only one who doesn't care about Facebook?"

That has been a theme on HN longer than I realized.

Maybe you should compare random /active pages from years back. I don't know, I only found out about /active this last month, but I'm guessing the problem will be more visible there.

Most of us that have been around since the beginning (this is my second account) are too busy to participate as regularly as we once did. I'd rather spend time on my business than get hooked on discussions here as much as I did in the past. I will jump in here every so often and I still think HN is relatively great. Doesn't everybody remember the big debate over even linking to Techcrunch articles? We have always been whining about what's wrong to realize that this place has consistently been pretty great.

And sure, we might be like that dance club that's a bit too mainstream. But pretty soon those early cool kids grow up and are also not cool enough to discover the next hot thing. The next HN will come about and it won't be somebody from here who starts it.

I agree. This place has always been mostly good with a sprinkle of fluff pieces every now and then. The one thing that has changed a bit is the comments. Sadly the comments are being muddled a bit by the new kids on the block. Not that the new kids are doing anything intentionally bad, nor are they really bad comments, but I do remember a time where the people replying to comments were actual domain experts in a certain field or they wrote the application that you use every day, etc.... It's a problem with getting a larger user base. I do think that this site has aged about as gracefully as possible though.

Additionally, I think newer users are conditioned by other sites to use voting as a way to agree or disagree with what is said, rather than as a way to rate the quality of the point being made, regardless of what the point is. Which I find very frustrating, and I know people say 'whining about declining quality has always happened', but I really feel that this voting behavior used to be less pronounced.

I think newer users are conditioned by other sites to use voting as a way to agree or disagree with what is said, rather than as a way to rate the quality of the point being made, regardless of what the point is.

It hasn't just been new users. Rather it has been nearly everyone since the beginning. See what the founder said 1103 days ago about downvoting for disagreement:

http://news.ycombinator.com/item?id=117171

I think this may be due in part to people learning how to target Hacker News readers with their article and blog post headlines, so while the front page would look similar, the newer linked content is more fluffy.

I mean, there has to be some explanation like this, because it just seems so clear from regular reading that the quality has gone down. Then again, maybe I'm projecting the decline of comments onto the content.

The fact that HN hasn't been overrun by marketing zombies is why I'm not too worried about its direction. If linkbait titles are the big complaint about submissions, it's a good sign.

That's true I guess, but it seems like a low bar.

But it's a slippery slope, right? It's easier to worry now BEFORE marketing zomboids etc. arrive and bombard us.

You (pg) will not need this since you have the data.. ;-).. but for anyone else interested in doing analysis over time, HackerSlide stores JSON representations of the HN front page every hour (starting Oct 15 2010).

http://hackerslide.com/data/YYYY-MM-DD-HH.json gets you what you want. For example: http://hackerslide.com/data/2010-10-15-23.json

Wow great! http://hackerslide.com/data/2010-10-15-05.json appears to be the earliest.

Do you provide this as an "api" and allow people to build apps on top of it? If so, are there any limits?

No. At least, no more API than what I just explained ;-) It's not my data. It's publicly available because the main http://hackerslide.com/ site uses it and if you want to grab it every now and then, that's fine by me.

If anyone grabs the same file over and over or anything crazy like that, they'll be blocked, but if you just poach the data and store your own copies, go for it :-)

I wonder what Google's reading level rating mechanism would make of changes over time? Disturbing thought -- what about comments by poster?

The frontpage content is fine. I'm much more concerned about the comment quality.

That's always a serious concern. It's not surprising that many bloggers either heavily moderate or don't even allow comments on blog posts. It's a difficult issue to solve for sure. Better comments means you need better people offering thoughtful and insightful comments.

There was a submission a couple of weeks ago of a site that randomly arranged previous HN submissions of a certain threshold from the past with each refresh. Was really useful and sadly neither Searchyc.com or Google is helping with finding the post.

Perhaps the submitter sees this comment and shares the link here again?

I have recently begun to realize that there are 2 aspects to "website degeneration". One is the obvious new people showing up because they heard it's cool psuedo python paradox situation.

The other comes from the old-timers themselves. I'm not particularly that old-timer, but in the 3something years I've been here, I can say I have experienced this:

I've grown.

The "5 awesome vim tips" or "super deep closure stuff" articles that today annoy me as fluff were deep and new to me back then. HN was a fantastic site introducing me to amazing things, and it is introducing a lot of people to the same amazing things now. As such, the quest for truly new content needs to go deeper, but the people who need the lighter content are greater in number and enthusiasm, ensuring the fluff rises.

When the older crowd doesn't notice their growth, or the lack of change in content (or the combo) you start to get "it's gone down-hill" type comments.

edit: I mean the quest for truly new content for me needs to go deeper, not that the site necessarily needs to go that way.

A similar effect: The average age on the internet is younger than you, and will be for the rest of your life. Think about what effect that has on the density of quality user generated content.

Haha great... I'm barely 30 and you punks are already calling me old. :)

College students started calling me old at 27. Just wait til you hit 40!

Heh, I'm 26 and I've started calling myself old.

I'm 22 and feel like an ancient grumpy scottish engineer.

Umm, what? In the limit of everyone going online the average age of internet users becomes the average age of the population at large. Perhaps younger internet users would be more active, especially on social media sites, but that's a different matter.

Current estimates put the average age on the internet at about 37.

His point is that while you used to love some content (teen angst poems, political ramblings by 17 year olds, etc) as you've grown older, you're probably smarter and more mature. The quality of teen angst poem hasn't necessarily declined, but your view of teen angst poems has changed and you view them as worse because you're smarter and more mature.

I think this is a very plausible explanation, we are all at different points in the learning process and can only enjoy certain material. All but the most trivial programming articles are above my head right now, and so while I might enjoy them a lot; the HN community would say they are fluff pieces.

http://marcel-oehler.marcellosendos.ch/comics/ch/1992/07/199...

Calvin and Hobbes, 16 JULY 1992

" One of the joys of being a kid is that experiences are new and therefore more intense. For example, I'm about to stick my nose in a jar of mustard and inhale deeply! Let's see what it's like. WHOOP!!

See, whed you are oder, you dake your sinuses fo granded.

Some of us prefer to.

Random highly-ranked hacker news threads: http://www.skrenta.com/hn/

Yup, this is the one I mentioned in this tree a little earlier.

There are plenty of cached pages in the Internet Archive:

http://waybackmachine.org/20070221033032*/http://news.ycombi...

The first one saved was on Feb 22, 2007, just after the site launched:

http://replay.waybackmachine.org/20070221033032/http://news....

Notice that the second link is to pg's announcement post.

Second cached page, Feb 26, 2007:

http://replay.waybackmachine.org/20070226001637/http://news....

And so forth:

http://replay.waybackmachine.org/20070405032412/http://news....

http://replay.waybackmachine.org/20070406195336/http://news....

Bounce on the ▸ icon to page from day to day.

This one has a gem on it from paul:

http://replay.waybackmachine.org/20071001102019/http://news....

The archive is not complete, but it does give a reasonable glimpse into the recent past of HN.

[Edit: I think I'll go crazy before getting the line breaks to look correct.]

Ah yes, the cycle of website life.

* Hot new community forms at Site X.

* Site X residents refer to themselves as the New Wave of whatever. Much better than older Site W because of features/members/dynamic/demographics 1, 2 and 3!

* Site X's reputation spreads to former hot new sites T, U, V and W. Site X begins to attract more and more new users.

* Site X denizens begin linking articles at T, U, V, W and vice versa.

* Site X begins to exhaust natural topics of conversation. Denizens of more than 3 months standing become sick of 100th "What does Site X think about AlphaGamma?" post and begin to slap down newbies.

* Someone reminisces out loud about the Golden Days of Site X.

* Discussions on Site X become more and more about Site X. Extremely intelligent individuals begin to earnestly argue that their proposed feature will save Site X from itself.

* Someone proposes or launches Site Y. A how new community begins to form there ...

I've been watching this same story play itself out since Slashdot circa 1998.

This comes across as snide. Slashdot circa 2011 is largely unreadable. So is Digg, so is main Reddit. The cycle may be real, but so is the phenomenon driving it.

Sure. All sites experience regression to the mean[1].

Usually this starts happening about 12 months after I decide to become a regular. In HN's case I've come in halfway through the cycle, about 3 months ago, so apparently the magnitude of my destructive powers is increasing.

[1] and also the snide.

>I've been watching this same story play itself out since Slashdot circa 1998.

And every social subculture in history. See: the constant evolution of the social scenes around music genres. It's just a fact of social psychology we have to live with.

"Dubstep was sooo good back in 02 before the students found out about it! I mainly go to future-garage raves now man..."

And Linux was so cool before the Ubuntu kids showed up.

Good point.

On reflection it resembles flocking behaviour. I imagine that website members can be modelled as simple agents with two rules:

1. A cool site has few, impressive members.

2. Move towards the cool sites.

Given an arbitrary distribution of "impressive" members, this should cause a constantly moving flock of members to move from site to site regardless of its purpose.

Add to that:

3. A member's threshold of minimum community coolness below which they will leave, is correlated to their impressiveness.

I figured I'd covered that in rule 1, but upon inspection, it's really two different rules. Your rule 3 is a better statement of one of them.

It strikes me that a logical step to prevent #2 is "allow only new contributing members that have been vouched for by n old members".

Yes, I'm that elitist. And yes, I'm well aware I probably wouldn't get voted in :)

But here's the other part - keep the site open for reading. That means it (hopefully) self-selects for quality, and other people can observe a high-quality discussion.

Do you have any examples of a site where treehousing has improved things?

Apple comes to mind, app-wise.

I was thinking in terms of communities.

Metafilter is often cited. They have a $5 fee for accounts.

Then again, Something Awful is $10...

That's clubhousing, not treehousing. Treehousing would be putting social barriers on entry. Clubhousing would be financial.

These terms are 100% arbitrary and invented an hour ago, but these seem like good definitions.

Dribbble, then.

It'll be interesting to see how that model holds up to another 2-3 years. The few invite-based communities I've been a part of started to degrade as the social graph grew out and started to pick up the occasional undesirable.

It was harder to get rid of them because they knew someone who knew someone, and trying to push them out had a network effect. It might be obvious to someone removed by 3+ levels that a bad element is bad, but the person who invited them might not see it.

Advogato investigated the problem of transitive trust networks pretty thoroughly as I recall. Anyone here able to comment on how they turned out?

I am seriously considering pruning branches, not just leaves of the social graph to keep communities healthy.

I.e. if you invited a bunch of bad apples, you will be pruned too. I have no idea if it would work, but it's pretty obvious that open social graphs beyond a certain threshold are not conducive to high-level discussions.

Zed Shaw's Utu had that idea. Doesn't seem like he's still working on it.

I'm not surprised that I like how Zed Shaw thinks :)

This is how many private bitorrent sites work, or at least claim to work. You are responsible for your invite tree.

A better way might be to nuke the undesirable, put their branch on notice if they start to cause trouble, and remove them if necessary.

It avoids unnecessary disruptions. Not all branches will shake if a leaf falls.

There is a story making rounds about hiring policies of Saint-Petersburg Mint (back in Royal Russia times). Every new hire not only needed three recommendations from current employees, but that should the new person ever get fired, those who recommended him would be let go as well.

Brutal, but brilliant... even if it is not necessary true :)

I assume, of course, that this process wasn't fully recursive, lest the entire mint be fired for one unfortunate hire.

Even then - it's only going UP the tree, not traversing down again to the siblings :)

This is similar to how many private BitTorrent trackers work. If someone you invited gets banned, so do you.

But here's the other, other part: keep the site open for writing as well, but make the non-elites' contributions only visible when you are logged in as a non-elite. Then the elites can continue their excellent discussions unhindered, and any passing stranger can browse the elite discussions: yet everyone can have their say in the agora.

Let's not forget it happens in the actual cities we live in as well! Hipster-ism gives way to gentrification and then the cool kids move because there are way too many yuppies. Same things happens with suburbs - they get too crowded and people start moving to the country to make new suburbs.

This is just human nature, it manifests itself in the clothes we wear, the cars/bikes we drive, the places we live, it certainly also applies to social websites. I'd be surprised if it doesn't - we'll see how facebook fares, or if there will be a coolkid exodus as the rest of the world gloms on.

It's basically a (sub)cultural arms race that everyone fights in.

Rent control has been a pretty effective way of preserving the identities of communities in cities. It's not perfect, but it helps. Concepts similar to rent control can and should be applied to sites trying to preserve certain characteristics.

In my experience (SF, LA) the rent control is there to prevent the gouging of yuppies, not to keep the poor people in. The idea is the city wants to keep rent just high enough to retain the rich people for the tax and spending base but low enough that the landlords to squeeze them out, which they most likely would do if given the chance.

It might be different in places other than California though, like New York.

You can't squeeze out rich people by raising prices without ending up with empty units, which landlords don't want.

Lots of landlords are stupid and will raise rent beyond the market clearing rate just to make a quick buck, resulting in empty units and shitty communities.

No, actually it's rent control that gives way to that. Landlords don't rent at a loss, and prefer keeping their apartments empty, resulting in shitty communities, then squatters, etc.

Both mechanisms seem somewhat plausible. Either of you got any evidence about how often each actually happens?

From the Wikipedia page ( http://en.wikipedia.org/wiki/Rent_control ):

Most economists believe that a ceiling on rents reduces the quality and quantity of housing available.[41][42] This view is based on analysis of empirical evidence as well as the understanding generated by theoretical models.[42] Economists from differing sides of the political spectrum, such as Paul Krugman[43] and Thomas Sowell,[44] have criticized rent regulation as poor economics which, despite its good intentions, leads to the creation of less housing, raises prices, and increases urban blight.

It probably works both ways, depending on exactly how low or high the rents are held. I've lived in some really, really nice areas that are rent controlled (SF Pacific Heights, Santa Monica, Downtown San Diego), apparently others have lived in relatively crappy neighborhoods where rent is also controlled.

Now that I think about it further, it seems obvious that rent control would be a tool that can be used by a municipality to control rents in either direction, depending on their goals.

Here in SF the geniuses of the Lembi Group bought a bunch of apartment buildings on credit (something like 1 billion dollar's worth), used questionable/borderline illegal tactics to squeeze people out of rent controlled units, and then jacked the prices way up. Then the real estate bubble popped and they went bankrupt.

Off-topic, but rent control has terrible problems. It lead to some grave NY neighbourhoods problems from the 60s to the 80s; back in the 20s in Paris...

Everything in moderation.

A big part of political-economic maturity (for me at least) realizing that just because the supply/demand lines from your Econ 101 class don't line up, doesn't mean that something is worthless. It could be that you're not measuring everything. Culture matters, preserving social mobility matters.

Well, to be perfectly exact, in the real-world, for a limited duration, controlled rent can work fine. It just has been applied without discernment too many times.

Sure, you can screw up anything, and I certainly wasn't saying "everything should be rent-controlled".

Just kinda irks me when someone comes in with the Econ 101 argument regarding a policy that's being implemented by people with advanced degrees in city planning. These people took Econ 101, and know more about the subject than you or I. They could still be wrong, of course, but it's not for a silly simplistic reason like that.

Big mistake mentioning future-garage, now I looked it up on Google and know what it is. Pretty soon I'll be showing up uninvited to your raves.

On reading this, the Evaporative Cooling effect comes to mind.

http://blog.bumblebeelabs.com/social-software-sundays-2-the-...

While I basically agree with you--I too have seen this cycle many times before--what I don't understand is why people feel the need to slap down newbies in an attempt to recapture some since-passed golden age.

If I get bored of a site or simply get nothing more out of it, I just move on.

If I see topics that are boring or simply have no interest to me, I just ignore them.

If I see rude, insulting comments or personal attacks, I just flag them and move on.

I really don't understand why people need to express so much hate.

Loss aversion, maybe?

People who are deeply invested in a site want to defend their view of how it should be. Change is threatening, and it's easy to react with knee-jerk hostility.

> Site X begins to exhaust natural topics of conversation. Denizens of more than 3 months standing become sick of 100th "What does Site X think about AlphaGamma?" post and begin to slap down newbies.

This implies it's hostility to "old hat discussions" that have already been around the block that people are worried about.

I don't think that's primarily it. The civility levels have slipped a lot... this place used to be as civil and friendly as meeting a bunch of people at a dinner party where everyone admires the host.

Still super-civil compared to the rest of the internet, but the levels have come down a lot. Raw outrage, profanity, and outright insults used to get downvoted to oblivion even if people agreed with the general argument the commentor was making. Rudeness really wasn't tolerated, so new members learned quickly that you had to think for a minute longer before replying.

It's still a great site, but it's not elitism and reminiscing that the old guard is scared of. It's comment quality slipping to rest-of-internet level.

How about, instead of a single community, the site runs as a set of "cohort" communities that each only allow direct sign-ups for their first year in existence, but then run a bit like leagues thereafter? If people like you on HN/2011, you might be invited into HN/2010, then HN/2009, etc. It could also be automatic/karma-based: if you earn enough karma on HN/2011, you start again on HN/2010 with 1 karma point (though you can also stay on HN/2011, marked as an alumnus/boddhisattva.

Pretty interesting idea. Has anything like this ever been tried with other communities?

I think this would just breed resentment, or at worst Lord Of The Flies/Robbers Cave style anarchy.

[1] https://secure.wikimedia.org/wikipedia/en/wiki/Realistic_con...

When you have two groups that have no reason to not be the same group, the fact that they're separated breeds resentment. When you tell them that there's a criteria they were ranked upon to put them into these groups (in this case, age + karma), they understand. For example, there's no big rivalry between grad students and undergrads; one graduated to become the other.

>For example, there's no big rivalry between grad students and undergrads; one graduated to become the other.

All undergraduates know that they will soon be at least graduates, if not later postgrads. The transition is inevitable.

Any kind of stratified internet community would have to have the eventual prospect of transfer from one strata to the other, as an inevitable consequence of good/bad participation.

As soon as it becomes set in stone, or the transfer becomes a deliberate choice rather than an inevitable consequence of behaviour, I think people will forget about the criteria that originally divided them and slowly regress to hatred of the other.

A recursive treehouse sounds like an idea worth trying.

Just because this has been the typical website lifecycle in the past, doesn't mean that it has to be the typical website lifecycle in the future. We increasingly have better and better tools to manage how people interact with sites and communities.

Upvoting/downvoting, filtering, disemvoweling, moderation, tagging, reputation systems, etc. are just a smattering of the tools we have available and with time it's likely that we will see more and better tools for community management available in the future.

The most important thing is for "us" to define exactly what current and past features of the community we want to preserve and also decide what kind and amount of "chaos" we want to permit to allow the site to evolve and improve. It makes the most sense for PG, YC founders and very high-karma users to be responsible for defining these community characteristics, ideals and values.

The problem is that many of those tools are affected negatively by the regression to the mean as well. An average comment, that would never be upvoted during the early elite, will be upvoted by average people during the mean. Reputation follows.

The question to be answered then is "what mechanisms of reputation and combinatorial game mechanics can prevent regression towards the mean?"

I don't have an answer to this off the top of my head, but succinctly describing the problem as you just did oftentimes can be half the battle.

Another formulation:

"What system can allow a site to absorb N new users without causing regression to the mean, with the constraint the membership is open?"

I'm pretty sure the answer is: no such system exists.

Eigenvalue/SVD upvoting. Comments and articles are ranked for you according to their ranking with similarly voting/submitting/commenting users.

The frontpage (depending on how you set it up) could look like Reddit's, based on the majority of users, but to logged-in users it'd be more refined to their tastes. Perhaps you could look at the site from others' perspectives.

Defining the model, computing likenesses and finding the appropriate material could be quite resource intensive if you didn't think long and hard about the implementation.

(You'd might also want to avoid up and down arrows, because "like" and "dislike" quickly supplant "good comment" and "bad comment".)

The very first time I saw a site like this, many years ago, that is what I assumed all of the voting was about. I know a lot more now than I did then why resource wise it is the way it is, but I occasionally think about how one might go about doing a more customized version.

Superficially at least, it seems to be a good way to put everyone into a community whose size relates to the commonness of their preferences. I think this would occur as long as comments/articles beneath a certain threshold just weren't visible to a particular user.

Unfortunately it seems like a lot of calculating to do per hit, but in thinking about it, it doesn't seem to hurt Netflix too badly.

If it's too much complexity to track each user's preferences in terms of every other user's, you could also take a Bayesian filtering approach, similar to junk mail controls, and compute a custom front page based on each individual user's prior upvotes/downvotes.

In either case, the function of the voting mechanism will ultimately shift from generating the public front page to generating a custom view for each user. At that point, you could re-purpose the public front page as a selection mechanism to draw in the highest-quality users - perhaps by choosing a cohort of existing users to have public-upvote rights, or even by just applying old-fashioned editorial controls.

The core issue here (IMHO) can be summed up as:

* We all (and I'm aware I'm a pretty young account myself) vote on stories and comments to rank their quality

* Newer users vote less in line with the historic ethos because they're less aware of it

* All our votes have equal power (the downvote filter here is a crude effort to suppress that but isn't powerful enough to be fully effective)

* Network effect means a successful site's growth will be closer to exponential than linear

* With exponential growth the new who are unaware of the community standards are quickly able to overpower the existing who are aware

Hence almost inevitable degradation in quality which people here are complaining about and which I can partially see myself.

Two ways to partially address this spring to mind:

* Express vote power as a function of reputation - an upgrade to the downvote system. This has the problem that it provides a direct incentive to align with groupthink and post the easy but commonly held stories and threads so as to attract repuation and so increase one's voting power and.... It also, I think, requires floating point reputation and scoring, and so increases computational load.

* Display voting based on a similarity function - 'users who liked this also liked....'. This permits cohorts of users to work en masse without prior coordination to lower the ranking of content they find less interesting / valuable and so get what they want more easily. This means though that it increases fragmentation and reinforces groupthink again by design, which is clearly not ideal. It's also again significantly more computationally expensive.

TL;DR: Computation can be thrown at partial improvements but they're expensive to operate, open to gaming and likely to reinforce groupthink. I'm not sure it's socially or technically possible to keep the noisy brats out without severely damaging what attracted the Grand Old Users in the first place. Welcome to social groups...

What about disabling voting until a user reaches a certain amount of karma? Of course, you'd have to give the upvote ability to the original members (hand selected?) so that the system could actually get off the ground.

It would be similar to the "invite only" community idea where older users have to actually approve new members, but less restrictive. Perhaps even have upvotes on users instead of just comments. It would be similar to an "invite", but just for allowing them to vote. I'd say also keep the hellbanning that we have here. There will sadly be some false positives, but it seems to work very well for keeping out the worst.

Perhaps upvoting users could affect the reputation of the folks doing the upvoting. If you upvote a bad user into the community, your reputation suffers as well.

Now, this does nothing to solve the problem of groupthink, and in fact somewhat enforces it. However, I think some amount of groupthink is inevitable if you want to create a strong sense of community. Because you're trying to cluster users around a core set of values, it's probable that they'll think similarly in other ways as well.

Edit: I think multidimensional voting helps as well. Ignoring the comments voted "funny" on /. helps to filter out what people liked just because it was amusing. I think "agree/disagree" voting would be useful as a separate thing, because too often that winds up conflicting with upvoting/downvoting for quality.

Interestingly, this isn't a problem related to any particular category of site. Hell, look at 4chan: bitching about newfags and how /b/ isn't good anymore, how /b/ was never good, different factions with different ideas on how to fix the problem (or what the problem is, or if there even is one), spinoff sites where the early adopters migrate to...

This isn't an HN problem, it's an Internet problem. pg could probably do some things to ward off dilution, but it won't last forever. Interestingly enough, Something Awful seemed to be pretty damned successful in this regard by doing the unthinkable and charging people to post there. I'd hate to suggest it as a solution, but it has been effective in the past.

pg could probably do some things to ward off dilution, but it won't last forever.

I think a paraphrase of this sentence has appeared in HN comments 100's, perhaps 1000's of times.

yep. and in all this time, pg has managed to keep the site hovering just above the shark, as he once put it.

SA has internal problems too. There's a constant slow cycle where the individual topic forums become boring and straightlaced because the users get proud of how mature they are, so they start new subforums where people are allowed to make "bad" but interesting posts as long as they pretend to be doing it ironically. Then everyone moves into those and it collapses, because they really did become bad posts.

Sometimes they move offsite instead, usually because the site admins didn't actually like the topic enough to make more forums for them, or because the admins banned all the users because they had formed secret IRC cabals conspiring against them.

The last stage is that the admins change the forum CSS so it's unreadable. I haven't figured out that part yet.

The gist of that (forums "cancer") is that those users "infected" with it have their posts become harder to read outside of those subforums by lightening the font. A quick mouseover returns the text color to black.

In my opinion, the method doesn't really work because they aren't reading their own posts. It should probably change everyone else's text color instead of their own. Greasemonkey scripts and the like would still fix it, but it would be a lot more annoying to the shitposters that way.

Still, for a site that's been up since the Dark Ages ('99), and relies almost entirely on its users to keep it relevant, it's been remarkably successful.

I wonder how different the website lifecycle is from the cycle of nightlife hotspots.

I agree. One thing tho that never seems to come up: we people from older waves ignore the fact that we spent the last $N years growing, getting better, getting deeper into stuff, while the site has not grown with us. So even if the site hasn't changed, our perception of the topics and such has -- intro to X used to be deep, since we were little, but now that we are big, we just see a puddle :)

If you are right, a website could automate the cycle as follows:

1) Every user belongs to at most two (or k) classes.

2) Each class can have a finite number of users n.

3) When a class fills up, a new one is opened.

4) There are always two (or k) open classes at a time, and new users join both of those two (or k) classes.

5) The classes are staggered so they don't close at the same time; e.g. if k=2, n=10000, the first 5000 people to join belong to class 1 and class 2, the next 5000 belong to class 2 and class 3, the next 5000 to class 3 and class 4, and so on.

6) Users can only post to classes they are in, and can only comment on posts in classes they are in (or maybe newer ones too).

7) By default, users only see posts to classes they are in (or maybe their classes + older ones). They can opt in to see more if they want.

The idea might work better if it is combined with some more onerous mechanism to join classes late - a fee, a requirement for an endorsement from a member or someone. Charging a fee for access to the more '1337' earlier classes might be a good way to monetise the site; the fees could either be flat, or could escalate (exponentially?) with the age of the class and / or the number of people who pay to join it late (possibly with a discount applied as active members leave).

Classes might get too small due to attrition - coalescing older classes when they get too small might be a solution to this - it might also make sense to increase class sizes (exponentially?) as the site gets more popular.

For what it's worth: I feel safe saying that most high-karma users of HN have a variety of severe concerns with it. My experience asking this question over email has generally been one of getting gigantic essay-length responses.

In my official capacity as "representative of people dorky enough to have karma this high", we do officially declare: stuff's broken. Needs unbreaking.

Did you forward those essays to the powers that be?

No. I'm high-drama enough as it is.

Well, based on what I've seen so far, PG usually asks for datasets that support the point that one is making about the degradation of the community. If the essays that you have are just long rants, they will be easy to ignore. However, if there exists a dataset that supports those various points, as well as suggestions on how to change HN, PG usually seems very reasonable about trying them out.

If there is one thing that causes an internet community to break down, its when a subsection of that community has too muh communication out-of-band in another medium that the rest of the members are unaware of. Especially if that other medium is more synchronous than the primary medium.

Many forums have been driven to civil war by a clique plotting on a secret IRC channel. (Note: I am not suggesting that is happening here! Just that I've seen it before.)

What was the point of this comment? (I'm asking seriously; I want to know what it is you're specifically trying to communicate here).

It was an innocent observation: I was sharing an anecdote about my experiences on other internet communities.

The mention of the fact that the higher-karma, "ingroup" members of HN seem more likely to communicate with each other on twitter and in private emails than the rest of the population reminded me of bad situations I had seen on other forums.

I then thought how glad I was that here on HN it seems to happen on twitter which is still largely public. You commented quickly here about private conversations regarding HN with other high-karma members. It is reassuring that you were open, and happy to be open about this.

This gives me faith in what I find to be an excellent community that the inevitable psychological group-dynamics type phenomena are playing themselves out here in a relatively benign way, due to its high calibre membership. Your later comment "I'm too high drama for that" came across as an amusingly wry confirmation that you feel the same way.

As I said, this put me in mind of situations I have seen in lesser communities where this dynamic had led to ugly scenes. I posted about them. I then felt bad that you or someone else might misinterpret this as an accusation against them. Thats why I added my parenthetical note. Perhaps I should have reconsidered the whole thing.

For all the newer people, I encourage everybody to come by the HN secret cabal IRC channel, #startups. I found that the term "secret cabal" really sounds much more exciting than the actual channel ;)I try to go by at least once a week, though. It's fun to be in more of a real-time discussion.

I would also encourage people to look up folks' email and email them with questions and such. Great way to make new friends. Many a time I've tracked somebody down from an HN discussion only to have it continue via email. (It also brings the noise factor down over here, which is always a good thing)

I know a lot of guys follow everybody they can from HN on Twitter. I don't know if I would personally go that far -- Twitter has a hecuva lot of noise and I get easily distracted -- but it works for some.

So yes, there's a lot of out-of-band chat going on, but as far as I know, none of it is exclusive. If you'd like to participate more, the same channels are open for anybody. I know I keep my Twitter, Facebook, and email addresses in my profile for just that very reason.

I encourage everybody to come by the HN secret cabal IRC channel, #startups.

Do you mean through here: http://www.ircnet.org/ ?

If so, either I'm doing something wrong or the Secret Cabal is not in session tonight (darnit).

I think it's on freenode.

Yes, it is on FreeNode. Which is (in my opinion) to other IRC networks what HN is to other link aggregators.

There seems to be a regular effort to help HN posters connect with each other. Doesn't seem like a month or two goes by without another site that offers that.

Still my favorite site on the whole internet, but yeah, things have definitely changed.

Stories, yeah, they're a bit worse but that's not the main thing. I skip stories that don't appeal to me, and you can always go a few pages deep if you want more stories.

It's the comment quality sliding down that's scary to me... it's becoming much more okay to engage in standard-internet-level discussion here, which is to say, much less civil than HN used to be.

Perhaps liberally killing any comment that's primarily insulting, sarcastic, pedantic would help?

For instance, I just saw a link to this two year old thread today, look how civil and friendly it is:

http://news.ycombinator.com/item?id=554889

The only halfway insult on there - "Dunno but she has a Twitter page. Not sure why PG would quote such a weirdo." - was voted to zero. And the 2-3 jokes/sarcasm stood still at 1, none of it voted up.

Also, I'd love to see people who just consistently left substance-less comments get put on auto-kill, but I know that'd never happen for a ton of reasons.

I'm thinking of one guy who just posts about how outraged he is about anything remotely politics related, and nothing else. Joined in the last six months. Actually, that probably describes a lot of users now, which might be part of the problem...

Case in point: here's a thread from 2 years ago where we all talked about atheism. Civilly.

http://news.ycombinator.com/item?id=236808

You couldn't get anywhere near that today. Discussing tough and interesting subjects civilly with some of the smartest people I know wasn't what brought me to hn (startups were), but it was a big factor in keeping me here. I knew if I posted some crazy half-assed startup idea that might piss people off that the community could talk about it logically and dispassionately. That allowed each of us to open up more.

Those days are gone.

Those days are gone.

I blame karma. I know it existed back then too but I don't think people talked about it. There should be no "top users" page. Having a measuring stick drives competition and there should be no metric to civil discussion besides the rewarding feeling of discovery and helping people.

That thread contains this comment decrying the state of HN:

http://news.ycombinator.com/item?id=237127

Admittedly, it wasn't voted up, but there was some agreement with it. You're looking back wistfully at a point where another user thought HN had fallen off precipitously.

I don't see a significant difference between HN now versus about 3 years ago (when I first started lurking), and I believe the community could and (on most days) would have the very discussion you've linked. There are cycles where many of the submissions aren't interesting to me. I usually chalk those up to slow news days. And there have been situations where I've been appalled and infuriated by (what I perceived as) the poor quality of articles/comments that the community was voting up. Some of that is the community offending my own biases. Some of that is just bad days. You can't discern the climate by just looking at today's weather.

The other cycle I've noticed is these efforts at self-policing and reflection. I'm almost annoyed by them because I don't see HN as getting worse. But I'll allow that others are more sensitive to HN's quality. Perhaps these reflective posts are what keep HN's quality high enough for my tastes. Or maybe, as I suspect, they are more about the tastes and perceptions of the long-time users changing. It's more than a little reminiscent of the "golden age" thinking of older generations, where the past was simpler and more innocent because they were younger and insulated from the time's complexity and injustice.

(Sorry for rambling there.)

You're looking back wistfully at a point where another user thought HN had fallen off precipitously.

Yes, I think that's what made it a good example.

The other cycle I've noticed is these efforts at self-policing and reflection. I'm almost annoyed by them because I don't see HN as getting worse.

Yes, it's almost an inside joke. One of the "classic" HN stories is the meta story where we all lament the decaying state of HN. The "naval gazing" stories. You learn either to ignore them or vent (politely) After watching them for a while, I think they serve to help release a lot of frustration people have.

I would note that many of the folks early on who said HN was fine are now also saying there are problems.

To me, HN's core problem was the problem you always have when speaking to more than five people: trying to anticipate reactions. You end up trying to cover every possible response, and it kills the tone. One of the reasons my comments run on so long is that I have to cover every obvious way they could be misunderstood. You're stuck either not replying to anybody, writing long comments, or getting in endless debates. All of that is distasteful to me.

One of the hard lessons I had to learn blogging was not to tease people. I used to put pictures of girls in bikinis up or joke around about making money tricking people online -- to me I was just playing. I live by my reputation: being an internet porn mogul is not on my list of future plans. But there were a heckuva lot of folks who didn't understand my posts, who thought that made me some kind of evil person. It was really sad to have to give up the playful part of my personality simply to get along with people because of the many ways things can be misunderstood. I hate that.

But the real problem is that everybody's personality will piss somebody off somewhere. I know I probably rub thousands of people the wrong way just because I'm the way I am. Others do the same. So then I end up commenting something like "Looks like it is raining outside" and I get 3 or 4 downvotes. Sure, the upvotes outweigh them, but it's just silly nonsense. Vengeance voting.

Hey now I've rambled too, so we're even :)

In my official capacity as commenting wind-bag, I'll second that.

HN started as a way for YC guys to communicate. It was about startups, the tips and techniques to make one, to grow one. Once PG opened it up, it became "stuff hackers like", which meant cool deep tech stuff. But the tech stuff was there because after a while, all the startup stuff just ran together.

Then "stuff hackers like" kept drifting wider and wider. I was happy with that, but now that the traffic numbers are through the roof, what I'm seeing is the 1% snark factor is out of control. If each member of HN said something truly snarky only once a month, there would still be thousands of snarky comments a day on here. It's just too much noise.

HN has always been driven by emotion: the only reason to click that button is to express your emotional response, good or bad. Sure you can rationalize that in various ways, but in principle it's about how the thing makes you feel.

Now we have the love-fest stories, the hate-fest stories, the snide jokes that get upvoted, the trashing of "rate my startup" posts (I still can't believe how some of the startups are treated). It's becoming less of a community and more of a mob.

I think maybe the story titles might look mostly the same, perhaps, but the quality of the stories and the quality of the conversations have changed quite a bit.

Excellent point and thanks for all your thoughtful comments.

I use to comment regularly on HN (member since Apr 2007 and former #1 on leaderboard in the ancient days, haha), but have found that I very rarely do any more.

When I look at my thought process now when thinking about posting something that may be of value, I find myself very reluctant. This is due to the snarkiness and hot-headed emotion that often disregards logical arguments as worthless and instead values breathy spittled anger.

Cowards become brave behind anonymity.

Who enjoys "lawyering" their comments? Basically trying to anticipate and address all the possible concerns hot-head users will bring up. Yet, I feel like I've had to do that more and more. In the end it seems an exercise in futility - particularly when I can just move on to interacting directly with the thoughtful hackers I care about and avoid those that spit when they talk.

I can understand the lament of the OP since I've really enjoyed the community and associating with the gentleman-hackers I've met here.

Thanks especially to those who have added true value here - I've learned a lot from the thoughtful ones who share and help.

Maybe upvotes/downvotes could be weighted by the karma of the person giving them, thus putting more emphasis on those with high karma (which might sound elitist, but the only reason those people have high karma is because people think they talk sense).

This would concentrate power slightly in the hands of those long time dedicated commentators ('the community'), and slightly lower the strength of newcomers who are still getting used to the ettiquette ('the mob').

Hopefully this would go someway to lowering the effect of newcomers, until they rack up enough karma themselves to 'graduate' into the higher karma ratings.

Wasn't this the same mechanic behind Digg?

Agreed -- I've always enjoyed my debates with you. I'd go so far as to say I sort of miss them. But I've now grown so frustrated by the pedantic replies that I get on HN nowadays that I've had to start deleting my comments after I write them. And I've stopped visiting as often because it takes too much effort to write a response that I feel avoids the trollish responses I fear I will get.

One of the final straws for me was when the title of my submission about being replaced as CEO was changed -- a topic of significant relevance to this audience I think. What frustrated me most was that I had picked a useful title and the not only was the new title worse in my eyes, but it was surrounded by rather inane posts about the "top X reasons for Y" or something about Apple or Google. It just didn't add up for me.

And on a tactical level, there is unquestionably a tendency for people to respond to the top post rather than start a new top-level thread. The lack of collapsing of threads is quite frustrating on long topics.

For now, I'm mostly done here.

In my official capacity as "a relative newcomer", I agree that stuff's broken. I think one of the challenges the site is facing is a question of how much to continue to cater to the old guard and those who share their perspectives, and how much to evolve? The start world in general (and YC in particular) is at a very different place than when HN started ...

I also think there are problems (though I've not spoken with you specifically). However, I think this is a universal problem that doesn't necessarily need a solution. Most sites seem to have a group of hardcore participants with axes to grind. Top Wikipedians have lots of issues with Wikipedia. Top eBay sellers have problems with eBay. Even on MetaFilter it's not always plain sailing..

All of this has happened before, and all of this will happen again. HN is succumbing to the problem pg tried to address: dilution. Thing is, reddit already came out with an excellent solution with their subreddit system. This simply wouldn't be an issue if there were different sections for tech, hacking, programming, startups, science, finance, and general interest. Keep all of the deeply technical stuff in one place, the cruft in another. Let's face it, the people who are complaining about lack fo deep tech are also likely to read and enjoy one of Spolsky's blog posts. There's no need to ban the latter to protect the former, just keep them in separate sections.

Now, HN isn't trying to grow, so there's no need to have user-created subreddits (sections, I suppose). Just make 8 or so that people care about, and add another if there's sufficient demand.

I really shouldn't be crediting reddit with this, as the solution existed long before them. All HN needs to do is follow the forum model and have different sections. It's too big to only have the front page.

We have become exceedingly efficient at ruining online communities.

As a longtime MeFite and a longtime member of this community, I believe that most of the issues could be dealt with by having obvious and active moderation.

As MetaFilter, not only do we know who the mods are, we know which mods are on call at what times. (And there's 24/7 coverage.) HN relies very heavily on a flagging system, but it's just not as responsive to stuff that is broken as is a human who's responsible for what's on the front page and what's in the comments. Having a handful of humans who are responsible for curating the front page (and possibly also pinning really good stories from new onto the front page) would solve most of these problems. Is this less democratic? Sure it is. Would the unfairness be worth it? In my opinion, yes.

This problem just isn't solvable with code; it takes benevolent dictators.

What's the threshold HN crossed where it started needing active moderation? Because '08 HN threads were just better than '11 threads. Is it just a number of users, past which no set of guidelines restrains pathological conversations? Why? If you can trace it back to what thing happens when you get to your 50,000th user, maybe there's some passive moderation mechanism you can identify to keep it in check.

I'm not sure it's number of users--- Kuro5hin's downhill slide roughly coincided with a net loss of users, and its heydey was probably also its most populated era.

'08 HN seemed pretty similar to me, though, though I was just a lurker. It was a mixture of good technical content, startup-scene celebrity watching, too many TechCrunch posts, a vaguely political thread every few days (sometimes with a weak "hacking" justification), and the periodic thread once every 3-4 weeks about how HN is dying / has too many political posts / is turning into reddit. Just fewer total comments and votes (by a good margin).

Some random examples: http://news.ycombinator.com/item?id=179755 , http://news.ycombinator.com/item?id=105739 , http://news.ycombinator.com/item?id=109052 , http://news.ycombinator.com/item?id=80234 , http://news.ycombinator.com/item?id=437727

Theory on the threshold: the point at which groups that social engineer social news sites decide the site is worth their time. This is also a point where the population has grown to a certain point and the discourse has regressed, providing a larger and more easily manipulated audience.

For me, I noticed it when people started posting things from investor blogs and the VCs and angels themselves started commenting here. I think that spurred some small handful of users to start showing off how smart they were in hopes that they'd get a response from fredwilson or whoever, and that just naturally spread the way it did. (There's a certain high when you get fredwilson or msuster to respond to your comment on their blogs; I think it's the same dynamic here.) I'm not blaming the VCs at all here - but that's when I started noticing it. (joshu and yegg are the exceptions that prove the rule because they're hackers first.)

Also, I've been told the Something Awful community is surprisingly pretty awesome, although I'm not a participant there. If there's some intersection of structure/moderation between MeFi and SA that HN could implement, I think that'd be a great place to start.

sighs We complain because we care. It doesn't mean that we have all the answers or we're pointing fingers. It means we're so invested in finding a solution to this problem that we won't just move on. Given just how good HN was (and remains), minute changes are more noticeable. I don't think a comparison of then-and-now really gives us useful data. There's just a gestalt, a feeling, you get that something's not quite the same. The closest I can think of is when you're out to dinner with someone you've fallen out of love with; your routine isn't any different, but you just know that it's not the same.

Something Awful and Metafilter both charge money to join. Something Awful is pretty awesome...but is double-sided. The people from Something Awful are the same people who founded 4chan. ADTRW (Anime Death Tentacle Rape Whorehouse) is one of the boards of Something Awful and they were the first people on /b/ the day it opened along with 4chan and have set the bar to where it is since.

They are basically a grown up 4chan...amazing things like Copenhagen Suborbitals http://www.copenhagensuborbitals.com/ but aren't all pretentious and srs business like Hacker News likes to pretend it is.

SomethingAwful was (is?) good because of the membership fee, and because of the viciously strict moderation.

You could have your account deleted, with no warning, for:

    * Persistent lack of punctuation or grammar
    * Posting tired memes
    * Persistent inability to cite sources for claims (in some subfora)
    * Dozens of other things I can't remember
    * Basically anything the mods decided to ban your for

The reason this worked is because the mods were drawn from the community. The admins would look at a subforum, see who was trusted on there, had good judgement and was there a lot. They would then make that person a mod, without asking them or even telling them! That person would just log in next time to be confronted with lots of extra widgets on the UI.

Because the community was so cohesive across the site, that person would then know immediately it was their duty to post a thread saying "Hi, I'm XYZ, your new Moderator" and christen their new powers by stickying the thread.

Again, they were not told to do this, but they would immediately and without fail, because the criteria for being chosen as a mod means that anyone who is chosen would be aware of this tradition, and in fact every other aspect of their role, having learned by example.

The SA forums were operating lots of quite deep game mechanics stuff nigh on ten years ago. It's very interesting. Most of the measures wouldn't be appropriate for here though, a lot of them involved strategically fostering antagonism for the benefit of wider community cohesion. It really worked over there for a long time though.

EDIT: I seem to remember that for a long time they only took payment by actual credit cards, visa and so on, and not debit cards, paypal etc. This was deliberate to prevent kids from signing up.

I think there's a real disconnect between HN and Metafilter. Metafilter is about making good posts. There are tens of thousands of active members and only a few dozen posts per day because you are supposed to make good posts. The motto is "best of the web" and it is a serious motto.

By design (I think?) HN is not about making good posts; it's exactly the opposite. It's about posting whatever link you happen to stumble across. The hope is that maybe the good posts will rise to the top via voting, and that comments will add value to bad posts. This is a really different idea! I'm not personally convinced that it's possible to get a high volume of visible good posts using this idea in an open general-interest community, because no matter what you do, there will always be more people ready to vote for popularized, lurid, and flamebait posts than good posts.

I think that distinction has much more of an effect on content than moderation. (Although if you've decided that you only want good posts, you probably need moderation to enforce your decision.)

I'm probably one of those darned newbies who isn't a real hacker and is screwing the place up. (Sorry.) So I wasn't around in 2008 (or whenever the Glory Days were). I don't feel like this article or other discussions about the issue have really given me a good idea of what HN supposedly once was that it isn't anymore. I wish I could get such info. I think that kind of information would hold out some hope of figuring out a real solution -- a means to raise the bar or deepen the discussion or whatever it is that people are wanting.

I know there are other large forums on the internet but this is the largest one I have personally participated in. I think such large forums are breaking new ground, socially, in ways that do not compare to sites like Facebook. Where else can I actually speak with my 80K closest friends? If I am in a room of 500 at work (and not on the stage, because I am not one of the big wigs), only a handful of people around me can hear anything I say. We all can listen to the presentation, but we cannot converse. Here, any and all of us can converse. It is unlike anything you can do "IRL". I suspect that is part of the issue: No one really has a model for how you manage that kind of social interaction. And the models we do have break in that setting.

Just thinking out loud.

Part of "the glory days" was a much more interesting debate. Currently, lots of HN comments are either going for the cute/snide/sarcastic joke, or argue dogmatic points.

Worse - the snide remarks get modded up. Being guilty of them myself, let me post an example: http://news.ycombinator.com/item?id=2202958

The discussion was about Google's two factor auth. Somebody asks for a good API, poster #2 suggests OpenID. I joke "you read the part about 'good' API". It's by far the highest-modded comment I have. (Which makes me feel guilty every time I see it). Sure, I don't have a lot of comments or karma, so it might be a fluke, but it exemplifies what's wrong with the discussion.

HN'08 would've modded me to 0 for that and moved on. HN'11 rewards playing to the audience.

A couple of real quick and dirty thoughts:

A) Meatier comments/debates require a certain level of trust that the debate is about ideas and not about pecking order. That is enormously hard to achieve in most settings. It gets harder when the community grows. A smaller group is much more able to know each other and all that.

B) My best understanding is that formality is what older cultures with larger populations moved towards as a solution for such problems. I know I find some stuff at work endlessly annoying because of the disconnect between the setting and the social behaviors/assumptions. My mom is European and my dad was career military and my ex husband was career military. Europe and the American military are both more formal cultures than the American Deep South where I grew up (and live currently). I find some of the assumptions of more 'casual' cultures to be very uncouth.

I don't know how to raise the bar on formality for a place like HN but my best guess is that would help with this particular issue. I like a good, healthy debate. I rarely engage in it because most "debates" are really fights (ie they are "arguments" in the other sense of the word -- emotional, social, ugly), not intellectual discussions.

Anyway thanks for tossing me a clue. That does help me think about the issue, unlike most of what I read in these discussions.

That is an interesting thought.. I've never looked at it from that angle, but formality is certainly a buffer against "pecking order" fights. Then again, as much as I'm happy in a less casual setting, I wouldn't go so far as addressing fellow HN posters as "Sir" or "Ma'am" :) (But maybe I should...)

It's also interesting that in the last 6 months or so I ran across a couple of sites that take your HN karma as a predictor of your geek-worthiness. Well-intentioned, I'm sure, but it encourages gaming the system. HN karma now is not only something valuable to the HN community, but outside.

It would be very interesting to see if kuro5hin/slashdot/reddit "decline" periods correlate with wider acceptance of their karma metric.

I don't know that formal culture necessarily has that much to do with using titles (though that can be a part of it). I think it just doesn't assume familiarity. Informal cultures assume a degree of familiarity which you logically can't achieve with your 80K closest friends and I think that is one of the roots of evil here, so to speak. I read a story once about a young American man who had an affair with a British woman and then ran into trouble because when he met her in public, she was offended that he behaved in a familiar fashion merely because they had slept together. Her view: "We've never been formally introduced."

That may make no sense to many people here. I can't think of a better example though right now. Formality gives people a certain degree of social space that a "small town" mentality (for lack of a better phrase) fails to provide. That space reduces friction enormously.

The snide remarks got voted up in 2008 also. For a long time, my top rated comment (posted in 2008, 50 upvotes) was nothing but snark.

http://news.ycombinator.com/item?id=318690

(In contrast, my most valuable submission to HN ever only got 14 upvotes: http://news.ycombinator.com/item?id=250027 )

Forking of sites has the same problems as forking open-source projects. It exacerbates conflict, forces people to choose sides, and ultimately both forks typically end up poorer because of members lost.

Instead, the best solution is to evolve Hacker News as a product.

My personal opinion is that we should put Hacker news in the hands of the YCombinator alumni. Founders and first employees (CEO, CTO, lead designer, first engineer hire and first design hire) of YC startups would probably make the best moderators and admins.

In fact, I would say that it's probably time that PG spin off YC as a full-time startup, assigning control of the design and codebase to one talented UI designer, one talented developer and one talented product manager.

For the site to keep growing in a way that maintains quality, it needs more functionality that it has. The two features that lack the most are filtering and combinatorial game mechanics.

Filtering is necessary so it is easy for the the hardcore tech articles to be easily found by high-karma members, so they can vote those articles up. If it's not findable, it's not voteable. Filtering is also necessary for people to extract the most value out of hacker news. Most users don't want 100 front-page articles everyday. They probably want 10-20 of the highest value articles. Less is more.

Combinatorial game mechanics like those on StackOverflow would help as well. Upvoting/downvoting is limited in that it will always fall victim to the masses. Giving special voting/tagging/burying rights to distinguished members (very high-karma users and YC founders and employees) would go a long way to helping eliminate the crap.

I think I speak for most members here, when I say that I don't want Hacker News to be a democracy. I want it to be a technocracy. I want the smart and accomplished people to control what is good and should be visible to all. I've got only 260 karma points, and personally I don't think that should be enough karma points to allow me to upvote a submission. 500+ karma points should be the threshold to be able to vote an article to the frontpage.

Even game mechanics have a problem.

Say your valued members have a certain mindset, ethos and style. Let's call it YC-meme. Everyone with YC-meme contributes in a manner that exemplifies YC-meme. If someone comes along and doesn't have that meme, their posts tend to not be what the memesters want. So, members use their voting powers to reduce that person's karma. Eventually they either leave, get ignored by all, or become infected with YC-meme.

Now, open up the community to everyone. At first, YC-meme is still dominant. Only 1% of people without it are able to write posts in a style similar enough to gather karma. Let's say they have RD-meme, which has about 80% overlap with YC-meme. Great.

The problem is, soon, there isn't 1% of the community with RD-meme, there's 10%. Who can (and do) re-enforce RD-meme posts, because it's what they want. People can even get lucky and simply contribute 'good' posts, long enough that they get a decent karma score, and so can re-enforce different memes.

What makes it WORSE is that what's acceptible is what's visible, so with more RD-meme posts, RD-meme posts become more likely, as the probability that they get 'punished' decreases.

End result: Community-dilution runaway.

There are a few 'cures' I can think of, but most are fairly prescriptive:

1. A concrete set of rules and behaviors that anyone can reference to moderate from. Gross. 2. A very high bar for moderation, with active participation. Given that the people most likely to have YC-meme are also likely to be busy, this might be doomed by design. 3. "Proof" required for contribution. Great way to kill off your community. 4. Invitations. Either so easy to get as to be meaningless, or so hard as to stagnate the community. 5. Close ranks to just people at/gone from YC. Sadly, that'd mean I was out. May still be the best alternative.

Maybe the community is self-aware enough to start consciously moderating better. Perhaps a post like this every so often might serve as a gentle reminder. That'd be the best outcome, but needs to be chosen by a majority of the user base. Do YOU want a stricter HN?

2. A very high bar for moderation, with active participation.

The reason this is doomed is: there is currently much more stuff needing moderation than the selected few can handle. They would soon wear out and you'll end up with a site with almost no moderation at all.

The trash needs to be stopped at the front door. When it gets into the house, it takes too much effort to clean up.

both forks typically end up poorer because of members lost.

I strongly disagree. Off the top of my head: XFree86/Xorg, Mozilla/Firefox, Debian/Ubuntu, Rails/Merb... these were all cases where the existing community had calcified in some key area and a fork was necessary to create a space where important work could get done.

And in most of those cases, not only was the fork stronger than the trunk in (some) specific areas, the fork actually led to the trunk getting stronger. Not only were patches merged, but entire projects, community values, development practices, etc were.

XFree86 is an exception, but organizational paralysis was what necessitated the fork, so it's not surprising they were also unable to respond to the fork.

But I just don't buy your notion that forking is somehow unhealthy and destructive. It's one of the most valuable ways we have to keep open source code, ideas, organizations and practices fresh, healthy, and vibrant.

I agree that there are exceptions. In the case of HN, I don't see how forking would be of benefit, especially since HN is so strongly linked to YCombinator, the startup incubator/school that tends to attract the most talented up-and-coming startups.

Another reason HN may be boring: we've beaten a lot of the common topics to death. It only takes a handful of articles about "how to pitch a VC" to soak up most of the relevant advice on the subject. While posts often present a unique combination of previously mentioned ideas, it's becoming increasingly rare to actually find something new if you are a regular here.

This may also explain why even older members are posting things further away from the core hacker news topics. Something about health care or law or economics might seem more interesting than a pitch a VC article because we have all read many many opinions on the latter.

I think good technical stuff though is evergreen because there are always new languages, libraries and techniques.

It's worth noting that this is also true in Magazines, where they have strong editorial review, and an actual staff of people creating content.

Once you've read 12 months of Runners World, you've basically read every Runners World that has been, or ever will be, written.

As a low-karma, long-time lurker, I'm not sure I've ever really seen the kind of submissions he is looking for. Can someone provide examples of submitted content that would meet his criteria of deeply technical discussion worthy news?

I keep reloading this comment because I'm in the same boat as jmm57 and would love to see some examples as well.

Out of curiosity, is Hacker Monthly (http://hackermonthly.com/) what al3x and other long-time members want? Or is it the kind of discussion & content seen on a StackOverflow?

http://news.ycombinator.com/item?id=1304354

http://news.ycombinator.com/item?id=1321443

How to solve the signal/noise problem? Amplify the signal.

Call it undemocratic, but insight and perspicacity is not uniformly distributed so it's absurd that pg/$whoever_you_respect's upvote on an article counts as much as anyone else.

As a simple experiment, it would be interesting to see a view of the frontpage based only on the upvotes of people who are above a certain avg-comment karma threshold (since the site is predicated on karma as a quality indicator) and the idea that people who write insightful comments won't upvote crap stories.

Hm... Story upvotes modulated by users Karma might be good enough. It's an interesting concept...

Stackoverflow operates under this principle, more or less, but it doesn't seem to entirely stop the slow decline.

The problem with using karma is that it is affected by the regression to the mean. As more "average" people come in, more "average" people will give them karma. Solve that problem, and you might have a solution to the whole problem (I, for one, am not smart enough for that ;).

Maybe some mix of karma, account age, and the number of upvotes received from first-year members to give a person's votes weight.

Alternatively, perhaps we can let users select (follow) a set of like-minded people and make the up-votes of the people they pick weigh more on a "customized" front page.

But then people may have not like having their up-votes be so transparent..

But the interesting issue here is that those who write interesting commentary may not be good at discerning good quality. (according to whatever standard we're holding)

This problem applies to all communities using votes to sort user-generated contents: HN, Reddit, StackOverflow, Quora, etc.

I think something like PageRank can really help. The same need is to rank things based on a directed acyclic graph. The only difference is we have multiple kinds of vertexes: users, submissions, comments.

What about using baysian filtering? We already have a lot of data on good and bad comment styles, baysian filtering could give us an indication if a comment is violating HN guidelines.

I am not so optimistic about submissions because the data we currently have is tainted by submissions we don't want. However by using the technique you have described, we could probably achieve better results.

If you really wanted to make the site more like it was, you could look at the voting history of people here at some distant time in the past and find other members whose votes highly correlate with the old timers. Some people just quietly upvote, so they don't have much karma. If you want the upvoting to be more like it used to be, award people who vote similarly like how older users used to vote.

It would help take care of the effect where older uses leave due to changing interests etc.

Note: I don't actually think it's a good idea. I think just figuring out _what_ you want exactly, in vaguely quantifiable role, is sort of a hard problem. I suspect a lot of people have different 'golden ages' in mind when they think of HN.

When I came here originally it was pretty startup heavy, which was sort of remote from what I was up to at the time. I liked it when it got a little away from startup business related stuff (my perception) and toward more generic technical issues. My point is even if everyone agreed that ait has gone done hill (an open question), not everyone might agree on what it should be like. It's not a democracy of course, but even as an individual for my own preferences that seems as hard as the technology issues.

Gated communities are effective means of preserving the identity of communities, they all employ some bar of entry whether racial, religious, ageist, or economical.There are many social maladies that are also unique to the gated community, the insularity often breeds all sorts of creepiness. Preservation all to often morphs to Stagnation.

Artists are often the shock troops of a neighborhood gentrification, after the studio loft, comes the artisan coffee, some renegade youths, a young lawyer or two, and before you know it, the neighborhood just ain't what it used to be.

I would Posit that a website calling itself "Hacker News" immediately opened itself to all kinds of interpretations. The term "Hacker" seems to be as hotly debated as "Artist" and justifiably so.

The Hackers, introduced others who identify with the Label, and still others who probably do not, but nevertheless find it of value to their venture.

When the neighborhood changes, you are free, within your means, to move to another place. Sometimes you yourself change and require a change of scenery.

When a startup grows to a full company, many times you lose something while gaining another, and vice versa. Many in this forum have made those choices on their own, so it should be familiar ground.

It is almost heretical to mention it here, but perhaps there is no algorithmic solution (if there is a problem) to the complexity of human relation, expression, and motivation.

More people, more heat, Entropy.

Come now, I can't be the only one that finds the Hacker News quality "good to great"?

If Hacker News is about hackers in a startup sense, it's good that the front page has everything from: Movies being in decline - Ruby concurrency explained - A torrent meta search engine - Windows 7 SP1 launch - iPad2 being unveiled.

There are far more elements to hacking than programming, just as there are far more elements to startups than programming. And I dig that Hacker News is so varied.

I think there is a vocal minority of people that get irritated by bicycle shed debates (+1 from me to allow collapsing comment threads on my machine) or people wanting to only read about programming or hacking - the latter of which is laughable because I am pretty sure you'd be sick of Hacker News if it was 100% a specific topic (I have some scars in the field of sorting content users will enjoy...)

Guess what: there are millions of non technical silent people on the internet, and a huge amount of those people visit Hacker News every day - and love this destination. The amount of random non computer scientists I meet in Cambridge that love Hacker News is staggering.

It's definitely not what Alex is semi-proposing but I've been running RubyFlow - http://rubyflow.com/ - for a few years now and it totally stole the MetaFilter model, just for Ruby-only stuff. No "votes" and points scoring - just interesting posts from people in the Ruby community coupled with me editing posts for format and deleting anything that's blatantly spam or offtopic. Seems to work though I have been tempted to go in the voting/Reddit/HN direction with it.. maybe I shouldn't!

So long as you have the personal bandwidth, you will almost certainly be a better editor than any algorithm you can come up with.

Thanks for the link, by the way.

Simplest solution

Limit the number of links submitted per account per day to 1.

Why

Prevents spammers and karmafarmers from submitting the entire TechCrunch\Wired back-catalog at a rate of 25+ a day.

Further Analysis

Increasing the scarcity of a resource (link submission ability) will increase the value of items it is traded for (links).

HN already gets the independent code submissions people want. They just die an early death on the new page due to overcrowding by webzines\newspapers with builtin linkbait titles. This reduces the rate of dropoff for independent news.

If the crowd has cycled so much, I wonder if maybe this isn't the best solution for the desired outcome.

Granted, I haven't been visiting tech-specific boards for more than a few years, but I'd generally agree that the more technical articles are what I'm interested in.

I think I'd be interested in a board that was geared toward programmers/hackers, but didn't use a typical karma/point system. I'd like to see one that perhaps utilized karma, but under a collaborative filtering system. So, in a simple for-instance, if a small subgroup of people tend to upvote articles that I do, those articles would be given more weight, and similarly those who downvote articles I upvote would be, from my perspective, given less downvote weight, while at the same time there might be a different subgroup that was weighted to value their downvote more. Perhaps give people the ability to tweak the tolerances of their collaboration. Give them the ability to say "if this guy has X karma and ignores someone's articles and votes, then I want to ignore them too"

Of course, this might be 1. a completely naive idea, 2. an idea that's already been tried and failed 3. an idea that's already being used 4. something to time-consuming for people with real work to do or 5. an idea that's unworkable and that I'm only having because I just started reading books on, and experimenting with, machine-learning ;)

Though even if it existed, I probably wouldn't use it. I already waste half my day reading the few articles that interest me on hacker-news, heh

it sucks that when you design any system or any set of rules, and humans are going to interact with it, you have to think "how are these shady bastards going to subvert my beautiful creation?"

I think that if you stay at any online community long enough, you begin to perceive a drop in quality - even if that drop does not exist.

IMHO opinion, there is plenty of signal in the stream. What has happened is that the interests of the community have diverged. I'd be far more interested in ways to focus on things that I was interested in, within the stream, than narrowing the flow of information.

On my wishlist is a way to pipe the HN stream through a Bayesian filter based on articles I've enjoyed, and make an RSS feed of articles I'd be interested in.

Where is the data that shows HN has degraded? We aren't seeing kitten pictures. A scan of the front page shows the mix of articles being programming, startups, tech.

I'm not sure what Alex wants? More discussion around PG's hackers and painters?

I've come to realize in my short time here, that dissenting opinions are dangerous. I've learned to respect the karma gods and pander or, post my opinion and delete it before taking too hard a karma hit.

Its sad to see that an informed, but non-conforming opinion is taken as fact and karma-nuked.

Agreed completely.

Please cite examples. Your post is practically an example of karma-whoring without them - anyone can make this claim.

It's simple really. Digg was good, then it got popular, Reddit was good, then it got popular. I've increasingly visited HN more than Reddit to get my old Reddit fix, as I assume others have, and so popularity has increased and now the quality is degrading as people want their karma fix or 2 cents.

There should be a brainstorm on this. I'm starting to realise I want comment submissions from well known or quality submitters. Not just your average kid or someone who is trying to troll.

The other issue is one-off opinion pieces on some guys blog. HN feels like every programmers chance at 15 mins of fame. Why Ruby On Rails is X times better than this (adudecodingblog.com), My way of speeding up Python (pythonlover.com), etc. having someone like pg, of Joel, or big wigs viewing items or articles like these, offering actual real world advice, and providing comments.

Maybe a subscription based hackernews, where the kudos goes to the legends of the industry, interns are made, and I get my intelli-fix and boredom disguiser because I'm stuck in a cube-farm polishing PL/SQL wondering how the hell I got here and when can I play that stupid COD:Black Ops with its really crappy hit detection. Why do I keep playing it?! Why haven't I asked for a bigger paycheck? Why am I not contracting? How is it that the kid I use to teach programmer is now earning more than me? Oh well, keep surfing...

The Illustrated Guide to Flame Warriors is a handy reference: http://redwing.hutman.net/~mreed/index.htm

In theory, centrally planned things make a lot of sense.

In practice, democracy usually comes to a better solution, even if it is not perfect.

HN is driven by votes, the community is getting what the majority wants right now. The only way to really improve HN is to change or limit the community. You can tweak the rules only to limit certain actions to high-kharma users, but if there is pent up demand for some kind of story it will make its way to the front page.

IMO, a tag system like StackOverflow or Quora seems a good way to go.

Tags concede defeat, fragmenting one dysfunctional community into a whole spectrum of subcommunities. How will we handle the tag for "good enough for '08 Hacker News"?

If you can sideline all the people who want to bullshit about how the ipad2 is the future of personal computing or the quality of the writing at TC, you've already won.

No, what Reddit does is fragmenting community. How much does the tag system in StackOverflow or Quora fragment them? Tags helped people to somehow filter their already overwhelmed input stream, focus on what they really interest in, and more importantly, ban the items they genuinely don't like to see.

That was my thought too, they both have quite stringent moderation, but that doesn't solve the increasing the quality of participation, unless of course it's decline is what drove some people away.

I'm fairly new, but I think HN is great. The level of discussion is way, way higher than other similar communities I've seen, and I think the general focus on tech issues with splashes of other topics is perfect. Seems like a successful self-regulating community if I've ever seen one. People just like to complain.

It seems there's always been reluctance to add features, for the sake of simplicity. Personal messages, for example, would have been useful in many, many instances, but instead we find ourselves checking out the plain-text profile and finding alternate methods of communication.

This limitation has also sprouted ancillary sites attached to the HN Tree of Life, such as searchyc.com, hackermonthly.com, and hnrecap.com as mentioned in the post.

In a similar vein, carving out a sub-HN seems to be: a) downloading the source code, b) bringing it online at another domain and c) announcing via "Tell HN".

All in all, unless someone with >10^5 karma decides to take the time and add some community features to HN (for various values of "community" and "features"), we're all going to continue and see more noise and many different signals.

As an aside, I wholeheartedly appreciate the name, "Bloomfilter."

Everyone here has seen this same lifecycle play out at just about every online community there ever was. Doesn't matter whether it was open or closed. It's a fact of life.

The solutions offered are top-down culture modification and just plain don't work. Adapt, and wait for the next HN to come along. You can't stop the train.

Almost all sites that have comments and user moderation concentrate on the comments and up voting / down voting them. Normally, when dealing with people, I don't remember the individual quote that made me think they were brilliant / a troll, I wrap that up into my sense of them. If I'm flipping channels and see someone who has struck me as a brilliant commentator, I stop based on the visual cue of their face. Names are kind of hard (is this the guy who called me a $%$% or was he the one who really knows python?).

I guess I wonder if the same thing keeps happening on "comment moderation" sites, isn't it time to look at the ways your view could be based on your (not the group's) opinion of your fellow commentors? I don't have a technical suggestion, but I will probably think a lot on it.

Perhaps pg is too busy these days to really tend to HN? God knows I would be if I were carrying his load.

Nope, I got an email from him the other day telling me not to editorialise in submission titles after I posted that Nissan Leaf HTML5 extravaganza with the heading "Behold: We don't need flash for anything anymore. Ever".

I'm sure that post itself rankled our OP.

Question: Why'd you give it such a sensational title?

Well, I guess because I was editorialising!! I found it sensational at the time. That was the first time I had looked at a site and said to myself "well that must be Flash(TM)", and then a second later in my head thought "well fair enough, you'd HAVE to use Flash(TM) to get that kind of experience for a major brand", then right-clicked on the page only to see the standard context menu... no "Zoom in"!!

I was really quite blown away. I could have just put it on my blog then posted that to HN (which is actually what pg said I should have done when he emailed me), but my initial thoughts, were that I really just wanted to share this with my pals on HN (similar to the way someone else had posted Ben the Bodyguard as a "Gorgeous HTML5 implementation" or something) and that putting it on my blog then linking to my blog from HN would just be viewed by the community as me jockeying for traffic and being self-promotional!

Did you forget all about the "More" debacle?

Gods know it was a terrible decision, but it showed some active participation by pg.

I must show my HN immaturity here by admitting I don't know what that is :)

This should get you up to date, then: http://news.ycombinator.com/item?id=2118325. :)

In response to some of the points being raised in the thread, and to provide some sorely needed data, here are some snapshots of the HN "newest" page taken since Feb 2009:

http://news.ycombinator.com/item?id=2254397

"... I think HN does a crappy job with general tech news and a so-so job with content that’s specifically relevant to startup founders and employees. These days, HN does a downright terrible job with deeply technical topics; that’s the area I hear the most complaining about on Twitter and in private. Since deep tech is HN’s weakest point, let’s go after it. ..."

The weakness of the argument is that the engineer/developer/programmer view is a subset of the interests hackers, founders and entrepreneurs. I draw a clear distinction between tech guns for hire who only want depth as opposed to those who want to solve technical problems and maybe innovate which requires both depth & breadth.

It's important to separate concerns, and I think there are three main things people are complaining about:

1) quality of submissions 2) quality of comments 3) quality of community

It's the second and third that I think have declined. It's not because the people are any less smart, it's that there are just too many of them. It becomes difficult to keep a mental model of everyone in your head, so you start seeing everything as disembodied text, rather than human beings speaking to one another, with a history of shared experiences.

IMO, this is a solvable problem. You can use avatars and display our locations next to our comments, or even just make our names a little bigger. Anything to humanize the conversations.

Every single community driven website on earth have being thru this kind of cycle the only way to stop this is to start a new one that will end getting into this cycle again. It's a matter of the amount of people using it, on the beginning only very early adopters and people passionated about that specific topic (here tech startups)are in, as years go by more people that have more interests start to join and post things they think is good, and then the topic changes to a more general subject.

I like this community and I think the quality will always fluctuate but the most of it will always be very good content for people in a hurry.

Thanks for all of you who help this place being nice.

Well, obviously posts/discussions like this can actually be contributing to the "problem" as some see it, but I'll make one point I don't see made here.

For me personally, I've learned a lot and grown a lot over the course of the 4 years I've been lurking and occasionally contributing here. So for me, a smaller percentage of the stories/articles/posts/discussions appear as insightful as they once did. I don't mean to knock HN in any way, in fact my point is that that fact is not a "problem" to me. New users are joining everyday and everybody who makes the effort to learn and contribute gets something out of HN.

It's what brings me back 17 times a day.

HN would be better if it were invite-only like Dribbble. I'm not a member of Dribbble, but it's a good example of why restricting community growth is beneficial.

We're too late for that here. I don't think PG has enough bandwidth or interest to truly solve the problem. New users will continue to join, adding noise to the signal, unless HN changes course. It's going to become more generic and more biased the longer the site stays open.

I hesitate to suggest more moderation as some posters suggest. I'm already uncomfortable with the murmurs of unfair moderation in the system here.

Actually HN is the reason I don't have enough bandwidth for other things, not the other way around.

One solution can be to give read-only access to new users and charge a very low one time fees for upvoting, submitting and commenting. Only people who are serious about contributing will pay for it.

Honestly I'd love to pay for a Hackernews account if it meant weeding out some rather distasteful people. Not saying that is the be all/end all solution to the problem though.

There is no hacker news problem if you realize that all sites up to this point in internet history are either heavily curated for quality and limited in scope or self-moderated with a one-dimensional, imperfect karma game, and thus chaotic and ephemeral.

Such is the nature of suchness until someone figures out a better game that more properly engages human nature.

Crying over the demise of HN is like crying over a naive hill-climbing algorithm when it gets stuck.

Very well written article. I think he has some good points. I'd really like to see a 'deeply technical' alternative to HN and hope he'll find the good guys !

Could this phenomenon be in any way attributable to nostalgia? Personally, I find any online community loses attractiveness after a certain period of time.

One thing worth noting is that reversion to the mean doesn't have to be a bad thing. For example todays mean level of education compared to the mean level of education a few hundred years ago is very different. A good question to ask might be, would submitting this increase the mean level of discourse on Hacker News? It is the same sort of thing as what is in the guidelines, but reworded for greater relevance.

Put some teeth into karma. Make more karma mean bigger upvotes and downvotes, say 1 extra point for every 500 points of karma. It doesn't have to be linked to when a person joined. It's elitist like a journal or university, but at least anyone can read HN, and good posters will rise. Problem solved?

EDIT: Oh yeah. 1 week comment lockout for negative karma, with a grace period for newbies to learn how to comment.

I just want to speak up as a relative newcomer that feels I've learned enough about this place to speak my mind without being afraid of retribution and can do so with a basic understanding of what works/doesn't work here.

I really like it here, and it's my 1st stop after Gmail every day, and often more than once per day. Nothing is perfect, but as far as I'm concerned, this is as good as it gets.

This has probably been said before (and/or above), but perhaps each article should be tagged and users should subscribe to only those tags of interest to them. In addition, a social component to HN would be useful -- for instance, allow me to recommend a story for someone or for stories of interest to my friends to be ranked above the norm. Our friends make great filters!

I agree with Alex, I think the success of HN has led to an overflow of people dissatisfied with HN who have made HN a success in the first place (and some with YC itself.) I posted two ideas that could be of interest to such people here:

http://news.ycombinator.com/item?id=2253752

Why can't HN force categories on every posts? Have the community create and curate the categories, and select their own categories on HN to read. Just a blanket vote up/down button hardly measures how valuable anything is for any particular group if that group is not constant.

This was discussed before: http://hackerne.ws/item?id=1934367

I had a comment which I guess I should link to rather than repost: http://hackerne.ws/item?id=1934605

I would very much like to have http://startups.ycombinator.com/startups/ and have a more narrow focus on startup related news and entrepreneurship.

The constant stream of front-paged political arguments and noticeable increase in mean-spirited commentary in the threads has caused me to spend most of my time on HN logged out. It used to be that I'd read the comments before I'd even read the story to see if the story was worth reading. I wonder if it's possible that pg doesn't notice the degradation in comment quality as much because the trolls have been here baiting him since the very early days.

I don't think HN is irrevocably broken; I'm glad that pg is helming the ship and I think he's doing an admirable job of it so far (I think the ranking algorithm in use for the front-page stories is one of the best anywhere). But HN used to be great, and now it's merely good.

I think that a lot of people who have been here for a long time have thought about what's changed here, and how it could be fixed. I know I've littered more than a few mailboxes with lengthy emails about what I think is wrong, and what I think the solution is. Reading this thread kind of tells the story - a point has come where the community is large enough to have factions that value different things. "Anything that good hackers might find interesting" works when you have a small group of people engaged in conversation. It's less useful when you have mobs of people who have come with different ideas of what they want to get out of this site.

In the early days, HN felt like it was a problem solving tool; a way to find out what cool things people were working on, and occasionally to ask for advice. The community was humble, competent, and full of people who actually made things. Those people are still here, but there's a self-aggrandizing element here as well. The group of people who seem to think that someone else's success somehow reflects poorly on themselves, the bloviators and blowhards who believe that a volume of arguments somehow makes up for the measurable factuality of arguments. I don't really know what the solution is to this. I thought if there was a way to ignore people it might make a difference, but after some experimentation I think that that's a dead end - there is too much chance of missing something truly interesting from doing that.

All this being said, HN has had an immeasurable positive impact on my life - The people that I've met through HN (both in person and virtually) are some of the smartest, most amazing people I've known. I'll get to use the things I've learned from HN (and more importantly from the people in it) for the rest of my life. I can't think of another site on the net that has come even close to making such a huge impact.

I can't imagine missing out on all of this if HN had been invite-only when it launched. I didn't know anyone when I first came here. I didn't even know who Paul Graham was.

Instead of complaining about it, I think those of us that have been here for awhile owe it to pg to actively try to improve the community. It's become too large for him to handle on his own. Yes, there are moderators, but they're an invisible hand that only act as a corrective force.

We're a creative lot. I'm sure we can figure out some way to improve this community from the inside.

Or may be after a while our perception about something changes disproportionate to the actual change?

That's basically boredom--and it can happen even if you consume something good for a long time. That "good thing" doesn't change so much as your perception of it.

Maybe experienced HN (read as a mix of high karma + there since the beginning), could have a bigger impact on which articles are chosen. I honestly don't mind a "dictatorship" selection where chosen members could remove useless post / select useful one.

Hi, I'm taking on this challenge. Al3x, can you send me an email at jkwon.work@gmail.com?

Also, I'm taking suggestions for seed users. There will also be a HN Karma cutoff where everyone above a threshold can join. You can nominate HN users or yourself here.

Wait... he complains on his post about not submitting his blog posts to the community.. which then gets submitted to the community and upvoted (albeit probably not by him). Does that sum it up?

Political stories simply need to be killed with prejudice.

Part of the problem is the basic UI look, hence I am working on a new UI for it: http://peri.me/2B1A/

what about tagging? automatic or crowd-powered. Seems a great way to sort through the noise. tags could be "technical", "startup", "YC" etc.

Another HN challenge... as discussions get longer (like this page), it gets more chaotic and more difficult to follow.

Remove points from users (keep on submissions/replies for positioning).

Problem solved by changing motivations/behaviors.

Pretty soon you guys will be telling the kids to turn down that damn music and stay off your lawn :)

I now check 'new' articles more often. There are a lot of gems in there that are not voted up.

Does anyone here think crowd-sourcing due-dilligence on startups would be a good idea?

Sounds like (s)he wants a Less Wrong for startups.

My suggestions for tweaks to improve the site:

1. hard ban on purely political news ("Egyptian leader stepped down! OMG!")

2. hard ban on gender-specific things ("i'm female, went to bar during hacker conference, got groped, OMG!" -- yes it was hacker conference, but gosh subtract the 'during hacker conference' and you have real life, it's independent of tech, not specific to it or due to it, just a life thing with guys and gals)

3. particularly if hard bans (enforced by a set of trusted admins) on the above topics are not added, then allow submitters and admins to add/edit content tags for each post; then allow logged-in users to submit content filters so that when they see, eg., the front page, it can suppress all posts with certain tags (eg., pure-politics, gender, sports, religion, etc.)

4. optional for-small-periodic-fee premium accounts, which allow those users to exercise extra features like smarter content tagging/filtering, sorting, user following, user submission/comment filtering (so you can blacklist blowhards and pedants from what you see, even if they are not banned from the site overall)... I'd personally love to blacklist anybody that ever does a comment reply to me that is (a) rude, or (b) idiotic, or (c) overly pedantic (some is fine, we're nerds, goes with territory, and some precision is valuable, sometimes). Blacklists could be flat files, one user per line. We could share them among each other privately. I've bookmarked a few "ahole-or-idiot" users but I'd love it if I could have them automatically stripped from anything I see on HN in the future. Actually, I'd love to have this feature on all social/forum/news sites I visit.

5. fix the "type comment, hit submit, get error page saying something doesn't exist, so you have to go back, copy your text, hit Refresh, paste the text back in, hit Submit again" bug/feature. that drives me nuts. feels like impl side-effect rather than intentional UX

6. don't have the up/down arrows so close together when viewed on iPhone

7. don't allow just anyone to downvote any comment. or at least, they can't downvote it beyond 1 point, below which is penalty land. right now, any dumbass can downvote a comment of mine from 1 to 0, which then reduces my overall lifelong site karma by 1. Just because they disagreed with me. Or they're an asshole. Or they accidentally hit the downvote button (see 6). Instead, have a minimum karma requirement to issue downvotes, and/or only admins.

HN is great, despite it's imperfections. But I'd gladly pay up for premium features. HN Gold? HNGold.com (YC-W11)?

EDIT: added a few items

(In response to 7,) you already need 500 karma to downvote anything.

I also find I can't downvote direct replies to my posts.

My suggestion would be simpler : have a "this is not hacker news" queue. It's like the frontpage, but it's for the stuff that is "not hackernews". Whenever something hits the frontpage that people feel does not belong, instead of whining, they just hit the "this is not hackernews" vote button. With some algorithm that can combine vote numbers, influencers or whatever, when it crosses the threshhold, it gets moved over to the alternative page. So there is the community moderated hacker news and there is the alternative everyone hacker news.

good idea. i've been using 'flag' for that now, but your idea is better because it more precisely signals why one is flagging something. makes distinction from reasons like obscene, duplicate, etc.

The basic complaint is that social sites grow into mobs. The solution is rather obviously: halt growth. You have to limit the number of active users to some vaguely Dunbar-ish number or you inevitably wind up with a lowest common denominator mob.

Metafilter did this, right? For a couple years they said "No new accounts."

I think scaling a social site to a very large number of members without deteriorating badly is impossible. It's a matter of human nature and mobs.

The solution, ultimately, is for the site to wither and die, and be replaced by something else that will have the same fate. This is what happens to all things, and to all human social groupings in particular, from ancient Rome on down to your nuclear family.

If you're tired of it, start something else. Or hang out and jump ship when the next great thing comes along. Trying to preserve the golden age is rearranging the deck chairs on a sinking ship.

Case in point - this article.

If sites want to claim some sort of community and continuity, they're going to have to place newbies into virtual reeducation camps. Want to see the newest links? You have to read through 10 comments from an '07 post first. Posted a link to an internet meme? Back to the virtual reeducation camp with you.

Reeks of elitism.

This.

Reply on news.ycombinator.com