The Value of Downvoting, or, How Hacker News Gets It Wrong
blog.stackoverflow.com
blog.stackoverflow.com
We can thus safely assume a nonlame set of articles, and we also (so far at least) assume nonlame voters. And if you only have nonlame voters voting on nonlame articles, upvotes should be enough to pick the winners.
Honest question, and I do not mean this as a flame, because generally I quite enjoy Hacker News.
How, exactly, is the current top-rated story on HN, "How to Stop the Drug Wars" ( http://news.ycombinator.com/item?id=507509 ) related to.. news of hacking?
But I can't.
That's okay; the models are really different. Stack Overflow needs to scale more because to be useful, there need to be thousands of questions asked every day, and every question needs to be able to get a response. Hacker News can be useful to me even if it only gets a hundred new stories a day, and it's not really a problem if many stories get axed for a bad reason.
I think I've flagged something....once. Yup. Most everything on the front page is interesting, the most interesting of which I upvote, and the few things that aren't I just don't read and let die. the front page changes enough to allow me to do that anyway.
http://ycombinator.com/newsguidelines.html
> Off-Topic: Most stories about politics, or crime, or sports,
Isn't "How to stop the drug wars" about politics? I know it is exactly the kind of article I expect to find on Reddit. I was surprised to see it on the front page here. From the linked article:
> That is the kind of promise politicians love to make. It assuages the sense of moral panic that has been the handmaiden of prohibition for a century. It is intended to reassure the parents of teenagers across the world. Yet it is a hugely irresponsible promise, because it cannot be fulfilled.
This is not an article about politics?
Right now only the people who upvote it get to have a say.
Not so. The people who flag it also get to have a say. And if you observed the site more closely you would notice that their say is quite effective.
I understand how voting works, but how does the flagging mechanism work? Once it's flagged by (n) people, what happens? I checked http://ycombinator.com/newsguidelines.html and http://ycombinator.com/newsfaq.html and I couldn't find anything.
I think flagging is a very different (and basically silent) axis of expression; you can't counter an upvote in the case of controversial, marginally on-topic questions like the drug war one.
I totally agree that some of the powers like voting down and flagging have to be earned through participation, by the way. We do the same thing on Stack Overflow.
I think the silence is a feature not a bug. The meta-noise reached its peak just before flagging was introduced, and it got way better after that. It still flares up intermittently, like now. An interesting observation is that it's mostly new(ish) users who post complaining meta-comments. Perhaps after they've been around for a while they notice that those discussions are always the same, as are the "sky is falling" threads.
Edit: uh-oh, the right margin is fast approaching. And damn it, I had managed to go at least 6 months without getting sucked in to this meta business!
So flagging in and of itself does not trigger any automatic removal of the article. There has to be a human action to remove the offending article.
What next, some really intelligent analysis of Ron Paul?
There have also been intelligent, interesting stories about Ron Paul on Hacker News. For example this one ;-)
This site is vastly related to programming and startups, but other topics that are of interest and thought provoking tend to rise up as an interest.
We don't have kittens on the front page, so I consider it a victory.
I think that's pretty healthy. It means that Paul Graham gets an article to the front page one day, and the next day a rebuttal to that article goes up right next to it. Doesn't matter if all the people who agree with one want to downvote the other - as long as it's good, we all get a chance to judge for ourselves.
This is not an article about politics?
My comment was plainly not about the article but about a false description you made of this site. The article itself strikes me as a borderline case. It can't really be called "evidence of some interesting new phenomenon", but neither is it narrowly about politics. It's an intellectually respectable piece about a thorny social issue. For me, it passes the test because of the interesting historical content of the first paragraph, which I was curious and gratified to learn.
If I could make one thing go away from HN it isn't egregiously off-topic articles, which the flagging-and-editing protocol handles just fine, but rather the incessant "The sky is falling, it's just like Reddit" meta-noise.
But that's precisely what happens when you give people no other option to vote against something: they write comments complaining about it. It's a direct and very literal consequence of the design decision not to allow downvotes on article submissions.
Besides, I only brought that up because pg said:
> The reason HN doesn't need downvotes is that HN, unlike Reddit, kills lame articles.
Which is generally true, but clearly not true today because the top rated article is, as you said, borderline. And from my perspective, it is not at all borderline, and I'd vote against it in a heartbeat.
The problem is that the upvoters get a highly disproportionate say in the matter. Let's say
- 30% of the HN community finds that drug war article interesting. They all upvote it. - 20% don't care or have no opinion. They do nothing. - The other 50% think the article is only marginally on-topic. (I would say it is not at all on topic, but let's give the benefit of the doubt.)
For those HN users, the article isn't spam or evil or lame (it's the freakin' Economist!), so lumping it in with that sort of nastiness by flagging it seems extreme -- a bit like the nuclear option. I can't find any description of how the flagging works, so I don't know if flag counts are public or not.
What you end up with is a plurality of public upvoters balanced by a (I'm assuming) silent and mostly invisible minority of off-topic flaggers. These are very different axes of expression. They don't really balance each other in the case of a marginally on-topic submission.
What would help, particularly in these marginal cases, is a reciprocal downvote.
And on top of that it is clearly politics, which isn't fun to discuss or engage in. Unless you enjoy other high-risk, low-reward activities like, say, juggling chainsaws..
This effect is probably muted on SO which is primarily technical.
What we really need is a system that works most of the time. The cost of a false negative with down-voting (losing a genuinely insightful article as off-topic etc) is simply too high, IMO.
As regards the comparison with Stack Overflow: I find it rather pointless; the sites exist to serve different purposes altogether.
Perhaps you could rethink your initial thoughts given all the comments above and provide an even more interesting analysis of how user behavior is modified by the presence of certain features... :)
But you never give any reasoning for your own disproportionate weights beyond what you personally feel is fair or seems to work. Granted, HN gives no little rationale as well, but that makes it at least as valid, having survived longer. It may be that the ideal weight for downvotes is even lower than 1/5th, or it may be that merely having downvotes at all hurts a site more than not having them.
The message an upvote sends is "I like this and I want to see more of it". Conversely, you seem to think a downvote should say "I hate this and want to see less of it", but when the site adds them directly to upvotes, the downvote message becomes "I hate this and I think everyone who voted it up is an idiot." A site using this mechanism sends the message "We are willing to allow controversial material as long as an enthusiastic subset of users votes it up." I would expect this to encourage clique voting over time.
By contrast, the message the flag system sends is "I don't think this belongs on the site; what do you think?" It defers the decision of what belongs on the site to editors. This is more work than letting the market decide, and it probably doesn't scale as well. However, it also ensures that the market doesn't take your site in directions that you don't want to go.
On the other hand, with a downvote, there's no opportunity for discussion, and the end result is that worthwhile stories about out-of-fashion concepts like Perl or .NET or software patents get suppressed.
It's worth mentioning how often the intent of voting is misconstrued; is it "I disagree with this" or "this adds no value"?
Or "Karma payback time, dude"
In practical terms, it means all of that. What it is "supposed" to mean, I guess, is stuff that PG likes to see (or not)
I agree with you, stories about drugs aren't what I come to HN for.
I've been friended by some people on digg - and they constantly shout battles at me: bury this, digg that. The bury brigades have gone to work on this, etc, etc.
You can't do that here, someone who hates a topic simply can not affect it. All you can do is say there are X number of people who are interested in something. It doesn't matter that some people hate the topic, or are opposed to it. They have no voice.
Example: If you have a very controversial topic that 50% of people agree with, and 50% of people disagree with. You will never see it if you have downvoting. But since there is only upvoting, you will. And I think that that is very very important, and it's one of the reasons this site is so much better than digg and reddit.
I've always thought that digg should frontpage also controversial stories, not just upvoted ones, but ones that have an almost even ratio of down/up votes.
And BTW you don't loose half of the information about votes for a topic - the algorithm here is that stories fall down over time, so time itself basically acts as a constant down vote. You have upvoting to counter that, and ignoring a topic to assist that.
Bill Gates once said something like: if I only read the stories that I am interested in, I'll never learn anything new - so I make a point of reading the entire newspaper. (I hope I remembered the quote correctly, it was from before the internet.)
Editor control should be used to put a stop to that, IMO. Democracy is great, but it went too far with Reddit. HN would become much less useful to me if it went in that direction.
That wasn't really his point so much as yours. ;)
> Political issues often result in uncivil discussion. There is also probably significant audience crossover with reddit that caused that post to rise up. Hackers may have an interest in libertarianism or economics, but does that mean that this site should become one about Ron Paul if people feel that way?
That is in fact a good question to ask, and I wasn't precluding it from being asked, but rather reiterating the stated goals for Hacker News. Such questions ought to be asked.
I am a big fan of hacking social and economic problems. If a topic is not of interest I ignore it. A thread rarely stays on the front page more than a day or two.
I originally came to HN for Web 2.0 news. But things have evolved.
Which is good for everybody who uses HN as something other than a place to get tips about programming, which I expect is the vast majority of the HN user population.
http://ycombinator.com/newsguidelines.html
"Stories on HN don't have to be about hacking, because good hackers aren't only interested in hacking, but they do have to be deeply interesting.
What does "deeply interesting" mean? It means stuff that teaches you about the world. A story about a robbery, for example, would probably not be deeply interesting. But if this robbery was a sign of some bigger, underlying trend, then perhaps it could be."
(Funny sidenote: In forums with alleged [real or fake] groupthink tendencies, like Reddit, it appears that referring to and criticizing the groupthink is very often voted up by the community. This is why I believe that groupthink emerges naturally as a result of the availability of the downvote. No one notices what's going on and attempts to correct their own behavior.)
Killing articles and comments solves the problem of lame articles and comments. However, it does not solve the problem of lame voting hiding useful comments. I suspect that this problem will require a solution soon; I myself have noticed an increasing trend toward Redditism here.
Additionally, I believe that the comment system is probably even more important than the article system. In some sites sites, including Hacker News, I get more from the comments than I do from the articles. Thus, ensuring that comments are good and fairly balanced is just as important to me as ensuring that stories are good.
The reason this happens is because of the following. Let us say that an ordinary story that is completely relevant to Reddit and is worth reading is upvoted by 50 people and downvoted by 10. This gets the story +40. Now, let us take a very controversial story, say, a blog post on how much git sucks that is obviously fishing for links. This story gets upvoted by many more people, say, 100 people... and downvoted by 300. It doesn't get near the front page.
Now, take the same on HN. The first story gets +50, the second gets +100, despite the majority of people believing it should not be on the front page.
This is not to say that the Reddit system is better, but rather that the HN system is not perfect.
(In my defense I thought they were good articles. But the system appears to work from my experience.)
I like making a distinction between something that I simply don't find insightful and something that is actually trolling/spammy/false. In HN's case, my lack of an upvote is equivalent of not finding something insightful... my flagging is an indication of the latter.
EDIT: [About the editors, it takes an hour and sometimes more for a link to be killed, at least when I flag a link and I see it for hours - happened today. So maybe automation will help in any editors shortage]
For example, someone few days ago submitted your article "Why TV Lost", although it was already submitted, and we already discussed it! But he was able to do it, obviously because the submission detection script does not think that "www.domain.com" is the same as "domain.com", and treats them as different links.
Also if a link is submitted previously and it was dead because it's spam, then if a user tries to resubmits it again it should be killed automatically. Like Google homepage, myspace and all of these bla bla bla.
There are zillion techniques for preventing spam, and you know it better than me for sure. But there is a vision that you have about spam on this website, that I don't really understand yet.
Those two are not the same. Just like mail.google.com isn't the same as google.com.
I am talking about the "www" which is the acronym of "world wide web", and it's automatically assigned to any domain name, so usually domain.com and www.domain.com points to the same host, unles if the sysadmin changes it which rarely happens.
I am talking about the "www." and NOT any other subdomain that the sysadmin/webmaster can add/change or remove.
I am sorry if you misunderstand what I am talking about.
Thank you so much for your explanation, I really appreciate it.
However, all of this is missing the main point: for a site to be popular (lots of eye balls) and be able to sell itself to a large media company for a large sum of cash, it needs to appeal to more than a small, coherent community. Reddit changed because their founders wanted to cash out. HN isn't changing because you have no (apparent) goal of cashing out, and are thus willing to keep HN as it is.
On Stack Overflow, at its core, you have posts, and people reply to that post. That's fine. You can do that whether you're logged in or not (although you need an OpenID provider to sign up, which is a pain when it /could/ partner with a provider to seamlessly give you one on registering).
You can mod up and down the posts. And mod up and down the comments. But only if you have a certain amount of karma for each. But the selected answer by the author of the question is at the top, ignoring the karma completely.
You can have favourites. But only if you're logged in, despite the fact you can submit posts whether you are or aren't.
And then you have comments on each post. But you have to click to view the comments. And you can only add comments if you have a certain amount of karma. This kills a lot of conversation needlessly.
And you can edit other people's posts - if you have a certain amount of karma.
And you can flag posts, but only if you're logged in.
And then too often you view posts that make no sense until you see the context which is buried amongst the tags, rather than the site being naturally split into natural obvious 'sections' which will interest people who know about certain technologies.
It's all very confusing, and completely needless. Hide things or at least make it obvious what I can't use before clicking. Reduce the arbitrary karma limits to add more distinct classes so that the majority have equal abilities.
Well, the funny thing is that the complexity is there for the some of the same reasons that HN explicitly doesn't document its behavior -- it's a small barrier. But these are much lower barriers than HN, as I'll describe below.
> Hide things or at least make it obvious what I can't use before clicking
Ah, but this is a problem -- we tell you what you need to earn to do something when you click on it. So a) you have an idea that the feature exists, and b) you can do it at some point in the future.
> Reduce the arbitrary karma limits
Our limits are quite low, far lower than HN. We require the equivalent of 2 karma on HN to upvote, and 10 to downvote. More here, just divide by 10 to get the HN equivalents -- http://stackoverflow.com/faq
Imagine you're a new user visiting Stack Overflow, when things you both can and cannot use are shown in exactly the same way as a full user until you click on it and then it tells you that you need X karma to do something. I think this is a bad thing and that the correct fix is to either hide the things you can't do (because features I can't use don't really matter to me) or to make those things look different. This is where we differ, because you think it's not a bug, it's a feature, because it's a "small barrier" by confusing new users as to what they can do out of the box. I personally think that's /insane/, much like someone who decides to make all their links black as a "small barrier to new users".
I think you also misunderstood what I meant by "reduce the arbitrary karma limits". I don't necessarily mean "make them easier to attain" (although I'd argue reducing barriers to entry unless you have a good reason as it generally encourages new users), but rather "reduce how many of them there are". There are 10 different 'levels' of karma where features are unlocked to /users/. Why does a user need 15 to upvote but 100 to downvote? Why do you need 500 karma to retag questions but 750 to edit community Wiki posts? Most sites have a maximum of 3 levels of hierarchy (Admin, Moderator and User), your site has /at least/ 11. I guess with the badges system I should be happy that it doesn't use the same ridiculous system as Team Fortress 2.
And no, I wouldn't regard HN as good at usability either. Functional rather than ideal, even if it is trying to make creating a new poll difficult.
C'mon! The guy doesn't know what he's talking about, yet he is quick to say that HN "gets it wrong". Others show him he doesn't know the rules and he replies that the rules should be more explicit, when the fact is this very idiosyncrasy is what helps HN not to grow too much: it takes someone who is really interested in participating in the community to stick around and learn how things work around here.
I'm flagging the link. I never liked Jeff's "from-the-hip" style of blogging, but now more and more of his articles is plain linkbait and a way to get attention to Stackoverflow.
(Btw, Jeff, the whole spiel of writing even when there's nothing to say is annoying and makes more noise than signal. Quantity does not always trump quality. Practice makes it perfect, yes, but all I can say is that you're getting really good at producing crap.)
To have a post on HN with such an inflammatory title and factually wrong claims is downright offensive. It's no different than me going to his office and screaming how stupid he is.
In fact, I'll cancel my rarely-used StackOverflow account.
2) comments can be voted down, but it's uncommon, and the UI is invisible (literally) to the point that a casual user wouldn't even see it happening on most posts (very few posts go negative). This is a good thing, but it's not at all clear to a new or unsophisticated user that the possibility of being downvoted is on the table at all. Particularly given the front page which does not allow downvotes on submissions.
3) isn't your post the very sort of inflammatory stuff that you're claiming I write?
This is as silly (or arrogant, or plain stupid) as writing a critique of quantum physics just because you know how and when to use Schrödinger's equation.
2) It's not uncommon to downvote comments at all. Again, you are passing your perception as absolute truth. What you may see is few comments that have a negative score. Few downvotes and few negative-score comments are two completely different beasts, and you're inferring wrong things with the information you have available.
3) http://www.merriam-webster.com/dictionary/inflammatory : tending to excite anger, disorder, or tumult.
My comment was a response to your post. It comes off as angry, but it is a reaction to the sort of thing that you write. The post has more than 100 comments, few of them actually accomplishing anything except discussing with you and pointing out why you are wrong.
So, the answer to your question is no; it's your articles that provoke reactions in others and excite disorder. It's your article that is inflammatory, not my comment.
- - - - - - - - - - - - - - - - - - - - - - - -
There is no "right way" here, no "better side" to belong to. The problem lies in taking these opposites to radical extremes, and this is what is happening to you. While hardcore introverts end up suffering from analysis paralysis and never get anything done, you end up doing things carelessly and using "social brute-force": you say whatever comes to mind and expect people to throw you "warning messages" and point you to a better solution.
Your blogging style is the equivalent of the guy that trolls in IRC channels (http://bash.org/?152037) to see if he gets responses from the gurus. This is seriously irritating: not everyone wants to be your "idea compiler", and you consume much of other people's time and energy that could be better used elsewhere.
You may even think that you are being successful with your approach: each post you write will probably make your readership increase more than decrease. But if you believe that, you'd be again relying on the idea that Quantity always trumps Quality, and again you'll be wrong by missing the point about the purpose of something like Hacker News.
Quantity trumps Quality on SO - you are more interested in having hordes of beginner and intermediate techies to drive traffic than lots of experts that don't need to seek technical help as often - but HN is about insightful news and comments, and for that you need the participation of above-average and like-minded people; it's all about quality, not quantity.
This is really a criticism of me, personally.
You don't like me. I don't have a problem with that at all.
The idea that everyone has to like everyone else is unrealistic and dishonest. If everyone likes you, you probably aren't doing anything very interesting.
Calm down.
Design decisions are simply tools to accomplish these goals. Jeff says disallowing downvotes is "harmful to the community" and explains why he theorizes is the case, but why guess? We have HN, we can examine it and decide for ourselves if HN has been "harmed."
I am all for him beating his chest and extolling the virtues of his complicated system. But I do not see the merit of his comparison to HN: It strikes me as fairly run-of-the-mill Coding Horror-style link-baiting.
When a clearly political post about the drug war stays on the front page for two days, that's harm in my book. I expect you will see more and more of this over time as the community grows.
Sorry you feel that way about the post, reg. Just meant to be a discussion, and I went out of my way to say positive thingsa about HN. It feels almost like a personal attack the way you stated it here.
Although I still maintain that upvote and "off-topic" are not symmetric actions.. but choosing between public visibility of data vs "cabal of secret editors" is a no-brainer IMO.
I agree the potential for damage is much higher with downvotes, but there's an asymmetry there.
The trick is to make downvotes cost a little bit of karma, IMO. And while I'm on the topic, upvotes shouldn't be as free as they are, either.
Having said that, the 2-1 downvote scheme you proposed does put some of us within spitting distance of taking nickb down to zero, so I support it wholeheartedly.
Having a secret cabal of editors, whose behavior and rules are not documented, is simpler than spending karma to vote against things? Really?
It's like people used to say about micropayments; all these little knobs exact a mental cost; I have to spend 5 cycles deciding what to do --- mod up? mod down? what are other people doing? do I need to care?
Seems to me like Hacker News is doing the Simplest Thing That Could Possibly work. A good thing.
Wouldn't it be simpler to automate the whole thing, and let the algorithm do the work?
Having a downvote on stories simply accelerates the divergence of good stories and bad ones... it doesn't add any information, and in fact gives undo power to early voters. An early downvote would prevent a story from ever having a chance at the front page. (It's hard enough to get people to read the 'new' page as it is... they aren't going to read something that is five minutes old and already at -1.)
If comments could 'fall off' the page like stories can, they wouldn't need a downvote either. Also, there are fewer stories than comments, so they are easier to watch by the editors. Downvotes on comments let users help out the editors a bit.
Personally, I'd rather not have downvotes on comments either. Being able to flag trollish things is enough. If I simply don't agree, I should reply or vote up an opposing comment rather than simply downvote with no explanation.
In my defense:
1) There's definitely an element of "if the user can’t find it, the feature doesn't exist" with downvoting here.
2) In practical terms, downvotes are so extraordinarily rare on HN that they almost don’t exist.
Anyway, I'm encouraged that downvotes exist on HN, but the threshold could be considered so absurdly high that they're nerfed into oblivion.
See:
http://news.ycombinator.com/item?id=507278
http://news.ycombinator.com/item?id=507604 (the downvoting on this does seem questionable, although it is an extreme position)
(Edit to add: I should mention I'm not talking about foul-mouthed racial slurs getting tolerated out of a total desire to let it all hang out. Its more that views which are quite possibly locally unpopular get tolerated, particularly when not obviously trolling. "Rails docs suxorz" would probably end up gray. "The quality of documentation in Rails is subpar, and has been for years. The platform changes so fast that significant features of the most recent releases are documented only on blog posts, if that. The wiki is an unusable mess, bordering on unreadable and filled with information which is out of date or merely wrong." would almost certainly not end up gray, despite the fact that Rails is very popular with the crowd who hangs out here.)
Another reason is that while the signal to noise ratio is not quite infinite the signal to "#$%!"# ratio is pretty close to it. I have omitted the word I wanted to use almost entirely out of my concern that it would be inappropriate for HN. That says something right there. To see it, click here:
http://www.penny-arcade.com/comic/2004/03/19/
The following are taken directly from another popular news site, at random from the #1 story today.
[Ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha. That was rich.]
[It'll be as big as that book of his, that totally revolutionized science! Oh, wait, nevermind.]
[The next Cuil!]
These three comments were heavily encouraged by the community on the other news site. They would not be encouraged here. This tends to prevent other people from seeing "Aha, sarcastic one-liners are what is valued, I should probably work on my sarcastic one-liners" and drowning most of the value-adding discussion.
And they don't exist for article submissions, which is problematic for all the same reasons I listed in the blog entry.
that suggests they are overutilized elsewhere and work pretty well here, I am not adverse to opposing viewpoints, I just dont want to see plainly dumb comments, if you overuse it, then it loses meaning.
(taking this thread as a test data, while I dont agree with a lot of what people have said, I wouldnt have downvoted any)
It sounds like you didn't actually understand the dynamics of the HN community. You sound silly insisting your initial impression is even partially right, when the whole community is telling you the opposite. (This applies only to posts, of course. Downvotes still don't exist for stories themselves, outside of flagging.)
As for your assertion that not disclosing the karma threshold for downvoting is 'bizarre', I'd argue it's useful not to provide an exact number. Enough video games have 'achievements' listed out for people to obtain... I'd rather have people participate honestly than just to gain more 'abilities'.
There aren't article downvotes-- but there doesn't seem to be a need for them. The moderators kill stories that are plainly inappropriate, and the ranking algorithm makes it difficult for things to reach the front page without a decent number of upvotes in a short period of time.
My point is: this is an interesting and subtle topic, and it might be in your best interest to study it in some detail, rather than just glancing superficially at what others are doing and forming conclusions thereupon.
2) That's largely because few users make comments lame enough to be downvoted. But when they do, they get downvoted: http://news.ycombinator.com/item?id=507604
It's not because you don't see them directly a lot that they don't have an impact. A comment that ends at +3 with 7 upvotes and 3 downvotes would have ended at +7 (or more because of the "momentum effect") if there weren't downvotes.
And all because someone went around downvoting all the comments for a particular user they fell out with. It's a sledgehammer, pg! Come up with something better.
If you want to participate, you should write comments, or, better yet, submissions of your own.
But it's not a big problem.
The only other site I can think of that's close to as consistently a high level of quality in the posts and comments is MetaFilter. And unfortunately there are very few technology discussions over there. :-(