Probably too early to tell, but it would be nice to see what happens when you take karma away from comments. I am trying to imagine reddit without it and it doesn't seem quite as interesting. Some hard data would be cool to look at.
Fundamentally, removing them has changed the balance of what karma is all about. Karma previously had a function for the reader. It highlighted and bubbled up the good. It showed where there were good counter points. Now...well it's just for the account owner. It feels just like it's there for your own rpg-levelling style dopamine hit. It feels pointless.
I really feel this is a mistake. Comment karma had a function for readers, not just for account owner's inner satisfaction. This role has now been lost. It's making browsing comments - especially for very popular topics with lots of comments - much harder. I'm skimming a lot more or actually skipping the comments altogether because I don't want to read them all to find the goodness. It also means people can't learn from others what is or is not a valued contribution.
So I really hope this will be reversed or at the least make a better way (more than just what is the top comment) to make karma benefit the community not just the individual.
Also, I was reading the nginx 1.0 post where I saw some short but technical observations, and I had no idea if the community deemed them valid or informational. At first I though this was bad, but upon further reflection, and after reading some technical comments with more beef, I'm now on the fence. The beefier comments brought along their own credence, so if removing karma scores prompts folks to write longer more explanatory comments, that might be better.
The curation still happens without numbers attached.
Other times, I don't have that much time and simply want to glean from the best comments on a post.
And then there are the posts in subject areas that are outside or on the edge of my expertise, where I have no business evaluating the comments. I can still benefit from reading them though, and the karma helps me give weight to the good ones and disregard the bad.
For me there are a number of answers to your question:
- I love HN because I get to learn about lots of stuff I know very little about. For example I'm a business guy not a progammer, but I love to learn more about the non-business things. I genuinely can't judge the 'good or bad' on my own on some of this stuff. The karma gives me a quick and simple way of the community - this wonderful, intelligent community - pointing me in the right direction.
- It also shows where my comments might be off base - not because my comments karma is high or low (which I still see) but because I can see if the counter arguments are rated highly. For example I can't see if your comment is highly regarded or not. If was +40, compared to +1, I'd be more likely to think 'hmm, maybe I'm wrong on this after all'. Sometime is not just the power of the argument that changes minds, it's the weight behind it. I can't see that weight now.
-sometimes I quickly browse HN with some minutes of down time. Sometimes I read in depth. I now feel I'm being forced all the time to read in depth. Whilst that sounds good, in practise for me it isn't. Instead of more in depth reading I'm just cutting out the browsing, so I feel ultimately I'm getting less out of HN.
I find that I often instinctively head to upvote a comment that has hit its critical mass in terms of karma votes regardless of its content. I have to take a second, back off from my instinctual action, and evaluate. Often I end up upvoting it anyway upon reading it, but at least then I have taken the time to evaluate it myself.
Not having the points present makes this problem go away because there is no implicit "lots of people like this, I probably will to" reaction. The jury is still out on whether or not I think things are ultimately helped or hurt by the move but I believe it is worthwhile for the sake of experiment if nothing else.
What I recommended would encourage more voting, so you could just read the most highly ranked comments with some confidence about quality -- poor comments would probably be downvoted quickly.
For instance, the top comment at the top of the page may have 10 karma and 50 replies, while the next valuable comment has 15 karma (but was submitted 3 hours earlier), and isn't visible until halfway down the page. Without the karma displayed, it's much more difficult to skim the thread. And for all I know, the rest of the comments only have 1 point.
Plus, I recall from my reddit days that the earliest comments are way more likely to receive votes. So I always ended up trying to divide the karma by the time since post, which just distracted me.
Different strokes I s'pose.
Yet because the comments are still sorted somewhat intelligently, I'm not really being drowned in the cruft like one would expect.
My experience is the opposite of yours. I find karma as it has become here recently is not a reflection of good counterpoints. I feel it has become a reflection of the popular points. Well argued counter points, if unpopular, are "voted down".
Assuming the thread sorts by karma, you can still read in order, and get what you're looking for, and maybe you'll read things that are less popular, not just the majority-upvoted comments.
Another comment here talked about score, a gaming term. I don't like the term "score". Like you, I value thoughtfulness and interestingness, whichever side of a point it comes from.
In user profiles, karma can work as a gaming feedback mechanism to encourage desired behavior, while in a comment thread, it can work to surface insightful discussion and bury non-contributory content. A display of "score" isn't needed for the second mechanism, just good sorting.
However, it seems that a refinement of seeing your own 'score' on your own comments could be valuable towards the first mechanism: "How was my comment received?"
That dopamine hit is valuable to foster recurring contribution.
A good compromise would be to have a button at the bottom of the page that says, "OK, I'm done voting now, please show me the tallies", and the voting buttons would disappear, and tallies would appear in their place.
If you guys want to fancy it up, you could implement the feature where users could indicate they're done voting on the thread and see the tallies early, and you'd only have to keep track of x days worth of user thread status flags before you can purge the data.
Or any two words that intellectually interrupt the "vote up / vote down" reflex to vote down unpopular or dissenting points of view no matter how well thought out.
And pg & friends probably have some different goals than what some hn users are used to a online forum having. HN doesn't need to be monetized or have huge page views for it to be a success. All pg needs hn to do is attract the right people and scare off the wrong people. And I imagine that creates some metrics that are more quality based than what most hn users are user to.
- I spend substantially more time wading through comments because I can't pick out the good ones. This is particularly true of nested threads.
- I often vote based on the current score. If the score is as the comment deserves (in my opinion) I leave it, otherwise I upvote. I rarely downvote.
- I can't pick out whether a comment is valid (based on scores, and thus the opinion of other users). For instance if a user writes in a comment that joshua schachter never invests in B2B startups I don't know how valid it is.
- When someone responds to a comment I like to see the votes. This is particularly useful when someone disagres with me - if the response gets a lot of upvotes I can be fairly certain I'm wrong. If I don't see the votes I won't know.
Besides, if we don't see scores they're not much good to users. My bet is that over time people will vote less, since they don't immediately see their impact anyway.
Do people here think this is a forum for entrepreneurs? Do people think it's OK to link to the HN thread and not the post? Without the votes I simply don't know.
Or how about the distribution of upvotes over comments under one post -- are upvotes more uniformly distributed?
Do the comments on the bestcomments list have greater or fewer upvotes on an average?
If the post is offtopic/noise why is -4 a magic threshold? If that threshold can be justified why not incorporate it into HN as policy? If it's just to protect egoes, then HN can just secretly use something like a log scale or some other relationship when displaying karma to users.
If voting patterns are more independent of a post's current karma, the measurement should be less biased. The same is true for positive karma--I often don't bother vote up comments that already have relatively "high" karma.
Everything else should get ignored or warrants further discussion. In some cases it is even important to upvote "wrong" comments.
Same sort of deal when people are factually wrong, but aren't being a jerk about it: I don't think they should rate that highly, but I wouldn't likely vote them into negative territory either. And yeah, I might leave a comment too.
I always thought that the saving grace of /. was the metamoderation. People here downvote based on personal bias here frequently enough, and it would be nice to have the bad actors dinged for it. I'm not saying you do this, but it is done. Far better to have a highly upvoted correction.
Downvoting obviously wrong comments is very simple too. And useful. Like I said though, in the past I've tended to do so because it's incorrect, not to "punish" as I might with a mean-spirited comment, so I would never go past -1.
Naturally, the other end of that is a problem too, wherein people upvote for all the wrong reasons. I all too often see an comment that is wrong, but perhaps clever, and is upvoted highly. The most interesting aspect, I think, is that if you put a well-reasoned explanation of why the comment is wrong, you'll often see the trend reverse -- the wrong comment will start getting downvotes, and the correction will start getting upvotes. This might SEEM right, on first blush, but it also isn't.
The thing that irks me most though, is perhaps the hero-worship. I think if we were able to hide usernames until after we'd voted, then we'd almost certainly have better results. I cringe every time pg posts a one-liner that adds nothing to the conversation; Not because it isn't perfectly within his right to do so, but because I know it's going to be upvoted to the sky.
Besides, for a lot of things, what constitutes "wrong"?
I personally used to use downvoting to sink comments that are of no particular interest to me, and to promote other branches of the thread that I'd like more people to look at and comment on. So there's gotta be a mechanism that would allow for rearranging comments in a more popular order, but without punishing their authors unnecessarily.
I like having the community filter articles, but when it comes to a discussion, I like reading the arguments and deciding for myself. Scores do influence my expectations around what's important and what isn't.
Besides, there is already a filter in place for really bad comments. If something isn't at -5 or so, I really ought to give it the benefit of the doubts and read it.
My algorithm was to read the first several high rated karma comments than the article if interesting. I usually got the picture from the peoples perspective quickly and it also eliminated lot of the repetition and noise.
Too bad, my hope and wish is that the karma on comments is coming back.