The Quality, Popularity, and Negativity of 5.6M Hacker News Comments | Hacker News Reader

The Quality, Popularity, and Negativity of 5.6M Hacker News Comments

minimaxir.com

82 pointssinak20 comments

> The minimum average points score of any given comment was about 2 points, meaning that all comments received atleast 1 upvote on average. This trend has been increasing until 2011 when it peaked at about 4.5 points. Since then, the average has trended downward, with a particularly large drop starting at 2014.

> Therefore, starting in 2014, both quantity and quality are on a downward trend.

The conclusion is unwarranted, because there are other changes in the system and access points. Most significantly, points are no longer displayed and the shift to mobile devices. This could affect voter's behaviors over time.

For example, people may be less inclined to vote when they do not see points (the numbers could trigger voting in some cases). Voting on mobile phones is also a pain as arrows are very small there, and a lot of mistakes could be made (clicking down arrow when intending to vote up).

If people are less inclined to upvote, then they would be less inclined to downvote as well, offseting it slightly.

Regardless, if the decline could be attributed to mobile devices, it would have happened way before 2014.

> If people are less inclined to upvote, then they would be less inclined to downvote as well, offseting it slightly.

That's not exactly true. Since more people can upvote than downvote, the threshold for downvote privileges is > 325 karma. So downvoting is reserved for those that frequently interact with the service in a positive manner. Downvoting could have had almost zero impact done to it due to the small set of people with that privilege.

As quantity increased, and newer less frequent users stopped upvoting, because there was no way to discern one's value in that interaction. Downvotes could have held at a similar frequency, because of it's smaller group. The decrease in upvoting could be accountable for a large portion.

Also, I rarely used a mobile device for interacting with HN, until 2-3 months ago, because the interactions are quite crappy (one of the reasons they mentioned for opening the API up.)

Edit: It's actually interesting, that the upvote/downvote ability wasn't considered impactful, since you mentioned it further down in the article.

> Edit: It's actually interesting, that the upvote/downvote ability wasn't considered impactful, since you mentioned it further down in the article.

In retrospect, I should have mentioned the downvote threshold. However, it wouldn't be helpful since I don't have access to the karma values of the upvoters/downvotes (e.g. are the majority of upvoters people who are under the karma threshold? are the majority of downvotes those at the karma threshold or far above?), so I wouldn't be able to make an accurate inference.

When was the karma value of a post hidden? I can remember it happening but tried a few search phrases and couldn't track down the announcement and resulting discussion.

I would be interested to see a how the "negativity index" behaves with other emotionally charged topics. What other sorts of topics lead people to use a more emotionally charged vocabulary? What if instead of titles containing (women|female|diversity), you did the same analysis with other topics associated with toxic discussion? Maybe:

(israel|palestine|gaza)

(gentrification|rent control|eviction)

or even

(windows|microsoft)

The data seem to indicate that people use more emotionally charged words, both positive and negative, when they talk about something emotional/personal like diversity. I wonder if other topics lead to the same asymmetrical increase in negativity?

Seriously, I found a few of the conclusions drawn from the data here to be pretty odd. I was hoping for some interesting insight but the article is really disappointing.

> "The language in the former is more neutral and about the content of the article (apparently Hacker News users really like to talk about Open Source software), while the submissions about gender and diversity trend to talk about tangent topics."

Does the author _really_ find it surprising that combining all topics will lead to, on average, more "neutral" language than a topic that (ostensibly) doesn't require any expertise/knowledge of esoterica? It should be obvious that the number of people who have an opinion about diversity in tech will be higher than the number of people who feel the same way about "Show HN: Virtjs, an ES6 emulation library " or "The amazing progress of LEDs" (both from the front-page right now).

Note: I'm not suggesting that the latter two aren't as _interesting_, but that contributing a comment of substance is obviously going to be more difficult and thus have less people go ahead and do so.

Is it just me who finds the latter two far more interesting?

It seems the biggest problem with HN becoming more popular is that it's picked up all of these politically charged topics which were very seldom before (from memory). If anybody could run the numbers on the percentage of topics featuring 'diversity' from 2014 compared to earlier that would be very interesting to me.

There's no shortage of places to discuss diversity and the usual political issues, but far fewer places to discuss difficult business and tech issues and having HN taken over by these 'interesting' topics is very unfortunate. Is this the case though?

Per your suggestion, I created a word cloud of (windows|microsoft), and compared it with a word cloud of (google|android) as a counterpoint: http://i.imgur.com/CD3rFRR.png

...I did not expect these results.

Just a quick FYI for the author:

> "The average amount of positive words in a comment made in thread about gender and diversity is 2.48 words, a little higher than the average, and is also the most frequently occuring value. However, The average amount of positive words in a comment made in thread about gender and diversity is 2.10 words, a much higher increase."

I think you mean:

> "However, the average amount of negative words"

Good article though, great graphics and cool insights!

Nice timing coincidence: recently I decided that I would not comment on a story if I didn't have anything positive to say about it. I found it that it's a lot easier for me to focus on one tiny flaw and point it out that actually making a contribution to the article. My number of comments has really gone downwards since then, but hopefully their quality will go up.

I enjoyed this a lot. I do think the author is tending to make his assertions too strongly e.g. saying things like "this data shows that..." instead of "this data suggests that..." It's still a fun read and thought-provoking.

I made the worst comment of the month list for a comment I actually thought was good (!), and now I'm really curious about the context...

Is there any way to recover the thread ids or parent comments for items on the the worst comments list?

Found it quickly on https://hn.algolia.com/ it was a thread titled "Steve Jobs - To All iPhone Customers" and here is the link: https://news.ycombinator.com/item?id=51226#up_51333

But I agree, I read some of the worst comments and would be nice to get a link to the discussion for each of these to get some contexts!

Very thorough. Two things I'd like to see are comment score distributions in the 3 months or whatever before and after scores were hidden (the purple histogram), and a heat map of article tone versus comment tone.

I am slightly disappointed in the lack of mention toward posting over a 24 hour period, and by extend to toward the timezone coverage of the users. Although I have seen article pointing toward this in the past.

I had done timezone analysis for Hacker News submissions, in February: http://minimaxir.com/img/hn-submissions.png

More likely than not, comments would show similar behavior. (I didn't include it in the post because those types of charts are much harder to make)

Thanks for this.

Third paragraph from the bottom, you say "However, The average amount of positive words" when you meant to say "negative." Second from bottom, you misspell "necessarily."

Fascinating dive into the data-really appreciate the sentiment analysis of comments made about women and diversity.

> It’s worth noting that on Hacker News, you can be downvoted for being factually wrong.

Also for having a differing opinion, the incorrect number of characters in the post, misaligned chakras, irregular weather patterns, cosmic rays, etc.

Reply on news.ycombinator.com