Daily happiness averages using Twitter

From http://hedonometer.org/about.html :

> “Why does the day of Osama Bin Laden’s death have such a low happiness score?”

> Many people presume this day will be one of clear positivity. While we do see positive words such as “celebration” appearing, the overall language of the day on Twitter reflected that a very negatively viewed character met a very negative end. It was a day of complex emotion which is best explored in the word shift for the day, rather than the single number of its average happiness.

That's.. a pretty dodgy statement. A more straightforward and honest explanation would have been "We use a bag of words approach which relies on the assumption that people use negative words when they feel down and positive words when they feel good. This sort of models gives good results in most simple cases, but doesn't handle complex cases, and can't take into account that people sometimes use the words "death" and "dead" in a positive way."

There's nothing wrong with using these sorts of basic models. That's what pretty much everyone who provides sentiment analysis does, and it's good enough for most cases. But there's no need to hide the limits of the system either.

Remember these are Twitter users, not the world.

There is a trend in thinking that Twitter, Facebook and other social users are representing the world population.

Hedonometer gives some explanations here: http://hedonometer.org/about.html

"Tweets represent a non-uniform subsampling of all utterances made by a non-representative subpopulation of all people. However, there are hundreds of millions of people presently using the website to express their activities and interests, and as such it is an important social signal."

May 2 2011. Did "death" really mean sad or happy?

"I suppose I should be expressing some ambivalence about the targeted killing of another human being. And yet, uh, no. [...] Last night was a good night for me " - Jon Stewart

I recently had a similar idea (i.e. plotting contentment) related to keeping employees engaged / detecting how events at work affected the general mood: https://news.ycombinator.com/item?id=5646466 This post's great as it shows a really good way to model that data (e.g. their use of different colours for days of the week helps to determine if you had a good day for a special reason or if that day people are generally in a better mood).

A surprisingly good holiday detector (Christmas, valentine's day, mother's day etc.)

Other major events also clearly stand out (earthquakes, tsunamis, hurricanes etc.)

So now Twitter really is out there as the biggest social science database there is. And much of what we get are descriptives (and analysis on the stockmarket). What are interesting research questions that need tackling (let's say: non-financial) and can use a dose of massive sentiment analysis?

I would like to know how you can use global and local sentiments for all kinds of analysis of social welfare.

I wonder how do they deal with negative phrases:

* not happy

* far from being happy

* I could use some happy moments

* ... etc

Reminds me of http://twistori.com/#i_love which has a nice screen saver too.

I wonder how much happiness stems from aggregate "good" weather across the locations at whcih people are tweeting. I would wager that it correlates quite nicely.

Then I would, once again, question why the hell people live in gloomy places when most of them enjoy sunshine (and lollipops?)

Wow, Tuesdays are killer.

Edit: At first it seems we are trending down in happiness, but surely there is some bias here. One thought is that the number of new or fresh twitter users has declined, and so the overall eagerness and excitement around tweeting has declined as well.

Very good, I was considering doing a similar experiment, but possibly grouping happiness by country. And I was also thinking some way of determining what triggered the expression of happiness, you know like , 'I am happy now, I just did a skydive' etc..

See also: hourly happiness averages, using a dedicated research app: http://blog.mappiness.org.uk/2011/11/17/project-update/#more...

I'm not that surprised by the low score of 5/2/11, because double negatives are hard. It's actually hard to separate sentiment when interactions (such as double negatives) come up:

"A bomb killed a terrorist." Good news. Three negative words.

Many psychologists believe that the quickest parts of the human brain don't process double negatives at all-- that's why thinking "this is not going to kill me" doesn't help during a panic attack, but "this will end" does-- which is probably a small part of why news-watching (even positive news like Osama's death) makes people unhappy.

"My best friend has been killed by a heart attack." Three positive words. One negative. One (attack) that is slightly-negative but has energetic/positive connotations.

"My best friend defeated cancer" vs. "Cancer defeated my best friend." Similar tokens; opposite meanings.

What really surprises me is that it seems contrary to economic trends: in late 2008, when the economy was going to hell, the sentiment average goes up in a major way. Across 2009-13, while the economy slowly recovers, the sentiment level declines. Day-of-week average differences are very slight, but the more people are working, the more unhappy they are. This could mean that structurally unemployed people are self-deceptive, or tweet happier things because they have more time per tweet, or it could genuinely mean something.

is it me, or it looks like it is going down?

So if I say 'merry christmas' does it think I'm feeling merry? That would be an unfair assumption.