Hacker News Reader

Comment by joeblau | Hacker News Reader

Fascinating project, I worked for a data analytics company and we always had a basic test that we'd try to throw at any Sentiment analysis engine.

// Negative > This is shit.

// Positive > This is the shit.

Most engines can't sort that out. I'm definitely going to take a more in-depth look at this.

View on HN

"This is the shit" is a very specific idiom. Unless the engine has been specifically trained to understand it, it is not surprising that it doesn't know what to do with it.

It doesn't look like this one was able to sort those two out either. It gave me strongly neutral signals for both sentences, but that may be because 'shit' isn't currently tagged.

"This is shit" isn't inherently negative. Consider the question "what is the brown substance in the toilet?" and you can answer a neutral "this is shit".

Presumably if you throw a realistic corpus of written text you will have much more occurrences of the idiom.

Sentiment analysis implies attitudes, which imply context. A robust sentiment analysis system must either be trained on a targeted corpus (in which entities/concepts and attitudes are well aligned) or include some way of establishing a more targeted context after having been trained on a general-purpose corpus.

That is to say, people generally aren't describing the physical material occupying a toilet in a corpus of movie reviews.

I was referring to the previous comment, where the two lines about shit were used as a test for any sentiment analysis engine. Your point about context is what I was trying to illustrate.

I imagine that someone using the word 'shit' to describe the brown stuff in the toilet is probably negatively inclined towards it. Someone discussing it in a more neutral context (like a doctor or scientist) would probably be more likely to say 'feces'

Reply on news.ycombinator.com