Hacker News Reader

Comment by somehnreader | Hacker News Reader

Could you elaborate on news & NLP in regards to stocks?

We tried sentiment analysis in uni a few years ago and had no good results:

The idea was essentially: news says: 'stock A is great' -> it goes up shortly thereafter

We tested our algorithms against classifying Amazon reviews & Tweets by sentiment. Those are filled with sentiment and easy to detect if it is a 5 star review or a 1 star review. The news articles we parsed all had near neutral sentiment. We ended up building a classifier that could detect the news category of an article quite easily instead.

My initial idea was sparked by the Gulf spill and the subsequent dip in BP, I wanted to detect and capitalise big events like that, but the news sources we parsed always seemed to significantly lag behind the stock movement, too.

View on HN

I just did a project on this, but for Bitcoin instead of stocks where I examined news, Reddit, forum (Bitcointalk.org) and IRC sentiment using some simple ML algos. The goal was to determine whether this data has any predictive causality.

I scraped the above sources over a full year (2015) and then had the data annotated on positive, negative and neutral sentiment.

The problem with labeling sentiment data is that there might not be a single 'true' label due to varying interpretations and ambiguity. So at best you'll get to 80-85% accuracy there. The less formal (News > Reddit/Forum > IRC), the lower your accuracy due to lack of context.

I Then matched the annotated sentiment to market data and did some causality analysis. What I found is that interestingly, you can't just say positive news = price/volume goes up. It is way more fine grained than that. For example negative Reddit sentiment leads price movements, but price movements lead positive sentiment. For news its the reverse.

All in all I didn't incorporate this into any trading strategies, but found it interesting to see the differences between online sentiment channels.

NPR's Planet Money is doing that by following Trump's tweets: http://www.npr.org/sections/money/2017/04/07/522897876/meet-...

I recall reading a paper (sorry, can't find the link now) where the volume of news mentioning the company was what was relevant.

Need to measure this in terms of average volume of news for all tracked market participants, and relative to the normal volume of news for that stock.

That way, if the return has recently gone up and there is substantial volume increase its probably because of positive news.

Reply on news.ycombinator.com