I was a bit bummed I thought this would be more inline with Shannons entropy, or a measure of how much an article reveals about the topic.
They use Shannon's entropy in the paper, no?
I would summarize it as the entropy over distribution of topics over the whole news spectrum, rather than a the entropy of a particular piece of news.
I guess this is rigorous confirmation of the slow news day = talk about anything, crisis = talk about Michael Jackson's death for six months model of news broadcasting.
> I guess this is rigorous confirmation of the slow news day = talk about anything, crisis = talk about Michael Jackson's death for six months model of news broadcasting.
Well, yes, but it is also implying "slow news is good news".
I’m working on a commercial side project along these lines — if anyone is interested in helping or collaborating, shoot me an email!
When I've tried to predict social unrest or market volatility from the news, I've found, than news actually trailing important events and are useless for any sorts of prediction.
> When I've tried to predict social unrest or market volatility from the news, I've found, than news actually trailing important events and are useless for any sorts of prediction.
It might be useful though to be able to distinguish actual social unrest from the astroturf variety.
Luckily, that’s not my application area if these ideas right now. :) But it might indeed fail for some of the same structural regularities!