Twitter May Have Predicted the Election | Hacker News Reader

Twitter May Have Predicted the Election

technologyreview.com

41 pointsjonbaer31 comments

Right now 100% of the comments in here are about "predicting" the election, when the word "predict" isn't even used in the article. Probably an over-eager editor who changed the title.

The article itself is about a very interesting network analysis of establishment vs non-establishment behavior. I'd like to see more discussion about the content of the article.

edit: downvoted within 1 minute for trying to spur interesting discussion past the crappy headline :(

“Our analysis was showing something, but our beliefs were different,” says Vishal Mishra, CEO of Right Relevance, which sells a research platform focused on influence. “We thought, how is Trump going to win? But our analysis kept showing us.”

He doesn't say "predict". But it is very clearly implied.

I noticed the establishment v. non-est. behavior as well. How much of this network property is really a predictive signal more than an explanation for "shock" at the results? What are the conditions for it being one or the other?

Any global predictor that claims to have accurately determined Trump's victory should keep in mind that he lost the popular vote. Only a state-based predictor can be plausibly said to have been able to make such a prediction.

Yeah it really comes down to what you base your hunch on. when I saw the first exit poll saying that many voters had opted for a 'strong leader' I knew it was over even thought he polls hadn't closed anywhere.

I have a great many hunches and some facility with visualization of directed graphs and other analytics, but am not clear on how to scrape and normalize the datasets I need.

I've seen one of those predictors. It looked at an honest signal, captured in a way that would be fairly representative of the population at large. Since the data had some location information attached, you could attach every point of signal to a congressional district. The model was simple, comparing the signal per district, and then building an electoral college map. It didn't nail every state, but it wasn't off by much, and gave Trump a slightly smaller victory than he actually had.

I won't mention the signal because I want to keep my job. However I'd be surprised if there weren't at least a dozen companies that have access to some honest signal with such a good sample of the population at large as to be better at predicting the election than any pollster.

It's interesting to think about where to find this information.

Big ISPs have access to these "honest" signals on a large scale.

Could you elaborate on your point? Im not sure I understand.

An accurate global predictor with no regional information should have predicted a Hillary win (since Hillary won the popular vote by a fair margin). So if you had a global predictor predicting a Trump win, either (1) it must have taken regional information into account (in order to understand that the distribution of votes was concentrated in such a way that the electoral college would not swing Hillary's way), or (2) it must not have been an accurate predictor. Most of the polls or methods that predicted a Trump win did not take regional information into account, which means they were just inaccurate predictors.

Ah, I don't disagree.

But there might be "hidden" location information in datasets that dont appear to have it, such as Twitter.

Word choice carries regional and social information.

So it's conceivable Twitter data could correctly predict the election with ML techniques that essentially are estimating the electoral college without being trained on an explicit model of it.

This makes sense to me. I saw similar things in newspaper comments sections. I have a weird habit of always scrolling to comments sections of news articles. It gives me an interesting snapshot of a huge range of opinions. On political articles, I consistently saw so many comments from Trump supporters, with some consistent threads of opinions (as well as more fringe or extreme ideas) that just weren't being represented strongly enough in the general media. I also of course saw comments from Hillary supporters and Trump haters. Their arguments and thoughts WERE already being represented strongly in the general media - there was not much new that I learned there. Something that Obama hasn't addressed is just how biased our mainstream newspapers and magazines were in the run-up to the election. As someone living in a highly liberal area (and who is not aligned with Trump's politics - before I get criticized to infinity) I felt I gained a lot of understanding for why people were supporting Trump, not from the dozens of mainstream newspaper articles I read each day, but from comments sections. I view that as a media fail. I was the only person of my peer group who was not shocked about Trump's win. I'd also add - the media strategy didn't work. Marginalizing people's voices and points of view - often people who felt marginalized in other ways (economic etc), strengthened their resolve to vote for the person who DID hear them (which was Trump.)

Where's the "predictive" part? Just visualizing twitter networks isn't actually a prediction.

Sounds like a "prediction" just like the results you get from training your stock picker on historical data.

At least you can break your data in to training and testing sets for stock picking, this is just hey guys we found a correlation. 20/20 hindsight is not a prediction.

Exactly. Furthermore, is this company claiming that they can predict electoral college distribution? Because we all well know that Hillary actually won the popular vote. So in fact the popular sentiment was actually on her side.

Any analysis claiming that they could predict Trump's victory without even addressing the disparity between popular vote and electoral college is a fail even before it begins.

actually Google Trends predicted it:

https://www.google.com/trends/explore?q=vote%20for%20hillary...

it predicted obama as well in 2012

https://www.google.com/trends/explore?q=vote%20for%20romney,...

Interesting how Hillary is the only one who was referred to by first name. "Vote for Clinton" didn't register as much as "Vote for Hillary"

maybe it was to differentiate her from bill clinton?

She's the only one that had a spouse that was formerly President and or otherwise very famous. Hillary's campaign and supporters heavily marketed her by first name rather than last name to try to bolster her own political identity.

My impression is that this was a choice of her campaign (differentiate from Bill Clinton? make her seem friendlier?).

Jeb Bush and Bernard Sanders also had campaigns centered around their first names.

I find it odd that you refer to "Bernard" when he's almost universally referred to as Bernie, but simultaneously call John Bush "Jeb". I'm missing whatever the intended meaning of the way you wrote these names was.

I did not know that Jeb wasn't his real name, that's all there is to it.

holy cow. mind blown. did not know that wasnt his real name either.

you mean Jeb! right?

I have noticed there is a strong bias that women are more often referred to by their first name compared to men in the same position. For example in discussing college professors, a single-word identifier for a male professor is most often their last name; for a woman it is much more often their first name. Keep your ears open -- you will hear it!

Google Flu Trends "predicted" the flu season, and then in 2013 it "predicted" it incorrectly:

https://www.wired.com/2015/10/can-learn-epic-failure-google-...

For the election, Google Trends isn't showing any data on the US state level, and you might recall that Hillary won the popular vote averaged over the entire US.

How much computing power, work and money is it to produce visualizations like this? I'm handy with cytoscape and DAG concepts but I don't know where to get real time datasets like this, or how to automatically construct them for other forums (eg say I wanted to study the social networks on Reddit or HN). Ideally I'd like to be able to work with something like this interactively although real-time operation isn't too important.

One thing that jumped out at me from this brief article was that the use of the term "media influencers" -- they certainly weren't influencers in the conventional sense of being gatekeepers.

Hindsight is 20/20

How do these people predict the electorial college?

They don't. It's a complete PR sham.

Reply on news.ycombinator.com