Why Nate Silver, Sam Wang and Everyone Else Were Wrong | Hacker News Reader

Why Nate Silver, Sam Wang and Everyone Else Were Wrong

quantamagazine.org

8 pointsdigital5510 comments

Everyone claiming that polls or forecasters were wrong should really do their homework and think what 'forecasting' is all about. It is not about predicting future with 100% certainty. It is about calculating odds of outcomes. You can calculate odds of Trump winning and put them at 40%, you can put them at 1% or 90%. In neither of cases you are wrong or right when he wins. Odds of winning are not 'verified' in any way. If you say that there is 50% chance of coin flipping on tails you are not wrong when it flips on head.

Perhaps, but I still think the fact that so many polls had Clinton's chances of winning well above 90% is indicative of some incorrect assumptions within their models. Especially considering how they were off in nearly every single swing state. Most forecasters had Clinton winning Wisc/Michigan at over 90%, and considerable advantages in most the other battleground states.

It would be like if I gave you a quarter and said it has a 75% chance of it coming up heads (ignore physics), then you flip it five times and it comes up tails each time. Sure it could be a run of coincidence, but I'd also put serious consideration into my statistical model.

the polls were not expecting a globalist establishment incumbent vs a nationalist populist outsider. the polls treated him like GOP candidate, when he was more of a parasite to the party, using its skin to get on every ballet as a "top two" candidate.

at least Nate Silver's part two to this piece will include "i was less confident than most others" https://fivethirtyeight.com/features/how-i-acted-like-a-pund...

Silver really took Trump's ascension to the GOP throne as a lesson. http://www.politico.com/story/2016/11/nate-silver-huffington...

Didn't Nate Silver acknowledge that this was a tough election for his methods because of the highly unusual nature of Trump's campaign, among other things?

EDIT: found some indication of this from May of this year: http://fivethirtyeight.com/features/how-i-acted-like-a-pundi...

I've been listening to the 538 daily podcast for the last couple of weeks (as a Brit it's been a great way to get an insight on how the US system works rather than just the politics on the surface) and, when on the show, Nate would not be drawn on saying who he thought was going to win. He really seemed to be going out of his way to say that the model is based on polls and those polls can be off, the model is only as good as the information that's put into it.

The model had a pretty low Hilary chance a few days ago and then it ticked up at the end to over 70% for her but Jody Avirgan and Harry Enten were warning about pollsters' tendency to herd (I think that's what they called it) at the end and so the final polls can be less reliable due to this.

I'm looking forward to their wash up podcast and then I'm checking out of politics, between Brexit and this it's been a bumpy year.

Also notable that they mentioned repeatedly that:

1. Polls in key swing states like Michigan have been consistently bad over the years, and there hasn't been a lot of high quality polling in those states during the election, which meant there was a lot of uncertainty in those states.

2. State polls tend to be correlated. That means if the polling is off in Michigan, it'd probably be off in Pennsylvania and Wisconsin as well, and in the same direction. And, what a shock, all of them were tipping point states, and all of them had polling errors that swung in Trump's favour.

3. Trump was well within the polling error of winning by election time. That is, while FiveThirtyEight had Hillary with 2-1 odds, that was with significant errors bars due to uncertainty, and it runs both ways.

To be honest, listening to their podcast and reading their posts, I was preparing myself for a potential Trump win.

And he was getting flack for giving Hillary only 2-1 odds. And he also indicated there was error bars that work in both candidate's favour.

IMO, if anyone was at least sorta close, it was FiveThirtyEight.

You can't claim they were wrong at all. If you throw a die and I say there's a 67% chance you wil throw a three or higher, and you throw a one or a two, you cannot claim I am wrong.

It's true that you cannot claim I was wrong if I said that probability was .00000000001%, either, but it sure feels different, and statistical tests will also bear that out.

Actually, the LA Times made a fairly big deal about forecasting correctly.

https://twitter.com/latimes/status/796360500091252736

They predicted Clinton would win the electoral college.

Reply on news.ycombinator.com