Some personal news
natesilver.substack.com
natesilver.substack.com
> Much of FiveThirtyEight’s vital intellectual property — such as the election forecast models — is merely licensed to Disney. The license term for these models expires with my contract this summer. I still own these models, and can license or sell them elsewhere.
This is one hell of an agreement to have with your employer. That was my next question given that most of FiveThirtyEight was Nate's own work, and he had very good insight to have an escape hatch.
I really hope to have the power to do this one day.
Isn't this what the Skype guys pulled off with Microsoft too?
Disney wants the talent. It’s just not as valuable to them as keeping Nate happy
They're also not just after "entertainment IP". In fact I'd say they are one of the premier proprietary data analytics orgs with respect to park management, fast passing, queuing. Same with Market Research on film/animation, they've quite literally been doing it since it's been a thing.
https://www.semafor.com/article/05/01/2023/abc-news-struggle...
As others have mentioned, he's very wise to have negotiated contracts so he keeps some IP, but those sports models might be what generate the most revenue.
Are there precedents for this in the tech world, where somebody can legally bring IP with them to a new role but the old employer/competitor keeps an exact copy? Seems like an unusual arrangement that'd be hard to sell the upside to somebody competing with Disney/ESPN.
Also, that [poker bluff](https://www.youtube.com/watch?v=9cVrlVzoh48) was very entertaining to watch. Apparently Nate's quite the poker player (https://en.wikipedia.org/wiki/Nate_Silver#Economic_consultan...)
This is actually the standard arrangement, depending on how you define "tech world". In California for example a Proprietary Information And Inventions Agreement will include a provision along the lines of:
> If, in the course of my employment with the Company, I incorporate a Prior Invention into a Company product, process or machine, the Company is hereby granted and shall have a nonexclusive, royalty-free, irrevocable, perpetual, worldwide license (with rights to sublicense through multiple tiers of sublicensees) to make, have made, modify, use and sell such Prior Invention. Notwithstanding the foregoing, I agree that I will not incorporate, or permit to be incorporated, Prior Inventions in any Company Inventions without the Company’s prior written consent.
Meaning anything you created before remains yours, even if, out of the kindness of your heart, you allow your employer to benefit from it.
What's unusual in Silver's case is maintaining that arrangement for a) the founder of a company, and b) through an acquisition.
The competition would get value from having him host a show about it on their platform. Disney… I wonder if they even care about the model, or if they just took a license to it/current copy thing because, I dunno, maybe he offered it for really cheap.
I mean can you imagine, 538 but without the main character? It would be incredibly lame.
I assume Disney keeps 538's fursona, Fivey.
...do they, though?
I'm actually not so convinced in the journalistic value of FiveThirtyEight. It's something I've thought about before.
FiveThirtyEight's political forecasts absolutely provide a desirable service! Who doesn't want to know who will win an upcoming election?
But is it good for people to know who is more likely to win an election? Are these predictions beneficial to society or democracy? I sometimes think they may actually be harmful, decreasing voter turnout by convincing people (correctly or incorrectly) that the result is preordained.
That's not to say Nate Silver is doing anything wrong by making FiveThirtyEight available, but he tends to talk about it as a sort of public good. I'm not sure whether that's accurate.
They’re just the guy who went to the stables before to see who was eating the most hay.
I honestly mostly follow the sports odds and I’d argue that Nate’s models are some of the most biased out there for that.
EDIT: (cause I can't reply to the below comment) my question was what model beats 538, not which betting market beats 538. yeah, duh, the Vegas line beats 538, because it beats literally all publicly available models. It'd take a really disastrous gambling market failure for anything else to happen.
Nate put 2021/22 warriors at 0.5% to win, Vegas was +900.
Nate put 2020/21 bucks at 10% to win, Vegas was +550.
If you can’t even come close to beating the line, you’re not a good model. He’s just transparent about his variables.
> EDIT: (cause I can't reply to the below comment) my question was what model beats 538, not which betting market beats 538. yeah, duh, the Vegas line beats 538, because it beats literally all publicly available models. It'd take a really disastrous gambling
Lines aren’t built that way. They’re set by books to offset their risk. They’re based on a bunch of (usually drunk) wetware computers with huge biases (don’t believe me, look at any Dodgers line when they have an obviously bad pitcher matchup).
If your model is worse than that, then what’s the point of a model?
To the point of bookie models, those are based on how they think people will bet, not on who they think they will win. The point of books is to be outcome neutral.
They're initially set by bookies based on their models, but then pretty quickly evolve to a market-based mechanism.
It's really not, which is why quantitative models have real value. You don't have to argue that he's biased, you just have to point to the evidence. But your whole comment is indistinguishable from someone with sour grapes who is simply casting aspersions on his track record without being willing to analyze it properly. At the very least, you ought to specifically identify what kind of biases his predictions have shown; he leaves himself wide open to being proven wrong, so it's only fair for you to do the same.
He makes bad models.
Nate put 2021/22 warriors at 0.5% to win, Vegas was +900. Nate put 2020/21 bucks at 10% to win, Vegas was +550. If you can’t even come close to beating the line, you’re not a good model. He’s just transparent about his variables.
Edit: To be clear I don’t gamble, I just follow sports and like data analysis, so this isn’t a bank account thing.
(That said, yes, I think the models clearly have net social value, too.)
True. But millions of them will hear a radio announcer, or TV person, or tiktok video or whatever else say something like "In the latest polling, <X> has pulled ahead of <Y> ..." where "the latest polls" will mean (among other things) 538.
I do believe that this has some effect (if small) on patterns of voting (particular by encouraging or discouraging turnout).
No, it won’t; 538 doesn’t produce polls.
Nate silver didn't exactly invent political polling either. That's doing a disservice to the enormous time and money invested in polling over several decades of elections (and the learnings when things didn't go correctly). Polling will be around regardless.
That's fair. I was responding to ways I've heard Nate Silver talk about the model and polling in general in e.g. podcasts, which is why I've had the question floating around in my mind. But you're right, that's not quite what he wrote here.
> And in fact, even if there's a net negative societal value that doesn't mean it shouldn't be around.
To be clear: I don't mean to imply that FiveThirtyEight shouldn't exist. There's lots of things which serve a human desire but aren't necessarily a social good. Slot machines for example.
A lot of public discourse is creeping on several layers of meta, to where we are constantly reacting to how people react to other reactions.
Sometimes I wish we would just let ideas stand on their face instead of constantly engaging in acts of national navel gazing.
In an election, the vote is the measure of popular support for a candidate/policy/platform. Democracy doesn't necessarily need polls.
I've found interesting some related essays by Jill Lepore, an American historian, on the advent of polling earlier in the 20th century:
- https://www.newyorker.com/magazine/2015/11/16/politics-and-t... - https://www.newyorker.com/magazine/2020/03/09/the-problems-i...
It definitely is, I tried to make the same point a week ago :-)
Data illiteracy, not the existence of a forecast, is the core problem.
> I sometimes think they may be harmful, decreasing voter turnout by convincing people (correctly or incorrectly) that the result is preordained.
This is an old tactic that is used to the hilt by both sides whenever it serves them, and media outlets already make forecasts, often without 538's scrupulous emphasis on how often they're wrong. I don't think 538 makes this worse.
1. People want to know who is winning. It doesn’t matter whether you like it or not. People want to know, and thus there will always be a market for it.
2. Polling is imperfect and it’s a lot of work to understand how to interpret the various polls out or to forecast in lieu of polls. Political journalism before FiveThirtyEight was filled with even more crooks and charlatans coming up with nonsensical ways to forecast elections or interpret polls. There was a lot more noise in the system. FiveThirtyEight shut that stuff down to an incredible degree.
3. FiveThirtyEight has introduced more folks to probability theory than I think any other journalistic collective. What has always made them different is that they provide probabilities, not certainties.
As in, there is no way that polling or analysis of polls will ever be stopped, so why even debate its societal value?
It’s like saying “the world would be a better place if everyone started/stopped doing this one thing”. Okay?
Does it actually work though? I don’t think they were able to predict the past 3 presidential elections, so not sure where they provide value.
It's generally not understood to have positive connotations. In part because there is indeed some evidence to suggest that your concerns about it harming the democratic process are valid. For example: https://www.journals.uchicago.edu/doi/10.1086/708682
On top of that, while 538 initially provided good and solid insights, it has recently been successfully gamed by right-wing players by standing up bogus polling organizations and getting their bogus results into the aggregator's (like 538) results. The goal of course was to make their candidates seem more popular, and the "reliability" scores did not work, as the aggregators results were significantly off in that direction.
Safe to ignore NS and any of his output (and more reliable). [0] https://en.wikipedia.org/wiki/Election_silence
[Citation needed]
Lots of people raised that concern at the time, predicting the effort would bias aggregators, but for 538 at least, I’ve seen no analysis indicating a significant swing to a right bias in 2020, and overall the 2020 results were quite accurate.
I was watching it in real time. The effect was mostly in 2022 (they hadn't really gotten it going in 2020)
Here's the 538 headline before election day about how the Republicans were in very good shape, even potentially a blowout win [0]
In fact, it was the exact opposite.
Yes, there was a lot of statistical handwaving about how they were really right, but, not really. BTW, this was just the top result from a DDG search, and is only one of the things I remember.
[0] https://fivethirtyeight.com/features/2022-polling-error/
Even if you're building your own model you'll likely be turning to FTE for the latest data.
In addition they're data driven journalism is often paired with amazing data sets that they open-source for free.
> But is it good for people to know who is more likely to win an election? Are these predictions beneficial to society or democracy?
Two points:
1. The model DO NOT tell you who will win the upcoming election.
2. It is absolutely beneficial to society and democracy, having a poll-free & rigorous-analysis-free democratic society means an environment where even the most apolitical citizen can be bamboozled into thinking that an election was "stolen" simply based on their close associates and news sources. Polling and aggregation, when executed with a modicum of accountability, prevents a devolution into a Soviet-style culture of paranoia particularly among the elites. It has tremendous value, that value will be backfilled by rumour and innuendo and that is very bad for society.
> who doesn't want to know who will an upcoming election?
In particular, politicians want to know. This to me is the interesting upshot. Polls are non binding, so the public could use them to signal to candidates how to tweak their platform.
I’m always curious if there’s any real good data saying turnout patterns change. The thing is, voting because you think your one vote might change the result, is already completely irrational. I always think of voting of being like when you go to a concert and buy the band’s CD even though you already have Spotify: it’s more about find a way to personally participate in an important life moment than practicality.
Especially in a world of liars.
For plurality voting (a.k.a. first-past-the-post[0]) with more than two candidates, voters have to cast their ballots strategically[1]. This is especially common in primary elections and non-partisan positions.
Unless all elections switch to a method that is not subject to the spoiler effect (like approval voting or instant runoff voting), voters need more information about which candidates are most likely to win, not less information. Sometimes candidate competitivness can be inferred from things like endorsements or fundraising numbers, but polls -- and aggregated models based on polls -- are an important tool.
[0] https://en.wikipedia.org/wiki/First-past-the-post_voting
Don't sell your company unless you are prepared to walk away and do something else - because at some point you're going to be forced to do so whether you want to or not.
Given that we're on HN, I probably should have thought that thru a bit more.
Either way, very glad that Nate was smart enough to license everything. Kudos to him for being a shrewd negotiator.
> "In July 2013, ESPN (owned by Disney) acquired FiveThirtyEight, hiring Silver as editor-in-chief and a contributor for ESPN.com; the new publication launched on March 17, 2014."
> "BB-8 was first seen in the 88-second The Force Awakens teaser trailer released by Lucasfilm (owned by Disney) on November 28, 2014"
Han Solo: Excellent analysis. Chewie, Leia, prepare for surrender.
Sounds like he had some good leverage going into this deal. Why would Disney have agreed to a temporary license, as opposed to a purchase of this critical IP? Regardless, good for him.
It was probably a risk hedge from them.. he would have wanted a lot more for the full enchilada and they didn't know how it would perform in the future. And now his work is not hard to reproduce so it wasn't a bad call. Every political data team I have heard reporting on essentially concedes that whatever they do ends up more-or-less matching 538.. so it is hard to extract surplus value but also easy to hit that mark roughly.
I follow the NHL and there are a lot of models, they disagree in certain details but they largely agree and the "winner" at the end of any given season isn't winning by much.. even private data isn't a big boost.
538 has a very clear value proposition outside of exposing nerdy polling and forecasting breakdowns for a small group of wonks. It's the place a lot of other folks source for rigorous tracking+forecasting of elections, sports, etc. It's good.
Disney, however, seems to be putting out a lot of crappy content on their Disney+ platform that rates poorly and is expensive to produce. The Mandalorian + Andor are great but the other recent Star Wars IP is horrible and cost them a fortune.
This is one screwup that can't be placed at the feet of their previous incompetent CEO.
Here you go: Biden over Trump, 66% to 33%
I just saved Disney millions.
I think Nate Silver and his team did an amazing job making statistical analysis interesting and transparent and trustworthy.
On the other hand, 538 was a weirdly sterile website to actually read. On the rest of the internet you have raging and bombastic news designed to elicit feelings and (most importantly) clicks. 538 tried their best to fit in this world, but you were still just reading data analysis about other news or topics.
This seems odd that Disney would buy FiveThirtyEight but not its IP. How common is this?
If I were buying a data journalism company known for its models, that seems like something important to purchase. This seems like a curious contract negotiation and seems like a mistake on Disney’s part. Or a future lawsuit if there’s a difference of opinions.
Interestingly, the deal with the NYT was that the entire blog was licensed. For the deal with Disney, it seems like Silver gave up a lot more, and only kept control of the models.
When I typed “define journalism” into Google, this came back:
> the activity or profession of writing for newspapers, magazines, or news websites or preparing news to be broadcast.
So yes, 538 was journalism by at least one definition of journalism.
Here’s one more definition:
> Journalism is the activity of gathering, assessing, creating, and presenting news and information. It is also the product of these activities
Another yes! We’re 2 for 2!
Yes, it was, and still "is" until they take the website down: https://fivethirtyeight.com/
538 is a bunch of opinionated statistics and bad editorial. Squarely in the center of journalism.
Artifacts of another age. This stuff won't work anymore.
A basic free gmail account is very stable compared to almost any other option. L
The risk of having the gsuite paid services shut off are about the same as any other high dollar value paying customer using gsuite or office365 as an email service.
At least, I hope not.
In about five minutes I came up with an algorithm that defeated most of the obfuscation tricks of the time:
Strip whitespace, convert "five" to 5, remove special characters, and look for ten consecutive digits in the body text. Maybe a couple of small tweaks after that like removing text between digits.
Most people, when they invent their little unique scheme, invent one that is already defeated by the algorithm above. At least for phone numbers.
Anyway, all that to say you're right, it never worked.
There are at least \d of us!
Looks weird :)
> Artifacts of another age. This stuff won't work anymor
Its not a protection scheme, its repeating the content of the mailto link with emphasis.
(If it was a protection scheme, there wouldn’t be a mailto link.)
Analysis after analysis about how Hillary was a sure thing. It was an absolute echo chamber and the election completely lifted the curtain for me
Really? I think I remember him giving Trump 30% on the night?
EDIT: Got it - some people struggle with incorporating uncertainty into their reasoning.
A pollster claiming Trump:Hillary at 99:1, 50:50, or 33:66 could all claim to have been "right".
I agree that we don't have a way of assessing whether the percentage given was "correct", but over a bunch of predictions we can keep track and see how he does when he gives various percentages, or compare against others with a proper scoring rule.
I don't make any claim that the odds he gave were right, but just that if they were we should still expect to see his "prediction" be wrong pretty often, so the fact that his prediction was wrong this time only counts so much against him (and to make up for that, only counts so much in is favor if he's right with low confidence).
To them Silver could have said Trump had a 1% chance of winning, and they would still be saying “yOu dOnT uNdErStaNd stAtIstIcS!!1”
Basically, a lot of people really really really wanted to believe Hillary winning was a sure thing. They ignored any and all data saying “maybe she won’t”
And now you can see the effect looking back - these people are still so salty about that bitter loss that they come in here and say that giving Trump a <30% chance of winning was absolutely the right call and if you don’t see that then you’re an idiot who can’t grasp statistics
Literally - half the comments responding to me was this vitriolic idiocy. They can’t think, they’re emotions have overridden that
That’s why they downvoted you for accurately pointing out that there’s no such thing as knowing that a statistical guess was “right”
They’re downvoting you because you upset them by making them accept reality. That they were wrong and the Hillary couldn’t win.
https://www.vox.com/2016/11/3/13147678/nate-silver-fivethirt...
I think because what's the point of following the projections if at the end of the day they're indistinguishable from a coin toss? This applies to all polling. It's vaguely interesting but when (in Trump's case) a candidate can be consistently far behind for months and then still win because ¯\_(ツ)_/¯ that's how probability works, then what's the point of having followed the polls?
Usually, when things aren't quite so close, polls are a more useful gauge since they're more likely to be correct about who'll win as the gap widens.
[1] https://projects.fivethirtyeight.com/2016-election-forecast
I'm not arguing that something can ever be treated as a certainty, just that there's "no point" in following the polls since even a lopsided 71-28% prediction will go either way, and is indistinguishable from a 50-50 prediction on election night.
In this case the prediction is pretty close to just a 50/50 coin flip, but that's quite unusual and reflective of the chance legitimately being close to 50/50. Therefore I wouldn't say it invalidates polls in general.
Silver, as well as all other mainstream polls/predictions in the race had Hillary winning by “the greatest margin in presidential history”
You can try and whitewash it but we all remember that. We all remember how hopeless trump looked and how bulletproof hillary’s victory was
To say Silver and all the other polls weren’t framing this as a landslide victory for hillary and a humiliating defeat for trump is just living in a false alternate reality in one’s mind.
And judging by these responses that seems to be the more common state of mind for folks here to be in…
We don't need to remember though; we have the Internet Archive.
We can look back at Silver's prediction page on the morning of the election[1], where it's predicting Hillary winning by only 3.5% of the vote.
[1] https://web.archive.org/web/20161108000845/http://projects.f...
Go back and get an aggregate sense of everything he said. Look at 90%+ if the articles he wrote saying what I said compared to the one or two that y’all want to keep referring to
For people discussing stats there’s a very strong inability to grok averages here…
Its easy to check what he was saying too, we can just look at the archive from the same day of Nate Silver's posts : https://web.archive.org/web/20161108082405/https://fivethirt...
They're all about how it's a very close, uncertain race. You might point out that the top one is "Clinton Gains...", but even that one is just talking about "Clinton’s projected margin of victory in the popular vote has increased to 3.5 percent from 2.9 percent."
I did try looking up your quote - "greatest margin in presidential history" - but the only Google result was your comment.
Communication is a skill. Like the skill of a great sportscaster like Scully. Election reporting is, I’m sure, a similar unique skill set. Silver doesn’t have it, and I never want to listen to him again.
Nate Silver the pundit and TV personality and Nate Silver the constructor of statistical models are, oddly, like completely different beasts.
Maybe you need to compare it to a weather prediction. If you hear "30% chance of snow" where you're about to vacation, would you pack a winter jacket? What if it's 85%? If it's 0.5%? And if it's at 25-30% for months on end, that doesn't mean that "No chance it'll snow!".
It's not like a sports league where one person is behind for months and then suddenly wins it. Indeed what is the point of following the polls? For one thing people were probably addicted to the rollercoaster of "Oh shit my side's losing! " and then "Oh yeah my side's gaining!". At least for the campaigns themselves they should've been able to adjust their work in hopes of moving the needle (e.g. leaking stuff about the other side).
Who does it help? People who face regulatory uncertainty between the regimes. Mostly businesses, nonprofits, etc.
You might say even then, how do we know the result didn't just fall into that 0.5% remaining chance? But polls can and do still give some useful idea of what the result might be.
Here he spikes the football over it: https://fivethirtyeight.com/features/why-fivethirtyeight-gav...
I remember this. Tough spot for NS. I remember thinking that 2:1 Hillary and three language around it, came across a useless hedge. In retrospect, I think he'd have gotten more respect if he shouted from the rafters that Hillary doesn't have a lock and Trump could really win. Then, he'd have been heralded as a genius. Strong forecasting, weak marketing.
Their mistake was that they treated each state as an independent event. Treating each state as a separate biased coin toss leads you to put Hillary at 99%.
Nate knew that these are not independent events and if republicans are being under polled in one state, it is more likely they’re being under polled in all of the states.
That's approximately what Silver's team was saying. No idea where this "sure thing" idea of your's comes from.
Many people mistook these numbers to mean Clinton was going to get 66% of the vote and thus were expecting a landslide, but that is not what his numbers meant.
Now I wasn't reading any of the analysis he or his team was putting out, but the modeling seemed reasonable to me.
Also, for 2020, it turns out the actual vote split between the two US presidential candidates was pretty much exactly in line with polls.
The accuracy of the polls is another matter entirely. You could estimate the chances that your polling was wrong, given that the selection of the sample is a random variable, people lie, and intentions change, but that's not at all what the general public thinks the number means.
You can calibrate it by looking at their other election predictions and seeing how generally on target it is.
"given 100 predictions by Nate Silver, how many will be correct?"
that's what you just said.
In other words, if we forgot what actually happened, picked one past prediction out of a hat, asked "so was he right?" and then checked the results, he would be correct 95% of the time. That's what you're saying.
Well, here:
It's telling that he uses MLB games, rather than Presidential elections, which are not covered at all in that article. Why?
There are many thousands of MLB games. There are a few thousand US House elections. There are only a handful of Presidential elections. It is not possible to measure their accuracy on those with any statistical rigor.
Not very well.
> It’s telling that he uses MLB games, rather than Presidential elections, which are not covered at all in that article.
All the types of predictions 538 does are covered in the Brier skills chart on the overall summary; there’s detailed analysis covering each type of prediction (with further available breakdowns for subtypes available once you’ve gone into the type) available from the dropdown at the top of the page.
All that graph shows is that their "probability" is roughly in the ballpark with huge errors in the middle of the range, which is probably the swing states we care about most.
I'll suggest one concrete case where their probability of a candidate winning a 2-person race has some exact meaning:
*You'll pick a number randomly from 1 to 100. If it's less than or equal to candidate A's "probability" of winning, then you bet $10,000 on A. Otherwise you bet $10,000 on B* (we need to put real skin in the game)
There is, in fact, detailed analysis for Presidential elections.
> although I guess you mean that each state is a data point.
No, each individual prediction of each of the 56 (enumerated upthread) distinct elections for one or more Presidential electors is a data point.
> Are you willing to abide by that rule? If not, you don’t really believe the probability.
Well, no, not being willing to abide by that rule means either not being rich enough that winning and losing $10,000 are roughly symmetric in utility or not being an SBF-style nutball and demanding a non-zero risk premium. As it turns out, I am in both categories.
how many predictions are there for each election? does 538 issue more than one per election? Or how many data points are there?
as for the last paragraph: I guess you don't really believe in their probabilities, then, which was my whole point. I have no idea what you're talking about, re "non-zero risk premium."
One for each batch (which may be as small as one) of data (polls specifically in the polls-only model) added to the data; each takes into account current and past polls, with time.-based decay in weighting, distance from the election, and other factors. This includes polls for other contests in the same cycle, because the models accounts for correlation between them.
> does 538 issue more than one per election?
Sometimes multiple in a day for an election (especially the Presidential general election.)
> Or how many data points are there?
Well the scale for the bubbles on the Presidential election night calibration plot runs from 30-300 per bucket, and there’s 21 buckets.
> I have no idea what you're talking about, re "non-zero risk premium."
If you don't understand risk premiums and the asymmetric utility of nontrivial quantities of money, you have really no business talking about when a bet is reasonable.
Yeah, but that's what we care about - the past has already happened so we only care about future events.
Then if he says "Hillary is likely to win" we can have 95% confidence he's right.
If he says "Hillary has an 80% chance of winning" we ignore the 80, and just observe that it's more than 50.
Or if you do want to review the past, you can look at the error for a category of elections or that entire year rather than his whole prediction career.
I'm not seeing a formula there.
> One could note the number of times that a 25% probability was quoted, over a long period, and compare this with the actual proportion of times that rain fell.
it still depends on many samples, or "over a long period" in your doc.
You can't escape the fact that there are only one or two samples, no matter how much math you throw around.
And there are several example rules on the page.
> You can't escape the fact that there are only one or two samples, no matter how much math you throw around.
That depends on what question you're asking. "How well calibrated are the electoral predictions that FiveThirtyEight makes?" is a sensible question with a lot of data points, seems to speak directly to the crowing about the one call being bad, and seems well suited to the application of a scoring rule for comparison between people making predictions about the same things.
With a straight face you’re going to say that in 2016 you saw a bunch of polls coming out showing trump winning?
Absolutely not. Nobody, and I mean nobody, gave him a snowflakes chance. Everyone including Silver mentioned “the greatest election landslide in history”, or some variant.
If there were polls showing trump winning they were hidden from ans downplayed in the msm. And to say anything else is a very poor lie.
https://projects.fivethirtyeight.com/2016-election-forecast/...
The last poll there from Nov 8 showed Clinton winning the popular vote by...wait for it...3.9 percent. She won the popular vote by...2.9 percent. National polls weren't far off at all.
The electoral college was decided by less than 50,000 votes overall. No one predicted the margins in those key states would be so small. It was a razor-thin win.
Statistics always have a margin of error and give probabilities. Polls didn't have to predict Trump winning. Instead, they give a probability of winning. FiveThirtyEight predicted a 28% chance of winning[2]. That's based on Monte Carlo simulations (that's where they run, e.g. 100 random elections and count how many times each outcome occurred). It actually doesn't matter if that estimate was low; it was high enough (1 in 4 chance) that the outcome wasn't insane. Even three days after the election, they already had an analysis of what they went wrong with polling[1]. I mean, you could read that or continue to trot out tired polarized arguments.
[1] https://fivethirtyeight.com/features/why-fivethirtyeight-gav... [2] https://projects.fivethirtyeight.com/2016-election-forecast/
You’re not getting it. The analysis, the “28%” - those were wring. those were poor analyses
Yea they came back after and made up some bs about “oh we know what we did wrong now!”
When I was in school all the people who were cheating their way through math did the same thing. They couldn’t solve the problem, and the second you gave them the answer they magically “got it”
Fortune telling is a business of luck. Unless you can convince your followers that you’re right even when you’re clearly wrong
It's more of a business of being connected to some forms of future reality. I can understand if people think nobody can do better than pure chance, but even so, I'm pretty sure the folks who believe statistics could give them an edge are quite misguided.
[1] https://projects.fivethirtyeight.com/2016-election-forecast
538 consistently gave Trump a better chance than much of the mainstream media, with him showing a roughly 30% chance for much of the year, including right up at the election. At times it dropped as low as 15% or so, but... that's still not some insanely rare occurrence.
As for what they were actually saying? Silver and 538 never claimed it was a sure thing. Let's look at some actual articles:
https://fivethirtyeight.com/features/election-update-dont-ig...
"At the same time, it shouldn’t be hard to see how Clinton could lose. She’s up by about 3 percentage points nationally, and 3-point polling errors happen fairly often, including in the last two federal elections. Obama beat his polls by about 3 points in 2012, whereas Republicans beat their polls by 3 to 4 points in the 2014 midterms. If such an error were to favor Clinton, she could win in a borderline landslide. If the error favored Trump, however, she’d be in a dicey position, because the error is highly correlated across states.
There’s also reason to think a polling error is more likely than usual this year, because of the high number of undecided voters. In national polls, Clinton averages about 45 percent of the vote and Trump 42 percent; by comparison, Obama led Mitt Romney roughly 49-48 in national polls at the end of the 2012 campaign. That contributes significantly to uncertainty, since neither candidate has enough votes yet to have the election in the bag.
To be honest, I’m kind of confused as to why people think it’s heretical for our model to give Trump a 1-in-3 chance — which does make him a fairly significant underdog, after all. There are a lot of ways to build models, and there are lots of factors that a model based on public polling, like ours, doesn’t consider.3 But the public polls — specifically including the highest-quality public polls — show a tight race in which turnout and late-deciding voters will determine the difference between a clear Clinton win, a narrow Clinton win and Trump finding his way to 270 electoral votes."
https://fivethirtyeight.com/features/clinton-probably-finish... is about as bullish on Clinton as I can find from the archives, and even there Silver does not talk about how Hillary is a sure thing, and instead spend several paragraphs discussing reasons why Trump could still win and what those would look like. And as those things occurred, the model shifted back from being as bullish on Hillary, for the reasons Silver outlines.
I don't know why you would read a 70% chance of victory as being the same thing as a sure victory. I wouldn't want to bet my life on a 70% success rate!
70% chance of success is about a 1 out of 3 chance of failure.
Can you run the 2016 election 1,000 times and count the times Hillary wins? Of course you can't.
Unlike estimating "what's the chance it rains tomorrow?" you can't even find 1,000 elections in the past with roughly similar conditions.
Unlike a baseball season, you don't have enough samples for the true probabilities to come out.
So, yes, despite all your downvotes, you're right: he pandered to the people who want a percentage number, despite the fact that it's meaningless and they will misunderstand it.
No, they aren't.
(They can’t be validated for a single event in isolation, but 538 predicts lots of election events, so their models can, generally, be assessed, and are historically quite accurate.)
Real life is harder.
Similarly, probability is not the result of conducting repeated experiments (though the results of repeated experiments can be used to estimate probability). It's the measure of an event relative to the measure of the set of all events, eg, it's how much of the total volume in the space is taken up by a given event.
I don't even know what that means.
The second paragraph, I can't take issue with; the problem is "the set of all events" has cardinality 1.
If you have a question I'm happy to clarify. If a restatement is helpful, the problem is that this definition of probability is limited and fails to capture the complexity of the problem domain, not that the problem domain is meaningless.
You're misunderstanding the terminology, "events" refers to sets of potential outcomes (eg, when rolling a die, the outcomes are 1-6 and the event "even" is the set of 2,4, and 6).
(I don't know 538's actual methodology, consider the below an illustration rather than a literal description of 538's model.)
The outcomes here would be each way the electoral vote could possibly be allocated, based on the number of states and the idiosyncracies of each state constitution. The events would be the division of these outcomes by who wins the election in that instance. The model then seeks to attach a weight to each outcome based on how likely it is.
You can think of these weights as the volume of each outcome. We use some kind of model to assign them. The most straightforward model would be to repeat an experiment many times and then assume the rate at which the outcome appears in your sample is approximately the same as it is globally. Other times that isn't possible and you need to use more sophisticated modeling techniques. I'm not going to offer a defense of statistics and machine learning writ large, but I think looking around it should be pretty clear these things can work well even if there's a lot of art to getting it right.
You can imagine comparing the probabilities of each event by pouring their constituent outcomes into separate graduated cylinders so that you can compare their cumulative volume. If you divide their volume by the total volume of all outcomes, that's the probability of that event. Or rather, that's the estimate of the probability, given our model and the data we have.
This is more or less my drunk history version of measure theory and probability theory. I imagine I've gotten some details wrong, so I'd encourage anyone interested to check the subjects out for themselves. Here's some YouTube videos:
https://youtube.com/playlist?list=PLBh2i93oe2qvMVqAzsX1Kuv6-...
https://youtube.com/playlist?list=PLUl4u3cNGP60hI9ATjSFgLZpb...
If you think that evaluating their model based on binary outcomes is the best way to do it, then, well, this argument makes sense.
If you want to evaluate the model based on all of the details available, then, well, it stops making nearly as much sense. What explanations were given for potential Trump victories? What information was available at the time? What information was available after? Do we have data around if those explanations did align with Trump's victory?
I've not done a deep dive on this - 538 is entertainment for me. But from what I remembered in 2016, what I read today in digging up some example articles for another comment, and looking at the actual outcomes... yes, those explanations were closely aligned with reality.
Humans have to come up with probabilities and make decisions based off of them all the time, even when we will never get a chance to get 1000 occurrences. These might not be provable in the same way as something we can run 1000 times is, but the idea that they are meaningless and without value is bizarre.
I assume you mean "in real life", but otherwise this seems to be close to their actual methodology. You can see the outcome frequency charts under "What to expect from the Electoral College."[1]
[1] https://projects.fivethirtyeight.com/2016-election-forecast/...
So, the "actual methodology" is not real life, but some abstraction (like, "assume no friction or air resistance")?
That qualitative difference is fundamental. It suffers from the same problem when results of simulations are applied to the "real world". The quality of the simulation is paramount, and there's no way to determine whether the simulation is good enough besides assertions that the simulations are really sophisticated and Nate is very smart.
Clearly. Its not a difficult concept.
But those same people are wholly unable to see that all the other verbiage and propaganda that was behind/infused with Silver was, in no short terms, a demoralization campaign against the trump voters
And since they can’t see that all they can do is project their ego to say they aren’t wrong, its that the people who don’t agree with them don’t comprehend very basic statistics