Disney is shrinking FiveThirtyEight and Nate Silver (and his models) are leaving | Hacker News Reader

Disney is shrinking FiveThirtyEight and Nate Silver (and his models) are leaving

niemanlab.org

71 pointsAmorymeltzer83 comments

Incredibly smart of him to hold the rights to his models personally, and only license them to 538 (and hence ABC/Disney).

Absolutely. Good planning and foresight on his part to enable him to retain ownership of his hard work.

Unlike W-2 work where EVERYTHING you do on work machine/time belongs to the organization. It's almost a hostage sitt

Even as a startup founder, the best practice is to sign over your IP rights to the company since investors want to see this. So I can say even with a large equity stake in a new venture that's not w-2 work, you still sign over IP rights to an org.

I’ve been told all information should be free lol. I’m glad there are some sensible people.

If the models were Free, they also wouldn’t be locked up at Disney when he left.

It's a political model and as such, it most likely will be gamed as the cost to manipulate people at scale using the patterns of behavior it follows is not disentangled.

Might as well divest FiveThirtyEight or shut it down if Nate Silver is leaving.

Nate Silver is 538.

Rephrase title to Nate Silver (and his models) are leaving and Disney is shrinking FiveThirtyEight. Construct is so important in English (even without the And being a Hence)

Especially given his Twitter handle:

* https://twitter.com/natesilver538

I wonder if TheRinger can afford him. My impression is that sites like 538, Grantland, Andscape, etc don’t make any money for Disney, so this isn’t too surprising.

Nate Silver is a guy who got lucky one year, and managed to convince everyone he's a genius.

Not making a judgment about his methods or not, but he was pretty accurate more than once; in 2008 he predicted 49/50 states correctly in the US presidential election, and in 2012 he predicted all 50 correctly. You might be right that he isn't as smart as people think, but "got lucky one year" is an exaggeration.

I'm a big fan of Nate Silver, but I think this idea that he "predicted N/50 states correctly" in 2008 and 2012 is mostly a myth.

(1) The website doesn't "predict" a state when it reports that the probability of (e.g.) an Obama victory in Florida is 50.3%, which I believe is what 538 assessed in 2012. If you think that a state's probability distribution is 50/50 between Candidate X and Y, and the outcome is Candidate X, obviously you don't get any points for a good prediction. You didn't make a prediction. And if you say it's 50.3% probability, it's essentially the same situation. You didn't make (much of) a prediction. Which is okay!

(2) In terms of Brier Scores or entropy scores, which compare the probability with the outcome and give more "points" the stronger the prediction was made (vs. what actually happened), FiveThirtyEight's predictions are generally just middle-of-the-pack compared with other websites and mechanisms of aggregating these polls. (See, e.g., https://web.math.princeton.edu/~sswang/Wang_Origins_of_Presi...) There's no strong evidence that 538's predictions are much better or worse than other reasonable ways to weight and aggregate the recent polls that anybody else might come up with.

(3) And... almost everything that tries to quantify the accuracy of the predictions is talking about the state of the 538 website ON THE MORNING OF THE ELECTION. But that's probably the least useful day to be looking at FiveThirtyEight. By then, you can just wait a few hours and get the actual results. :-) The predictions that are really valuable are the ones that are made some time in advance, e.g. one or two months.

But (a) it's hard to directly measure the "accuracy" of a prediction like that, because... things change and we can't rerun the universe, and (b) it's not like the model is somehow time-invariant to be able to measure the accuracy indirectly by proxy; there are various "time until election" factors included in these model itself, the dynamics of the election itself probably have a "time until election" dependency, AND the website operators, including 538, are generally manually tweaking the model parameters or behavior in ways they don't always disclose as the election evolves.

So basically... it's hard to say. I think 538 has done great work, and in general it's incredibly useful to have a consistent world model that you can rationally update to discuss the effect of recent news. The FiveThirtyEight political models are as good as any for that! But I don't think there's much strong evidence they're better.

That's fair! I probably should have realized that, given that I already knew that his 70-30 prediction of Clinton winning in 2016 didn't mean he "predicting wrong"; I'm guessing I was just less aware of how his models worked back in 2012 than I was by 2016, so I didn't really consider what it meant to "predict the states".

I think many people subconsciously conflate a "70% chance of winning" with a "70% of the vote" prediction, when the reality is much closer--a 70% chance of winning could very easily go the other way, given some systemic polling error and/or late-breaking news!

Yeah, it's definitely tricky to communicate. The chances of _either_ candidate getting 70% of the vote in any recent presidential election would be incredibly small; even 55% would be a huge anomaly. Most people wouldn't feel very enlightened looking at a probability distribution of potential vote percentages for each candidate on a CNN graphic though, especially when they'd basically just look like giant spikes around 48-52%.

> But it’s hard to directly measure the “accuracy” of a prediction like that, because… things change and we can’t rerun the universe

Its hard to measure the accuracy of a prediction like that, but with a big collection of predictions (and since every cycle Silver predicts a lot of individual elections – not just each of the state elections for presidential electors, including both the statewide electors and the by-district electors where those exist, but also house, senate, gubernatorial, and some other races – each election cycle, this exists) you can bucket them in similar probability buckets and say, for instance, and see how often things Silver predicted in each probability bucket came true vs. the probability range of the bucket, and get a good idea of the accuracy of the predictions.

> Silver was named one of The World's 100 Most Influential People by Time in 2009 after an election forecasting system he developed successfully predicted the outcomes in forty-nine of the fifty states in the 2008 U.S. presidential election.[2] His subsequent election forecasting systems accurately predicted the outcome of the 2012 and 2020 presidential elections with a high degree of accuracy, and - at the time - gave Presidential candidate Donald Trump the highest chance of success the day of the 2016 presidential election of reputable polls at the time.

- https://en.wikipedia.org/wiki/Nate_Silver

> gave Presidential candidate Donald Trump the highest chance of success the day of the 2016 presidential election of reputable polls at the time

This makes a fundamental error of confusing “polls” (which is not a category Silver’s work falls into) with statistical predictions based on aggregates of polls (which is what it is.)

I think it's poorly worded - there were a bunch of other aggregators at the time who said things like Clinton was 99% likely to win. IIRC 538 assumed that polling bias could be correlated between states while most analyses were assuming that each poll was completely independent, but that's probably an oversimplification given the vagaries of memory.

Your recollection matches mine, fwiw.

At FiveThirtyEight, their goal is to deliver high-quality quantitative analysis by utilizing record-level data. It's ironic that the type of comment you made is typically found on sports news websites that lacks the rigor and depth of a quality analysis.

> Nate Silver is a guy who got lucky one year, and managed to convince everyone he’s a genius.

Nate Silver is a guy who is very good at applying statistical modelling to a number of domains (including election analysis, where he basically revolutionized the field with everyone else doing poll-aggregate-based-prediction – which no one was previously – following in his footsteps, but mostly doing a worse job), but who morphed into another generic center-right Democratic political talking head. Whether that’s a structural demand of Disney that will go away now that he is leaving Disney or whether its what he feels his groove is now…will be interesting to see.

Genuine question, since I dont really follow him outside of elections, but in what way is he center right? I'm not american and I'm an immigrant/poc so I'm not especially biaised to the right but he always came off to me as pretty centrist and "liberal". When did that change?

(The 2020 election was the last time I really heard about him, and the most controversial takes he had was some very mild reservations on lockdowns and school closures... at a time when the entire world reopened schools except for the USA, which is why it didn't register like a hot/right wing take at all to me.)

He didn't get lucky.

He found a trend and published it widely.

He changed the very thing he was studying. Because politics was already a statistical game, his original models failed to reproduce.

He clearly published a single result.

His approach is pretty sound, and straightforward if you listen to his podcast. Proper treatment of fat tails via student t distributions and correlated variables with good (subjective) judgement on polling quality.

I find that most people with negative views of Nate's approach are innumerate with regards to statistics. They don't criticize the model because they can't be bothered to understand it, or even basic probability, and certainly don't rank his performance vs other modelers which wouldn't require understanding the model.

No: https://statmodeling.stat.columbia.edu/2020/10/24/reverse-en...

What are the consequences of this, in terms of prediction accuracy?

so basically what TFA says is if you build a new model on Nate's model outputs and use that to compute conditional probabilites of various improbable scenarios you see weird shit. State level polling data according to Nate isn't all that great (read: high uncertainty, moves around when a new poll comes out) unless it's a swing state. But hey when you go looking into noise you are going to see.... weird shit.

Did you Google any other good blog posts?

I’ve been saying some version of this to people for years.

But in his defense, I think he’s been more the victim of the secular decline in polling quality than anything else.

Very curious where he goes, if it'll be back somewhere mainstream like NYT, or if it'll be independent on Substack, or somewhere else more alternative/indie like Vox.

Also a shame he can't take the FiveThirtyEight trademark with him. He can take his models, but not the brand. Still, his own name "Nate Silver" seems to have just as much recognition as "FiveThirtyEight" so he'll do just fine.

When will the size of the electoral college change again?

When Puerto Rico or DC become states. Each would add two Senators and draw from the pool of Representatives (replacing DC’s third elector).

> When will the size of the electoral college change again?

When:

(1) One or more new states are added, or (2) The law fixing the size of the House is amended/replaced.

I wouldn’t be surprised if those happen together, so that adding the new state adds on rather than redistributes seats in the House initially.

Note that D.C. being added (with no change to the fixed size of the House) as a state would actually shrink the electoral college by one, since it would add two Senators and zero reps, but D.C. already has three electors; any other state being added (with no change to the fixed size of the House) would add two electors for the two additional Senators added.

Or in theory 3) when the Constitution is amended in a way that changes the makeup of the electoral college, but in practice that's much less likely to happen than the other two (which are themselves unlikely to happen soon).

Ha!

Now you've got me genuinely curious as to whether launching e.g. FiveForty would be considered trademark infringement by the courts.

My hunch is it would still be infringement, since it's similar enough. In the same way Pepsi wouldn't be allowed to launch a cola called Coky.

This is horrible. Now how will I delude myself with forecasts about the Premier League? I'm going to need to wait for him to get a substack.

Somewhat related: The problem with Nate Silver and most others in this game is they don't treat the statistics they show as random variables. I want to see confidence intervals for all statistics. They just show 'Chance of winning is X" instead what they need to so is 'Change of winning is between X and Y'. It's such a simple change I'm shocked nobody does it including Nate Silver.

It's all what I call 'pop statistics' at this point and IMO hurts the public and democracy.

You do not have to dig far _at all_ on 538 to find that info. They prominently display the distribution of their results.

Huh, I had no idea Nate Silver got his start on daily kos.

Ad based media/journalism is hopeless. Stuff behind a paywall maybe can survive if you cater to a specific audience (politics junkies, businesses). The vast majority of journalism though will be explicitly public relations and a defacto wing of an economic entity.

Commentary and analysis should be fine as professions for the highest levels. You can make a pretty good living out of subscriptions and even multiple millions out of a contract with a streaming service if you are an A-level analyst.

But reporting requires some level of infra-structure that is harder to bootstrap.

Well there goes any somewhat reliable election forecasting for the US

The very idea of election forecasting is harmful to democracy and should be shunned wherever it appears.

If it's not being done in public, it's absolutely being done in private. Do you believe democratizing that data is a bad thing?

Yes, it’s quite possible that “democratizing” election forecasts is bad. A public forecast may influence the likelihood of voting while a private forecast would not.

I don’t think this is the case at all. The whole purpose of private forecasting is to judge how your efforts are doing with the electorate, and adjust your strategy, poll/assess again, rinse and repeat.

They don’t just forecast and shrug at the results.

Not having public forecasting influences voting.

Either way votes are going to be influenced.

The benefit of democratizing those forecasts is to reduce surprises / rigged election accusations.

this is an interesting take. why do you say that?

It’s pretty easy: if someone is projected to win, voters who were planning to vote for that person might stay home to avoid “wasting their time” since they’re going to win anyway. Then that person doesn’t win because the voters didn’t show up.

Or conversely, people don’t go out and vote for their candidate because their candidate is projected to lose.

People are going to make their own electoral decisions on their own terms, whether you, or I or anyone else agrees with them or how they came about their decisions. That’s about as democratic as it gets. Forecasts can inform those decisions, but they don’t decide them.

EDIT: I have no love for iOS’s tendency to capitalize “democratic” or “republican” against my wishes.

Which is remarkably stupid considering election day is voting for dozens or more positions.

It's really not there's a lot of places where publishing polls is illegal. Public opinion is easily influenced by perception of public opinion. That's basically all of style, design and fashion, and polls can always be crafted to get the results you want or outright lie.

Also what actual benefit does it serve to poll before elections if not to influence the outcome.

I think maybe internal polling could be ok. But is it? Should politicians have their positions and push them, why should they change or hide their views in response to polling. We should be able to vote on unfiltered versions of politicians not people who are malleable to whatever will let them win.

I thought the whole idea of a democracy is to carry out the will of the people through the politicians they vote into power. In that case, politicians following polls is the most ideal way. It's not like the people a large number of politicians they can vote into specific offices; almost always it's a choice between a small number of politicians.

While I readily admit to feeling like my comment could be seen as sealioning, but could you post some links and/or search terms I could use to try to research why one might hold an opinion like this?

I don't know whether to agree or disagree with this; I'm new enough to not have come across this particular opinion before (that I've noticed).

It's simple: Election forecasting is often used as a form of voter manipulation, if not outright suppression.

People should vote for the candidate that they want to win, not the candidate that the tv says has a better chance of winning.

Isn’t the 538 approach of rigorous modeling a lot less likely to be manipulation than than the often vibes-based approach it replaced? Their predictions are based on actual polling, so they won’t take a candidate with real support and propaganda the support away with their forecasts.

What purpose does predicting the outcome of an election serve in a democracy?

People want predictions for various reasons, but in general, predictions are good for making plans and preparing for the future. Elections are just one of many examples of an outcome you might want to predict, alongside crop yields, population growth, product sales, etc.

I don’t have any deep thoughts about how these predictions serve democracy, but if people are interested in that line of inquiry, they should go ahead and pursue it.

This only works where there is a candidate you want to win, who possibly can win. For example, where I vote in the UK, there are only two choices: Conservative or Liberal Democrat. Which you vote for has more to do with which of those you dislike least and not which is preferable (which would, frankly, be a golden retriever in a rosette, who would be more effective than either party).

> who possibly can win

huh, who told you that, the people forecasting the election?

No, a basic understand of history.

The last time someone that was not a member of the Conservative party, Liberal Democrat party, or one of the direct predecessors of these parties was 1857. If you count the Whigs as a predecessor of the Liberal party, that goes back to 1830, which is the earliest election listed on Wikipedia.

It is safe to say that Bath will not be represented by someone who is not a member of one of those two parties any time soon, regardless of what forecasts may or may not say.

Making stuff up.

Free speech, but not if it's consequential.

Would this opinion still hold in places where voting is mandatory (or more precisely, not voting is penalized)? https://en.wikipedia.org/wiki/Compulsory_voting

An attempt at a steelman argument is that it has the potential to influence the outcome by shifting voter attitudes. If we consider the 2016 election, since most predictions were leaning so heavily towards Hillary Clinton that might've caused her supporters to be less likely to go out and vote, thinking that she's already likely to win so there's no need for them to participate. If the messaging had been that the election is going to be extremely close and every single vote matters then you might've gotten improved voter participation. At least this is one variation of argument which I've encountered, but I'm not sure if there's strong empirical research to back up the claim.

After the lesson of 2016, messaging about election participation for 2020 was much more active which lead to the highest voter turnout of the century.

Edit: Upon further reflection, I'm actually not fully convinced of this line of thinking. Or at least I think it's not a fully accurate representation. We can also consider the other perspective in 2016, if Donald Trump supporters had thought the election was hopeless because of polling, they probably wouldn't have bothered to go vote. Although my understanding is that Republicans are already inherently less trusting of these kinds of systems, which might provide a plausible explanation for why they weren't as negatively affected.

Generally it is something like "Candidate X is winning already, they don't need my vote to help so I'll save myself some and skip the voting process" or "Candidate Y is winning, there's no way my vote will change that, I won't even bother."

Same argument for early reporting of early voting (early voting itself I got no problem with, but keep the ongoing tally quiet imo).

The best case for maximum participation is when voters truly believe their vote may matter. We see this recently with nearly 49/51 splits in many elections over the last few years/decades. Some see this as polarization (and I often agree), but it can also be seen as a result of increased participation. Whether increased participation in democracy is ideal or not, well... "a republic [...] if you can keep it." [1]

1: https://www.loc.gov/exhibits/creating-the-united-states/conv...

Ironically, FiveThirtyEight actually reviewed the literature on the effects of polling blackouts and concluded that the evidence was complicated and mixed. (See https://fivethirtyeight.com/features/does-knowing-whom-other....)

I don’t think reliable forecasting is gone. Nate Silver does not bring any new data to the table. However, he does have good models to translate a whole bunch of raw, often contradictory, data into something more representative of reality.

That being said, it’s not going anywhere. He will either self publish or join another firm. He might rejoin NYTimes for example.

That's heartening. Mainstream media election coverage is infuriating. Nate is the antidote for me.

https://election.princeton.edu/

Princeton had Clinton at a >99% chance in their 2016 election forecast.

Yeah, they used the obviously bad model that each state's results are independent rolls of the dice.

Which, flawed models can still be useful, but I seem to recall them being pretty arrogant about their prediction in 2016 too, before the fact.

I'm surprised to learn that Disney owned FiveThirtyEight and also that they kept Nate Silver afloat for that long. Dude's credibility was shattered after 2016 - a master of proving that getting high on your own fumes makes you high.

Seriously? What did he get wrong in 2016? Trump squeaked it out. 528 provides probabilities, and he expressed a very strong possibility of Trump winning with a 28% chance--much higher than almost everyone else including the Clinton campaign.

Also, the numbers with which Trump won (he was the only Presidential candidate, or at least one of a couple, to win despite getting fewer overall votes), it’s pretty clear his odds were indeed low.

There have been five times where the Electoral College winner lost the popular vote: 1824, 1876, 1888, 2000, 2016.

* https://en.wikipedia.org/wiki/List_of_United_States_presiden...

> (he was the only Presidential candidate, or at least one of a couple, to win despite getting fewer overall votes)

Of the 52 elections from 1816 onwards (1816 being the first election where a majority of states determined their electors from the popular vote), 5 of them ended up with the victor not winning the plurality of the popular vote:

* 1824: Andrew Jackson has a solid plurality, but fails to gain a majority of electoral votes, causing the election to be decided by the House vote. Ultimately, Clay (#4 in electoral and and #3 in popular vote) throws his support behind Adams (#2 in electoral and popular vote), and the Clay + Adams numbers would reach the majority threshold needed. (Okay, the House tiebreaks don't work like that, but this is long enough already).

* 1876: Hayes gains the victory despite Tilden having an outright majority of the popular vote (the only one on this list to do so), although the counting of the votes produced severely acrimonious results resulting in a commission to determine which returns were valid (which ultimately ruled for Hayes).

* 1888: Harrison wins the electoral vote despite Cleveland having the plurality

* 2000: Bush wins the electoral vote despite Gore having the plurality

* 2016: Trump wins the electoral vote despite Clinton having the plurality.

Out of 52 elections, 5 times the victor failed to achieve a plurality vote. That's about 10% of the time, which isn't all that rare when you think about it.

Bush should have lost the electoral vote in 2000. The Supreme Court handed him the presidency.

> The Florida recount and subsequent litigation resulted in major post-election controversy, and with speculative analysis suggesting that limited county-based recounts would likely have confirmed a Bush victory, whereas a statewide recount would likely have given the state to Gore.[6][7]

https://en.wikipedia.org/wiki/2000_United_States_presidentia...

May want to rewatch that election night coverage with him in it. He seemed surprised to me.

https://www.youtube.com/watch?v=RaWSe_uij74

So you select a subjective interpretation of his response over his statistical prediction model? Wild stuff.

Reply on news.ycombinator.com