Hacker News Reader

Comment by Wowfunhappy | Hacker News Reader

> It’s not just that I think these models have a lot of journalistic value

...do they, though?

I'm actually not so convinced in the journalistic value of FiveThirtyEight. It's something I've thought about before.

FiveThirtyEight's political forecasts absolutely provide a desirable service! Who doesn't want to know who will win an upcoming election?

But is it good for people to know who is more likely to win an election? Are these predictions beneficial to society or democracy? I sometimes think they may actually be harmful, decreasing voter turnout by convincing people (correctly or incorrectly) that the result is preordained.

That's not to say Nate Silver is doing anything wrong by making FiveThirtyEight available, but he tends to talk about it as a sort of public good. I'm not sure whether that's accurate.

View on HN

If anything, I suspect this detailed polling of making elections closer. Millions of voters aren’t reading 538. It’s a distinctly wonky thing to do. But people really into politics, the kind of people who volunteer for campaigns and make large contributions do pay attention and I think it helps rally their support behind realistic candidates for their party so that the candidates put forward for the general elections are as closely matched as possible. Well funded campaigns probably already had their own private polling and statisticians, but 538 helped make sure all highly invested individuals had access to decent data and could deploy their resources more effectively. If you keep that data out of grassroots hands, then you put the parities more firmly under the control of a very small party elite with all the experts.

> Millions of voters aren’t reading 538

True. But millions of them will hear a radio announcer, or TV person, or tiktok video or whatever else say something like "In the latest polling, <X> has pulled ahead of <Y> ..." where "the latest polls" will mean (among other things) 538.

I do believe that this has some effect (if small) on patterns of voting (particular by encouraging or discouraging turnout).

Generally speaking, you don’t hear this because 538 is not a poll, it’s a political analysis brand and competitor (ABC news). Most news orgs or journalists quote the polls themselves.

> where "the latest polls" will mean (among other things) 538.

No, it won’t; 538 doesn’t produce polls.

I've heard numerous news organizations/corporations cite 538 as the source of their analysis.

538 does produce analysis. They do not produce polls - they consume polls.

None of which matters to people who hear "analysis of the electoral race" on some media.

I don't think dragonwriter was weighing in on your broader point, but being a little pedantic (and a lot correct) about what work is done by what people in our electoral coverage.

And that will happen no matter what. Polls and prognostications will always exist and probably have always existed back to the first tribal chieftain election. It’s just that instead of getting totally inaccurate predictions that only serve a narrative, for a little while a few of us knew how to get a more unbiased prediction. Now even fewer people will know anything aside from their favorite outlet’s media bubble.

We have a word for this: horse race journalism.

It's generally not understood to have positive connotations. In part because there is indeed some evidence to suggest that your concerns about it harming the democratic process are valid. For example: https://www.journals.uchicago.edu/doi/10.1086/708682

In fact, horse race journalism has been observed to have such negative and self-fulling effects that some countries ban polling and/or electoral journalism for hours, days, or weeks before an election. Even in the US, it is illegal to do any "electioneering" within 100ft of a polling place.

On top of that, while 538 initially provided good and solid insights, it has recently been successfully gamed by right-wing players by standing up bogus polling organizations and getting their bogus results into the aggregator's (like 538) results. The goal of course was to make their candidates seem more popular, and the "reliability" scores did not work, as the aggregators results were significantly off in that direction.

Safe to ignore NS and any of his output (and more reliable). [0] https://en.wikipedia.org/wiki/Election_silence

> On top of that, while 538 initially provided good and solid insights, it has recently been successfully gamed by right-wing players by standing up bogus polling organizations and getting their bogus results into the aggregator's (like 538) results. The goal of course was to make their candidates seem more popular, and the "reliability" scores did not work, as the aggregators results were significantly off in that direction.

[Citation needed]

Lots of people raised that concern at the time, predicting the effort would bias aggregators, but for 538 at least, I’ve seen no analysis indicating a significant swing to a right bias in 2020, and overall the 2020 results were quite accurate.

>>[Citation needed]

I was watching it in real time. The effect was mostly in 2022 (they hadn't really gotten it going in 2020)

Here's the 538 headline before election day about how the Republicans were in very good shape, even potentially a blowout win [0]

In fact, it was the exact opposite.

Yes, there was a lot of statistical handwaving about how they were really right, but, not really. BTW, this was just the top result from a DDG search, and is only one of the things I remember.

[0] https://fivethirtyeight.com/features/2022-polling-error/

538 actually did a terrific job accounting for those polling phenomena, finding that in spite of that narrative around right-leaning polls arising, polls that cycle were pretty spot on https://fivethirtyeight.com/features/2022-election-polling-a...

FiveThirtyEight is beneficial, and it’s fairly simple as to why.

1. People want to know who is winning. It doesn’t matter whether you like it or not. People want to know, and thus there will always be a market for it.

2. Polling is imperfect and it’s a lot of work to understand how to interpret the various polls out or to forecast in lieu of polls. Political journalism before FiveThirtyEight was filled with even more crooks and charlatans coming up with nonsensical ways to forecast elections or interpret polls. There was a lot more noise in the system. FiveThirtyEight shut that stuff down to an incredible degree.

3. FiveThirtyEight has introduced more folks to probability theory than I think any other journalistic collective. What has always made them different is that they provide probabilities, not certainties.

Your #1 is affirming the consequent.

My #1 is really attempt to say that OP’s question is a moot point.

As in, there is no way that polling or analysis of polls will ever be stopped, so why even debate its societal value?

It’s like saying “the world would be a better place if everyone started/stopped doing this one thing”. Okay?

Those are good, meaty questions.

In an election, the vote is the measure of popular support for a candidate/policy/platform. Democracy doesn't necessarily need polls.

I've found interesting some related essays by Jill Lepore, an American historian, on the advent of polling earlier in the 20th century:

- https://www.newyorker.com/magazine/2015/11/16/politics-and-t... - https://www.newyorker.com/magazine/2020/03/09/the-problems-i...

Societal value != journalistic value. And in fact, even if there's a net negative societal value that doesn't mean it shouldn't be around.

Nate silver didn't exactly invent political polling either. That's doing a disservice to the enormous time and money invested in polling over several decades of elections (and the learnings when things didn't go correctly). Polling will be around regardless.

> Societal value != journalistic value

That's fair. I was responding to ways I've heard Nate Silver talk about the model and polling in general in e.g. podcasts, which is why I've had the question floating around in my mind. But you're right, that's not quite what he wrote here.

> And in fact, even if there's a net negative societal value that doesn't mean it shouldn't be around.

To be clear: I don't mean to imply that FiveThirtyEight shouldn't exist. There's lots of things which serve a human desire but aren't necessarily a social good. Slot machines for example.

The big value of polls is a backup check for discrepancies. A major difference between final polls and polls leading up to the election could be innocent (like unexpected demographics), but it could hint at voter suppression or even completely innocent issues with the voting systems. Paired with exit polling, there's enough information to find answers.

As opposed to horserace election coverage, which is the single biggest focus of all of American journalism for the year or so leading up to an election... which is of greater journalistic value?

Simply looking at 538 during election cycles makes it hard to argue they aren’t a horse race tracker (they literally started as a sanitized gambling odds site and Nate’s new book is about gambling).

They’re just the guy who went to the stables before to see who was eating the most hay.

538 absolutely is horserace coverage, but unlike basically all the other election coverage, it's rigorous, impartial, and intellectually honest.

Eh, eye of the beholder.

I honestly mostly follow the sports odds and I’d argue that Nate’s models are some of the most biased out there for that.

Which non-538 models consistently outperform 538's models?

EDIT: (cause I can't reply to the below comment) my question was what model beats 538, not which betting market beats 538. yeah, duh, the Vegas line beats 538, because it beats literally all publicly available models. It'd take a really disastrous gambling market failure for anything else to happen.

Literally the Vegas line?

Nate put 2021/22 warriors at 0.5% to win, Vegas was +900.

Nate put 2020/21 bucks at 10% to win, Vegas was +550.

If you can’t even come close to beating the line, you’re not a good model. He’s just transparent about his variables.

> EDIT: (cause I can't reply to the below comment) my question was what model beats 538, not which betting market beats 538. yeah, duh, the Vegas line beats 538, because it beats literally all publicly available models. It'd take a really disastrous gambling

Lines aren’t built that way. They’re set by books to offset their risk. They’re based on a bunch of (usually drunk) wetware computers with huge biases (don’t believe me, look at any Dodgers line when they have an obviously bad pitcher matchup).

If your model is worse than that, then what’s the point of a model?

To the point of bookie models, those are based on how they think people will bet, not on who they think they will win. The point of books is to be outcome neutral.

> Lines aren’t built that way. They’re set by books to offset their risk.

They're initially set by bookies based on their models, but then pretty quickly evolve to a market-based mechanism.

> Eh, eye of the beholder.

It's really not, which is why quantitative models have real value. You don't have to argue that he's biased, you just have to point to the evidence. But your whole comment is indistinguishable from someone with sour grapes who is simply casting aspersions on his track record without being willing to analyze it properly. At the very least, you ought to specifically identify what kind of biases his predictions have shown; he leaves himself wide open to being proven wrong, so it's only fair for you to do the same.

He regularly underperforms the Vegas line.

He makes bad models.

Nate put 2021/22 warriors at 0.5% to win, Vegas was +900. Nate put 2020/21 bucks at 10% to win, Vegas was +550. If you can’t even come close to beating the line, you’re not a good model. He’s just transparent about his variables.

Edit: To be clear I don’t gamble, I just follow sports and like data analysis, so this isn’t a bank account thing.

Horserace election coverage is the larger problem. But FiveThirtyEight is clearly a participant in that.

The models are one thing. It's the data that really matter imo. No other organization has created as thorough of a collection of polling firms, their polls, as well as their faults and past records as FTE.

Even if you're building your own model you'll likely be turning to FTE for the latest data.

In addition they're data driven journalism is often paired with amazing data sets that they open-source for free.

It’s definitely an interesting discussion.

> who doesn't want to know who will an upcoming election?

In particular, politicians want to know. This to me is the interesting upshot. Polls are non binding, so the public could use them to signal to candidates how to tweak their platform.

I’m always curious if there’s any real good data saying turnout patterns change. The thing is, voting because you think your one vote might change the result, is already completely irrational. I always think of voting of being like when you go to a concert and buy the band’s CD even though you already have Spotify: it’s more about find a way to personally participate in an important life moment than practicality.

I think the major parties and other interested organizations will always have forecasts like these available internally, and will leak them (or lies about them) when it's to their advantage. Having a high-quality forecast directly available to the public can't hurt more than that.

> I sometimes think they may be harmful, decreasing voter turnout by convincing people (correctly or incorrectly) that the result is preordained.

This is an old tactic that is used to the hilt by both sides whenever it serves them, and media outlets already make forecasts, often without 538's scrupulous emphasis on how often they're wrong. I don't think 538 makes this worse.

> FiveThirtyEight's political forecasts absolutely provide a desirable service—who doesn't want to know who will an upcoming election?

> But is it good for people to know who is more likely to win an election? Are these predictions beneficial to society or democracy?

Two points:

1. The model DO NOT tell you who will win the upcoming election.

2. It is absolutely beneficial to society and democracy, having a poll-free & rigorous-analysis-free democratic society means an environment where even the most apolitical citizen can be bamboozled into thinking that an election was "stolen" simply based on their close associates and news sources. Polling and aggregation, when executed with a modicum of accountability, prevents a devolution into a Soviet-style culture of paranoia particularly among the elites. It has tremendous value, that value will be backfilled by rumour and innuendo and that is very bad for society.

You seem to be arguing about net social value, which is a distinct and much more complicated issue than journalistic value.

(That said, yes, I think the models clearly have net social value, too.)

> But is it good for people to know who is more likely to win an election? Are these predictions beneficial to society or democracy? I sometimes think they may actually be harmful, decreasing voter turnout by convincing people (correctly or incorrectly) that the result is preordained.

For plurality voting (a.k.a. first-past-the-post[0]) with more than two candidates, voters have to cast their ballots strategically[1]. This is especially common in primary elections and non-partisan positions.

Unless all elections switch to a method that is not subject to the spoiler effect (like approval voting or instant runoff voting), voters need more information about which candidates are most likely to win, not less information. Sometimes candidate competitivness can be inferred from things like endorsements or fundraising numbers, but polls -- and aggregated models based on polls -- are an important tool.

[0] https://en.wikipedia.org/wiki/First-past-the-post_voting

[1] https://en.wikipedia.org/wiki/Strategic_voting

It has been fairly useful to me as an observer and voter in understanding how the elections go down. Understanding what the chances are of each candidate and how certain voting groups can influence the election can (sometimes) reduce being blindsided by an election. It may reduce my impartiality as a voter, but it definitely helps me process politics as a citizen.

This is something I think about a lot - the actual value of polling in general and public opinion polling in specific.

A lot of public discourse is creeping on several layers of meta, to where we are constantly reacting to how people react to other reactions.

Sometimes I wish we would just let ideas stand on their face instead of constantly engaging in acts of national navel gazing.

Even if he result appears dead-even there is a cost. Framing the election in terms of who wins instead of which candidate is best increases the tendency of voters to vote for candidates who "have a chance," even though there is no rational reason to vote for candidate that is more likely to win.

There is a very real reason to vote for the candidate during the primary who is most likely to win the general election.

If your vote is the deciding vote (i.e. the candidate won by exactly 1 vote), please buy me a lottery ticket.

All votes are the deciding vote, in that case!

I would think forcing people to better appreciate how probabilities actually work is better than avoiding the topic entirely. Learning that 70% is not certain is important in lots of contexts. 30% probability events happen all the time…

Data illiteracy, not the existence of a forecast, is the core problem.

The science is valid and they are a good aggregator of the various polls, it's perfectly legit and moderately beneficial.

Especially in a world of liars.

The model is only one feature of the site, you realize right? Have you read anything else they put out besides the election forecast?

> FiveThirtyEight's political forecasts absolutely provide a desirable service! Who doesn't want to know who will win an upcoming election?

Does it actually work though? I don’t think they were able to predict the past 3 presidential elections, so not sure where they provide value.

FiveThirtyEight has had astonishing accuracy since 2008. They don’t pick winners, they provide data analytics of polls and other factors. People may not appreciate probability theory but this is far more reality based than what passed for political analysis prior to 2008.

> it may even be harmful

It definitely is, I tried to make the same point a week ago :-)

https://news.ycombinator.com/item?id=35707786

The value is the hype not the journalism.

Reply on news.ycombinator.com