Nate Silver's FiveThirtyEight: The Emptiness of Data Journalism | Hacker News Reader

Nate Silver's FiveThirtyEight: The Emptiness of Data Journalism

newrepublic.com

17 pointsekpyrotic17 comments

I expect a lot of "opinion" journalists to pile on Silver, but this piece strikes me as particularly tone deaf.

> Is numeracy really what American public discourse most urgently lacks?

It ranks pretty high.

> Like Ezra Klein, whom he admires, Nate Silver had made a success out of an escape into diffidence. What is it about conviction that frightens these people?

It's not conviction that's frightening, it's conviction backed by ignorance.

Since an open society stands or falls on the quality of its citizens’ opinions, the refinement of their opinions, and more generally of the process of opinion-formation, is a primary activity of its intellectuals and its journalists.

It's hard to read that in a friendly manner, it sounds like it would be so awful to have people thinking for themselves.

I think it is maybe a little optimistic, but an 'open society' probably works better the closer you get to discarding the idea that some people are intellectuals.

(I don't mean that as a rejection of expertise, but someone with a deep understanding of the dynamics of region (or whatever) is really only better positioned to explain the probable consequences of an action, their expertise doesn't really give them more standing to choose what action is best)

I agree this is tone deaf, mostly. And certainly journalists have room for improvement in understanding data. However, I certainly agree with this statement:

>Neutrality is an evasion of responsibility, unless everything is like sports.

More data can also mean more noise. More data can also mean implying more causation where none exists. And the world isn't a giant game with sterile constraints and static rules.

"It's just my opinion" strikes me as the larger evasion of responsibility here. There's nothing irresponsible about reporting the facts of a story so that people can make informed decisions.

> More data can also mean more noise.

Nate Silver wrote a book titled "The Signal and the Noise." I'm sure he's aware of the danger. Data Analysis (Data Science, whatever you want to call it) is all about turning data into information. Reducing noise.

Forget about Silver for a moment.

What’s more dangerous, the bias in an opinion page or bias in a feature article masquerading as neutral fact?

More data, more samples, can produce more spurious relationships, whatever statistical massaging is applied. Given more Noise, there’s more opportunity to inject bias in finding your Signal. Journalists shouldn’t pretend their data doesn’t stink because they just figured out how to get Excel to crap out regression results.

Who is suggesting they should?

It's conviction backed by ignorance backed by unwillingness to fix the ignorance backed by mathematical illiteracy that prevents them from fixing their ignorance.

Oh yeah, backed by not owning up to their convictions when someone without all their deficiencies proves them wrong with fact (or at least strong evidence).

This is a 'controversial' article written for hits. There will be thorough, intelligent criticism of 538, but not until the website publishes more material.

Towards the end of the article, the author asks "Is numeracy really what American public discourse most urgently lacks?" I cannot believe there is a red blooded American who could sincerely ask this question. Americans are paranoid that our schools are falling behind the Russians, the Danes, or the Chinese. Education is part of the American dream. The knowledge and understanding people have of the issues of the day is a metric for the strength of a democracy.

The author leaves us with a final question: "What is it about conviction that frightens these people?" upon which he refuses to speculate. Although he is the literary editor of the New Republic, he does not recognize that the fox essay is in the genre of manifesto (I wouldn't have known it either, but it was written in at the top).

Manifestos are meant to be self-aggrandizing and they will always define their subject over and against what already exists. Mr. Wieseltier needn't worry about the op ed page, it will survive Data Journalism.

Well, it's Leon Wieseltier and TNR. My expectations are set correspondingly low. However, he actually stumbles not far from an important point -- Silver is fetishizing "the side of no side," and using the cult of pretended objectivity as a blunt instrument to bash his critics with.

Silver isn't doing LHC observations here: he's doing political journalism, and data in politics and economics are neither neutral nor objective. The data we select, the models we use, create toy realities that we can use to justify certain policies and philosophies.

What makes Silver particularly pernicious -- and I count myself a fan of his approach to weighting election polls -- is that he pretends that data and modeling means 538's articles are endowed with an Olympian objectivity. Yet his hires suggest that his articles are going to be animated more by ideology than rigor.

For example, he chose for his science writer Roger Pielke Jr., a political scientist whose climate-change denialism (usually published as popular books, op-eds, and blog posts, not peer-reviewed papers) has been thoroughly debunked by practicing scientists. Not a promising beginning to his site; if I wanted to read worthless contrarianism written by unqualified hacks, I'd just renew my New Republic subscription.

I'm not familiar with Wieseltier, so I don't share your negative bias against him (and maybe I should), but the "important point" he "stumbles not far from" seems to me to be the main idea of the piece.

>Many of the issues that we debate are not issues of fact but issues of value. There is no numerical answer to the question of whether men should be allowed to marry men, and the question of whether the government should help the weak, and the question of whether we should intervene against genocide. And so the intimidation by quantification practiced by Silver and the other data mullahs must be resisted.

This writer makes some good points, but I don't think Silver is attempting to use statistical analysis to answer these questions. Silver's point about opinion journalism is one about bad incentives-- those people are rewarded for attracting attention, not for being accurate or even thoughtful. He's attacking the George Wills of the world for saying things like "Romney will win because Americans have realized that blah blah blah" when all the available polling data indicated a landslide win for Obama.

Of course there is room for opinions and certainly an appetite for it, but the problem for the New Republic is that the authority bestowed upon "publications" is rapidly eroding. This obviously hasn't been helped by their decisions to compete with the Buzzfeeds of the world in the race to the bottom for pageviews. Now that anyone can say anything in public, the fact that the New Republic has decided to employ you no longer makes your opinion automatically worthy of consideration.

> He knows many things. He has no priors.

So what is Leon Wieseltier saying here about Nate Silver, exactly? That Silver considers himself to have no priors? That he, Wieseltier, thinks that Silver has no priors? Both interpretations seem ignorant. Further, how can someone "know many things" and yet have no priors, when a prior is a representation of prior knowledge?

I find it hard to believe that Wieseltier would be so dumb by accident, which means it's actually on purpose. The article is an attack on the sort of public discourse Silver wants - arguing over likelihood of outcomes under uncertainty - in favour of "repetition" and "persuasion". Are those "what American public discourse most urgently lacks?"

Come on, it's rhetorical. Silver complains about the columnists being stuck in or ignorant of their so-called "priors", and isn't Wieseltier making your exact point against him? That is, saying "well, what's the alternative?" He is also sarcastically criticizing the smug application of a technical term ("prior") to a domain where it doesn't hold nearly as much value as Silver accords to it.

This seems like a straw man, the issue isn't that opinion journalism should be abolished, the issue is that it is overprevalent. News has become too much a mixture of pack-like reporting of one exciting incident after another (often stories with little value beyond entertainment), and opinion. In order to form a judgement about something, you have to have experience of it, and data journalism is, or should be, about allowing people to understand the reality of what is actually occurring beyond their immediate bubble. i.e. at its best, it should be 'empty', and as close as possible to an observation. Once you have at least some factual basis, then you can start forming opinions, or forming judgements about the validity of other people's opinions, and it is at that stage that opinion journalism ceases to be about providing superficially plausible rationalizations for your ideological preconceptions, and begins to be about finding some structure to make sense of the nature of the problems, and the nature of the possible solutions.

Americans really need to pay attention to the numbers more. Military spending. Prison population. Educational standards. Numbers of police. When numbers like these go haywire, that's how empires fall.

I don't understand why people listen to this guy. You can be right many times and profile political currents and opinions in a very relevant manner, not belonging to any party, it's still behaving like a prophet, and that's what people want to hear. That's just dangerous.

Well-written, but still pointless opinion fluff.

Reply on news.ycombinator.com