Nate Silver Went Against the Grain for Some at The Times
publiceditor.blogs.nytimes.com
publiceditor.blogs.nytimes.com
Anyway: The New York Times should've worked harder to keep him. They should fire Tom Friedman and Maureen Dowd and put those resources into the kind of analysis Nate Silver espoused. Y'know, the "actual journalism" kind.
Their offer to him was already grating to many others at the Times, I imagine. They offered him quite the little staff, and expansion into any topic he wanted.
I don't think his decision to leave was about their offer, that, or ESPN/ABC just dreamed bigger and gave him a bigger offer.
Either way, I'm psyched to see that he's getting brand separation and returning to his website along with a staff and the ability to write about whatever he wants.
Actually Krugman is also a poor fit. I wish he'd walk away and just keep doing what he does independently. At least then I wouldn't have to go into privacy mode to read his articles.
Read this NiemanLab article from 2010. It was always clear (to me at least and a few others) that the NYT thing was just a step along the way (partnership vs acquisition - 3 yr contract, etc):
http://www.niemanlab.org/2010/06/articles-of-incorporation-n...
Or, he could sign up with ESPN/Disney, and do sports on ESPN, politics on ABC, and so on.
He'll have a much broader level of impact at ESPN/ABC. I suspect NY Times just couldn't match that.
And get better results in the process.
Now that MSM has seen what's possible when logic and statistical analysis is applied to journalism, I'd imagine they'll be able to find a team to reproduce or at least mimic what Silver did in 2012.
How did he do that?
If anything, I would argue the opposite, because his model is proprietary and his explanations (presumably deliberately) vague. 538 is awash with numbers but the algorithms used to generate them are the very definition of "eye-of-newt and other secret ingredients".
Silver basically said "here are the exact probabilities we have calculated for these races and here is the gist of our methodology [1]". If that doesn't cut through the rhetoric, opinion, and 'expert analysis' that plagues MSM political coverage, I don't know what else possibly could.
I agree. I much prefer the (simpler) work of Sam Wang from the Princeton Election Consortium (http://election.princeton.edu/) who did release MATLAB code of his work and his data sources so you could do it yourself. Somehow, Nate Silver got all the media attention though.
Yes, somehow the young, geekily attractive popular sports author, whose blog on an extremely popular alternative news site grew a significant cult following, got all the mainstream media attention. It is extremely surprising that said media attention did not go to the duo of crusty, obscure, highfalutin Ivy-league academics instead.
Somehow, that happened. How, we'll never know.
He definitely brings in good writing talents and a possibly unmatched inquisitiveness...but his methods and predictive analysis aren't irreplaceable. And yes, part of the draw is the brand that he's worked tirelessly to build...but this is a brand built on verifying results...I.e. being conclusively right....if the Times were to bring in another blogger who predicts the 2014 Congressional race to a T, and explains his/her methods and shows a real love for it, I'll subscribe to that blog and not give a whit about how many years of blogging they've done.
Contrast this with the irreplaceable Roger Ebert. He was incomparable as a writer, but his brand was built on something very subjective...and thus, once he's gone, it's hard to justify going back to RogerEbert.com, no matter how great of critics there are to replace him. Roger's brand is based more on long-built loyalty...Silver's brand is more based on making verifiable hypotheses and being correct...time and time again.
Also, you're probably right, there are at least three other prominent electoral college forecasters who write well, were as accurate as Nate, and who could possibly be contracted to provide the same analysis and commentary: Andrew Tannenbaum [1], Drew Linzer [2], and Sam Wang [3].
Unlike Nate, forecasting is not their primary career, but it's clearly a labor of love for them and who knows what could be worked out with the right offer. If I were in charge at the NYT I'd be starting up conversations with these and any others doing similar work, stat.
http://www.ire.org/resource-center/listservs/subscribe-nicar...
More importantly you can get the same predictive power by just directly using recent polling data.
That's not true in Nate's case at least. One of his biggest wins was debunking some of the BS "narratives" that pundits tried to spring during the election long before election day, the one about "Romney's momentum" in the final weeks of the election being the most memorable one.
Linzer also did a postmortem where he looked at, among other things, accuracy and predictive power of the model at different points in the election [1].
>Is that really useful?
An order of magnitude moreso than traditional punditry, not least because it's honest about what it can and can't tell you, namely "if the election were held today, this is the most likely outcome". No more, no less.
>More importantly you can get the same predictive power by just directly using recent polling data.
You do realize that's exactly what these guys do, right? The problem is, which polling data do you use? They provide a scientific answer to that question. Instead of cherry picking, they use it all, and weight it based on past accuracy and other factors.
I believe strongly in quantitative analysis of election data. I place zero value in punditry, especially from main stream media. What I don't believe is that some of these complicated models, Silver's in particular, are meaningfully superior to predicting the election by trivially applying recent polling data augmenting it perhaps with a simple weighted combination of polls based on recency or sample size.
I take as a given that Silver's model is better than punditry. I am skeptical that it is better than the trivial model which any undergraduate stats student would cook up.
> The problem is, which polling data do you use? They provide a scientific answer to that question. Instead of cherry picking, they use it all, and weight it based on past accuracy and other factors.
What I dispute is whether the "other factors", which is the secret-sauce that allows Silver to give the impression he has a uniquely predictive model, have any real value.
Thanks for the Linzer link. I have to take time to read it carefully but on first glance it again shows one of the things I take issue with: If you are going to claim that a model is accurate you should be asking, compared to what? How can you justify a complex model if you aren't even going to try to show that it is better than some trivial baseline?
This seems quite easy to test. Has anyone done so yet?
That would be interesting to know, but also easy to find out.
I think most of the extra value you could add would be from analyzing the poll data to get a sense of which polls were unreliable.
But, the Times really did a great job of supporting his work. The infographics and visualizations and interactive graphs that they produced -- typically with D3.js, which was cool to see -- were really second-to-none. (Who can forget http://www.nytimes.com/interactive/2012/11/02/us/politics/pa... ?)
Visualizations don't seem to be Nate Silver's bailiwick, so that leaves me feeling a little disappointed to see him leaving.
Nate's a clever fellow though, I'm looking forward to seeing what he does with ESPN.
As a long time Nate follower, he is very good at posting basic graphs alongside very strong explanations of said graphs. However, he has very little skill when it comes to more creative, targeted, unique graphs like the one that you linked.
However, he's a smart dude and ESPN has plenty of resources. I'm sure he'll be fine.
I've on occasion wondered if it would be beneficial for companies in other fields to adopt such a role. Could Google regain some trust and goodwill if it appointed a "public editor" with a suitably credible background, a degree of independence, and a mandate to advocate for the public interest?
[1] http://www.nytimes.com/2003/07/31/business/media/31PAPE.html
[2] http://www.nytimes.com/2003/10/27/us/the-times-chooses-veter...
You're right that the position is toothless...she has no ability to fire people or allocate resources of her own. What she does have in terms of power is getting to say whatever she wants, about anyone at the Times, without fear of being fired. She's like an internal affairs cop (who themselves have limited direct power) but who is guaranteed a spot in print and on web in America's biggest paper.
What other companies have such a position? If you're thinking, "X tech company who got hacked and wrote a lovely apology letter about it"...well, then you thoroughly misunderstand the public editor's province. Her equivalent at tech company X would be someone who interviewed the engineers who fucked up and then put their names and quotes into print...not someone who does PR apologetics for the company.
She was among the first under the Times masthead to point out that their incoming CEO had some major problems and to say what many were privately thinking. No, she didn't get him fired or removed...but again, that's not her purview. Her role is to raise awareness and stir the pot in a way that most people internally will never do. She is most certainly not a "pawn"
And yet, your comment provides no relevant information I didn't already possess. Shocking that two people with identical knowledge can reach different conclusions, isn't it?
Do you really think a corporation would allow serious injury to come to its reputation especially when its reputation is critical to its business?
Do you think if the public editor starting leveling serious critique at the NYT in the vein of FAIR, Media Lens or Chomsky they would retain their position? Do you think the NYT would ever appoint members of those organizations as public editors?
So no, they're not going to hire someone who is either actively bent on discrediting the newspaper...because that would imply the newspaper execs believe that the newspaper is something to be destroyed. The position is meant to be one that improves the newspaper and its accountability to the readers. Many of her columns raise questions and doubts about Times' coverage, but the expectation is that this makes the newsroom a better place...similar to how transparency in government ostensibly makes it better in the long run.
But, on the other hand, if Sullivan became the David Souter of the Times...she would be for all intents and purposes, difficult to fire without creating a shitstorm. I'm not sure what's in the NYT's actual 4-year-contract with her, but barring her committing some journalistic crime (like plagiarism), it's not likely she can be removed for the content of her criticism.
The intent of real criticism is to improve the actual role of the newspaper in relation to the public. The NYT doesn't hire people who do real criticism because real accountability to the public isn't the main incentive of the NYT: revenue and power relations are. The illusion of accountability is necessary, which is why they have the public editor in the first place, because otherwise people wouldn't buy their products, be influenced by their messages and see their ads.
It is the height of absurdity to believe that someone who gets paid by an organization would be unbiased in its criticism of it or that they would act in the interests of a third party, in this case the public, from whom they receive no direct benefit.
I think what should be happening is that investigative journalists in the financial sector should do a better job of finding sources within public companies, who are willing to leak information about practices and policies that are against the company's publicly stated positions, and/or are hurting shareholders, employees, the environment, and so on.
Example: why don't we see detailed leaks about NSA actions at the ISP's, carriers, and large web companies?
Sign of a more general change?
There is room for both, IMO. The 40+ year old crowd are largely comprised of people who are "done learning things" (a quote my father actually told me when I told him about my baseball research), so you need to have the Joe Morgans of the world.
But since all of the winning teams in baseball rely on an analytical approach, it's never been more obvious that nerds dominate the level of the sport where decisions are made. And commentators are shifting as a result of that, since it's just natural.
But you do need both, often because the people in the 2nd group tend to have worse social/presentation skills than the individuals in the 1st group. No slight intended (I belong mostly in the 2nd group, for god's sake), but it is not an unfair generalization.
That said, it also makes him an interesting counter to some sports pundits when there are invariably differences between the numbers and perception of players and teams. This will make for some good television. Heck, ESPN just started a show called "By the Numbers" a few months/years ago, so I think there really is a move to numbers-based analysis in sports. The key is just making digestible to the general public, which Silver has been very good at on 538.
Possibly, ESPN had some struggles this year [0].
I hope they're looking to change up the formula. I find the traditional sports pundit and the fans who ape them unbearable.
0: http://deadspin.com/espns-ratings-are-way-down-this-year-wha...
Nate Silver was ahead of his time on those advanced metrics but he was undoubtedly one of the pioneers. If people had been more receptive to his brand of statistics five years ago he likely never even goes into politics. Sports was his first love (along with poker, which he goes out of his way to not mention these days, unfortunately. He even obscured his username on the largest poker forum[2], so it would be harder to find with search engines).
Completely random thought that probably means nothing, but is worth noting: he's openly gay[3]. With the tide on gay athletes turning sharply and quickly, I have to think that factored into his decision. Perhaps he is hopeful he can play a role in that, being the champion of the gay athlete or at least the one guy at ESPN they know will be on their side.
[1]http://en.wikipedia.org/wiki/Wins_Above_Replacement
[2]http://forumserver.twoplustwo.com/members/4548/
[3]http://gawker.com/5969477/sexually-gay-but-ethnically-straig...
The new school argument is something like: the Triple Crown is an overvalued and superficial accomplishment and giving a player the MVP based solely on that is making a similar error to the scenes in Moneyball where the scouts cared about irrelevant things like whether a player has an attractive girlfriend or how he gets on base (walks vs hits).
I have no problem with using WAR as something to include in the discussion, but it should not be the complete deciding factor.
Interesting side note, apparently Oakland's (famous for using sabermetrics) model for WAR had Cabrera on top - http://bleacherreport.com/articles/1570961-mike-trout-vs-mig...
Everyone is happy to point at "stats." Almost nobody wants to hear about confidence intervals.
Offline, fans of some sports are almost exclusively about statistics and numbers. Baseball is famous for this.
So you are right about the typical role of stats in sports. Commenters like chasing and lnanek2 are mistakenly keying on the fact that sports, having well-defined rules and human-constructed elements, are very easy to measure and so lots of numbers exist. As you correctly point out in reply to chasing, these numbers are simply thrown out there without actually being used in a coherent analysis.
I would not be surprised if the "how to block" mini-game could be productively analyzed in a game theoretic fashion in terms of a few discrete tactics and empirical success probabilities given the tactics choices. From that data, you can compute equilibrium mixed strategies for both the offensive and defensive positions. This was kind of analysis was by economists for the determining equilibrium strategies for making/defending penalty kicks. http://www2.owen.vanderbilt.edu/mike.shor/courses/game-theor...
Of course, the blocking in American football situation is much harder to analyze/collect data for, because it's not just a 1 on 1 successfully blocked or not, but rather the outcome of the play that matters.
But the point is that detailed blocking foot work is not adequately captured by such a framework (unless you're going to do detailed numerical simulations of every part of everyone's body). There is no concise way to explain exactly why chop blocks work well in certain situations, and why a hypothetical block which seems mechanically similar is completely useless in the same situation. Human bodies are complicated, and the answer is usually just "empirically, that's how it works out".
On the flip side, bookies have always been taken seriously.
The average sports fan might find the commentators more amusing or moving but if you ask them who has a better handle on what's really going to happen, a professional bookie or a commentator, I bet you'll get the right answer.
ESPN is owned by Disney who own ABC, on which Silver will sometimes appear to do some political commentary.
It's much more of a package deal than the vertical venture a lot of people make it sound like.
I'm not at all surprised he moved on - I suspect that they expected him to be grateful, when the reverse would have been more reasonable.
Back then he was just a novelty with a website it looked like his 15 year old cousin built for him one weekend. And one who was going to find it very challenging to remain relevant until the next election cycle.
Now he's arguably outgrown them, having become a household name to anybody who follows politics closely, become a TV personality etc - but lets not rewrite history. He definitely stood on their shoulders to get where he is today, at least for the first year or two.
And this is all coming from a pretty huge fan.
I'll be curious to see what he says about why he left. It's very easy to imagine several reasons the he found it hard to work for the Times. It was very interesting to me to see how hostile some folks were to him during the 2012 election cycle. He obviously threatened the viability of a great deal of the(mostly useless) political punditry that we see. I think it's hard to separate which parts of that hostility were conscious reactions to a perceived threat and which parts were an unconscious reaction to ideas of how the news should work.
I'm sure Nate will do a great job with ESPN and I hope he likes it more than his time with the Times.
Sounds like the NYT did everything they could to keep him, but against money and mass audience, prestige only takes you so far, especially when a stodgy old guard is fighting a rear guard action against you.
http://editors.talkingpointsmemo.com/archives/2013/07/whyd_n...
http://www.forbes.com/sites/kurtbadenhausen/2012/11/09/why-e...
Seriously, it may not have been about money or fit. If someone came along and said that you could continue to code software and get paid to do it, but didn't have to work for CrushSoul, Inc., anymore AND it was in something you found insanely fun and interesting, wouldn't you do it?
So it's no surprise to me that he'd find it difficult to work for an organization like the NY Times... The NY Times was not only complicit in the Iraq war, but came out strongly against Julian Assange through a series of horrible articles.
The purpose of the NY Times is to serve powerful interests and to prevent the embarrassment of powerful people.
Just skimming many of the regular op-ed contributors and editorial writing gives a window into the mentality of the paper. It's coasting on past glories and continually offering a message to readers that the world is full of illegitimacy and suffering outside the US.
Setting aside all creative considerations, the Avengers remains one of Hollywood's most impressive acts of longterm chutzpah, building individual franchises out of B-string characters helmed by way outta left-field director choices and bringing them all together in one film years down the line.
They could, of course, rest on their laurels and churn out a triptych of Avengers movies, each with their orbiting solo films. But Disney thinks bigger than that. Picture the scene in Avengers 3 (after the credits, natch) where Galactus looms into view over Mos Eisley, Hans Solo all wtf. Turns out the next three Star Wars films have all been building to an Avengers/Star Wars merger, dragging in its wake 15 tv series, 45 ancillary films and a trail of comics so large that collectors turn collectivists in order to purchase vast communal warehouses to store their collections.
As the credits roll on Star Wars/Avengers 3, we'd probably forgive Disney if the final card announced that they were following billg into philanthropy. They now account for 10% of US GDP and President Cory Booker is terrified that the gravy train is about to come off the rails and plunge the US economy into recession. BOOM. Audiences gasp. It's a post credits sequence. The camera tracks slowly through a darkened office, over a minimalist, uncluttered desk, towards a high-backed chair that's facing the wall. The phone rings. A hand reaches out to put it on speaker phone. We hear an efficient sounding secretary: "Connecting Mr Fett" and then a pause, the sound of breathing perhaps before Fett says "I'm sorry I failed you". The tension is palpable, as the chair slowly begins to rotate towards the camera. OMG. It's Nate Silver, wizard. "No no, Boba, it's all exactly as I planned it."
Cue the incorporation of 538 into the Star Wars/Avengers franchise, a development freighted so subtly in the accumulated oeuvre to date that only Nate Silver's super algorithm can track and maintain a consistent canon, further fuelling a blog empire.
The science of predicting election outcomes may continue to be improved, but perhaps it's essentially solved in a way that sports prediction isn't.
But did want to mention that ESPN is 80% owned by The Walt Disney Company (which also owns ABC)... and 20% owned by Hearst.
Here's the history of ESPN's ownership, from Wikipedia: "In 1984, ABC made a deal with Getty Oil to acquire ESPN. ABC retained an 80% share, and sold 20% to Nabisco. The Nabisco shares were later sold to Hearst Corporation, which still holds a 20% stake today. In 1986, ABC was purchased for $3.5 billion by Capital Cities Communications. In 1996, The Walt Disney Company purchased Capital Cities/ABC for $19 billion and picked up the 80% stake in ESPN at that time." http://en.wikipedia.org/wiki/History_of_ESPN
I've met with Hearst a few times, and their executives love to mention their 20% stake in ESPN!
I hit the ESPN 'premium' wall far less often than at NYTimes. 'Grantland' has been mentioned as a model for the Silver subbrand at ESPN; are any 'Grantland' stories premium-only?
Don't forget who has forsaken their credibility. Never forget that. It is only by remembering who has lied that we avoid future lies.
Would you lend money to a bankrupt company? No? Why then do you read the NYT?