Discoveries of Data Scientist Who Spent a Year at the New York Times | Hacker News Reader

Discoveries of Data Scientist Who Spent a Year at the New York Times

contently.com

54 pointssabon13 comments

The linked post by the data scientist is more interesting than this interview: http://brianabelson.com/open-news/2013/11/14/Pageviews-above...

That (the article linked to the scientists actual blog) was an interesting read. Doesn't need to be revolutionary to be meaningful, and after maybe a bit too much introducing, I found the explanation of his work clear and the visualizations both meaningful and engaging.

Great bit of work the data scientist did.

A lot of jargon. Not much information. It did not inspire me to visit his blog.

I totally agree. Would have liked to see more of the data used, the information gained and the the methodology used.

I totally totally agree. A lot of buzzwords, and fancy jargon ... and things that try to make the writer look intelligent. But nothing substantial.

Agree. I should have checked...comments, first.

The interview is meant to be for an industry news, journalism and marketing audience. Brian's blog is a place to find more in-depth reading. As these things go, this is on the light side of jargon.

The "wildly speculative" Felix Salmon piece [1] referenced in the contently post takes a look at some of the actual data that Abelson collected. Of note is the chart showing how relatively accurate predictions can be made about how many page views a given article will have based on a few simple variables (e.g. did the main NYT Twitter account tweet a link to the article, etc).

However, I'm not sure I grok Salmon's angle of it being a bad thing that the NYT almost exclusively promotes proprietary content over stories off the AP/Reuters news wires.

[1]http://blogs.reuters.com/felix-salmon/2013/11/15/how-the-nyt...

I'm not sure I grok Salmon's angle of it being a bad thing that the NYT almost exclusively promotes proprietary content over stories off the AP/Reuters news wires.

"Felix Salmon is the finance blogger at Reuters"[1]. There's his angle.

[1] http://blogs.reuters.com/felix-salmon/2013/11/15/how-the-nyt...

This guy thinks news outlets aren't using data to make editorial decisions as often as they could. Maybe there's a reason his colleagues at the Times don't defer to his data. If the new york times made every editorial decision based on its effect on metrics like pageviews and social shares, you'd have the Huffington Post. The online equivalent of a tabloid. Newspapers have a responsibility to the public that transcends profits and popularity.

That said, we data could improve news consumption if used properly, especially in terms of testing user interfaces. Data should be used to design how people consume news. It shouldn't be used to chose what news to report.

Here's the challenge for the next Mozilla Knight Open News Fellow: http://aronpilhofer.com/post/57733248022/from-documents-to-d...

I feel so terribly antiquated, as a mere statistician.

The amount of A/B or other systematic testing in the news industry is...well, a bit behind par compared to the tech industry. So Brian's work is probably one of the most methodological studies done within a news organization.

Yes, it is true that media companies have long been obsessed with ratings and circulation figures, but these numbers have been, IMO, pretty "fuzzy." Before the Internet, there were also audience surveys and tests that purportedly tracked how readers progressed through headlines and stories...but how could such measurements be precise without computers?

Even on a news website, the testing is not so straightforward. Does one story get more eyeballs than the other because of a great headline, placement, wording of a tweet, use of a graphic? Or did it get more eyeballs because it was of a particularly salacious or notorious event? Since the number of stories that a news site can produce in day is relatively small...around 100 to 1000...and the number of variances between topic, length, media assets, time of day, is so large...I think Brian's analytical strategy was pretty keen.

Anyway, here are the original links from his blog. I think there as thorough of analyses as you'll find in other domains:

http://brianabelson.com/open-news/2013/03/18/A-Metric-For-Ne...

http://brianabelson.com/open-news/2013/11/14/Pageviews-above...

Reply on news.ycombinator.com