Why don’t journalists link to primary sources?

Here's the problem: Almost all media outlets are in crisis mode so they're cutting down on their staff, meaning that a smaller number of journalists have to write the same amount of stories as before to fill the paper (or website). There simply isn't time for proper research if you have to write five articles a day.

So many journalists resort to "borrowing" from other news sources, which in turn often have borrowed their article from somewhere else. An article may start as a published paper in Nature,which is then reported in popular science, which is then reported by Reuters, which is then reported in The Times, which is then reported in The Daily Mail.

By the time it reaches The Daily Mail it's been through so many filters of busy journalists that the meaning has often become distorted. The journalist from the Daily Mail has no idea that there even is a primary source.

At DocumentCloud, we're trying to change that. Here is a list of newsrooms that use DocumentCloud to share their sources -- you'll notice most of the usual suspects in there:

http://www.documentcloud.org/public/#search/

Here's a good example of a major 5-part story that cites its sources heavily. Take a look at the source documents page, and try reading one of the articles, and clicking on the links:

http://www.lasvegassun.com/hospital-care/

As for that class of secondary sources (e.g. review articles and editorials) that is provided for free in a dumbed-down form, then, yes, journalists should provide links.

A custom of providing links would be very helpful in the "chat about science" writing industry, so much of which is based on opinion and political view.

[1] and obviously here I'm using a very broad brush -- Ben Goldacre is a good example of a young doctor / writer who really gets the web and there are quite a few others.

That's a very difficult thing for technical people to grasp. Sometimes I think we view everything online as various versions of wikipedia, and it's not like that at all.

In practice, sure, they do a bad job many times. Sometimes they just copy and paste Press Releases. Many times they bungle a quote (Although most quotes are problematic to use verbatim. People don't talk in real life like they do in movies and books) But reporting is an art, not an algorithm. Reporters and editors are supposed to be very good at taking multiple sources, including the reporters own generated research, and mashing it together into something greater than the sum of the parts.

I don't have an opinion one way or the other about linking to primary sources. If I had to choose I guess I'd like for them always to be included. But the idea that somehow it's going to make a big difference doesn't add up for me. For researchers, sure. But not for the average reader.

With adequate tools for making the sourcing process as effortless as possible (see Jeremy Ashkenas' and Scott Klein's comments) it ought to remain an optional value-add for publishers: those who don't mind having their sourcing restricted to press releases, effort-obscuring generalizations or PR-driven 3rd party referencing won't encourage technologies that would enable their newsrooms leverage their in-house professionalism, regardless of its potential.

Linking/referencing is fundamental to good, original content, whatever the medium (see randlet's comment). If publishing software won't support easy source accreditation (particularly in a web-focused, hyper-linking context), journalists and editors are going to lose out - either by facing a 'keep my job vs. best practices' drama or by becoming disillusioned with the chance of success their organization has in maximizing the credibility of the staff input going into evolving their brand itself.

Taking their access to low cost, more modern publishing tools as a given, perhaps in more dynamic/educational reporting environments they should exploit software with prompts - asking authors to confirm that their content is 100% lacking of any derivative work, sources, or any mere cursory reference to any other material/commentary out there. This might even help less experienced journalists off-set some of the responsibility they have in bringing new ideas to the attention of their demanding readers ('this is my article, but here's some source curation for you, while I get my story straight'). Of course software that assumes any kind of pedagogical role is just as likely to become demonized by journalists who resent tools for their antiquated inefficiency. There's a fine line.

Services like DocumentCloud are vital in all this. As ilamont points out, there are also practical hurdles to overcome: even if you have the technology and inclination towards sourcing best practices, if you're running an online publication there's still the knee-jerk reaction to want to obscure your primary source, in case it out-performs your own content and thereby undermines your credibility. That's a question of quality and self awareness, though. The credibility loop will eventually hit your property, one way or the other.

That said, we usually followup with journalists and ask for a link and most usually don't have an issue with providing it until their editor overseers ixnay the idea.

I think it's often less about malicious intent than being clueless about the "link economy".

The Knight News Challenge gave out grants this year for people to tackle the problem of verification and trust in online news, and I met someone from Mother Jones who was working on a system for documenting primary sources.

I also gave a talk last year proposing a standards-based solution for this at a journalism/tech conference in Philadelphia. My thoughts about it are here: http://beatpanda.co.cc/blog/2011/01/31/show-your-work-fillin...

More importantly though, I'm seeing a lot of potential customers for disruptive news startups right here in this thread.

Look at how many people don't trust the media. The challenge, if you want to make paying customers out of these doubters, is to not only deliver the news, but to give people a reason to trust what you're saying.

Journalists showing their work would be a good first step towards that.

I worked at a newspaper for 5 years, and their editorial system - this would be the software that reporters write their stories in, editors edit the stories in and proof readers and copy editors lay out pages in, not to mention ad placement, classified pagination and much more - was pushing 10 years old when I started. It wasn't integrated with the website content management system. It was slow, outdated and very annoying.

But it worked, everyone new how to use it, and we knew that we would be able to publish multiple publications every day, 365 days a year. The fact that the system wasn't fully integrated with the CMS meant it was difficult to add links to stories, usually requiring going into the CMS and hand-coding links, after waiting for the publishing scripts to grab the story and post it, a process which could take up to 30 minutes, depending. Typically an online editor or copy editor, far removed from the research and work the reporter had done, would manage this for the entire website, since it makes more sense for the reporter to be out on the street doing actual reporting.

So the 'simple' task of adding a link isn't as simple as it seems when placed in the context of a fast moving newsroom. Typically the battle isn't "why isn't there a link to that source document?" but rather "what do you mean you can't get anybody to speak on the record" or "we need a story for the 1A centerpiece now."

And then the next day the same battles start all over again, the process repeats and another newspaper is printed. It's difficult to change the process because the process is so critical to meeting print deadlines to be able to deliver papers on time.

To add to this, newsrooms are getting cut down at the same time they have to do more. My newsroom once had ~60 people at a time when all they did was print the paper. When I left, it was about half that size and we were posting regular online updates, tweeting, linking to stories from facebook, monitoring comments online, posting photo galleries, shooting video and creating interactive flash presentations on top of it all.

So, yea, if the reporter has time, he reads the report. If it's a story about calf length, it's more likely he'll skim it to be able to bang out a space-filling (or traffic driving) story. And since it's a light human interest story, and not something that's likely to get the paper sued, it gets a quick once-over proofread, then posted to the internet ASAP, then printed, and everyone is on to the next big story to feed daily monster that is daily publishing.

I still think some of these organizations will get there. The New York Times is leading the pack, but the slower publishers will take awhile to replace their antiquated systems and change the daily process that has been refined and perfected since the dawn of the printing press. Of the ones that don't, new organizations that can manage the digital realm will replace them.

But if you were to point out that primary sources also includes published research, almost to a man or woman they would say A) they don't have the time to read it B) they don't have access to the journals or C) they are not aware the research exists. A few might concede D) even if they had access, they wouldn't be able to understand the research, which points to the fact that most journalists didn't major in science/technology in college and academic writing can be difficult to penetrate.

Of the above factors, I think C presents an opportunity for academics and startup publishers. On the academic side, it's pretty clear that the traditional method of reaching out to reporters via press releases and personal contacts is becoming less viable as newsrooms cut staff and the remaining writers have less time to network/talk with sources (travel budgets to attend conferences are very restricted these days) and write up stories based on those encounters.

Some researchers have seized upon blogging as a great way to not only reach their peers, but also a wider audience, and of course, other media (including journalists, specialist blogs, etc.). Group blogs written by researchers and experts are another great way to highlight new research and discuss ideas, too. Terra Nova ( http://terranova.blogs.com/ ) is one example focused on virtual worlds; I am sure the audience here knows of many others.

But the problem with individual and group blogs is they are still largely unknown outside of a relatively small group of people. In order to make a mass audience connection, there needs to be a way for these ideas to be presented in newspapers and television reports (which is how many people still learn about the world around them), or on media websites.

An arrangement to republish blog content or for the blog authors to prepare easy-to-understand summary reports for a mass media audience are possibilities, but the processes and incentives need to be worked out -- preferably in a way that takes the load off of editors, who don't have the time to find the right bloggers and deal with the freelance contracts and payment issues. One startup idea would be to create a "marketplace" to match publishers who are seeking an informed report about a specific scientific topic (for instance, how a boiled water reactor works). Another avenue for a startup would be to set up a "science wire service" which prepares timely, relevant coverage (including blogs, video, and features) about new research and developments every day. Media companies could subscribe to the service and editors could browse the service and use as much as they like, just as they do with Reuters, Bloomberg, AP, etc.

As for the specific issue of not including links, this partly relates to the awareness and access issues mentioned above, but also to the fact that content management systems used at many newspapers and magazines are optimized for print publishing, not online publishing. Inserting links typically has to be done after the article has been written, often by different editors or producers who know how to use Wordpress/Drupal/homegrown tools. I think there's a startup opportunity here as well, but unfortunately it also requires a rethinking of newsroom processes and control.