How to Read a Research Paper [pdf]
eecs.harvard.edu
eecs.harvard.edu
1. Read title and abstract 2. Read introduction 3. Read conclusions 4. If the content of the 3 previous steps makes sense, and the paper is relevant: - Read the body of the paper - Archive it and write bibliographic record card (yup... that old school).
Worked me wonders in my PhD.
I have to dig through a lot of research papers for my blog, most of the papers are dead-boring, but I feel obligated to read them with a fine-tooth comb to better understand what's being reported.
Filtering irrelevant or poor documentation by following your process seems like a no-brainer. Thanks for sharing!
Conclusions in CS papers are generally worthless. The contribution of the work is typically not new knowledge, but a thing - a system, a technique, an algorithm, a language, a framework, etc. So the conclusions in CS papers tend to be "We presented a foo for a blah blah blah." I joke that conclusions in CS papers are mostly vestigial, and exist only because they're expected to exist. (Please note that this bothers me, and I try to include actual conclusions in my papers, which will take the form of general principles we can learn from this new technique or system that we're presenting. But such discussion is not needed, and is often cut due to space limitations.) So, skip the conclusions sections in CS papers.
The introduction can also usually be skipped, if it's an area that you are already familiar with. CS papers tend to have to tell a "story", and the introduction sets the scene for why the work is relevant and who cares about this sort of thing. These introductions can start out very broad and general, trying to motivate the work from larger trends in industry and society. (Yes, even CS systems papers will appeal to larger trends in society to motivate their work. There are a lot of CS systems papers which will appeal to the prevalence of, say, online social networks to motivate their work on the systems required to support such things. Such as graph processing or real time data management.)
If you are familiar with the field, you can typically skip all this. You already know it. Please note that introductions are great if it's a field you're not as familiar with.
A CS systems paper typically has the structure similar to: 1. Introduction; 2. Background; 3. Design; 4. Results; 5. Related Work; 6. Conclusions.
If I want to quickly assess if a CS paper is worth my time, I typically read the title, the abstract and then skim the Design section and skim the Results section. (Of course, if it's my field, I often can't resist just skipping to the Related Work to see if they cited me.)
If the paper passes my first look, then I'll typically start by reading the Design. If I hit any difficulty in understanding the Design section, then I'll jump back to the beginning to get the proper context. I don't bother reading the Results section until I understand what they did, and think it's reasonable. (If your Design section tells me how you implemented the best 3-wheel car in the world, I'm not going to bother reading your Results. I don't care to try to reason about the performance of your 3-wheel car, because 4-wheel cars exist - unless you can convince me of situations where 4-wheel cars are not an option. That's a hard sell, though.)
1. Read the title and abstract 2. Read the methodology 3. If the methodology isn't totally asinine, then read the rest
I discovered after reading hundreds of papers that most of them are total nonsense.
I found myself reading them and updating my "knowledge," then getting to the methodology section and realizing it was bullshit. I felt like I couldn't fully "un-update" my model when that happened, the damage was sort of done.
They make claims and they have "evidence" that sounds compelling enough to slip by some filter, but then the methodology is totally bunk. The n is way too small, or limited in some other fundamental way, the experiment design is idiotic cargo cult stuff. Obviously just going through the motions of publishing because they have to, rather than having some valuable insight. Then papers like that cite each other, and build this whole wobbly network, full of sound and fury, and signifying nothing.
I've learned to totally ignore any paper that I haven't read the methodology for first.
They are called reference managers, and will extract the title/year/author/abstract so you can quickly glance at that obscure "iccp2012.pdf" you downloaded last month and know if it's relevant or not to your current task. Provide full text search on your entire collection, synchronization between home and work, etc.
Coming from an engineering background I had no idea these existed, but it sure will save me a lot of time and frustration.
1. Envision what you would like software to do to help you with some task.
2. Figure out what software like that would be called, and what search terms/search engine it would be found on.
3. Enter terms into correct search engine, find that the software exists, install it and start using it.
I agree, Papers is great. I have been using it since the earliest version, both on my Mac and on my iPad. Neither has ever given me trouble.
When they started, the two founders were both still mainly post-docs at the Netherlands Cancer Institute [2], and that's what I remembered, reinforced by the copyright notice I mentioned above.
[1] http://www.papersapp.com/careers/
[2] https://web.archive.org/web/20070213053257/http://mekentosj....
They released a major new version (Papers 3) just under a year ago. One of those 'fresh start' kinda releases; a re-think of what a reference app should be like, and substantially different from Papers 2. I still haven't upgraded because it doesn't do much more than Papers 2 did, but if you're coming to the product for the first time, it really is a great release. Highly recommended. Definitely the best Mac client out there, and they have a good iOS app too, along with solid Dropbox syncing. (Haven't tried their Windows version.)
I started it back in ~2002 or so, and it's been kept running by a small group of contributors ever since.
Not a multi-user or web-based solution, but it has accumulated quite a few features (including searching many databases) that can make keeping a personal BibTeX file up to date much less of a pain.
--- The time budget is extremely sensitive to how much you already know about the subject. If you're generally up on the literature in an area, it is often much easier to isolate the chunks that are genuinely new to you and thereby grok the paper very quickly. In a new field, even several hours to really read a paper might not be enough, depending on how much backtracking you need to do to understand core ideas.
--- In physics, the conventional wisdom (which I agree with) is that the main thing to do when checking out a paper is to look at the figures. These are generally chosen to highlight the most important points of the paper and will often quickly convey what was measured/calculated, how the effect scales, etc.
I see each article as a patch. Our minds are both running operating systems and decentralized repos. Each of us are specialized and running a different custom OS, so may require different sets of patches.
Science is specifically concerned with our mental models of how the world works. Different fields in science are like different levels of abstraction in programming.
My specialty leans more towards higher levels, so subjects like psychology, philosophy, biology, are more relevant to my mental model than lower level mathematics, chemistry, and physics.
When I read research articles, I'm primarily interested in extracting an abstract high-level idea, which I can apply to my repo. I start with the end of the abstract, then discussion, conclusion, results, to find the main point, then only look at details if I can use them.
Once I have a simple idea, I merge it by connecting it to related ideas in a functional way. Each idea then becomes like a code snippet of a function, and my process of learning is like coding, where although you may copy/paste a snippet off the web, you still need to reason with logic to connect it into your code so it actually works.
If I can't make something work right now in my repo, it doesn't commit, so is temporarily stashed, where I may either forget about it, or come back to it if I find the missing pieces to fit it into the working directory.
I think asking for a one-page detailed example and illustration of the paper would force a more careful read, and would be more interesting for sharing with peers.
Writing Technical Articles http://www.cs.columbia.edu/~hgs/etc/writing-style.html
Whiteside's Group:Writing a paper: http://www.ee.ucr.edu/~rlake/Whitesides_writing_res_paper.pd...
Know what you read!
surely that means accepting that a bulk of research papers are written exceptionally poorly with excessive wordiness and jargon, a lack of good paragraph structure and not very much real content.
if you have to tell people who can read how to read something, its probably written incorrectly.
If you think that everyone just knows how to do this, take a close look at your last set of reviewer comments; were all of them pithy, accurate and demonstrating a close understanding of your work?