A Cornell professor’s trick for getting 1700 peer reviewed publications
andrewgelman.com
andrewgelman.com
The way he operates usually goes like this: he has a hypothesis (something that is usually trivial in my opinion). Then he recruits grad or undergrad students to work with him on it. After that, he let his teammates to do all the development work (coding) and experimentation (recruiting subjects and sitting with them in the lab to run the experiments). We needed to recruit human subjects because our work was related to usability of assistive technologies. By the time the data was collected and analyzed, the guy would show up with a vague draft of the paper, in which he has liberally cited his own (and his grad school mentor's) old publications, without knowing a damn thing about what his teammates did. Then we, as his teammates who ran the experiments, fill out the experiment and results sections. He then would write up the conclusion and fill up the rest of the holes. After that, he submits the paper taking the first authorship (because he came up with the hypothesis). That's the way he was able to publish >25 papers during 5 years of grad school (until he got his PhD).
Now that guy is an assistant professor--with ~50 publications--at a well-known University in one of the midwest states (of the US) at the very young age (I think he's just above 30 years old). If you look at his publications, he liberally cites himself and his mentors.
Kudos to him, he knows how to play the game.
*edit: Getting a few down-votes on this one. I was being mildly sarcastic but on the other hand it takes some skill and smart thinking to pull this kind of thing off. Also in my experience, bull-shitting is a very valuable and necessary skill at work and in life in general. Arguably it gets you to a position where you can do work of actual substance and get recognition for it.
It's very unlikely that someone who's spent their entire professional life being one way flips a switch after tenure.
And if we assume that some of those experiment involved master students then for a few of them that was their first publication. It look like he was kind of a unmoral presence from many point of view, but you cannot ignore that he probably helped many people get more experience.
Again I am not claiming that what he did was right, but it is neither necessarily a detriment to science.
Of course other groups have also done related things, and you should cite them as well to create an honest image of the research landscape.
In the case the OP mentioned, this is how a lot of research is carried out. The main part that sticks out to me as as scuzzy is that he took first author without being closely involved in the work.
But as a PhD grad looking for a faculty position, you will be evaluated by the number of first-author papers you have published.
Part of the problem is the review quality is almost as inconsistent as the article quality.
From experience I can tell you that rejecting papers is a lot more work than accepting them. It requires arguing things in such a way that it survives editor scrutiny. This in turn requires actually reading the paper, doing some background checks, etc. Of course editors getting involved also means work for them. So negative reviews tend to trigger a lot of work. So much easier to just rubber stamp the reviews that give a thumbs up.
And editors have the problem of needing to fill their journals with stuff. Nobody reads these things cover to cover. So, there's a lot of filler content that will never be cited that got rubber stamped by reviewers taking a five minute glance at the text. As long as the article is not too blatantly bad, nobody cares. A common practice is to invite accepted authors back as reviewers. So, there's a notion of bad articles leading to equally bad reviews.
It is not surprising that there are researchers that game the system by publishing in friendly publications where the reviewers/editors are known friendly. They all need to keep their numbers up.
A possible fix could be to review out in the open. Publish reviews along with the article. Allow scientists to challenge these as well. It's one thing to say "this is probably fine" anonymously than it is to say, please publish this with a message stating that I approve and endorse the content of this article.
Also, journals catre more about important results than correct ones, until they get burned too much. But much of modern science is too hard to disprove, because the claims are too subtle and complicated.
That system essentially can't work unless 1) you make 1st-year grad students do the review and 2) you somehow make people OK with publication delays being extended by months.
If you read papers from like the 1700s you will see stuff like this. Basically you wanted respectable members of the community to vouch for you.
Say it was some experiment with dogs and blood pressure. It would say "My servant Mr. X recorded the readings I read off the dial while Mayor Y restrained the dog and Pastor Z observed it was correct".
Universities are typically more concerned with their professor's abilities to raise funding than their academic integrity. If the numbers look good (money coming in, publications going out) they are not likely to complain or apply much scrutiny. Their assumption here is that peer review should be sufficient. As I was arguing it is not. So that is in fact the core problem. Universities don't care, reviewers don't care, journal editors don't care, publishers don't care. They all benefit from inflated numbers. The system does not work. Cornell is complicit here and part of the problem.
Yes, lower tier (but not scam journals) can be the homes of lower quality work, but most often they are there because 1)it is a journal for a subdiscipline (Child Development, for example), 2)useful but not groundbreaking research (e.g., establishing validity of lab's protocol), or 3)research that was conducted well, but the results were not clear or are complicated.
The latter is what often irks me the most about science: The need for clear results to get in a good journal. The journal quality should be determined not on results but on the quality of methods and research questions. That is why I am a proponent of pre-registration.
I still say cut government funding for research by 90% and watch it get fixed automatically as all the bad actors stop joining (you still probably need to wait for the current ones to die/retire).
And usually then things make more sense. We haven't fully figured out interviewing. Interviewing is hard.
We haven't fully figured out vetting academic papers, perhaps vetting academic papers is hard
Glad to see that the scientists have caught up.
The h-index, the measure used by researchers, is surprisingly gamable on other hand. For example, two researchers can buddy up and self cite each other few hundred publications in junk journals with DOI and will suddenly become the most prominent scientist of all time. It is mind boggling that even Google Scholar doesn’t use pagerank.
Today, they cross validate the search results against DNS queries in a specific region and from their tracking scripts embeded in pages too. They are getting a lot of data from ISP, Wifi access points, their own 8.8.8.8 DNS service, and even their mobile operator project fi.
So, even if you manage to trick the linking part, you'll not pass their other filters.
It's much harder now but still doable provided someone is willing to foot the bill for such an operation.
Papers cannot cite future papers, so you are left with trying to value older papers from a number of newer citations of unknown quality, by essentially counting them. Until enough citations become available, the quality of the new work is usually proxied by the prestige of the journal, which in turn make these papers more wide read, influential and cited, and this perverse power drives the whole train wreck of academic publishing.
There are precious few exceptions to this rule unless they are very old or are Paul Erdos.
I'd much rather hear 'Professor X's most important contribution was Y.'
In any case, if this kind of thing is reported in the popular press, blogs, etc, it's because a scholar caught it and reported it, which means there are checks and balances that don't let this behaviour run wild all the time.
And then again, there's going to be differences between fields- some will suffer from this kind of problem more than others, etc. There's no reason to "tar everyone with the same brush".
If a PI has 3 collaborator PIs and each PI has 4 students writing a paper every three months then the PIs can get 48 publications a year. Over 20 years, there's your O(1000) publications in a model with no substantial PI time contribution.
Given teaching loads, home life, university admin, academic duties like reviews I'd be surprised if a senior academic got 3 days per week to do research. I'd say at that rate the 4 first-author level contribution publications per year model would be optimistic. So a more realistic cap on the quality work someone could do in 20 years might be 80 publications.
I doubt the parent academic had O(1000) first author publications, nor is that mentioned in the article.
But to track back to what was getting me bothered, it's the implication when Professor X is discussed or introduced somewhere that they personally thought up and wrote O(1000) publications.
I know good PIs guide the research activities of their groups as well rather than just creaming off publications.
But while I understand where you're coming from, stepping back and thinking about it 'authorship as a reward for leadership' could be a sign of unhealthiness in academia. Why can't authorship mean authorship and leadership be recognised separately? 'Professor X runs a highly successful lab of N researchers, who produced M papers last year, and manages a budget of Z dollars.'
I actually wonder if journals should start offering a second list of names on the first page, or even three to allow both group-leader and technical-work credit. Has this ever been tried?
I don't know how to square this expectation with my experience, as a PhD student (currently). My own problem with my thesis advisor is that I'm concerned that I don't contribute enough to our joint papers, because he's doing much of the job- most of the ideas are his and he writes at least half of each paper, and codes the odd implementation. And he's been doing that for the last 30 years or so (though not with me, obviously!).
I have heard the rumours- that career academics let their students do the hard work and just put their name on the finished paper. However, that presuposes that PhD students are already capable scientists who can be trusted to write a publisheable paper entirely on their own, even in their first year. I think that anyone who's been through a PhD, or helped guide someone through theirs will know how rare that is. Even just figuring out what an original contribution means in your chosen field can take a long time- unless, that is, you have someone at hand who understands the field, knows the bibliography and can recommend a promising research subject and also methods. At that point, that person has already done 1/3 of the work for you- figured out what you should try to publish. The other two thirds are to do the research and actually write up the paper.
Btw, in the UK were I study, the done thing is that the student's name goes first in any joint papers, while that of the advisor, or in any case, the most experienced member of the research team generally goes last. The advisor will still get citations to their name of course, but so will the people preceding them - and the first author, who is usually the student, will appear as the principal author whenever the names of the researchers are referenced (e.g. in author-year citation formats, or in slides, etc).
I tend to see this as a substantial boost to my own career as a researcher. Maybe even too much of a boost, in a way. I don't like to think I'm riding on someone else's coattails. But, the fact of the matter is that at the start of your research career, inevitably, that's what you are doing.
Academics has been infamously cutthroat for a while now. The iron law is that anything which can be gamed for advantages will be.
The system needs updating so that what is gamed for is closer to the actual desired results. I suspect actual logic being incorporated into the judgements will be required which only makes things fuzzier. It is easier for more concrete fields to fall back on industries or failing that just numbers if it gets too theoretical like say maximizing energy extraction from a solar system.
You can prove that say Kant has fewer followers than Ayn Rand but that says nothing about philosophical merits.
It's rather funny to me that for years, Cornell's biggest professorial scandal was Daryl Bem transitioning from groundbreaking psychology work to ESP studies. It was a much-discussed embarrassment for a while, then he started putting out successful ESP findings with uncommonly-good practice and people started worrying about what else you could prove in the same way.
These days, it looks like Bem may come out rosier than some professors there who were still on the rise while his ESP work was causing outrage. Most of Bem's results are disputed at best, but they were actual contributions which led to further study. His methodology and basic integrity have never been seriously indicted, which puts him leagues beyond Wansink and Sternberg.
With Bem, it's easy enough to talk about his most important 2-3 contributions and analyze their merits. The others look more like they were being paid by the word, and that should have been a warning even before the specific problems were found.
Authorship and citations are basically academic currency, and as such have undergone inflation just like real currency. Professors get their name on a paper as a form of payment for future or past services to other professors (often related to funding). There's just no incentive for anyone to stop this as far as I can't tell, and appealing to ethical conduct in authorship and citations is not enough.
Currencies don't just undergo inflation for no reason. Central banks create that situation on purpose to discourage saving.
(Another, more-natural (Austrian), cause of inflation is a contracting economy so dollars chase fewer goods.)
If understanding/knowledge is being accumulated by a field the new publications should make those that they cite even more valuable, not less like in the case of a fungible currency.
That sounds like people who are not Euler engaging in dubious behavior.
Incentives for reviewing are broken. Currently researchers have to see it as an intrinsically motivated activity for the good of the field, there are no quality checks per se.
But of course, when the double-blind review process is not used, then it makes perfect sense for a reviewer to check the authors' other published work. In journals that require a certain percentage of original work, I don't see any other way.
Or journals could just submit all papers to the plagiarism-detectors used for undergraduate work, like Turnitin.
I have some limited experience in this... they do.
Or have a low level teaching job like that guy who proved a twin primes result a few years aago.
I'm sure there's still unhealthy pressure to publish for grad students, but I had the impression that it was comparatively insulated from the usual patterns and receptive to rare-but-superb work.
(It's a good question, though. How would Temple Grandin or B.F. Skinner fare in psychology today?)
A commenter asks why reviewers don't use plagiarism-detection software. With most everything online (and Sci-Hub) it'd be pretty easy.
Many journals do these days, and won't send a paper for review unless it passes
It will also flag overlap with your own works, but obviously it's not exactly a huge stain on your reputation if your thesis seems to borrow some paragraphs from your own papers.
Unlike scientists in academia, I'm explicitly (and well) paid to do both; it's in my job description. If do a sloppy job reviewing, and let a bug through, I feel the heat because something breaks. And what I work on is likely much simpler than what scientists work on.
Unfortunately it's become less about 'science and knowledge' and more about getting the grants to survive and eventually be able to work on 'real' research.
However I do not believe this is specific to Academia, any 'game' can be hacked (at least for some time).
Perhaps there's something "ironic" about that...
[0] http://steamtraen.blogspot.com/2018/04/some-instances-of-app...
But the article mentions peers might not be getting the works either: " Bobbie Spellman, former editor of the journal Perspectives on Psychological Science, is confident “beyond a reasonable doubt” that Sternberg was not telling the truth when he said that “all papers in Perspectives go out for peer review, including his own introductions and discussions.” Unless, as Spellman puts it, “you believe that ‘peer review’ means asking some folks to read it and then deciding whether or not to take their advice before you approve publication of it.”"
I would not say this applies to most cases.
I have got many papers reviewed, have done many reviews, and have got questions from people doing other reviews, and people usually put quite some effort into it. There are exceptions, but bad reviews are not the norm, and you can usually catch them when it happens and talk with the editor about it (either if you are the author or another reviewer).
Searching for similarities should indeed be part of the job, but it is not always easy, and I agree more work should be done in that aspect.
This may be different in other fields or specific journals but, please, don't generalize.
The interview is really interesting.
This professor has simply optimized an unethical path through the broken incentive system. He's not alone by a long shot.
I don't believe anymore that this stuff is a new phenomenon though. It's more like a group of people trying to discredit university these days.
His main research interests are in intelligence, creativity, wisdom, thinking styles, teaching and learning, love, jealousy, envy, and hate.
Physicists don't assert that they study "physics science", nor do chemists describe their subject as "chemistry science".
It's only a rule of thumb of course. There are exceptions (computer science, neuroscience, ...) but I think it works in a lot of cases: "social science", "management science", "data science", ...
It's a bit like how if a country includes the word "democratic" in its title, it's less likely to be a real democracy.
> Physicists don't assert that they study "physics science", nor do chemists describe their subject as "chemistry science".
Fortunately, most of the people in question study "economics", "psychology", "sociology", "anthropology", and so are exempt from this heuristic.
The technical issues are probably "easy". The ongoing maintenance costs/hassles could be solved. But the human factors would be very tricky to get right: anyone who maintained it would be under a lot of pressure in their decision-making -- your example of whether self-citations should be filtered is a good example of a decision which would have big effects and would be highly political.
One good thing about Google Scholar is (was?) that we didn't worry that Google engineers would tweak it to benefit Google researchers.