The Problem of Publication-Pollution Denialism

Nobody with much sense thinks that a degree from a for-profit "school" lends credibility, but they can nevertheless be used to further a career.

People who are familiar with the field don't care about these journals. The problem is that poorly fact-checked popular media are more than happy to cite these journals in click-bait articles touting "A Miraculous Cure for Pancreatic Cancer", which are widely disseminated to readers who don't have the scientific background to evaluate them rigorously. They see that the research was published in The Foobar Journal of Crypto-Oncology, and assume that it's reliable.

And if no reputable journal has published a cure for pancreatic cancer yet, the junk-journal articles will be at the top of the search results.

We could capture a network of trust among scientists, where individual scientists vet other scientists and articles. Think of it as ScienceRank, a PageRank where the nodes are individual scientists and the individual articles they publish, and the links are publish, review, reproduce and consistent-with events:

    - scientist A published article X
    - scientist B gave a positive peer review of article X
    - scientist B gave a positive peer review of scientist A
    - scientist B gave a negative peer review of article X
    - scientist B gave a negative peer review of scientist A
    - scientist C independently reproduced the experiment in article X
    - scientist C failed to independently reproduce the experiment in article X
    - article Y is consistent with article X
    - article Y is inconsistent with article X

Technology could be a multiplying force in the positive direction instead.

But with the democratization of scientific communication in the Internet Age, as well as the massive over-production of people with PhDs in the sciences relative to the number of academic jobs available, the number of accessibility of bogus publications has skyrocketed at the same time as the pressure to use them has increased enormously. Meanwhile, the amount of good science has stayed pretty much constant, so the good/total ratio is dropping precipitously.

So while insider information is still effective for us, it's much less so for others, and in particular the popular science press <em>and the majority of the people who read it</em> are interested entirely in page views on the one hand and being wooed by the "next amazing breakthrough!" on the other, rather than anything to do with actual science.

There was a comic someplace (should be xkcd but doesn't seem to be) saying of the average reader if "IFLScience" and similar sites, "You don't love science, you just want to look at its ass as it walks by." This is not the worst of all possible worlds (in other places they want science to never show its face in public, much less its ass) but it does create an environment where science communication is hard, and the proliferation of bogus journals and people willing to publish in them is problematic.

It wouldn't hurt for hiring committees to have an explicit rule that anyone who has published in a problematic journal has that publication counted against them, but mostly there needs to be some better form of triage than conventional peer review (which is weak enough as it is) or we're going to drown in papers debating the number of angels than can dance on the head of a pin, and lose that one weird trick that discovered by a post-doctoral mom that actually solves an interesting problem.

The purpose of peer reviewed journals is to disseminate scientific research that's been vetted so that readers don't have to attempt to reproduce the results to have some comfort level with the paper. That's not to say that published articles in quality journals shouldn't be approached with some degree of skepticism, but the sheer volume of science publishing precludes combing through the data of every article you read.

High quality papers will include enough information to reproduce any results referenced in the paper - video is no substitute for actually running the experiment.

These are places that publish papers without even reading the title. You could submit obvious nonsense—randomly generated gibberish—and get published. You can't do that in respected venues in most fields, especially in science. But for people outside of the field, it's really hard to tell which venues are legitimate and which ones aren't, and incentives are misaligned so that quantity of publishing matters far more than anything else.

Further, I wouldn't single-out academics when there are corporate funded studies which cherry-pick results and data to support their products/services.

The range of means by which that can be done is huge (though considerably less than "anything goes") and there is a definite role for work that is exploratory and speculative, up to and including stuff that is almost certainly wrong but worth publishing because a) the error is not obvious and b) publishing creates the opportunity for others to respond to it, hopefully putting the error to rest for good and all. Some of the early work on the "no cloning" theorem was motivated by publications of this type (it turns out if you could clone a quantum state you could use entanglement to communicate faster than light, and there was a series of papers in Physics Letters in the late '80's proposing to do just that.)

So trusting in the self-correcting ability of the discipline of science is fundamental to its progress, and therefore insisting on some alchemical standard of "proof" as the goal for publication would fatally cripple the scientific enterprise. For science to work we have to be tolerant of the publication of error.

But we need to keep the rate of erroneous publications down to a manageable level. Peer review and society membership were ways of doing this in the past. They have broken down today, and we are still casting about for new ways to keep the error rate well above zero, but not so high as to swamp everything else.