What an Audacious Hoax Reveals About Academia | Hacker News Reader

What an Audacious Hoax Reveals About Academia

theatlantic.com

81 pointszwieback79 comments

https://news.ycombinator.com/item?id=18134681

And previous: https://news.ycombinator.com/item?id=18127811

Thanks!

This article doesn’t mention the most ridiculous example: Affilia, a women’s studies journal, published a rewritten chapter from “Mein Kampf.”

https://www.google.com/amp/s/www.wsj.com/amp/articles/fake-n...

I think the most ridiculous example was the one where they advocated for "privileged students" (ie: white men) to sit on the floor chained in classes and not be allowed to speak, so that they can "experience reparations":

> I wonder if that ‘progressive stack’ in the news could be written into a paper that says white males in college shouldn’t be allowed to speak in class (or have their emails answered by the instructor), and, for good measure, be asked to sit in the floor in chains so they can ‘experience reparations. That was our “Progressive Stack” paper. The answer seems to be yes, and feminist philosophy titan Hypatia has been surprisingly warm to it.

That was mentioned in the Atlantic article, though they didn't mention that the journal rejected it conditionally because the proposed "reparations" of sitting chained on the floor and being spoken over weren't harsh enough for privileged students -- the author was offered acceptance upon revision.

Geez.

Source?

Not proof, but more commentary:

https://quillette.com/2018/10/01/the-grievance-studies-scand...

Poe's law in action?

> Poe's law is an adage of Internet culture stating that, without a clear indicator of the author's intent, it is impossible to create a parody of extreme views so obviously exaggerated that it cannot be mistaken by some readers for a sincere expression of the parodied views.

Where can I find this? Sounds like it would be funny to read.

Accepted, but not published, according to https://areomagazine.com/2018/10/02/academic-grievance-studi...

I haven't read Mine Kampf and I don't like Nazis or Socialism for that matter. But are you certain that every single chapter of that book is 100% genocidal rubbish? Because I'm guessing that out of the ~15 or so chapters there is probably one that is palatable.

The German's didn't go in to WWII because they were just terrible people. They had a very long list of quite justifiable grievances after WWI. If they hadn't been proud supporters of eugenics they'd be just another entry in a long list of bloodthirsty European states.

EDIT So here is a selected paragraph (from 3 random candidates) from said text. It isn't exactly anti-factual.

"Thus, from a certain vanity, which is always a cousin of stupidity, the great mass of politicians will keep far removed from all really weighty plans for the future, in order not to lose the momentary sympathy of the great mob. The success and significance of such a politician lie then exclusively in the present, and do not exist for posterity. But small minds are little troubled by this; they are content."

There are WAY too many compelling thinkers in the world to read to justify any of my time sifting through Hitler's writings hoping to find some nuggets that are not "100% genocidal rubbish".

I'm imagine he advocated for exercise and a good diet at some point. Why we need to try to find something useful the man said is utterly beyond me.

The claim is that a journal published a rewritten chapter of Mein Kampf and that journal is therefore bad.

I don't think that is a reasonable argument. It depends on what the chapter was talking about. If it made a big point about Jewish conspiracies and keeping the American people ethnically pure then that is a problem. If it was 1,000 words complaining about vision-less politicians then what is the issue?

If we are talking in the context of submitting it as a paper to a journal, then a problem is plagiarism. If we are talking about "does hitler have some non-genocidal ideas?" then like I said, not sure I care. "How was hitler so persuasive?" Interesting. "How did hitler outmanuever his opposition domestic and foreign" . Interesting.

But yes you have a fair point that just because Hilter said X doesn't automatically invalidate X. The old "the nazis drank milk so dont drink milk" fallacy

Not surprising that these kind of things happen in social studies and it's easy to make fun of.

However, lots of junk is also published in science and engineering, it's just not as egregious and apparent.

Indeed, "audacious" is not word I'd use for these kind of games, "lazy", "shooting fish in a barrel" would be closer.

Also, that junk is not as likely to trickle down to the news and the broader population.

yeah, it's not limited to social sciences, but it gets more media attention when it is because it feeds the narrative that the humanities are useless: https://www.nature.com/news/publishers-withdraw-more-than-12...

Engineering and science have an advantage, however, in that they are empirical and readily tested. Things evaluate to true or false, even if doing the evaluation is difficult. In the humanities, things often come down to value judgements and mere assertions, which cannot be validated one way or another.

not to mention a ridiculously high percentage of humanities papers are never cited once. Never. Cited. Once. http://blogs.lse.ac.uk/impactofsocialsciences/2014/04/23/aca... Who can see this and not admit the field is in crisis?

Depends on what you mean by crisis. If you have a student that really needs to graduate soon, this condition is a boon.

At least there's a standard for junk science -- it replicates or it doesn't.

it obviously skips the third option, which is the standard in most fields: nobody ever tries to replicate it.

Plenty of people try to replicate, in trying to build on work from other labs. What they don’t do is publish when replication fails, but the failure to replicate usually becomes known in the community.

If people don't publish failures to replicate, how do you know how frequently replication failures become known in the community?

The same way you know that you walked out of your front door this morning without it having to be published in a scientific journal.

To be clear, my question is how you know how many failed replications there are that you never find out about.

The 2nd part of academia that isn't being talked about here is citations. Getting published can be huge, but if your paper is actually garbage no one is going to cite it. That's a far better measure of the impact of a paper - who's using the data in it.

Witness this classic complaint from Peter Elias: https://oikosjournal.files.wordpress.com/2011/09/elias1958ir...

published in 1958 when he was editor of the IRE (now IEEE) Transactions on Information Theory.

Thanks, that's gold, apparently things don't change as much or fast as we think they do.

Why do you say "social studies" when the sociology journals didn't fall for it?

Yeah, I didn't quite know what umbrella term to use for the fields of study they hoaxed.

> Yeah, I didn't quite know what umbrella term to use for the fields of study they hoaxed.

Gender and Identity Studies?

There are much more egregious examples in computer science publishing. A paper that just repeated the words "get me off your fucking mailing list" over and over [1] was published in a journal. I also remember there being some automatic paper generators that resulted in several publications several years back. This is 100% an issue with bogus for-profit publishing, not with any particular area of study.

[1] http://www.scs.stanford.edu/~dm/home/papers/remove.pdf

Figure 1 really drives the point home

Not a journal of anybody cares about in any way whatsoever, just a scam journal that attempts to extract fees from desperate to publish researchers.

Which is why they were spamming people.

The same is true of the journals that published the hoax papers mentioned in the article.

I find it phenomenally interesting that people simply say what they want to be true with no consideration for whether it is or not. We are indeed moving into a post-fact world aren't we? [1] Here is the wiki page on the 'get me off your mailing list' paper. Here [2] is the page on the journal that chose to accept (after revisions) the Mein Kampf paper. You might notice certain differences. The other journals that accepted the fake papers are part of the T&F/Springer publishing networks, and some responded in this [3] article. They were not scam journals by any means of the word, just journals blinded by ideological bias.

[1] - https://en.wikipedia.org/wiki/International_Journal_of_Advan...

[2] - https://en.wikipedia.org/wiki/Affilia

[3] - https://reason.com/blog/2018/10/03/dog-rape-hoax-papers-pluc...

It looks like I may have been misinformed or working off of old information. I'll look into this some more, thanks.

The difference was that this was not a reputable publication, but a predatory scam.

https://www.theguardian.com/australia-news/2014/nov/25/journ...

These were "real" publication, with peer- and/or editorial-review. Not the same thing by a long shot.

It's my understanding that the hoax papers in the article were also published by predatory scam journals. Even the "get me off your fucking mailing list" paper went through peer review.

> “They told me to add some more recent references and do a bit of reformatting,” he said.

> It's my understanding that the hoax papers in the article were also published by predatory scam journals.

If the hoax papers were published in "predatory scam journals," they're engaged in an entirely different type of predatory scam than the "get me off your fucking mailing list" computer science journal. Sort of like the difference between fake news and herbal Viagra spam/scams.

> Even the "get me off your fucking mailing list" paper went through peer review.

>> “They told me to add some more recent references and do a bit of reformatting,” he said.

Obviously not any peer review process that deserves the name. That must have been an entirely automatic (or even automated) bit of feedback.

What makes the journals the hoax papers were published in different?

That’s not a fair comparison.

That’s not a peer reviewed journal — it’s a publication that will publish anything for a fee.

All of the papers in the original article are peer reviewed.

The papers in the article were published in journals that will publish anything for a fee. The "get me off your fucking mailing list" paper was also peer reviewed. The reviewer asked for formatting changes and citations, very similar to the reviews that these other hoax papers got.

No, that’s completely incorrect. The papers in the article above were published in top (or close to top) journals in their respective fields.

I have never seen someone claim that before. Can you back that up?

"Gender, Place and Culture" ( https://www.tandfonline.com/toc/cgpc20/current ) has an impact factor of 1.594. No idea whether that's enough to make it a top journal in feminist geography or not.

The retraction notice for the "Dog Park" paper is currently the most recent publication on that site.

That certainly contradicts what I've heard about these papers before. Gender, Place and Culture does seem to be a top journal. Thanks.

We approached this new effort by asking two central questions: Are we correct in our claim that highly regarded peer-reviewed journals in gender studies and related fields will publish obvious hoaxes? (...) And, if not, what will they publish?

We set out with three basic rules: (1) we’ll focus almost exclusively upon ranked peer-reviewed journals in the field, the higher the better and at the top of their subdisciplines whenever possible; (2) we will not pay to publish any paper (...)

https://areomagazine.com/2018/10/02/academic-grievance-studi...

The back story I heard is that the authors submitted to the top journals, were rejected, and kept going downward in perceived impact until the article was accepted. That is, that most of the articles (with perhaps one exception?) did not appear in the close-to-top journals. And they had many papers in the works at one time, and some of them did not get in at all despite the above "just-good-enough" optimization technique.

Although I kind of enjoyed the original Sokal paper - I was a subscriber to Lingua Franca, where Sokal announced the hoax in the first place - I think the message of this iteration is more that: "if you work hard to game the system, you can indeed find a hole and game the system". This finding is not that interesting.

---

Specifically (https://www.chronicle.com/article/Sokal-Squared-Is-Huge/2447...):

"Three scholars [...] spent 10 months writing 20 hoax papers [...] and submitted them to "the best journals in the relevant fields." Of the 20, seven papers were accepted, four were published online, and three were in process when the authors [stopped].

The real meat depends on the publisher. There are always publication mills that accept basically anything or push objectively wrong agendas like AIDS denialism. It is when the bellwethers publish absolute crap that it is a sign of true rot in the field and not just a publication.

A common response to this thread is "the review process must not have looked closely enough".

This tweet[1] contains screenshots of the review process of the dog park paper, and claims they received constructive feedback. As originally written, the journal wouldn't publish it.

https://twitter.com/RealPeerReview/status/104781375318043852...

Always fun to see that the Sokal Affair is reproducible science :). (https://en.wikipedia.org/wiki/Sokal_affair)

Edit: Hah — interestingly enough, The Atlantic refers to this as "the new Sokal Hoax" in the <title> tag and the URL, but as "an Audacious Hoax" in the page proper.

Other means superior to the natural sciences exist to extract alternative knowledges about stars and enriching astronomy ... incorporation of mythological narratives and modern feminist analysis of them, feminist interpretative dance (especially with regard to the movements of the stars and their astrological significance), and direct application of feminist and postcolonial discourses

I cracked here.

I think there are a number of variables that should be controlled for on probably separate studies.

1) how likely can renowned academic get published with gibberish

2) compare #1 to how likely can an unknown get published with gibberish.

3) how likely can an unknown get a serious paper published when it was actually the renowned academic who wrote it.

4) compare #3 to how likely a renowned academic can get a serious paper published when it was actually the unknown / lesser known peer who wrote it.

Journals are broken. A lot of people recognize that.

Peer review is also broken. A lot of people recognize that.

The fact that we don't publish negative results is a tragedy. We're losing so much collective knowledge that way. We learn by failing, but somehow we're ashamed to fail in public in academia.

I personally hope that arXiv and things like it keep taking off. And that we end up funding respected people to review articles publicly. And that we end up funding people to try to replicate interesting results, and publish their findings.

And that the news media learns to explain to people what p scores are, and what p-hacking is, and what the half-life of knowledge is, and that one new research article with p=0.05 doesn't mean jack squat. Basically the news media shouldn't cover leading research, it's just too noisy and it's difficult to understand the context and relevance of new research.

I think that's a good summary. My favorite relevant quote from Dr. John Ioannidis:

> “Science is a noble endeavor, but it’s also a low-yield endeavor,” he says. “I’m not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life. We should be very comfortable with that fact.”

https://www.theatlantic.com/magazine/archive/2010/11/lies-da...

Social Sciences are the quintessential BS job generator. It serves a purpose but not the one it thinks it does.

> It serves a purpose but not the one it thinks it does.

Pushing political agendas and keeping forever-students employed?

Somewhere around that. There is an "Angst" in society, deep inside we all know that there isn't a sunny place for everyone, while in the surface we pretend we can do it like the Baby Boomers. Making people SJWing around dog rape is a dummy load that keep them from demanding structural changes.

I am slightly cautious that this hoax is itself a hoax and no actual hoax papers were published by actual academic journals.

Unless you don't consider "Gender, Place & Culture" an actual academic journal, it did happen:

https://www.tandfonline.com/doi/abs/10.1080/0966369X.2018.14...

https://en.wikipedia.org/wiki/Gender,_Place_%26_Culture

Just as a comparison for folks - Gender, Place and Culture's impact factor is about 1.

Nature, one of the most impactful, is 40.

https://en.wikipedia.org/wiki/Impact_factor

> I am slightly cautious that this hoax is itself a hoax and no actual hoax papers were published by actual academic journals.

This is apparently the original account of the hoax. They had to call it off prematurely because they started getting media attention from places like the Wall Street Journal. I think the hoax is real.

I think anyone that was triggered by this experiment has self-identified themselves as part of the problem.

Let us see some absolute gibberish paper get published in a solid SCIENCE journal (e.g. Impact Factor > 5) before we all go "PEER REVIEW IS BROKEN ARGHH!!"

Relevant xkcd: https://xkcd.com/451/

I think there is a real disconnect between what peer review is and what the public seems to think peer review is.

The public seems to think that peer review indicates that something is true, but that's clearly not the case because we have no way of verifying any of the data used, etc.

The paper that got published in a top journal "Human Reaction to Rape Culture and Queer Performativity at Urban Dog Parks in Portland, Oregon", while silly, doesn't seem like it has to be rejected if it were an honest paper that actually did the experiments they said they did. It seems no more silly than the lobster nonsense Jordan Peterson peddles and conservatives eat up. I don't personally think it's a useful paper, but most individual papers rarely are, even in CS.

You mean this paper by E. A. Kravitz at Harvard Medical School Department of Neurobiology that was published in the Journal of Comparative Physiology in 1999?

Serotonin and aggression: insights gained from a lobster model system and speculations on the role of amine neurons in a complex behavior [pdf]

https://caspar.bgsu.edu/~huberlab/private/KingSolPub.pdf

I wonder if many of the people so acidly contemptuous of Peterson have ever bothered to actually read what he actually writes, or just viscerally react to the hot take media coverage. He's eminently reasonable, not an alt-right troll like Alex Jones or Ben Shapiro.

Peterson was a Harvard professor from 1993–1998 so he was there at the time this paper was being written. Now that he's at Toronto, I'm curious to know how much interaction he has with Geoffrey Hinton. He's made references to research related to the AI embodiment problem, and one of his business partners is Daniel Higgins who was at MIT.

I see him as more of a Ben Carson figure personally (a legitimate expert in his field that often answers questions about things he knows nothing about), but I agree that it's a little disappointing when his critics make fun of something where he's actually knowledgable.

>It seems no more silly than the lobster nonsense Jordan Peterson peddles and conservatives eat up

Actually that's based on real science. It's not social studies...

The methodology section of the article says that one person spent almost an entire year sitting in a dog park for up to seven hours a day (for a total of 1,000 hours). During this time, that person allegedly inspected up to 10,000 dog genitalia, or approximately one every six minutes. Even if the alleged institution was real (which it wasn't - they didn't even put up a website until after the article was submitted), the author had valid credentials (which they didn't), or someone knew of them personally (academia is a small world) there were enough red flags that the reviewers should have questioned the article's validity.

> Even if the alleged institution was real (which it wasn't - they didn't even put up a website until after the article was submitted), the author had valid credentials (which they didn't), or someone knew of them personally (academia is a small world)

Not sure about these journals, but a lot of peer review is blind, so the reviewers do not have access to any of this information intentionally.

Is there a full list of journals that published these?

This sort of hoax seems to happen every few years, but often to journals that will explicitly accept anything and let anyone who wants to give a talk, so long as it's related to the journal's mission (which means there's nothing alarmist about it).

Reply on news.ycombinator.com