Hacker News Reader

Comment by ralusek | Hacker News Reader

For those interested in this, take a look at the work regarding "grievance studies" by Peter Boghossian, James Lindsay, and Helen Pluckrose. They made intentionally unscientific studies with the intent of publication, and actually got themselves published on a few of their works. The objective was to demonstrate that not only does academia in the social sciences operate according to a completely different standard than the hard sciences, but is so ideologically driven that they behave in much the same way as a religion might.

View on HN

I think the main takeaway from the Sokal Squared hoax was that reviewers in this field are either a) extremely biased and approve any results that agree with their beliefs, or b) anti-science, or c) scientifically illiterate, because they couldn't even spot the painfully clear methodological problems in the hoax studies that were inserted on purpose to test exactly the quality of review.

Those aren’t mutually exclusive

They would have got a lot more published except their experiment was sprung by some reporter.

The social sciences (along with most of the humanities) appear to be beyond saving at this point and it might be a good idea to make a hard break with the past and start again with people trained outside the field.

Their "study" reminds me of Penn & Teller getting people to sign a petition banning dihydrogen monoxide (aka water).

https://www.youtube.com/watch?v=yi3erdgVVTw

Just another form of social engineering. Haha, we compiled footage from a bunch of randos to show how stupid people. Haha, aren't we smart!

It's the academic equivalent of tricking someone into walking into a punch.

People (perhaps you too) seem to have the idea that only social sciences are vulnerable to publishing stings like the one Boghossian and his friends pulled off, but that's not true[0][3]. Next, all the places the "hoax" papers were [mostly] submitted to were of low prestige (i.e poor ranked)[1]. The paper they wrote as a summary of their conclusions does not even rigorously define what field(s) in the social sciences they have issues with, instead they collect it under the nebulous term "grievance studies" - which is a term they made up themselves to deride academics (so much for the scientific spirit of camaraderie[4]).

The "hoax" they pulled off doesn't seem to be showing what they say it does, at least not to the degree they suggest; from here[2]:

>Let’s analyze the hoax a bit more carefully. The team wrote up 21 bogus papers altogether. (The essay starts by saying there were only 20; according to Lindsay, that’s because two of the papers were largely similar to one another.) Of those 21, two-thirds never were accepted for publication. The Areo essay dwells on several papers that had been rejected outright, including one suggesting that white students should be enchained for the sake of pedagogy, and another proposing that self-pleasure could be a form of violence against women. They take it as a sign of intellectual decay that such papers managed to elicit respectful feedback from reviewers, even short of publication.

Academics warn against doing what Boghossian and friends did for their own good[5].

>The hoax was cruel; it sought to discredit targeted journals by setting a trap that exploited the scholarly predispositions of their editors and reviewers. Moreover, because of the anonymous review system, once the tricksters revealed their intent, only the editors whose names appear on the journal’s masthead suffered the sting of adverse public scrutiny. More finessed responses by the disgruntled threesome would have employed tactics such as persuasion, insight, engagement with the actual scholarship, and good sportsmanship (Bergstrom, 2018).[6]

Most importantly, the researchers didn't include a control group for their study. How can they claim to be outing "bad science" when their own methodology is so poor and fails to prove anything other than anecdota and suspicion?

One of the reviewers subject to the hoax wrote:

>"Anyways, I guess I could be more critical in the future, but I assumed a grad student had written a confusing paper and I tried to be constructive. I'm embarrassed that I took it as seriously as I did, I'm annoyed I wasted time writing a review, and I'm glad I rejected it."[8]

P.Z. Meyers put the situation best[7]:

>If you can find a bad article accepted for publication in a feminist journal, please do jump on it and tear it apart. That contributes to the strength of the discipline. Don’t write a bunch of bad articles of your own, which are clearly intended only to weaken the whole discipline and provide a set of easy, straw-man arguments that you can use to pretend you’re a smart guy.

And for the icing on the cake: Sokal himself isn't all that impressed with Boghossian's efforts[9]. A good thread discussing the hoax is on the social sciences subreddit[10].

[0] https://www.newscientist.com/article/dn17288-crap-paper-acce...

[1] https://i.redd.it/qsi6i5rbv3q11.png

[2] https://slate.com/technology/2018/10/grievance-studies-hoax-...

[3] https://platofootnote.wordpress.com/2017/05/24/an-embarrassi...

[4] https://www.3quarksdaily.com/3quarksdaily/2018/10/bad-argume...

[5] https://www.sciencedirect.com/science/article/abs/pii/S03783...

[6] https://journals.sagepub.com/doi/full/10.1177/14733250198338...

[7] https://freethoughtblogs.com/pharyngula/2018/10/03/give-it-a...

[8] https://twitter.com/dwschieber/status/1047497301021798400

[9] https://www.chronicle.com/article/What-the-Conceptual/240344

[10] https://www.reddit.com/r/AskSocialScience/comments/9noxmp/is...

It did prove that the only people capable of pointing out there were problems with the hoax papers are people generally critical of the social sciences. That alone demonstrates a problem.

That's why the experiment ended early.

PZ Meyers is a doxing buffoon.

How is that true when a good proportion of their papers were rejected, and they kept trying in other journals until they were accepted? What about the fact that several reviewers wrote critical responses and suggestions for improvement? Why can't what you said also be said about hoaxes in the "hard sciences" like this[0]? HN user voidhorse has a good comment about their tactics[1] (quote):

> Sure, the criticism this hoax is trying to demonstrate may be legitimate, but the methodology is one designed to highlight the cleverness of its executors and diminish the credibility of a discipline, rather than point out constructive areas for improvement. Basically, it is a methodology that does not treat its targets as intellectual equals and is quite indecorous—you get the sense that a major point of this operation is to discredit the field and make its practitioners feel some kind of public humiliation or shame. A childish tactic.

[0] https://science.sciencemag.org/content/342/6154/60.full

[1] https://news.ycombinator.com/item?id=18899529

All papers get critical responses and suggestions for improvement. That proves nothing other than the submitted papers were considered to have some merit, and still possible inclusion in the various issues.

The problem is people (a journalist and RealPeerReview on twitter initially) identified quite a few of the hoax papers. And that does suggest there is a bubble within the field that has departed from usefulness and reality.

There is a problem within social sciences. Whatever is happening in hard science journals doesn't change that fact.

My claim was never that social science is immune to the replication crisis or that it could be more accepting of outside criticism; my claim was that singling out "grievance studies" (what the authors take to be sub-branches of critical theory) using poor research methods to make a name for oneself, childish tactics, not getting IRB approval isn't the right way to go about solving such a problem, and that using this as an example doesn't stand up to the large number of hoaxes/stings in hard sciences (see the first couple of links in my original comment). What suggests that the way to go about addressing this bubble (which can and does exist to at least some degree) is to publish more junk papers?

For a critical review of the papers' content, there's reason to believe that the hoax articles' premises may not be totally without merit[0].

"When Boghossian et al. describe their papers to us, the public, they do not explain what their bad arguments are, they only describe the “absurd” conclusions of the papers. So if the hoax is all about peer reviewers accepting bad arguments, then Boghossian et al. are failing to present the proper evidence, and propagating confusion about their own hoax."

[0] https://thingofthings.wordpress.com/2018/10/10/on-sokal-squa...

I think you are arguing from bad faith. Social Sciences has problems and it seems unwilling to do better.

You had to be shamed publicly and widely. People have a right to know social sciences are this broken.

Perhaps, then, the authors should have published a critical review of existing social science literature and discussed flaws in methodology. That would be academically and ethically honest, and probably prompt a discussion and response from inside the field itself. Instead, they ran with their article to online news sites. Why would they do that first rather than trying to engage the academia themselves? Why did the researchers need to be shamed? See the quote I provided in my original comment by one of the reviewers who was fooled. She was personally embarrassed and thought the paper had come from a new entrant in the field, so she was charitable. Why is it acceptable that making people feel that way is the first course of action? It is not at all in the scientific spirit.

People have a right to know, of course - but so do researchers. Researchers in these fields have a right to know exactly what Boghossian and friends thought was wrong with their methodologies and research topics. Boghossian decided that was beneath him.

Because the Medknow journals are notorious scam journals that will publish absolutely anything at all, without even a semblance of peer review?

There's a difference between non-prestige journal and vanity press.

>> Most importantly, the researchers didn't include a control group for their study.

I'm not sure what qualifies as a "good methodology"?

Reply on news.ycombinator.com