We tried to publish a replication of a Science paper in Science
slate.com
slate.com
I don't think there is any case of scientific malpractice here. Keep in mind a high-profile journal like Science is extremely competitive and receives many paper submissions. They have to filter what they choose to publish based on some reason. And it seems to me their reason is perfectly valid.
Now this is turning into a bigger dispute because the researchers started arguing with the Associate Editor on Twitter (I heard the verb "harass".) So he blocked them on Twitter. And this is making them more upset. Etc.
Could you cite?
Whatever replication come refutation there may have been seems to have eluded not only society, but also scientists at large. I do believe both her and Science that the study was refuted at some point, but if nobody knows this and people keep assuming the study is meaningful - then this refutation is completely meaningless. Science as a journal brings the eyeballs and thus has, in my opinion, an obligation to publish data indicating that extremely influential studies they have previously published are likely flawed.
This is quite true, but I'd be wary of what you justify based on this fact alone. Discounting all of the interpersonal politics that goes into paper acceptance, the overall editorial policy of "high-profile" journals like this has a considerable effect on the scientific climate.
It would be unfairly cynical (IMHO) to say that the editorial policies will be geared toward whatever the publisher expects to be the most profitable (even in the long term), but on the other hand it would be naïve to expect that they're even trying to do their best for the expansion of human knowledge and the betterment of society.
What I'm getting at is that choosing what to include in Science et al. is a subtle but far-reaching tool, and maybe one better put in the hands of someone at least somewhat impartial, even if it's just some simple popularity contest counting votes on arXiv or similar.
The authors of the repudiation article are, without much doubt, trying to pick a fight.
However, the larger issue is whether they should be. The article has some good points in it, the comments here on HN also pull out some good point (and some bad ones), and the 'replication crisis' has been raging for almost a decade now with any sign of slacking. The authors are being jerks, I feel, but dammit, they make a good set of points.
Look, I've been there too. The deadlines just keep piling up, the students stay just as needy, and the grant funding is just getting harder and harder to find (esp for new PIs). A bunch of older white guys trying to be asshats on purpose over in the corner and then pointing things out for political points (my opinion) is just like: Dude, really?, screw right the hell off.
Its just another thing on the giant pile of crap that you can't deal with at all.
But, we all know that US based 'Big-S/R1' Science is not a healthy place. It's never really been one, but we all know it now, and, to me at least, it feels like it's not getting any healthier anytime soon. The scathing blog posts keep coming, the #MeToo issues keep getting swept aside, the flagrant abuse of grad students is getting worse.
Writing these guys off is not healthy and is not going to help. Yeah, they're kinda jerks. But we have to listen to them and take them just as seriously as anyone else. Their points are valid. They are right to call out Science magazine for not publishing recalcitrant papers and relegating them to journals that often have insane paywalls.
Yeah, I know, Science gets enough papers by 9am January 1st to run for the whole year and never see a drop in quality or impact. I've heard that exact same phrase from every journal editor, it's like 'live, laugh, love', it's 'basic' now. And it's a crap excuse.
Trying to shoe horn these jerks as 'alt right' or 'political' is not helping. Honestly, breaking Science up so that it would be able to publish these smaller repudiation articles would be better, and we all know the older PIs wouldn't let that happen, as their egos are tied up in the metrics too.
Brushing these jerks off isn't the right thing to do. Science, as a R1 type thingy, is heading for a meltdown in the US. We need to take these jerk-bags seriously, along with all the abused grad students, the new PIs, the people of color, the women, etc.
They could probably run a page or two of capsule summaries of such non-replication papers, with full content on their website.
The authors argue yes. I'm not sure myself -- what is a refutation vs. a different perspective/interpretation or other addition and where do you draw a line?
But it's intriguing to think if this would be a meaningful force for better results -- and could be an incentive to replicate findings you think might not hold up, just from being in a prestigious journal.
It is important that we call on such publications and demand an explanation. If you or your organization is a paying subscriber, then demand to know their policy on this matter and let them know what you think. Otherwise they have no accountability.
Because the journal already made that decision, and that is their right.
> Then they have an obligation to at least facilitate peer-review
Reviewers work for free and are an extremely scarce commodity. Review of a work like this is effectively expending tens of thousands of dollars of people's time. It isn't something we can just indulge in for the fun of it.
Besides that, you might not appreciate exactly how much chaos would break out and how much it would undermine the academic publishing system if any paper rejected had to be reviewed just because the submitter demanded it. The system simply wouldn't work at all.
There are a lot of journals in the world and if the author's work is good, they should be able to get a different journal to publish it.
Further follow-ups need not be published in the same journal, but the first correct/convincing refutation of a published article definitely should be.
The authors are silly to think otherwise. It is also not surprising that Science published a flashy article without a lot of oomph behind it. It's what they do.
The field of study has nothing to do with it. If a journal publishes a result, they are ethically obligated to publish further work that refutes that result.
Too general. Few in science would accept this. The claim here is the weaker one that journals are ethically obliged to accept failed replication studies - and even that is non-obvious to me. (Should Science also be obliged to accept _successful_ replications? No? So it should accept papers on the basis solely of the results? Hello, publication bias.)
Rather than publish the entire paper, they could publish the single sentence:
> Issue 55, Page 8, the paper we published has been replicated with more data, and results do not support the original research conclusions. See paper entitled "Foo" published in "other journal" for details.
> If a journal publishes a result, they are ethically obligated to publish further work that refutes that result.
This isn't a violation of scientific ethics and I think its a silly argument to make. Journals aren't the parents of research results they publish and somehow responsible for all counter arguments or competing data points.
"Drinking goats milk gives X-ray vision𝆯, according to new study!"
...
𝆯 - Accuracy to be followed up in paragraph 42, subsection 3, in a subsequent journal from a third party.
So, the flashy headline gets used by "whoever" wants clicks (etc), but the follow up details are unlikely to be seen.
Sounds like a literal plan for knowingly spreading misinformation.
Side note - Anyknow know how to escape an asterisk here on HN, so it displays instead of starting an italic block? Only workaround I've found is to use 𝆯 instead, which isn't quite right. :/
Science/Nature/etc are broadcast journals. They're for publishing results that people outside your field would find interesting. Typically these are new and flashy results and sometimes theyre slightly suspect. Mostly theyre not. A refutation or null result typically isn't interesting to the audience of these journals- nor should it be.
This bit I don't really understand.
If someone is interested in X, why would they be ok with carrying around what turns out to (likely be) incorrect knowledge about it?
That approach is more equated with (say) Wired, who are famously inaccurate in many details "for a good story". eg Wired is entertainment.
That's not what Science (the journal) is supposed to be about though. Or is it?
Lots and lots of people outside the field continue read about this and remember it. Many millions of people. I remember when I first heard about it. I’m not in the field.
A notice in Science is at least noteworthy to that same public, even if it still won’t get a fraction of the same coverage. At least people who tried to track down the original research would find the follow up study as well.
Most of the arguments here (and in the slate article) have been talking about Science shirking their scientific duty. They are not.
Perhaps you can argue that they owe it to the quality of public discourse to print this article, but they certainly don't owe it to the scientific discourse.
For you to dismiss the fake news that this Science journal article generated, combined with the justifications of ignoring or dismissing arguments by all conservatives as essentially a scientifically proven mental disability is astounding.
I'm not dismissing anything here except the argument that a journal is required to print all counter research to all their published articles
> A refutation or null result typically isn't interesting to the audience of these journals- nor should it be.
In the context I don’t see how to read this as anything other than denying that it matters that millions of people believe this to be true. You literally say they shouldn’t be interested.
That context is one of a professional scientist who understands the caveats of new (i.e. not improved/refined) research being published in a journal like Science. That is the audience that Science's editors select their articles for.
The fact that (some of) these articles get gobbled up by uncritical media is a difficult problem to address. I doubt printing this refuting article in Science as opposed to FieldJournalA would yield much improvement to the situation.
I think you're wrong. The uncritical media does read Science but it won't ever read special journals. Thus, the wider public will be exposed to original bad research, but will never be exposed to the correction. The fact that people in the field who are specifically interested in this specific topic of research (all 25 or so of them) would know the truth is nice, but nobody would even think to seek out these people and ask for their opinion, when everybody already knows the "truth" (which happens to be false).
How do we know that? I mean, we know that science gets things wrong sometimes, and that's fine because it is usually correcting itself. But this knowledge relies on the fact that if group X does bad research, there's always groups Y and Z that would try to replicate and report if it failed. However, if you get your knowledge from publications like Science, then you'd only read about group X but never about Y and Z. So what would you knowledge that "most flashy results are not suspect" base on, given current replication numbers, strong incentive to publish flashy results ASAP, and a policy of not publishing refutations?
> A refutation or null result typically isn't interesting to the audience of these journals- nor should it be.
Why? As a layman with interest in science, to me "scientists from group X discovered that drinking camel milk will make you live forever" is substantially different that "scientists from group X claimed that drinking camel milk will make you live forever but 5 other groups of scientists say it's bullshit and that research is garbage". The former presents the result as unchallenged pinnacle of scientific knowledge, and the latter as highly suspect and very much possible baloney. I as a layman who has no way to verify any of that myself and read all special press would have a lot of value in distinguishing the two situations.
Indeed, legally and ethically Science is free to do what they want and should be held so by all external parties.
However, we, its consumers, are equally free to no longer pay attention to it, since its quality as a scientific journal is somewhat suspect. Anyone who truly believes in science (with a small s), ought to appropriately lower the amount of prestige with which they view science after such accusations.
Subsequent refinements and discovery of systematics and how to account for them improve the research, and often change the results, but those results don't make it to Science/Nature/etc because they don't belong there (usually).
So, really you should be taking the mindset that articles in these journals should be taken with a grain of salt. Whether that should have an impact on their prestige? I suppose thats a personal decision.
The answer is: yes, because that's how science works! And it's especially true if your journal is named "Science".
Science works best as a debate and not as a statement of truth from on high.
If you only publish "This is true" while excluding "No it's not because..." you're doing missionary work and career enhancement, not science.
https://www.bbc.com/news/science-environment-39054778
As such, this would seem to be a common occurrence and Science would have to decide whether to devote equal attention to every failed reproduction.
The only reason this is generating clicks is the politics angle. This is boring science. The original study was probably bad (as, let's face it, most psychology studies are), but they ran it anyway (because clicks). Now the other side gets its clicks on the "censorship" angle.
The science? Still boring.
I don’t understand the distinction you are making here. Why would it be insane to publish refutations? In this case, they refused to even review the refutation.
It's not at all insane to publish refutations. A really good refutation of a long-accepted result is great science, and this kind of thing makes news and runs in major journals all the time.
It's insane to demand that Science publish a refutation of anything they happened to run in the past just because your results (or, let's be honest here, your personal politics) disagree. Most refutations, too, are bad science!
The demand seems to be that "If Science ran an article they're obligated to publish anyone's refutation of that article". And that's insane. It only seems sane to you because your political outrage glands demand justice for someone using the previous study as spin, but that's not Science's problem, and it shouldn't be. Go run the outrage pieces on Slate, that's what it's for.
I did not get the sense that the refutation was seriously considered, and my impression is that the original article has not been retracted in any way, correct? So Science's public stance could be seen as the original article is still the best truth they've published on the topic, which feels wrong.
Wiggle room may be appropriate if the article is already retracted or has already been refuted.
But what is up with this "moral obligation" nonsense? Being wrong isn't "immoral", it's just a mistake.
You're trying to apply ideas of politics and rhetoric to argue that Science somehow "advocated" for a result that made democrats look good and republicans look bad, and that they're now "obligated" to make democrats look bad too. And that's wrong. Like I said if you want political argument fodder go to Slate (or, sadly, HN). That's not what Science does.
Being wrong is not immoral. Being wrong, telling bunches of people the wrong information, and then making no effort to fix your mistake once you have the chance is at least dubious on my personal moral spectrum.
I think the answer to this is a rather blatant “of course!”
If Science has the time to support false studies that retrograde the scientific cause, it certainly has the time to publish their corrections and refutations. The scientific process is not about how much stuff you can publish, it's about how much true information you can convey. A journal half the size of Science but twice as trustworthy is at least as valuable, if not vastly more.
It's not as blatant as you think. It takes experienced scientists to review submitted papers, and their time is an expensive resource. Is it interesting that a research team with a sloppy adherence to a methodology failed to replicate a study? Not really. Should Science give resources and a platform to people who are just careless researchers? Obviously not. It's easy to fail to replicate an experiment; just do a bad job.
Considering the reproducibility crisis, is it your position that people are worse at reproducing research than they are at conducting it? An alternative interpretation would be that people have more incentive to publish hastily-made "failed to reproduce" papers than they have "findings" papers, but I don't think that's the case either, especially considering the reply from Science to this case.
Not to every failed reproduction. Only to failures to reproduce results that Science had previously published. And if that means they end up publishing a lot of failures to reproduce, then perhaps they need to re-think their criteria for publishing the original results. After all, why do you think this "reproducibility crisis" exists in the first place?
As for why it got lots of press: It's the kind of study that would drive clicks/engagement, increasing ad impressions. So... Capitalism.
That's what they should have said to the first paper, but since they decided to publish it they can no longer say "who cares" to the correction. They should either retract the original, or publish this.
It's like any other paper that reports a statistically non significant result. The appropriate response is, "Well, OK. Sure, whatever." But yeah, we're not gonna take up space from something else that may be important, just to go back and forth over this statistically non significant result. You let them do that in your journal that crap will go on forever and you'll never have space for anything substantive. I'm sure liberal and conservative political types must have some other place they can do that at.
If you have a new result in some new area, that's one thing. But if all you have is the exact same result saying that the old non significant result is, um, non significant? I mean, I gotta agree with HN User "Gatsky". "Who cares?"
But I'm a research scientist by training and a stickler for protocol. So it's likely my age is showing and the rest of the world wants to loosen up these days? I'm just the old man yelling at the liberal and conservative kids to get off my lawn.
The same is true of all studies, but for some reason people lose their critical faculties when considering work published in the "top" journals.
If my friend asks me what the score in lasts night's game was, and I tell him 4-3, but then later realize that it was actually 4-2, I'll send him a txt to correct the mistake. If your argument is that a lower standard applies to one of the premier scientific journals, I think that's pretty messed up.
In reality what happened with the first paper is the editors of the journal sent it out for review, and whoever reviewed it was happy that it met the methodological standards considered sufficient at the time. This is not the same as being happy that it was true. You might consider that 'messed up' (and you are right, it is messed up), but that was how the sausage was made in psychology research in 2008. There is no galactic council that imposes standards of evidence in a journal. They rely on the reviewers, and the reviewers are active researchers in the field.
As an aside, publication in a top journal is not a guarantee that results are water tight, and should never be considered as such. If you want to believe a result, you need to read the paper and think about it carefully. Even then, this approach is susceptible to outright fraud, which also does happen.
We're allowed to question what a peer-reviewed article published in a top-flight journal says?
There might be a possibility that the science in Science might be wrong, and we need to work that out for ourselves?
This contradicts a lot of the messaging about science these days...
You're required to question!
-At least if you actually want to derive any benefit from the process.
Science is most assuredly a "use your (own) brain" kind of endeavor. ;-)
Climate Science isn't alone in this, though. Archeology also requires a PHD before you can comment on the purpose of any artifacts dug up by archeologists, even though archeology degrees don't teach anything about (for instance) textile production.
And then there's the various interesting parts of Social Sciences, which often require you to have a particular identity or experience before you can question any part of their conclusions.
I agree that Science should be a "use your own brain" kind of endeavour, but increasingly it is not, unless you have the appropriate letters after your name.
Science cares, clearly!
A better title for laypeople would be "We _refuted_ a Science paper that can't be replicated and Science won't publish it"
In pure formal logic they have at least proved it is not true for all cases, so it's a refutation in that sense at least
But in any case, "refutation" is closer in popular lingo to what happened than "replication"
With induction, one can only have negative knowledge (this is know as the problem of induction or the black swan problem), but it gets more complicated than that. Most studies are trying to look at probabilistic outcomes and are trying to get significant p-values. There is a myriad of problems with this beyond simply the existing probability that their sample was bias. E.g., their methodology could be improper, or even just less than ideal. There could even be unknown problems with the underlying assumptions and the model being used by researches.
To suggest that an opposite result refutes a previously obtained result deeply mis-characterizes how science and human understanding works, and the problems inherent in this thought process deeply overvalues the knowledge we have and undervalues the more-than-likely seriously uncertain data we work with every day.
If you're interested in these problems, i'd suggest Karl Popper's work on philosophy of science.
This is an interesting discussion but I'm unfortunately out of time now. I'd only point that most of science is abductive, rather than inductive, and that the whole concept of refutation will take us to discuss what is scientific _advance_
Social experiments by their very nature tend to pollute the data. This is part of why I don't believe UBI would actually work: Historical real world examples of the idea consistently fail. I think the UBI experiments are all nonsense.
Early settlers in the US promised to provide for everyone. When freeloaders eventually ruined that deal, they changed it to "If you don't work, you don't eat."
Every time this has been tried out in the world, it works for a small group of committed, like-minded people. When it is opened up to "anyone/everyone," they inevitably have to come up with rules to account for free riders who want to take advantage.
Another real world example: When Spanish Conquistadors sent large quantities of silver back to Spain from the Americas, it didn't make everyone in Spain rich. It merely fueled inflation.
That's what happens when you add money to the system without adding goods and services: Prices go up.
The current UBI experiments are so limited in scope that the conclusion is akin to saying "People with kindly rich uncles are better off than people without such." But when it stops being a small group of privileged experiment subjects and becomes everyone, the conclusions drawn from the experiments won't apply anymore.
The person who originally thought up this concept was Otto von Bismarck, a highly conservative prussian politician. The idea was that people who are not destitute but merely poor are less likely to start a (communist) revolution.
In more modern times, it means that if you fall on hard times, you need not immediately be written off from society. Eg. people who are made redundant during an economic down-turn will still be available to the working pool during an economic recovery.
Of course, the implementations are not perfect.
In eg. the Netherlands, one issue is that getting a job and going from social security to minimum wage actually leads to a negative change in net income. This means that a rational agent might actually choose not to work once they become jobless the first time.
What one might prefer is a situation where working will always yield a larger net income than not working. In such a situation, the rational choice for most people would be to always at least try to do (some) work.
In such a situation where funds are already allocated, and the downsides are already proven to be minimal; a form of UBI might be one way to provide that dynamic. In such a situation it may prove to be a rational tweak to increase the working population.
Instead, they tell glowing feel-good stories about how social workers could pay down debt and save a few bucks or whatever.
https://www.santacruzsentinel.com/2017/05/20/tech-giants-elo...
I'm looking at something that might be a good workaround to actually get to a workable implementation of Negative Income Tax: https://en.wikipedia.org/wiki/Negative_income_tax
... which is definitely financially feasible and quite likely a good idea (and likely to save money in so many ways). In itself it's just a little bit impractical due to the sheer amount of work that rewriting tax laws would take. Hence: workaround.
Also note that I am speaking from experience-on-the-ground in Europe, and my reasons to propose UBI are thus a little different from those of Elon Musk or Sam Altman.
That makes me think of this line:
UBI Is a Transfer of Wealth from the Needy to… Everyone
From this excellent analysis:
https://medium.com/s/free-money/after-universal-basic-income...
I'm not for UBI. If you are looking to convince someone otherwise, you are wasting your time.
I know plenty of things that actually do work to solve problems and bring down expenses. No one cares to listen to any of that.
But you do you and let's just agree to disagree. Some desperately poor American woman who thinks UBI is nonsense is zero threat to your plans to change European tax codes or whatever.
Adieu.
Likewise, if the original paper is a data point, a failed replication gives you two data points that cancel out.
I don't think they provide any compelling reasons why the same journal is obligated to publish replications of any arbitrary study they've published in the past. It wouldn't exactly scale.
Am I the only one here that thinks that there should be no overlap between the set of 'studies that are completely non-replicated' and 'the set of studies that need to be distributed at scale'?
1) Replication efforts need to be strongly incentivized. Science in modern times has been taking repeated hammer blows to its credibility. The systems we have in place seem to result in more and more simply 'fake' science being published, including in the top journals. This is not an easy problem to solve since replication efforts can be very costly and things like pre-publishing still ends up relying on good behavior, which is something we can no longer necessarily take for granted. The one small tool we do have against bad science is replication efforts. Every effort should be made to incentivize these. The willingness to publish significant refutations to influential studies is well below the bare minimum we should expect from journals on this front.
2) What is the purpose of a journal, outside of making money? It's to inform people by providing a filtered repository of the information the reaches a high level on the scale between reliability and relevance. When a journal publishes a study that may have been false, they end up doing the exact opposite of this - they are misinforming society. And this is especially true for the study which seems to have been refuted. It was not a footnote - it has been widely referenced and had a significant impact on public discourse. This all leads to a strong obligation to publish.
3) (another point derived from #1 while writing) - Publication of refutations aligns journalistic interests with public interests. In our current system journals are primarily motivated to publish the most 'meaningful' results which is ultimately a euphemism for the most shocking results. So they have an incentive to want to give these studies the benefit of the doubt. Yet these are the very studies which should be held to the highest degree of scrutiny and criticism. A journal that is willing to publish refutations of the studies it publishes returns this alignment since if Science found itself full of little more than refutation of 'meaningful' studies it published in the past, it would quickly lose its prestige. Yet at the same time if it was willing to publish refutations, yet they were few and far between, one could hold what was published within the journal to a much higher standard of reliability - which would be good for both the journal and the authors who were able to meet what would undoubtedly be their now increased standards.
Imagine if the paper that was refuted was one that claimed to show that e.g. humans were not playing as significant a role in global warming as thought. And then Science refused to publish a study that seemed to show conclusively that the original study was, at best, deeply flawed. Would people be responding in the same way?
How much of the journal should be given over to replications that fail, and then the replications that succeed where another failed, and any potential back and forth?
At any rate a replication study while important for the process of science is not so important for the process of news and as Science is a business and using their resources for publishing replications does not seem likely to increase their profit (but maybe even reduce it) while increasing their cost, I am surprised anyone would even think it should or would happen.
Perhaps what there should be is a reproducibility column, where studies attempting to replicate previously published studies are noted. A paragraph summation could be given. As reproducibility of a finding becomes accepted further replications would be dropped from the column as redundant.
on edit: grammar fix.
You're right that in business, there's a disincentive to publish replications, because business treats science publication as literary nonfiction: they're trying to publish the most sensational results, because the most sensational results sell journals. This is one of the strongest arguments that scientific publication shouldn't be a business: making scientific publication a business creates incentives for scientific publications to not publish science.
on edit: because it seems like your argument really is that Science the business should shut down.
Science the business should shut down, not of their own volition, but because the scientific community shifts away from for-profit publications and Science has the respectability of a tabloid.
A dishonest response is saying they should publish things that makes no business sense for them to publish, and the logistics of which really makes no sense for their business model, with the seeming purpose of upbraiding them when they fail to do this thing they would never do.
Science (the business) is being dishonest by pretending to publish science (the method for obtaining facts). It's not okay just because it makes them money, and it's not dishonest to upbraid them for their harmful behavior.
“Best Actor nominee argues why his performance is the most impactful,” etc, etc.
> “We still believe that there is value in exploring how physiological reactions and conscious experience shape political attitudes and behavior, but after further consideration, we have concluded that any such relationships are more complicated than we (and the researchers on the Science paper) presumed.“
That sounds more like a clarification or a letter to the editor. I don’t disagree that nuance is important, but I assume Nature believes they can get more nuance out of visiting new studies than reinterpreting old ones.
Using the time reversal heuristic, if we flip the order, so the replication came before the original, how would you feel about Science publishing a paper with a positive result, when multiple studies--with larger samples and the same materials--had not obtained the result?
If you think the original should still be published in Science in that case, why? Since the original is published and highly visible, it's dangerous that Science won't correct the highly publicized, unlikely-true result.
That happens all the time. Papers with null results tend to either sit in drawers unpublished, or never get written at all. Nobody knows about the multiple studies because nobody will publish them.
Some number of people (including eg. Roca) say that this is a problem - a "replication crisis" if you will.
Those people say that perhaps papers with null results should be published. Perhaps -they say- publication of null results could be (one possible) (part of) the solution.
Your reply sounded like you don't completely agree? Is that true, or did Roca read you wrong? Could you clarify?
But to not publish the refutation and not retract the refuted article is to make a deliberate decision to continue to endorse research which has been proven to be incorrect.
Why are these age old print journals still in charge of this process with print era processes??
All sorts of important questions are raised by this:
- Should we trust anything published by Science or other top journals at all?
- Is the lackadaisical attitude related to the subject matter? I've read that studies casting conservatives in a good light get replicated almost immediately, and psychology is completely dominated by liberal voters (>90%). Which leads to:
- Why did it take over ten years for such a widely cited study to be replicated? Nobody cared because it fed their own political biases?
And finally:
- What do we do about it? Can academia reform itself at all or is that a lost cause? If so perhaps psychology isn't really worth studying, given the large expense involved and the fact that the outputs so often seem to be noise.
But the statement is highly misleading, because what are you comparing it with?
Put another way: There's no such thing as a 100% reliable source for anything. You need to apply judgment in all things.
'Science' is actually a pretty decent journal. It didn't get its reputation for nothing. This doesn't mean that you can just assume they are a magical source of all truth; just like you can't assume 'Wikipedia' is (_definitely_ not), or 'Nature', or 'Britannica' or etc.
- Should we trust anything published by Science at all? Only to a degree. One must always be skeptical and think for oneself, no matter what the source.
- Why did it take ten years for the study to be replicated? Maybe it didn't. Perhaps these are just the first authors to speak up about it.
- What can we do about it? Continue working to improve the process, as we have done over the past centuries or millennia. The authors of this particular paper spoke up. That's part of the process too.
The claim that the field has moved on is complete garbage and nonsense. It is farcical on its face. The older work is still being quoted.
>We believe that it is bad policy for journals like Science to publish big, bold ideas and then leave it to subfield journals to publish replications showing that those ideas aren’t so accurate after all. Subfield journals are less visible, meaning the message often fails to reach the broader public. They are also less authoritative, meaning the failed replication will have less of an impact on the field if it is not published by Science.
Agreed, but maybe the issue here is journals shouldn't be prestigious. I understand journals' historical usefulness, but the Internet has made them obsolete.
Prestige and historical usefulness are practically synonyms. You're arguing that prestige shouldn't command respect, but this is like arguing against human nature.
"Science" should recognize its importance to the world at large and take extra care to publish failures of replications of papers that it reports. The more prestigious the journal, the more carefully they ought to follow this proscription, especially if they wish to maintain their prestige.
So many of the existing social and industry hierarchies were merely replicated on the Internet. As we see with all the popular print newspapers and magazines often being equally or more successfully online.
Maybe there were hierarchies for a reason and they were a good thing. Rather it's about building the better hierarchies rather than replacing them entirely with some academic idealistic pipedream (ditto for markets in economics).
I think this is a confusing way to word it; it implies that they successfully replicated the original study, and found no difference between their results and the original study. I might say 'they were unable to replicate the famous study about [...]' or 'they attempted to replicated the famous study [...], finding no difference between liberals and conservatives'.
They refused to send the null replication out for peer review. I think that is an important difference. The optics make it look more like suppression, rather than a rejection based on quality.
What's this twitter conversation they reference?
While people do have varying anxieties, knowledge, and reactivity to aversive stimuli, there's a stronger prior that political beliefs are based on a few different things.
1) Class Position: People with money or whom believe they are likely to ascend classes trivialize the struggles of others and ascribe problems to individual character rather than systemic design flaws. If an airplane crashes, do we blame the pilots?
2) Culture / Milieu: What ideas are floating around? What do people whom you speak to on a regular basis believe?
3) Personal Experience: Have you had a powerful experience that confirms or contradicts the broader culture?
4) (often but not always most importantly) What will personally benefit you or people you care about?
There's a few other things you could throw in there as well, but Marxists would say to liberals that the reason they are different from you is:
1) They aren't that different from you. You both believe in the integrity of the capitalist system.
2) Often they are from dominant racial or economic castes.
3) There's an intentional campaign on the part of the elites to divide the working class against each other, and one way is via scapegoating minorities and women. Everyone likes to say "No matter how bad it is, at least I'm not X".
4) There is an intentional campaign waged on the part of elites to tie American national identity to its military campaigns overseas.
5) There is an intentional campaign on the part of elites to use race and religion as a wedge issue to divide the people so they do not blame the people causing their problems.
That's not a scientific explanation, but my strong prior is that there's variation amongst people, but we're basically all the same and we're mentally flexible enough to think through adversity and come to our own conclusions.
"We conducted two “conceptual” replications (one in the Netherlands and one in the U.S.) that used different images to get at the same idea of a “threat”—for example, a gun pointing at the screen. Our intention in these first studies was to try the same thing in order to calibrate our new equipment. But both teams independently failed to find that people’s physiological reactions to these images correlated with their political attitudes." Emphasis mine.
Why they didn't used the same images?
> a very large spider on the face of a frightened person, a dazed in-dividual with a bloody face, and an open wound with maggots in it