Nature retracts controversial superconductivity paper by embattled physicist
nature.com
nature.com
Listening to someone who plainly had no idea what they were talking about is what got me interested in the field of education.
Maybe there should be a group of researchers that submit papers that seem believable with known issues to avoid scientific journals stagnating. Maybe the names and institutions of the group submitting papers should be hidden so that more reputable/famous people and institutions get the same level of scrutiny as everyone else.
That's a great idea, but who is going to pay for it?
Every scientist knows that it's more than possible to publish deliberately fraudulent work. Every non-scientist thinks that detecting this fraud is easy.
Eh, this working engineer doesn't think so. Freeform fraud detection is a hard problem in general, and when you're talking about (apparent) cutting-edge research, the pool of people who might spot specific details is tiny.
A low probability of detection does suggest that the penalty should be relatively high to have a deterrent effect.
In fact I know for a fact this has been done at least once before (at least with the big words): https://en.m.wikipedia.org/wiki/SCIgen
But it should be done in every field, regularly
The "alternative solution" to having a thing that wacks people on the head every Tuesday is...not having it.
This assumes code review's key purpose is to find bugs.
I always wonder about this when I review something, but at the end of the day there are many other things I'm looking more at than "is this a bug"
-Is this duplicating something
else?
-Should this be part of this
object?
-if I come back in a year, will
it take me 10 minutes or an
hour to understand it?
-is this necessary?
-should I approve this but add a
task to the tracking board to
figure out if this should be
done in the service instead of
the client?
-etcThe editor always has the final word on whether a manuscript gets accepted.
In terms of function though, the "hairy arm" is the complete opposite though: it's not to make sure reviewers are paying attention, it's to distract meddling reviewers from the changes you don't want them to veto.
> A feature added for no other reason than to draw management attention and be removed, thus avoiding unnecessary changes in other aspects of the product.
Sounds like it was for the same reason
You'd be better off pair programming everything and forgoing code reviews.
Sometimes I think the people that come up with this shit are sociopaths.
There was. People were not very pleased.
good idea: https://en.wikipedia.org/wiki/Sokal_affair
bad idea: https://www.theverge.com/2021/4/22/22398156/university-minne...
It’s called a scholarly hoax, and it’s been done more than once: https://en.m.wikipedia.org/wiki/List_of_scholarly_publishing...
The idea being that if the submission got published, the review process was obviously a total joke. I think they even had some examples that had actually been published.
Edit, Link: https://pdos.csail.mit.edu/archive/scigen/
(it would keep the students alert)
or maybe Van Halen's "CRC check", where their venue contract would ask for a bunch of things set up in advance, including a bowl of M&Ms with the brown ones removed.
(it was a test of diligence, that they read the contract and knew the requirements)
And he falsely claimed that his company had funding from OpenAI and Spotify.
Peer review is a relatively modern invention.
A vast majority of important 20th century physics, for example was pre-peer review.
But nonetheless there should be controls on such nonsense. The ease with which many academics get away with all this will make even a seasoned politician jealous.
There are ways of estimating percentages of bad papers, just as there are ways of estimating subsets of other populations.
This is science working as intended - the good reproducable work advances on the strength of the good papers and the bad papers fall away from lack of follow on.
Pretty easy to stay hired when you're a founder of a very small startup.
Edit: Looks like he's also on staff at the University of Rochester?[0] I can't snarkily explain that one.
[0] https://www.hajim.rochester.edu/me/people/faculty/dias-ranga...
the Springer Nature Group has a profit margin of 26% on annual revenue of ~€1.7bn [0]. If they were keen to publish this paper, they could have afforded to send an expert to the lab to actually see the process for themselves.
But that's not the kind of thing you do if your core business model is to publish splashy headlines...
[0] https://group.springernature.com/gp/group/media/press-releas...
that seems entirely too high for merely hosting and publishing papers. They don't do verification themselves, since scientific papers are meant to be peer reviewed (and unpaid peers, for that matter).
https://www.theguardian.com/science/2017/jun/27/profitable-b...
> In 2010, Elsevier’s scientific publishing arm reported profits of £724m on just over £2bn in revenue. It was a 36% margin – higher than Apple, Google, or Amazon posted that year.
It's the perfect rent-seeking business: grant bodies and governments judge scientists by approval of journals, scientists need to play the game of getting into 'reputable' journals, journals sit in the middle, outsource most of their work into unpaid work to scientists, and laugh all the way to the bank.
Then governments introduced mandatory open access for scientific research, for a moment it looked like journals' chokehold might fail, but they just adjusted: Nature journals charge authors ~$10,000 per open access paper. That's all taxpayer money going to for-profit organisations with little justification for the price.
so this is the lynchpin - how is it that such judgement is reliant on the journals, when it is peer review that gives papers their credibility?
Therefore, shouldn't the peer review system be the judge of quality? Which means by exposing more peer review and making it way more open and participatory, the gov't will no longer need to rely on reputation in journals to judge grants or progress?
But in the meantime:
The first step for acceptance is the journal editor - in lower tier journals, that's an unpaid Professor, in higher tiers it's a paid staff member (usually academic background too) - who decides whether the journal thinks this paper is a good fit or not. Most papers die there, too low impact, too niche outcomes, not enough wide readership etc.
If it has passed the editor it goes to peer reviewers, unpaid academics, who make similar decisions; is it technically sound? Is it impactful?
Impact is measured by the number of citations this paper will have, a bit of an educated guess. A paper on an obscure seagrasses will have few citations, a paper on a novel cancer-defeating mechanism will have many.
High impact factor journals are selective in which papers they accept (via editors and journals) to keep up the high impact factor. That double-gated decision is what turned into a stamp of approval; it's the journal's brand which is the lynchpin for the academics' career, not the peer review
What infallible sources or publications are you relying on?
Not sure what the point of that is, it almost sounds like ChatGPT
At a guess, it is copy/paste from MATLAB marketing materials, intended as a sort of appeal to authority – "we used MATLAB so our results must be good".
Here's what I got when I asked it to finish the paragraph "In summary Matlab's rich set of features, mathematical capabilities ...":
"In summary, Matlab's versatile capabilities, including robust mathematical functions, extensive libraries, Simulink for dynamic system modeling, support for diverse data types, and its user-friendly programming language, collectively create an environment conducive to efficient numerical computations, effective data visualization, and the seamless handling of complex scientific and engineering tasks. Its popularity in both academic and industrial settings, coupled with an active user community, cross-platform compatibility, and support for parallel computing, positions Matlab as a top choice for professionals and researchers seeking a powerful and adaptable tool for their computational and analytical requirements."
"diverse data types", "user-friendly programming language" are exact matches in the linked snippet, and the rest of it is very similar in tone ("strong community support" vs "active user community").
ChatGPT also likes to end on "In Summary"; when I asked it to list Matlab features it did just that.
That always tips me off too, its sounds like someone following a high school essay template
if there were challenges in getting people to care enough about reproducibility then we should address that
because the tabloid fanfare rightly deserves all the criticism it got and all the blemish to “science” if thats what people’s takeaway was
They're not paying attention to this.
No, sorry. Just no. Stop. "Scientists" were clear, loud, and nearly unanimous in their calls for caution about these results. People on this very site were in these threads saying "we should wait a few weeks, guys" and getting downvoted into gray oblivion for their trouble by a horde of kids hopped up on "proof" they saw via (and I'm not kidding about this) a multimeter screen on TikTok.
You can't fix that with science. Science did everything right here. We had a tantalizing result reported, investigated, and disposed.
I agree with your assessment, of course, but I don’t think it invalidates the comment you are responding to. Large parts of the population could plausibly think those things, despite how clear and unambiguous “scientists” were about the results.
Their trouble of being no-fun wet blankets? What trouble was that, exactly?
> "Science did everything right here. We had a tantalizing result reported, investigated, and disposed."
And the only thing you wish went differently is that you squashed more people's enthusiasm??
"And the only thing you wish went differently is that you squashed more people's enthusiasm??"
It's not about "enthusiasm". It's about scientific illiteracy. You can be excited all you want. "I saw a multimeter on TikTok, therefore it's true" and then shouting down folks with actual understanding for being cautious isn't enthusiasm, though. It's Dunning Kruger in action.
Academia is broken, in many ways even.
>"scientists can't be trusted"
You shouldn't trust scientists anyway, at least not without due dilligence. Science is built on skepticism.
The thing that has caused me to question this the most is crypto-currency scams. I don't mean generally, just the things we would all agree are clearly scams and kind of obviously so. They so often tell you: "do your own research". I think its a version of the poorly written Nigerian scam letter. The idea is a person with the right skill or knowledge will see its a scam right away and not bother them. People who are careless enough, dumb enough desperate enough will still be suckered and there are plenty of them. So the obvious scammers is completely comfortable saying "do your own research" and know that he filters out everyone but the suckers.
I wonder if sometimes most of us are better off just trusting the people who are experts most of the time? Skepticism is good but you can't really form your own exper opinion on every question you will face.
Whenever I was told this growing up, I would respond with "Why?".
It was not always appreciated.
noone can be an expert in all fields, but you can seek multiple opinions from multiple experts, and generally the consensus is likely to be correct. Esp. if those experts are far apart, and unlikely to be colluding or associated.
The trick is how to do it efficiently, and not to fall into confirmation bias (aka, seeking only experts that agree with your preconceived notions).
this is the reason why things like scientific journals exist. it's embarrasing that Nature had to retract this paper, but on balance they're still doing a way better job of judging this subject matter than i would.
It probably depends on what is at stake. I don't particularly care about superconductors, but I do care about my knees, and when one doctor several years ago wanted to operate me, I sought a second opinion elsewhere. Sure enough, the other clinic told me that this can be treated conservatively.
I canceled the operation and, 6 years later, I have zero problems, zero pain and can walk just fine, even up/downstairs. I am not sure what the operation would do; knees are sensitive joints and don't like to be cut open.
https://www.nature.com/articles/d41586-019-01673-x
Pons and Fleischman set off a very similar hype cycle about cold fusion in the late 80s.
It does trigger nuclear fusion, but the design cannot break even.
"No fusor has ever come close to break-even energy output and it appears it is unable to ever do so."