Potential fabrication in research threatens the amyloid theory of Alzheimer’s
science.org
science.org
One part that is really important that is mentioned in the middle of the article is that these systems are very difficult to handle, and it's almost impossible to make many of them nicely reproducible. Fibril and oligomer formation depends a lot on the environment and reacts to tiny differences.
I find this kind of fraud deeply frustrating, there is so much wasted effort in the wake of faked high-profile results.
Those same qualities that make a study hard to reproduce should also create skepticism that the first study was done properly.
The part that is difficult and annoying is that the variables are not necessarily known, it takes a lot of time and experiments to actually nail down those. And even then for some sensitive stuff every detail can matter, like the exact type of tube you did the experiment in or the vendor, batch and age of every chemical you used. Very often you can control this enough to have consistent results for a set of experiments, but that kind of stuff is really hard to control across different labs.
A journal that requires five confirmations to cite a work is a journal that will fast have zero submissions.
The main reason people attempt reproduction is to continue that line of research, and that often starts with reproduction plus a slight tweak. This is why you find a lot of papers like this. This is also how irreproducible results become sort of known about but not challenged in many fields.
It's an interesting idea, but it all comes down to how the incentive structures happen.
If you think reproductions have failed then it’s on you to try to have a cogent explanation for why if you’re personally motivated by a belief that the experiment is actually valid.
The particle physics folks also have a different approach these days where they blind the ability to see results and do multiple cross checks of each other’s results to validate the data (because the underlying experiment is so expensive to run). That can also work in theory although hard to say what the success rate of that approach is just yet.
indeed, the initial experiment is not immune to such concerns, and might itself have succeeded by making a mistake or failed because it lacked the technical capability
Is this common?
Extrapolation to a more complex field tells me that it's a norm there.
I am pointing out certain shady incentives in our field that might apply in another scientific field as well.
Is there a way to reward this type of research?
https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
It makes sense to publish this, maybe someone else can make it stable and reproduce.
1. is it common for a non-reproduced experiment to gain prominence as a leading hypothesis and spur related research?
2. if so, why? if not, why did this hypothesis take hold?
3. how can we make science more reproducible?
reproducibility is held as a tenet of science, but this example, assuming it is true, would violate the principle.
2. It does because some fields of science are intrinsically long-duration, highly sensitive to variables, or only testable at scales that exceed our experimental capability. That's just a consequence of physics and natural laws of reality. E.g. nutrition, chemical toxicity, economics, high energy physics.
3. We can cheat and find proxies or more tenable micro-systems to experiment on. But often those have their own problems (e.g. rodent models) or aren't feasible.
Reproducibility is a goal of science. It's not always an achievable goal.
When it's not, we do the best we can, as with drug testing pipelines.
given the prevalence of non-reproducibility, what other fields do you believe have suspect leading hypotheses?
Everything is extremely hype-driven -- it turns out that "cannot confirm X" isn't very compelling for journals, etc. etc. or even the news cycle. Journals, etc. thrive on exciting new findings... and that tends to lessen the critical looks.
Of course, there are lots of other things to this problem: Very narrow fields where people absolutely know who their "anonymous" paper reviewers will be, and so must include even extremely tangential references to those reviewers' papers, etc. etc.
Usually the sciences self-correct eventually[0], but that's only because there is such a thing as objectively verifiable facts and overwhelming statistics in science.
[0] Unfortunately often as slowly as "one funeral at a time" (Max Planck, I think).
The most exciting hypotheses tend to go one of two ways. They are rapidly supported with independent evidence, by being applied or replicated. Or they draw a lot of attention and money which helps them persist in spite of, or in the absence of, evidence.
Charlatans, purveyors of snake oil, and (most often) people who for whatever reason don't want to be seen to be wrong - they exist in every walk of life. Science is the same. The incentives in the system strongly select for these people in the 'leading hypotheses' space.
1. Twitter bot / misinformation research. A surprisingly large field in which papers are almost never replicable because they don't supply the actual tweets, but only opaque IDs and classifications. Trying to cross-check them is futile because by the time you tried months later many of the accounts have been suspended. They could easily supply the contents of the tweets they scraped along with account metadata, but don't. The few times I did deeper checks of these papers I always found some accounts that were identified as bots but weren't suspended, and on manual inspection were very obviously human. This field also has the problem of being increasingly based on ML pseudo-science.
2. Germ theory! It can't explain several aspects of the epidemiology of respiratory diseases. It's probably not wrong but is certainly incomplete. As we saw with COVID, models based on simple germ theory always make wrong predictions, but this problem pre-dates COVID. It was known for a long time that standard germ theory fails to explain the behavior of influenza, for example, why it's seasonal, why waves peak and enter decline before everyone is infected, why variants disappear totally instead of coexisting, why flu season seems to start everywhere in season almost simultaneously, why there have been outbreaks of respiratory viruses in totally isolated environments like arctic bases, and so on.
The issues here are deeper than reproducibility though. Even if bot papers were reproducible they would still be wrong because their methodologies are invalid and cannot support the stated conclusions. It's important not to get too focused on mere replication.
Huh? I thought that's so well-known (must be, if even I have heard of it) that it goes without saying: Spring-summer, people go out and breathe fresh air a bit apart; autumn-winter, everyone goes inside and coughs their germs at each other.
> why waves peak and enter decline before everyone is infected,
The more people are already infected, the fewer and further between the as-yet-uninfected are, so that seems quite reasonable. (Pretty much inevitable, really, isn't it?)
> why variants disappear totally instead of coexisting, why flu season seems to start everywhere in season almost simultaneously,
See first point above.
> why there have been outbreaks of respiratory viruses in totally isolated environments like arctic bases, and so on.
Someone always flies in supplies...
I mean, WTF -- "Germ theory"? Isn't that what it was called in the 19th century, when there still was any doubt about it? What next; "Gravity is just a theory!"?
And it would probably run into some practical problems if you tried to nail it down, like the fact that for many people they are office or factory workers and the amount of time they spend outside doesn't change all that radically during summer, certainly not enough to make the difference between explosive spread and total eradication.
"The more people are already infected, the fewer and further between the as-yet-uninfected are, so that seems quite reasonable"
That's why the wave slows down yes, but again, this ad-hoc notion doesn't work. Try it out on paper or code up a quick epi model yourself and then compare against real world case data. The waves always end long before predicted, even when there are still tons of uninfected people available to infect and people wandering around who should easily infect them.
This is one of the reasons why COVID modelling failed so badly. They coded up the exact simple germ theory model you're describing here, and of course it yields a single giant wave whereas what is seen in reality is a long series of small waves that start and end in ways that aren't predicted by the theory.
"See first point above."
See my reply. I can assure you, that the epidemiology of influenza really isn't as simple as "people going outside in summer". Take a look at some of the research papers from the 80s exploring this topic. There are still plenty of people interacting with each other closely indoors even in summer months, yet flu completely disappears. It's not something that ties in any obvious way back to accepted theory.
"Someone always flies in supplies..."
That's not what "totally isolated" means. In the cases in question there was no contact with the outside world whatsoever. In one case, at a British polar base, they mounted a very thorough investigation after an outbreak of cold virus after 17 weeks of total isolation. They checked if any new supply crates had been opened, etc, but no. They couldn't identify anywhere that new viruses might have been introduced to the base.
https://www.cambridge.org/core/journals/epidemiology-and-inf...
One of the more popular sub-theories (though largely ignored by epidemiologists) to try and explain these things is the possibility that many people are continuously infected at a sub-symptomatic level, that the immune system never 100% wipes out the viral infection in these people, just keeps it in check. Then something happens to slightly knock the immune system out of balance for a moment, like a sudden change in temperature, and the infection is able to re-gain a foothold and starts replicating out of control again. So that'd be why people just show up everywhere at once spontaneously infected without any obvious index cases.
And here I thought the goal of science was to find the truth[1] about how the world works, with reproducibility as one of the means to do that.
___
[1]: Or at least get as close as possible, hopefully asymptotically ever closer, to the truth.
And this is not rare. Sometimes it is fraudulent, people know things are faulty but continue anyway. In other cases it could be by ignorance of the existence of the phenomenon behind the faulty behavior or simply because that phenomenon has not been discovered yet.
For the purpose of funding, publication in important journals or conference proceedings is the important thing. And the public generally tends to treat publication as the standard of truth. But publication kind of works on the honor system and is susceptible to fabricated results.
Maybe there should be more funding towards reproducing (or not) important results?
The amyloid beta (Aβ) hypothesis has always been fishy but this paper is basically the bedrock of the current investigational trajectory.
The Aβ hypothesis was almost dead in 2006 when this method was invented and results posted and it sent shockwaves through the research world. Since then spending on Aβ research by NIH has gone from $0 to $290 million all it seems based on a lie cited over 2000 times in further papers.
The article is pretty convincing as is the independent verification that not only concurred but found additional evidence.
Sylvain Lesne has some ‘splainin to do.
Since papers tend to get published only if they have positive results, what does it mean for thousands of publications all citing a fraudulent paper? This seems really strange. If the first 1000 failed to produce results and were partially based on that original paper it should cast significant doubt on it, but again failures are rarely published.
What does this mean then?
A lot of papers that came after were producing results in the framework it established.
Because of the nature of a disease like Alzheimer's not many studies can easily measure the final effectiveness of their addition to the space on patients.
These types of cases tend to focus on the fraud, but then everyone eventually walks away and then there's no larger discussion of how this was enabled by the field to have such a hold.
I don't buy the idea that it's all so quixotic and involves so many variables to get right that who knows what's causing a lack of replication. I've heard whispers like this about other big effects in other fields. People know what's going on, and often it's right there in published results but ignored.
Imagine I post an article that’s fraudlent and ”proves” that small rocks of a certain size and color keep away tigers. You won’t then see 200 articles saying they tried to replicate this and failed. What you see is 2000 articles dealing with how to find rocks of this exact size and color, siting my paper to support why it’s relevant to be looking for rocks in that category.
That being said, numerous papers have been published undermining the 'amyloid beta as primary disease driver' theory already. Or at least, finding no statistical correlation between AB and neuron death or that sort of thing. The most valid hypothesis that I've seen is that AB is just a supporting element for another amyloid, tau, which does correlate with toxicity.
My paternal grandfather got diagnosed at around 70, and my father is 62 now, definitely getting more forgetful but normal for 60+... Still, I worry about him regularly. My father is a clinical psychologist, he's helped people all his life.
These news make me so angry I can hardly see straight. I genuinely hope this sociopath waste of skin gets Alzheimers himself, and has to relive every moment of his disgusting crimes as his brain slowly turns to mush and he forgets every person he ever loved(if he is even capable of love...)
Sylvain Lesné, has no Wikipedia page. He seems to publish roughly a paper per year while some famous scientists publish one every month. He is not at the origin of the amyloid-β hypothesis which is more than 100 years old.
Even more there are thousand papers published about Aβ*56 and Alzheimer's disease. How could this guy responsible for this mess?
And if what he published was wrong, why this was discovered only 18 later if thousands scientists are working in the field?
My understanding is that there is a need to find a scapegoat for the amyloid-β and pointing to an obscure guy is in the interest of many big fish.
[0] https://www.statnews.com/2019/05/21/alzheimers-disease-amylo...
Because he started it?
https://www.nature.com/articles/nature04533
But it is ultimately in indictment of everyone who went along unskeptically, collecting grant money.
This paper is from 1988:
How an Alzheimer’s ‘cabal’ thwarted progress toward a cure - https://news.ycombinator.com/item?id=21911225 - Dec 2019 (382 comments)
The amyloid hypothesis on trial - https://news.ycombinator.com/item?id=17618027 - July 2018 (43 comments)
Is the Alzheimer's “Amyloid Hypothesis” Wrong? (2017) - https://news.ycombinator.com/item?id=17444214 - July 2018 (109 comments)
It was a letter to the editor, by a doctor, describing a small study on hospitalized patients, and was seized upon by Purdue, as the basis for their entire sales pitch.
When there's money to be made, people can look the other way, quite easily.
The age adjusted mortality is declining in some countries, but when boomer generations age, the number of cases increases.
Are they supposed to do a forensic analysis of every paper they cite? It's sad if it's arrived to this point.
Reproducing the original result can be prohibitively expensive and difficult in many fields (ideally, this wouldn't be a concern - but generally, research groups aren't rolling in cash and resources, plus you need to be fast not to get scooped for the next paper and lose future grant money)
I know people in the field that do all this kind of research and do not know much about mice-experiments or even about the experimental techniques used to obtain these results. To me, they are victims of this fraud more than anything.
So I'm afraid to say that assuming the allegations are true, they will probably get away with it. It's absolutely standard when these sorts of things are discovered that everyone sweeps it under the rug as vigorously as possible.
Fixing it would be difficult. There are a huge variety of ways to produce fraudulent science. Even coming up with a law that captures half of them is fiendishly hard, and how to do enforcement in a timely manner? In this case capitalism came to the rescue because the investigation was funded by short sellers, so this has to be one of the best arguments for short selling around. But by the time there's a publicly listed company whose share price is dominated by research suspected to be fraudulent it's way too late.
There's probably also a fear of looking too closely. There's a culture of coverups in science that I've seen first hand. It's deeply unpleasant and breeds the suspicion that they do it so blatantly because it's become a way of life, because they know there won't be any consequences even if they're called out on it. If you start going after image tampering, well, it's only a tiny next step to say that if your paper reports a mean that's statistically impossible given the data set then that's also fraud. But then you'd have to investigate and mount prosecutions for a significant fraction of all psychology researchers. And then you're going to have to make it a crime to not share data on request as otherwise incriminating data is always going to be inconveniently lost. And then you're up to 90%+ of researchers facing action. Draining this swamp would be very hard.
the cynic in me suspects that they're saying that to maintain their - presumably funded - status quo, but not being even remotely knowledgeable in the subject, I have no idea whether that is well-founded
Masks were described as useless for countering COVID transmission because of what turned out to be superstition around "airborne transmission", itself finally traced to a result that properly only applied to tuberculosis.
Belief in ivermectin efficacy was a similarly widespread superstition among mostly non-scientists.
We have generally had much better results from science. Science was finally obliged to abandon its "airborne transmission" model by people who knew better publicizing correct information. But most ivermectin fans still cling to it.
What "the" science?
The scientific method is proven to be a very good way to make predictions about the world based on observations. The body of scientific work built up is immeasurably valuable and incredibly good at predicting things.
Some random corporation or politician or person claiming to have The Science on their side, and that anybody who disagrees or questions them is a heretic and an unbeliever? That's not reliable and it's not science.
Only politicians and policy advocating 'scientists' cough Fauci cough are confident enough to make proclamations about the state of science. To their credit, no one seems able to hold them accountable when they falsely declare consensus and silence the voices in opposition
Add to that a blatant case of forgery like this on a high priority subject, and you have the recipe for antivaxx, covid denialism and overall decline of the trust in science.
But, what "blatant case of forgery" are you inventing here?
Edit:
There is no such thing as 'the science'. There is science. Which is testing and questioning to determine facts. Anything more must be "trust in the science of..." some actual thing. And that actual thing is not public policy. We have sufficient evidence that the CDC and NIH are interested in appeasing their corporate sponsors and otherwise believe themselves to be beyond reproach
Reputable sources said, correctly, that there was no good evidence that ivermectin worked against COVID. Later, they were able to say they had good evidence it did not work, a stronger statement. They could not honestly say that, early on, and did not; but we all know that almost everything doesn't work. So, anything claimed to work deserves skeptical scrutiny.
Biochemically, it would not have been surprising if ivermectin helped some. But "not surprising if" is a very, very long way from "does". Reputable sources made, in the end, the correct call. Meanwhile, people draining the ivermectin supply did themselves no good, but made it harder for those afflicted with parasites to get needed treatment. Those using ivermectin instead of getting vaccinated made themselves carriers, contributing to spread and mortality.
I don't find this to be likely or relevant. Commercially available and otc worldwide, generics.
It does have an impact by reducing parasitic load and inflammation. This can be seen in countries with high rates of parasitic infection.
Reputable sources in the US hammered the one sized solution that runs contrary to immunization history and theory. Vaccination does not 'stop the spread' which is an absurd point to make at this stage. That is absolutely evident from case counts across the US as vaccinations increased.
In fact, ivermectin supplies really were depleted for quite some time.
Boosted and vaccinated are carrying viral loads longer than unvaccinated as of Omicron
And more importantly, it would seem that all deaths in recent studies of the later omicron variants were in the boosted and eligible for booster cohorts.
Weird. Reads to me as if you're saying religion is honest?
Merely two things that are true and referred to in the opposing sense.
Religion is built upon dogma, which science should not be. Honesty is built upon truth, which science should be.
[1] https://www.alzforum.org/news/community-news/sylvain-lesne-w...
hopefully this fraud didn't truly cause unnecessary delays in pursuit of a cure.
There a whole lot of the sorts of investor who are normally shorting stuff who must feel that this is somehow insider information
"“So much in our field is not reproducible, so it’s a huge advantage to understand when data streams might not be reliable,” Schrag says."
I am just a lowly engineer, but this alarms me. Why is anything that has not been reproduced considered valid science by anyone? Why aren't our standards higher?
If you can't reproduce an experimental result, it is useless information is it not? At least an experiment that can be reproduced yet fails to prove a hypothesis can teach you something. An experiment that cannot be reproduced yields no useful information. In fact, it can even mislead!
I just don't understand the motivations at play. These are obviously intelligent people who know that you can't fake reality, so why do they publish fraudulent papers? Just for short-term gain? Do they become blinded by belief in their hypothesis?
Most scientists consider most of their colleagues more or less incompetent, and even where they accept experimental results, often reject the experimenter's interpretation of the result, often correctly. Scientists advise us to ignore the abstract, ignore the interpretation, ignore the conclusion, and trust only the data, at most. But we mostly don't get to do that for fields not our own.
High prestige is detrimental in that it short-circuits this skepticism. This happens not only in Alzheimer Syndrome work. It put psychology research in the grip of behaviorism, statistics in the grip of non-causality, political science in the grip of dialectics.
This is hard to understand for software people, since code tends to behave reproducible as a default.
Basically the only way to reproduce a difficult finding is to learn the procedure at the original lab.
An example: A friend of mine could not reproduce his own findings in another lab. Turned out the precise type of the lamp build into the setup mattered.
Another example: I could not reproduce a finding the was something I wanted to build upon. Turned out the precise method to dissolve one of the chemicals in the buffer as the problem. It was even hinted in the paper, but who would describe in detail what he means by “ vigorously stirred” ?
There was an early, very good paper identifying all the details needed to make a valid maze experiment. Nobody cites it, so nobody reads it or acts on its results.
My understanding of the situation is that academics and scientists work in a weird bureaucracy, there is an incentive to publish, academics are very bad at detecting fraud and worse at punishing it and statistical manipulation is easy and endemic. These things explain why there's so much academic research that can't be reproduced and why some academic fields are basically the modern equivalent of astrology.
As a comparison: I did a bunch of math too and in some cases just accepted the proofs I build on.
Same for programming: I neither audited lapac not GCC.
For the engineering crowd software dependencies are a probably a good comparison. If they come from a credible source you trust them and build on top of them.
Nobody blames companies for not auditing the kernel if there is a security problem in it, but everybody screams: “You should have verified the result”, if some paper is redracted.
Granting agencies do the same thing, either directly via newsletters and press releases, or indirectly, by encouraging research teams to hire public-relations people.
Members of the public cannot be expected to be able to read, let alone understand, the dizzying array of specialized articles that are published on any given topic. Heck, this is a real challenge even for scientists looking at work that is slightly outside their domain of expertise. Some form of overview is necessary, but press releases, usually biased and ill-informed, are just not the way forward.
But will press releases go away? No chance. This is how students get attracted to universities, and it's how alumni get encouraged to donate for that oh-so-important sports complex.
This is infuriating.
There's so many elements to this story that echo things I've encountered, and read about in studies of trends. For every case of outright fraud like this might be, there's dozens of "soft fraud" (read: "questionable research practices") that get the benefit of the doubt or are never brought to light. No one has discussed this here yet, but part of the story is that the program officer on one of the accused' recent grants was a coauthor with him on this very research -- consistent with trends discussed in the literature (of the largest predictor of grant receipt being coauthoring papers with grant review panel members or officers). So there's this fraud, potentially involving someone who then goes on to help decide who gets research dollars.
Your reaction is something I've wrestled with a lot. Many times there are no consequences really. People talk about lost reputation or something, but in cases of soft fraud nothing really happens. It just all kind of evaporates into a fog of scientific dispute between parties. In cases of blatant, legal fraud, someone can be fired or lose a position, but that requires significant evidence.
For what? Yes, ego, citations, money, titles, accolades as a "rising star" or a "genius" or whatever it is.
The saddest thing to me really, aside from all those harmed by bogus treatments, or forgoing real treatments, is all the researchers with legitimate ideas, who go against trend, who are pushed out because doing the hard work isn't glitzy and has lots of dead ends. You end up in this system where the buzz is what matters, and riding waves of self-perpetuating hype is what gets you to the top. Appropriate skepticism and careful thought, meanwhile, costs you because that takes time and risk.
The amyloid hypothesis is one century old.
> In 2006, Schrag’s first publication examined how feeding a high-cholesterol diet to rabbits seemed to increase Aβ plaques and iron deposits in one part of their brains. Not long afterward, when he was an M.D.-Ph.D. student at Loma Linda University, another research group found support for a link between Alzheimer’s and iron metabolism.
Yet, fast forward to December 2021 or early 2022:
> Three of the papers listed Lesné, whom Schrag had never heard of, as first or senior author.
If Schrag was doing research in the same area, at around the time Lesné's results started to dominate, propelling his career, it's odd that Lesné's existence somehow eluded Schrag for 16 years.
Did Schrag switch to something else at around that time and never give it a second thought for a decade and a half?
In the time since he published, I lost my father after investing hope that a trial that is supposed to prevent amyloid plaques and now my mother-in-law is slipping further into dementia.
Maybe not direct image manipulation as in this case with Alzheimer's, but certainly there is always a lot of monopolistic rich-gets-richer behavior.
Nearly all professors at top universities I have met develop intimate relationships with funders and journals, which they use to steer the field in their preferred direction. As the posted article says "You can cheat to get a paper. You can cheat to get a degree. You can cheat to get a grant. You can't cheat to cure a disease."
I have been asked directly to misrepresent results on several occasions. In the most recent one, a professor who has received all prizes and accolades in his field threatened me and others when we refused to misrepresent research results. I could afford to do this, but my workmates who have families to support were on the brink of giving up to the bully.
Also, make a paper/proof trail and get it to court/media/government.
Kind of related, https://forbetterscience.com reports lots of research frauds.
The problem is the pressure to get funding by being productive. Negative results are wildly harder to get funded. Less productive labs are at a disadvantage for grants. Which means, like any metric upon which people's salaries rest, people are tempted to game it.
The funding for one year from this one agency amounting to 1.6 billion dollars is a stunning.
If what you say is true, promoters of the validity of the NIH should be a push for statutory changes that mandate NIH commit a percentage of the overall budget to confirming significant past research.
The big question is what do we do now? There's Tau, but that hasn't been a slam dunk either.
Just getting caught up in the practice of self-promotion and the trumpeting of one’s “novel and impactful” ideas while managing a lab from the top without having done actual frontline research work in some time can take you pretty far from scientific integrity IMO.
With no culture of being proud of reporting negative or contradictory results, I’d say “excessive scientific zeal” is an easy and common trap. Even a slightly forceful lab-leader or PI can end up swaying a group of people into some grey dishonest zone of scientific practice.
I wonder if intentional research fraud should carry legal consequences.
https://www.nature.com/articles/d41586-022-02002-5
"Exclusive: investigators found plagiarism and data falsification in work from prominent cancer lab" (July 20, 2022)
If academic science were in a healthy state, novel results would be validated 100s of times. The current system is setup such that only novel results are rewarded. There is no reward for validation. During my PhD I was actively discouraged from performing experiments/studies that weren't "new". Everything ultimately comes back to the funding model. Scientists are only funded for groundbreaking work, measured by the number of publications. In turn, journals will only accept previously unpublished work.
I’m not saying this person is faultless, but scapegoating is hardly a sound response to what is clearly a systemic issue of misaligned incentives.
Seems likely with Alzheimers that the amyloid buildup is a symptom not a cause.
The article points to how the main accused got an R01 approved AFTER this misconduct started to come out, and the guy awarding it was one of the coauthors of the first fabricated papers.
Richard Dawkins won't be impressed.
You could still search these retracted studies when doing research, of course. You just can't cite them.
Correcting the many incentive problems in modern American science would need a hypothetical body with significant funding leverage over journals & scientists to exert executive action. Sadly there is no such centralized funding body, so the problem must be unsolvable.
One potential problem is that it could become "the" authority on reproducing results. If they repro something and another scientist can't, or vice versa, would the other scientist get ignored?
Another problem is that it could take away the independence of science. The government might start saying what is good science and what isn't.
Any other institution, anywhere else in the world, is free to publish results which disagree.
Scientists should then have the necessary tools to figure out which one is likely to be correct, and try an independent third or fourth time.
It would be expected that sometimes there would be a failure to replicate which was a mistake in how it was replicated, that shouldn't be seen as being a failure of the goal of replication, and the original authors should be incentivized to reach out and discuss the issues with the methods.
You can share equipment, but if you share data, then it's the same experiment not a new one reproducing the first.
Sharing data would not only miss fraud, it would miss experimental errors (e.g. spilled beer in the petri dish), p-hacking, etc.
That sounds worse than what we have since it just eviscerates the significance of what replication means in the first place.
It was actually the incentives created by the free market that finally unravelled this whole thing, as per the article.
- any university receiving public funding must participate in mandatory replication
- some % of any grant would be reserved for replication
- replication labs would be licensed/accredited, subject to government inspections
- labs would not be able to choose who replicates their work
- replication labs would be rewarded for the work, not for the outcome
Wherein are you going to fund mandatory replication studies - some of which are massive cohort studies, so you're going to need a whole new population cohort - let alone the...citation police...to review every citation not just for its existence, but for its content.
My idea hacks around the problem by not diminishing that huge incentive to be first. Even if your paper is retracted due to lack of repro study, you were still the first, and if it does repro you are back to full credit.
I just add an extra incentive to get people who cite the study to verify that its reproducible. And at the same time, give the original author more incentive to include lots of details in their papers so it's more likely to be reproduced, and lock in their fame.
But given the tremendous complexities involved in biological systems one has to be very careful with the data. Quite frankly most in the field are not sufficiently trained or aren't rigorous enough in dealing with uncertainties and I wouldn't blame them it's mind-boggling and paralysing.
This of course can be exploited given the incentives stated above combined with the public's unawareness of e.g. statistical significance vs clinical significance etc., with the myriad mathematically correct ways of presenting data, and a handwaving attitude about reproducibility, this really opens up the floodgates for someone in a lab to become "creative", an entrepreneur. Which of course should be highly discouraged in the context of scientific research and in order to relieve some pressure the community once in a while condemn the most daring of examples without imhv looking too deep into the entrenched mechanisms enabling this.
A PhD student (biomedicine) once jokingly told me about the old HeLa contamination "problem"[0] in labs experimenting with cells. I must have reacted shocked as he laughingly added: Well, that was quite a long time ago, the problems now, only compounded.
So, in order to get the full picture it is "healthy" to zoom out and look at the other side of the spectrum i.e. longitudinal studies (with its own sets of limitations). One impressive one relating to Alzheimer's is the famous nun study[1].
You refute a paper by showing that part of it is incorrect. The research sleuths have provided incredibly strong evidence that the very existence of Aβ56 is demonstrated (in the paper) by a sham line on the western blot.
It's like somebody has demonstrated that a photo of a "ghost" is actually a double-exposed film, and you are stating "What's notable is that the entire article here never once refutes the most important finding of the original photo (that ghosts can haunt the waffle house)."
A doctor I know takes Kordon methylene blue which is pennies a dose.
https://clinicaltrials.gov/ct2/show/NCT03446001?term=TRX0237
I don't see why we should have any amount of optimism in this direction based on the evidence in front of us.
Do you hold stock in TauRx or its affiliates?
https://www.alzforum.org/news/conference-coverage/first-phas...
Scientific consensus has value, but science also requires that people be open to having their pet theories be validated through replication.
Consensus is an extremely time-consuming thing to build, and it's extremely important to be aware of when there exists a consensus, where there isn't one, and what the consensus is.
It isn't an appeal to authority fallacy, it's a form of deferral to expertise, and it's one of the most important heuristics we have.
In my alma matter field the question was whether her2 receptor recycles (has implication of all breast cancer antibody therapies that target her2) and a paper from a Genentech lab had ONE FIGURE (where different data points from different experiments) as proof. An entire sub field including projects in my own lab were spawned assuming this.
Whenever I point out this flaw in lab meetings I’d be shut down by my professors as “they know the authors they trust them”.
To say that everyone "believes in" a model could mean that everyone accepts it as plausible (and thus worthy of further exploration), or it could mean that everyone is justifiably certain that it maps properly to a real phenomenon.
I never say that something is "consensus" in the first case: IMO, the term should be reserved for the latter case, or appropriately qualified.
In any case, the situation you describe appears to lack meaningful triangulation. [1]
The experience of saying how you would do it better and getting it torn to shreds really brings you back down to earth.
What's really interesting to me is that such vast sums can be spent, a number of people apparently did not like the Aβ*56 hypothesis and had substantial reason to oppose it or try to find holes in it with substantial rewards for doing so, yet the ones to find problems were some dudes (without any funding at all) just poking around.
This is a well known thing scientific fraudsters do, “touche finale”. Any scientist knows preparing figures for publication is tedious, no PI in their right mind would routinely do that.
Presentations like these are very compelling:
I work in the neurotech/sleeptech space, and specifically in stimulation of slow-wave oscillations.
These SWOs decrease as we age, which is linked to the build-up of amyloid plaque, and increased insulin resistance. They aren't necessarily linked to each other, but rather to the decreased capabilities in the brain, from what I understand.
1: https://physicsworld.com/a/retraction-of-nature-paper-puts-m...
I guess he’s the Nobel prize winner, but I would’ve thought the immediate and obvious damage was unnecessarily delayed research progress while people are dying.
Over a decade? Who knows what the overall impact was.
"A neuroscience image sleuth finds signs of fabrication in scores of Alzheimer’s articles, threatening a reigning theory of the disease."
it would be better to see:
"A neuroscientist conducts an irrefutably strong study that upends 16 years of theory on the disease process behind Alzheimer's."
i don't care if someone cheated, i care that their results are discredited by scientific processes, not forensic review.
Wikipedia: "the scientific method involves careful observation, applying rigorous skepticism about what is observed, given that cognitive assumptions can distort how one interprets the observation."
So identifying fraud is the scientific process.
i guess the bigger point is that the act of topical inquiry should have inbuilt mechanisms for discarding ideas and approaches that turn out to be dead ends, without having to rely on them actually being fraudulent (they can be, and often are, completely legitimate, yet also completely wrong).
we shouldn't be discovering bad ideas in science by fraud detection, we should be discovering them by mainstream scientific process.
My un-scientific version of this is that if you have a large bowl of ice cream for desert every night for 55 years, you may develop Alzheimer's Disease.
I wonder if you could get the same unexpected result when copying images of Western blots?
It's probably not the case here but it could be devastating for a researcher to be accused of fabricating data by using an affected copier/image editor/file format.
A western blot is a direct measurement.
It's like asking whether the xerox example might explain why the splatter pattern is identical in sections of two putatively different Jackson Pollock paintings.
I've also seen very qualified people have a massive holes in their perception, making them somehow unaware that they're generating garbage, and even defending the garbage when it's pointed out.
Yes, it's good that science usually deals with such problems in the long run, but how is the average person supposed to trust that the latest scientific assurance, isn't 15 years away from being retracted like in this example?
Rational treatment might enable identifying individuals particularly at risk and not vaccinating those, but that option is closed to us. Instead, a random, suspicious fraction of the population pays particular attention to negative outcomes and avoids vaccination, to its detriment, and most of those at risk for problems get vaccinated anyway.
I don't think it was necessary at all, and instead is very counterproductive. Many people know they're not being dealt with honestly by the government and media, resulting in more distrust and resistance to vaccination, than there otherwise would be.
It is a tragic calculus. "Trolley Problems" are very far from theoretical in public health management. We are forced by distrust to sub-optimal choices that themselves promote distrust. Managing risk of a better population would be easier, but you battle pandemic in the population you have, not the population you want.
So tell the truth, build trust, and let people decide for themselves. That will save lives in the long run.
But even if you disagree, let’s talk about your claim that categorizing into two groups is stupid. Think from the citizen’s perspective of the CDC. The question is, “Do I trust them enough to take health advice from them?” If they have lied to you too many times about things that are important enough, the answer is no way. I never implied all that this lying group said was lies. Obviously one lie wipes out thousands of truths. Trust is gone. Listening stops.
And all sources are flawed!
You can look into major issues without needing authority figures.
> When the public institutions become untrustworthy, and this lose their authority
Trust is not binary.
No source is unbiased.
No you can’t. Not unless you are doing the study yourself. And even then, almost all studies rely on other authorities. Things like death certificates and cause of death, hospital reports, etc. all depend on authorities and you have to decide whether you trust them for this thing.
I never said it was binary. I said trust can and is lost through lies. If you can’t acknowledge that, I’m not sure what else to say.
"Obviously one lie wipes out thousands of truths. Trust is gone. Listening stops."
I would call this binary. I don't really care what we call it, though. Especially if downplaying counts as lying, then this policy is completely infeasible. It means nobody will ever be listened to. Trust is wiped out in all circumstances.
It's a really stupid way of handling a biased source. And all sources are biased, so it's a really stupid way of handling sources.
If you want to say that one lie adds skepticism to a thousand truths, that would be a massive improvement, because A) it works well to be somewhat skeptical of all sources, and B) you can still learn from many sources you're skeptical of.
The best we can hope for is people doing their best with what they have to work with. Very many do.
The worst make shit up, routinely. They hone their message to attract dupes, and always succeed. Many of them believe whatever pulls; most don't care what is true or isn't.
I have no idea why you are talking about perfect authority.
What exactly is “doing their best”? Is it lying, trusting the end to justify the means?
There are always an expected number of deaths in any period of weeks after (or without) a vaccination, subject to big random fluctuations. It should be obvious that (a) numbers cannot be interpreted correctly without education, and (b) people without such education finding numbers will insist on interpreting them anyway, some ignorantly, some with active malice.
What and how much to publish about numbers reported are hard choices I am glad I don't need to make.
Even publishing nothing, there will be spurious reports claiming to know official numbers, and spurious interpretations of spurious numbers. Your fragile trust is broken regardless, among people so inclined.
Science shouldn't even be engaged in trying to save as many lives as possible. Science should only be concerned with discovering and disseminating the truth as it is.
I do empathize with the people trying to minimize deaths from a raging pandemic in an atmosphere of politically-motivated disinformation that is actively contemptuous toward public safety. People with your attitude make their work that much harder, and cost more unnecessary deaths.
However, you were advocating for not giving people accurate numbers. Whoever is in charge of that decision should not lie. They should give accurate numbers.
What they should do as a matter of abstract merit, or of public perception of benignity, are two wholly different, generally easier and less vexing questions.
I have plenty of complaints about how public health measures are prioritized in the US. I am not worried about official dishonesty. Public officials are just as good at fooling themselves as everybody else, so they can be wrong without lying, and will be at times. Expecting somebody, anybody to be right all the time is a recipe for disappointment.
We are guaranteed pandemics regardless, just by how much international travel we do. What matters is the response. We are lucky monkeypox (actually a rodent illness) is rarely fatal.
And, some people will get in car accidents on the way home from the clinic, that would not have happened if they didn't go. People who get in line are exposed to random pathogens others in line are distributing, and to any pathogens injected via insect bites in that place.
People who do not get vaccinated are subject to similar risks, but are not counted.
Playing up these numbers does nobody honest any good.
This is sarcasm of course.
Economics is a field that has been particularly resistant to correction, but is far from alone. Geology and statistics are recovering from a similar handicap.
As Max Planck is often quoted, "Science advances one funeral at a time." Often vindication is finally delivered only after all the opponents are dead, and the ultimate victor has retired from a career blighted by them. Probably much more often people are driven out of the field and never vindicated.
Lynn Conway was driven out of computer architecture (where she invented out-of-order execution in the '60s, thus long delaying that advance) before finding success many years later in VLSI chip design methods.