How Much Science Is Fake?
osf.io
osf.io
That's why "show me the study" is such a midwit line of reasoning.
We need to go back to thinking more with first principles, received wisdom, and personal experimentation.
There's a place for scientific studies, but it's gone so far that many won't take an obvious point without a study behind it.
There are many such examples that are quite obvious but are hard to prove: processed foods are bad, screen time is bad for kids, starship won't fall on a shark etc. But people love to poke holes in obvious truths, especially in well educated spaces like HN.
It could be...
- unjustified and sloppy (not 'fake', but also not considered reliable evidence)
- unjustified and malicious (this i would consider 'fake')
- unjustified and gamed (again, 'fake')
- ...or just unjustified and under-specified (and would result in 'true' results if the conditions for replication were better studied and defined)
Almost no one is interested in having an honest discussion about whether or not the original paper actually says what it’s characterized to have said, and whether it was a good study in the first place.
So nowadays, when public policy is concerned, largely I disregard any scientific study that is introduced to support any position on the policy, and just do my own cost – benefit trade-off to determine my policy position.
But in general, I agree with you. It's ridiculous when someone pretends to shut down a complex issue by citing a random paper. However, an expert can still analyze the whole academic literature on a topic and determine what the scientific consensus is and how confident we are about it.
Unfortunately, they would. There are papers with thousands of citations that don't even have data samples, just models based on assumptions.
Maybe a lot of scientist really liked the proposed 'model' and/or some discussion on the assumptions.
and cited it in their paper proposing some additions or follow on work.
That is also fine.
"Computational modelling is a useful technique for predicting the course of epidemics [1][2][3][4][5]"
The cited papers wouldn't actually support the statement because they'd all be unvalidated models, but citing documents that don't support the claim is super common and doesn't seem to bother anyone :( Having demonstrated a "consensus" that publishing unvalidated simulations is "useful", they would then go ahead and do another one, which would then be cited in the same way ad infinitum.
But there are loads of papers like this. Then you have some literature studies which look at all these papers together and get result aggregates.
Then you get some “proper” studies which link to these aggregates, and several small studies, and you’re going to read these “proper” studies which are quoted often and deemed decent or good quality.
And at no point will you realise it’s all based on shoddy foundations.
This is for example what recently happened in social psychology
That’s not a science problem. That’s a political problem. Making choices based on a single paper that may not be replicated is the problem.
I think a better position is that we should have a higher bar of what level of study or replication is required for a given situation...whether that be health, housing, infrastructure or whatever policy is coming in...what kind of monetary outlay and timeline of impact is expected. I don't think most people here would be happy with a 6-person study, unreplicated, deciding policy...so what IS the threshold?
Take the climate. Assuming the science is correct (storms worse, oceans rising, etc), let’s do a cost benefit analysis of the proposed policies.
The proposed policies in the US all dramatically increase the cost of energy (and therefore of everything), but only slightly slow the progression of warming, AND the bigger contributing countries in the developing world, esp China and India, will continue or increase their CO2 output.
We already know how to build dikes, and people who buy oceanfront property already know the risks, and over the last century we as a species have gotten really good as reducing deaths due to weather, so I don’t support the draconian carbon reduction proposals, and instead we can just deal with the side effects as they arise.
"The proposed policies in the US all dramatically increase the cost of energy" - why? How do you even begin to conclude this without looking at some sort of (economic/scientific) analysis?
"only slightly slow the progression of warming" - again, how are you concluding this?
"we as a species have gotten really good as reducing deaths" - why should this trend continue? Why should it continue in the face of more extreme weather/climate change?
All I see are things you _think_ are true, and so to you your argument seems sound. But as the comment you replied to said, all I see is ignorant, sloppy science, since any meaningful analysis of these policies is by definition science. These cost/benefits you mention are not universal apparent truths.
As to your other questions, the answer to all (which any trivial web search would answer as I did) is “I can read”.
And I didn’t “cherry pick” some particular study or some blog that supports my preconceptions. All of the links I have provided are from sources that assume severe climate change is coming without serious interventions in greenhouse gas emissions; they are all on the “climate activist” side.
Dramatically increase the cost of energy, note that this is BY DESIGN, it’s the entire point of carbon taxes and markets: https://www.npr.org/2021/12/16/1064951646/why-the-cost-of-ca...
Only slightly slow the progression of warming: https://www.science.org/content/article/paris-climate-pact-5...
Weather related deaths down 98% in past 100 years: https://reason.org/policy-study/decline-deaths-extreme-weath...
And of course now I’m being down voted for my example.
I never challenged a single point of climate orthodoxy; I never questioned whether global warming was occurring, whether it was man-made, whether the studies and models predict warming accurately, or whether the impact will be as stated. I stipulated every single one of those points. But simply my personal conclusion that I don’t believe the policy trade-off is worth it, means that my voice is not worth hearing.
How much 'policy' are you happy with that you've never checked to validate that it was properly replicated beforehand? How much of your skepticism is against "the science" on new policy you start with an initial dislike to...compared to policy that we consider as standard, are probably comfortable with, likely consider to be 'beneficial enough'...and yet haven't adequately scrutinized?
https://www.science.org/content/article/many-scientists-citi...
First principles + received wisdom are counter balancing, not contradicting. Everything in life is a balance.
If you describe it as 'fake', I consider that to give the impression of 'the answer is NOT' this, and could lead to anti-policy.
If the description is 'unjustified and sloppy', that can lead to additional research to properly invalidate or potentially find something useful, so we don't throw the baby out with the bathwater.
If we start from first principles: - Any one individual is not trained or experienced in the vast majority of scientific disciplines, regardless of his/her expertise in any one field - Even if half the studies published have flaws (where did you get that stat? was that study flawed?), their body of references is filtered by decades of replication and paper ranking of importance - The people doing the studies have decades of study in the particular field they're publishing about
Concluding from first principles: Its more likely that a random individual's opinion about a random niche of scientific study will be wrong than that of an individual trained on that niche
And then also starting from "received wisdom": Joe Schmuck on the street doesn't know more than scientists about their thing
Also there are lots of Joe Schmucks that know better than a so called "nutritionist" that a steak is better than an Oreo. Not all Joe Schmucks, but a lot of them.
People of a given field will also often circle the wagons around bad actors to protect their own jobs and reputations. This introduces a very strong bias in the people who should be the most highly-informed on their own subject, and sadly makes them less trustworthy than people like Data Colada (https://datacolada.org/) who go beyond their own fields.
Most fields are really bad at stats which is one thing that is pervasive across just about all science. More than half of science wouldn’t exist without some sort of fraud, clear selection bias, or p-hacking.
It’s to the point that one sure way to lose friends and be shunned in academia is to be good at statistics.
It’s like the plagiarism scandal, academics have been getting away with it for so long it’s become normalized. It used to be that plagiarism would be the absolute end of any academic career. Now some academics are arguing for an amnesty on the basis that everyone did it and others are ignoring it.
Edit: I should point out like the other commenter that a lot of the fraudulent data is laughably blatant. Like pulled ‘random’ numbers straight from their head, not even using any type of random number generator. Or pulling random numbers from wrong distribution. Or not using realistic noise etc.
Well ... this intuition should definitely be true in theory, but in practice some fields are sufficiently distorted by biases (financial, ideological) that it's possible to be more correct as a random individual than an individual trained on a niche. The random Joe Schmuck may or may not have less knowledge, but they definitely have less career-related bias and that is often decisive.
There are plenty of examples where ordinary people manage to systematically beat the experts. A good recent example was the COVID lab leak theory. The entire field of virology and public health came down on one side of that issue, even though it was obvious that a lab accident was highly plausible, and then it turned out they all knew were all lying through their teeth and did it deliberately anyway. The rando nobodies who said "yeah, well, I don't trust the experts" came out ahead and this is now widely acknowledged. In the UK both Boris Johnson and his at-the-time health secretary have admitted that it is the most likely explanation for where COVID came from.
Ultimately, training is of no use if incentives cause people to mislead.
Are they? What kind of food is not processed that we should be eating?
I think I know what people mean usually when they use this term and like that in a sentence, but this term has always confused me even far into adulthood.
Still pointing it out, since I think it's such a bad or misleading term to use.
yes they're bad. It's not confusing, you're just overthinking it.
A processed food is any food or drink that has been changed in some way when it’s made or prepared. Most foods we eat are processed in some way.
Processing can be used to:
make foods safe, for example milk is pasteurised to remove harmful bacteria make foods suitable for use, such as pressing seeds to make oil preserve foods or help them last longer, such as tinned or frozen foods change how food tastes, such as adding salt or sweeteners create ready meals and snacks
Not all processed foods are unhealthy, but many ultra-processed foods are high in calories, saturated fat, salt or sugar.
Eating too many calories, too much saturated fat, salt and sugar, and not enough fruit and vegetables and fibre is not good for you.
----
Like why should processing something make it inherently unhealthy? Do you want to eat raw meat?
> people love to poke holes in obvious truths, especially in well educated spaces like HN
Someone makes a statement that they say is obvious truth, yet it seems false or misleading. You can't get away by later saying that someone will try to counter this "obvious truth".
It's also hiding what are the exact and true health problems behind certain foods.
Then you get those terms that are used and you get the anti-terms.
"All natural", that is supposedly good, etc. Except there are quite many poisons and things in nature that can kill you.
Then people get those strong feelings associated to those labels, and they get manipulated in various directions, because something can be considered to be of that label.
I probably had quite a few inaccurate beliefs in general because of this type of terminology.
Another common thing that made me have inaccurate beliefs was the "E" labelling in Europe. And I remember being told that "oh look at how many 'E' (evil?)" substances are on this package.
But also Curcumin, Vitamin C, etc are considered E substances.
So just so many inaccurate ideas all around. And I don't think this inaccurate labelling has helped much if people are unhealthy.
We can keep on going with all possible ingredients and all possible combinations and most sane people will agree on what is called "processed" or not.
This is disingenuous discussion dressed up in smarty pants talk.
Shall we continue or actually discuss something substantive? Like the fact that Americans are morbidly obese and eat 60 pounds of sugar a year. We look like crap.
You're not some harbinger of truth if we do that.
In fact, it's pretty evil (idc if you're Harvard, UCLA, or a commenter on HN) to argue over whether or not you can distinguish between a cooked carrot or an Oreo.
Which brings me back to my original point. That educated spaces need a study or even just a link to an authoritative source to accept even the most basic point. Here's a Harvard link for you: https://nutritionsource.hsph.harvard.edu/processed-foods/
Hope that's satisfactory for you. LMK if you need a UCLA or Stanford link.
I think it would still be better to not mention "processing", because it's not about the "processing", it's about what is there in the end foods that can cause issues. So I would say that excessive consumption of foods loaded with unhealthy fats, sugars and artificial preservatives, flavor enhancers are bad for you". It's bad to eat too many fats, sugars, calories in general and not enough fruits, vegetables which contain ingredients that your body needs to properly function. If you want to use a circular word you can also just use "unhealthy foods", not "processed foods".
> In fact, it's pretty evil (idc if you're Harvard, UCLA, or a commenter on HN) to argue over whether or not you can distinguish between a cooked carrot or an Oreo.
I'm not arguing about that. I'm arguing about what is healthy and what is not. Processing food inherently doesn't make it unhealthy.
Is orange juice a fruit? There, take some of your own medicine lol.
In all seriousness, if you're not convinced ultra processed foods are bad for you as a rule, there's nothing I can do except throw a bunch of studies at you, which I've already argued is insane for such a basic and common sense concept
I know you probably will take great issue with "common sense", but most people I know, don't. That's why it's common lol
And presumably you've already seen lots of these studies so I don't even know why we're arguing something that makes sense and is also backed up by many studies (though a large proportion of these studies are likely to be garbage)
I'm not sure what you are trying to imply here? That it is? Orange juice is not a fruit, it's a juice derived from a fruit. Which is healthy in moderation, but definitely not enough alone to cover all the optimal ingredients you would get from a variance of different fruits.
> In all seriousness, if you're not convinced ultra processed foods are bad for you as a rule, there's nothing I can do except throw a bunch of studies at you, which I've already argued is insane for such a basic and common sense concept
I'm not convinced because it all depends on how it was exactly processed and the amounts.
oreo = ultra processed
a raw carrot and a cooked carrot have different nutritional profiles.
In fact I claim "cogent" isn't even a cogent concept
How about "claim"?
See how these definitional arguments become un-useful and boring?
"what are your "let's just substitute a different word here" claims based on?"
Being annoyed that even when the comment is "people love to poke holes in obvious truths, especially in well educated spaces like HN." HN commenters begin to poke holes in an obvious truth- that there even is such a thing as processed foods. It'd be like arguing that heaps of sand don't exist because it's not "cogent concept"
Just ask anyone in the braintrust states...
Assuming you're being sarcastic. I never said anecdotal evidence. And indeed all 3 (first principles, received wisdom, and personal experimentation) are helping get to space, arguably more than research papers
There have always been lower quality work and journals that are inherently given lower weight in the scientific community. Many of these lower quality papers just aren’t considered by working scientists. It’s a bit like email… once you start seeing a significant amount of spam, you just get used to ignoring it. This is a feature of the system. Scientific consensus can change over time, but it takes a lot of evidence to get there. But some papers get more weight than others.
For me, there are two real issues - first, public trust in science. When the public thinks that 14% or “science” is fake, they are likely to discount all scientific findings, regardless of source. This can be dangerous, especially for a public who is already distrustful of science/academia in general. So, instead of looking at results with a critical eye, the danger is that a large fraction of the public will just ignore all results.
The second issue is in polluting AI training. When you feed extra garbage into a system, you’ll get garbage out of the system. As LLMs get more specialized having high quality inputs for training is critically important. A contributing force here is that lower tier journals (those most susceptible to faked data), are also the most likely to make the full text of papers widely available. So, the easiest data to get for training is also the most likely to have faked data.
Then, someone worked out a theory and replicable experiment (humans and guinea pigs are the only mammals that die horribly when not fed vitamin C) and with a replicable experiment in hand, you could use little s science and now no one gets scurvy, trust not required.
The argument that science should be trusted (believed) is insane, the whole point of science is questioning everything and deferring skepticism only in lieu of anything that can disprove.
Trusting science is just another form of religion, especially since science ("science"?) is often positioned as the rival to religion.
I'm assuming you trust at some point? Have you ever flown? We have theory, we have evidence, and a high level of _justified true belief_ (to refer to recent HN posts) for how flight works, but perhaps all _these_ replications are just fortuitous and 'fake'? We 'trust' Newtonian physics enough for society to _act_ and _progress_ ... and then we quantum comes along.
Enough evidence such that no evidence to the contrary can stand.
We agree that liquid water is wet because we can't find anything (yet) that says liquid water is not wet, it's simple as that and science is ultimately as simple as that.
>Have you ever flown?
Yup, I'm a frequent flyer actually and fly more or less every month or two for business (and pleasure once the business is done, but that part's a secret).
I find this a weird example of something that has "Enough evidence such that no evidence to the contrary can stand."
"Water is wet" is a definition, not a scientific finding/description. "Water tends to adhere to, and penetrate, materials." That's sort of a scientific finding. "We call this condition wet" isn't so much.
Science, by Popper's definition, is something that can be falsified but has not yet been so. If no evidence could contradict our current findings then, by definition, our current findings are not falsifiable.
I realize that I'm quibbling about definitions, I guess I just want to point out that your level of evidence is dangerously close to being a self reinforcing tautology. Similar to how any religion justifies itself.
The point is that our agreement is merely the result of there being no other conclusion as far as we are aware, and to expand our awareness we must be skeptical of our conclusions in perpetuity.
This is why trusting science is insane, because if you trust something you stop questioning it and if you stop questioning then you never know if it's true or false.
>Science, by Popper's definition, is something that can be falsified but has not yet been so. If no evidence could contradict our current findings then, by definition, our current findings are not falsifiable.
I'm going to ignore that you just defined "current findings" as not science, since I presume that wasn't what you intended.
> the whole point of science is questioning everything and deferring skepticism only in lieu of anything that can disprove.
Childhood learning is about increasing trust of science thru experimentation of gravity and energy (heat/cold, momentum, friction, etc).
Without that trust, there would be no learning. Perhaps that is the point of unreasonable skepticism.
The general public does not have the time to study climate models and review raw data on vaccine efficacy.
Without overall "trust in science" (or perhaps "trust in the scientific community") we lose the ability to form data driven policy.
The exception being when there is monetary value to be gained by fooling the public. IOW, be skeptical of studies funded by special interest, but
"Do not be so open-minded that your brains fall out."
I like making my own opinions, because that's how I become informed. If I'm just taking in what others say, I'm just an NPC acting along their programming.
Side note: the “1 in 7” claim from this paper is based on a straw-poll of N=12 existing forensic metascientific papers. I find it somewhat ironic that the author makes some strong conclusions based on this number which to me, though very plausible, is itself lacking scientific rigor.
Identifying fraud is critical but I worry a bit that baserates of such statistics get processed as an estimate of "what proportion of science is problematic", when the problems are broader than that.
An interesting paper would take the existing LLMs and see how well they can do flagging problematic work. A solution that works well at scale is needed.
Heck, an AI assistant to aid peer review would be great.
I would like to see this as well. I would be curious if LLM's start to see the same patterns I see having read half of nih.gov going down many rabbit holes of researchers scratching each others backs, pencil whipping reviews and such. Can LLM's pick up on such behavior or might that require something closer to human intelligence? I do not know enough about LLM's to even guess. Could it perhaps at least pick up on entirely fake papers from Youtubers pranking the system to see if they can get their silliest papers published? I report them but by the time they are taken down the Youtubers have had a large number of views and laughs.
Sounds incredibly legit, really motivates me to read the article https://osf.io/s4gce
Also, are we not hackers? If we can't sprinkle jokes into discussions of serious things without thinking that makes the whole discussion unserious, what have we become? Might as well go join HR or become a politician or something…
Maybe if it was something like "Once upon a time humankind discovered the scientific method, the most accurate tool that they had so far. This has made a lot of people very angry and has been widely regarded as a bad move. To help with that here we have spotted some inaccurate articles to appease the angry."
> ‘Fake’ science is either intentionally fabricated-where quantitative elements are invented-or intentionally falsified-where results are dishonestly engineered from real data.
2) The title "How much science is fake?" is clickbaity. The actual assertion of the paper:
> a preliminary approximation is that 1 in 7 published papers have serious errors commensurate with being untrustworthy.
3) The paper is a meta-analysis of 12 other forensic metascience papers. The approximation of 1 in 7 (14%) is an average of estimates derived from those other papers.
4) It's important to note that publishing papers is just one small subset of the massive endeavor that we call science. Moreover, as far as I'm aware, "publish or perish" appears to be a relatively recent historical phenomena of professional academia. We could probably do without many published papers, which exist merely to advance the careers of the authors.
5) In any human endeavor that involves money and/or power—which is almost everything—there will be corruption. It's unavoidable, given human nature. Thus, there's nothing uniquely untrustworthy about science. And science was never intended to be a religion: self-criticism is supposed to be a feature, not a bug, so my personal view is that the open study of fake papers is a sign of health rather than a sign of disease.
6) Is 1 in 7 really that bad? The converse is that 86% of published papers do not have serious errors commensurate with being untrustworthy.
Because they find 1 in 7 with serious errors, but their definition of "fake" requires it to be intentional. Where's the evidence it's intentional for all of these 1 in 7?
Arguments about intentionality are a bit of a distraction. Academics routinely try and excuse obvious fakery by claiming to be incredibly incompetent. At some point you have to treat unlimited incompetence as harshly as deliberate deception.
Just ask anyone in the braintrust states...