The Phrase “No Evidence” Is a Red Flag for Bad Science Communication | Hacker News Reader

The Phrase “No Evidence” Is a Red Flag for Bad Science Communication

astralcodexten.substack.com

77 pointsfeross27 comments

My take is these two meanings:

"We're actively looking for this, because if it's true it would be important, but we've not found it yet so we haven't moved to the next stage that will be required if/when we find it"

vs.

"People are spreading this lie and we don't want to seem super sure of ourselves because, hey, we might be wrong and we're not wanting to say that such a thing is categorically impossible but currently the experts say no"

And yes, it's unfortunate that in English these two meanings coincide to a degree. Language can be hard, especially if your trying to fit it into a tweet or headline and people are actively trying to misunderstand you.

But the meta point seems to be, if your political tribe all adopt very stupid opinions on something that you're an expert in, then someone outside the ingroup must be found to blame for that. Of course, if the outgroup all have a stupid opinion then it's proof of their fundamental wrongness and inferiority.

But, if we had to lose one, I'd go for replacing the second meaning. I'm quite happy for it to be "Expert consensus" instead, but is that defensible against someone who politically disagrees with you? Apparently not if you follow climate change science. Who is a journalist to decide that 95% of scientists in an area agreeing is consensus. So here we are.

Another phrase to avoid I think is "statically significant" or "significant". This is used even when the effect size is small, but people generally hear "significant" and think "a large amount". Small effect sizes can come from experimental bias/flaws that disappear when better experiments are done later too.

You also see it in the context of "the result wasn't statistically significant but was bigger than all the rest" as if that means something.

This is where we get, "Absence of evidence is not evidence of absence".

That can also be abused, but it takes some doing.

Sometimes absence of evidence really is evidence of absence, if you would expect evidence to have surfaced by now.

Nobody knows whether 5G will cause cancer. Nobody knew whether 3G and 4G would, either. We ran the experiment on the general public, and with very large N, it seems like not: no rash of cases seems to have appeared. Absent evidence is considered evidence.

But that tells us nothing about 5G. It looks like they are going to run that experiment, too. (Does anybody need 5G? Doesn't seem to matter. One thing we know is you and I will be paying for it, regardless.)

Logically absence of evidence would never equal to evidence of absence

It at most make the theory more and more convinced over time and/or other things

However the possibility of presence would not be eliminated by "absence of evidence".

Unless there is evidence that also eliminates the possibility of presence.

>Does anybody need 5G? Doesn't seem to matter. One thing we know is you and I will be paying for it, regardless.

Just bought a new phone. I wanted the cheaper non-5G version, since I believe the phone will be broken years or decades before Germany has useful 5G coverage. But the 4G version was sold out.

Nobody's suggested plausible mechanisms for low-power radio in those bands causing cancer, either. The only EM-induced cancer risk you should be worrying about is from the big EM emitter in the sky.

There is, in fact, plenty of evidence for nanoampere current causing profound biological effects. I personally attended a talk in 1987 by a biologist who induced insect larvae to grow to 4x normal size by E-M stimulation of such a current. And, there is George Becker's "The Body Electric", which anybody may read, summarizing a whole career, funded by the US Veterans Administration medical program, investigating those effects in detail.

Certain modulations of radio-frequency E-M in those bands can induce current of such magnitude, even at power levels such as are produced by cell phones. But the regulatory agencies do not admit effects of modulations; regulations admit only the effect of heating by baseband energy absorption, microwave-oven style. On the plus side, the modulations used for digital transmission are not generally expected to produce coherent currents. But nobody has actually verified any of that.

Generally, if you are planning to bathe an entire population in a form of radiation it has never been exposed to at any time in its entire evolutionary history, it should be your responsibility to demonstrate it won't cause any trouble. But, under present FCC rules, it appears not to be.

So, instead, the experiment will again be run against the general population, and what happens will happen. Fingers crossed?

We have numerous examples from the 20th century of blithely assuming that new physical phenomena are harmless or even beneficial, with subsequent harm. My mother, like many in her generation, had cancer in her nose from (ineffective!) x-radiation treatment for acne 50 years before. That was not caused by heating.

Thanks to the magic of modus tollens, absence of evidence can indeed be evidence of absence. "A 100kt nuclear bomb just went off in your front yard!" Me, looking out the window, nope. It depends, of course, on the argument being made. It could be true in some cases.

As cliches go this one is better than "you can't prove a negative"--not true. At least the "evidence of absence" one makes you think about causality and the nature of evidence.

If you look out the window, that is evidence. If you are heard to have reported looking out the window, that is itself evidence, because vaporized people don't report.

So, not an example. But, actually checking should be expected before proclaiming "no evidence". And, before parroting claims where there really is no evidence.

Until as recently as 2000, hospitals were routinely injecting quite large amounts of mercury-based neurotoxic disinfectant (thiomersal) into infants, when they got a dozen simultaneous vaccinations each preserved with the stuff. They stopped putting it into American vaccines about then, without admitting it had caused any harm. AFaIK it is still used in vaccines sent to poor countries. (Thiomersal is, anyway, excreted pretty quickly.)

> without admitting it had caused any harm

There's no evidence it caused harm. For example, quoting https://en.wikipedia.org/wiki/Thiomersal#Toxicology - "the World Health Organization has concluded that there is no evidence of toxicity from thiomersal in vaccines and no reason on safety grounds to change to more expensive single-dose administration"

> They stopped putting it into American vaccines about then

Because American worry warts were induced into a false panic by a fraud pushing a false connection between vaccinations and autism, leading to a specific belief that the mercury in thiomersal was the main factor.

The US authorities believed the precautionary response of removing thiomersal would increase public confidence in the vaccination system, even without solid evidence that it caused a problem. (The evidence by comparing autism in the US with a country that didn't use thiomersal was that thiomersal did not have a contributing effect.)

The US can do this because it has the money that a poorer country does not.

However, this precautionary removal caused people like you to believe the authorities were hiding a connection.

See https://www.cdc.gov/mmwr/preview/mmwrhtml/mm4826a3.htm for the stated reason. (" There is a significant safety margin incorporated into all the acceptable mercury exposure limits. Furthermore, there are no data or evidence of any harm caused by the level of exposure that some children may have encountered in following the existing immunization schedule. Infants and children who have received thimerosal-containing vaccines do not need to be tested for mercury exposure.")

> when they got a dozen simultaneous vaccinations each preserved with the stuff

That's exaggerating. https://www.cdc.gov/vaccines/schedules/hcp/imz/child-adolesc... lists 7 simultaneous vaccinations for most infants and 11 if including all high-risk groups.

However, the CDC link points out "Some but not all of the vaccines recommended routinely for children in the United States contain thimerosal". https://www.cdc.gov/vaccinesafety/concerns/thimerosal/index.... says MMR, Varicella, IPV, and pneumococcal conjugate vaccines have never contained thimerosal, which are 3 of the 11.

It did not, in fact, cause "people like me" anything.

But the topic here is standards of evidence. Before injecting a substantial quantity of neurotoxic material into newborns, the burden of proof should be on you to demonstrate that it does not cause any harm.

That it appears not to cause epidemic autism, particularly, is not the same as causing no harm. Being seen to pretend that it is the same inspires reasonable distrust. It reasonably leads one to think that you are pulling a fast one, because you, in fact, are.

Demonstrating lack of a particular harm might take the form of showing how large a dose, or or how long an exposure at the dose newborns get (e.g., by continuous administration, matching excretion rate), is needed to cause that exact harm.

There are a great many ways that a neurotoxin could cause harm. Identifying one that it does not cause is absolutely different from showing none of the other possible ones occur. Pretending otherwise reliably demonstrates dishonesty, and inspires well-earned distrust. Making fun of people inspired to well-earned distrust inspires more.

Earning trust after actively courting distrust may not be achievable. The fault for that is on people like you.

Nothing causes no harm. Water causes harm.

You need to demonstrate that the advantages are worth the harm.

> Before injecting a substantial quantity

And as I quoted, the stated view is that the amount of thiomersal was NOT a "substantial quantity."

> It reasonably leads one to think that you are pulling a fast one

Which is why PR efforts to throw shade on the US voting system - despite evidence to the contrary - gets transformed into false belief that the US voting system is corrupt to the core.

Which is why PR efforts to cast doubt on the negative effects of smoking, and on the negative effects on profligate fossil fuel consumption, are effective.

But that doesn't actually mean someone's pulling a fast one, only that people are trying to convince you that someone's pulling a fast one.

> The fault for that is on people like you.

You must also do your part and not make easily dismissed exaggerations, or omit important qualifiers, as else your entire point will be easily dismissed.

Too often "no evidence" means "no evidence that I know of" which really means "I don't know". Doctors in particular use "no evidence" this way all the time because they are trained to show confidence in diagnosis when the truth is they often don't know and are just trying things to get some clues. If you suggest something they are not familiar with they'll often say "there's no evidence that works". You have to show them evidence to get them to try it. The evidence may exist, they just don't know about it.

Even when it is true that there is no evidence, that often just means no one has actually studied the problem. So "no evidence we can cure the common cold" really means "no one has actually tried to cure the common cold because we consider it too complex a problem so we've never tried". Now that the pandemic has forced a lot of scientists to actually try curing one coronavirus they are actually starting to think maybe they could cure the common cold and the flu and even have some drugs in clinical trials. Curing colds could mean combatting many viruses individually, but that doesn't seem as difficult as mapping the human genome. So what if there are 200 viruses that cause colds? The sooner you start curing them all the sooner colds will be gone. It basically just became dogma so you couldn't get grant funding for it. So another interpretation of "no evidence" is "we haven't tried".

> In traditional science, you start with a “null hypothesis” along the lines of “this thing doesn’t happen and nothing about it is interesting”. Then you do your study, and if it gets surprising results, you might end up “rejecting the null hypothesis” and concluding that the interesting thing is true; otherwise, you have “no evidence” for anything except the null.

Not true. The degree of evidence for either hypothesis would be determined by its respective p-value. The fact that you then take into account the evidence on either side to make a pragmatic binary decision is a different matter altogether.

E.g., a p-value of 0.050000001 is not strong evidence for the null. A p-value of 0.99999 on the other hand, is.

> This doesn’t have the same faux objectivity as

This is the crux of the issue I suppose. If you don’t maintain this plausible deniability, you get accused of being biased when you report the actual truth.

There is some distinction if you know where to look.

“no evidence so far that the disease can be passed through human contact”

“No clear evidence Wuhan coronavirus can spread before people show symptoms”

“‘No hard evidence’ new coronavirus strain is more transmissible”

“At this time, there is no evidence of human-to-human transmission outside China”

These are all statements that were thought at the time to be plausible, perhaps even likely, but not proven.

In contrast, most of the unqualified “no evidence” claims are for statements that were thought at the time to be false: Covid is airborne; non-N95 masks are effective; Covid came from a lab (still considered doubtful), booster shots are necessary (would still be debatable if not for Omicron).

On the other hand, even the least probable of those statements is still orders of magnitude more probable than “homeopathy works”, so there’s definitely a missing distinction there.

SARS-CoV-1 is airborne, so why would an informed person think that SARS-CoV-2 was not likely to be airborne? Masks are effective in original SARS and influenza, so why would an informed person conclude masks would not likely be effective on the new coronavirus? Influenza needs boosters annually due to variants — why would a reasonable person know either way at the beginning that boosters would be unnecessary?

I think you drank too much government koolaid in your list of ostensibly false things. Worries about mask shortages, diplomatic relations with China, availability of vaccines in the global south, and inducing people to get vaccinated the first time all led to our governments lying to us and presenting it as science. I don’t think the actions were malicious, but they ended up being both wrong and bad public policy.

I usually use a word like "guess" to describe a prior that sounds plausible but hasn't been confirmed:

"Our best guess is that the moon is made of green cheese"

"Our best guess is that jumping out of a plane without a parachute would kill you, but there has not been a formal study"

There actually has been a formal double blind study on whether jumping out of planes with parachutes save lives. The result found no life saving correlation, but suffered from other experimental problems.

Study did not get "off the ground" so to speak due to lack of participants agreeing to participate in the at height part due to reluctance to accept the risk of being in the placebo'd control group.

This article is a mess. Zooming in on:

Is there "no evidence" homeopathy works? No, here's a peer-reviewed study showing that it does. https://pubmed.ncbi.nlm.nih.gov/30202036/

Except the study has been retracted. I quote: Following publication, the journal received criticisms regarding the rationale of this study and the plausibility of its central conclusions. Expert advice was obtained, and the following issues were determined to undermine confidence in the reliability of the study.

The in vitro model does not support the main conclusion of the paper that Rhus Tox reduces pain. The qualitative and quantitative composition of the Rhus Tox extract is unknown. Figures 1G and 1H are duplicates; and figures 1I and 1J are duplicates. The majority of experimental points reported in figure 3 panel A are duplicated in figure 3 panel B. The collection, description, analysis and presentation of the behavioural data in Figure 3 is inadequate and cannot be relied upon.

As a result the editors are retracting the Article. The authors do not agree with the retraction.

Not a great example really. It was at this point I decided not to give this any further attention.

> Is there "no evidence" that using a parachute helps prevent injuries when jumping out of planes? This was the conclusion of a cute paper in the BMJ

The BMJ paper doesn't say there was no evidence!

The paper explicitly says there was anecdotal evidence. Here's the relevant quote from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/ :

] The perception that parachutes are a successful intervention is based largely on anecdotal evidence. Observational data have shown that their use is associated with morbidity and mortality, due to both failure of the intervention1,2 and iatrogenic complications.3 In addition, “natural history” studies of free fall indicate that failure to take or deploy a parachute does not inevitably result in an adverse outcome.4

> Their [the BMJ authors'] point was that "evidence" isn't the same thing as "peer-reviewed journal articles".

How does Astral Codex Ten draw this conclusion? The phrase "peer-review" doesn't exist in the BMJ article.

The point of the paper is that observational studies should not be rejected out-of-hand. Observational studies can be and are peer-reviewed.

> Is there "no evidence" for alien abductions?

Some of the circled headlines used "no evidence" or even "'Exactly zero' evidence."

But others used "no hard evidence" and "no clear evidence", which are higher bars.

So, yes, there is evidence of alien abductions, but quoting an article I found using Google Scholar, it's well characterized as: "With so little hard evidence to support anything having to do with aliens on Earth — in fact, essentially no evidence outside personal testimony" - https://cpb-us-e1.wpmucdn.com/journeys.dartmouth.edu/dist/a/...

(It then points out other psychological mechanisms which might explain the evidence, ending "Whether their stories are true or not, their trauma is very real, and it deserves to be treated as such.")

> Is there "no evidence" homeopathy works?

Sure, there's evidence. But there's also counter-evidence.

Is there stronger evidence that homeopathy works than there there was evidence that COVID-19 could spread through the air-con system?

Probably.

So how is this example supposed to help make the author's point?

> Here we should reject journal articles because they disagree with informal evidence!

Q: What is the basis of the author's informal evidence? A: "a strong theoretical understanding". Q: Where did that strong theoretical understanding come from? A: Published research.

So the author is rejecting a journal article because it disagrees with other journal articles, after being watered through public perceptions.

That's not informal evidence!

> there is literally not a single piece of evidence anywhere in the world that anyone could use in favor of snake oil.

The author earlier accepted anecdotal evidence of alien abductions as a type of evidence.

How is the author so sure there is no anecdotal evidence that snake oil works?

It's because of essays like these I avoid substack links.

>> there is literally not a single piece of evidence anywhere in the world that anyone could use in favor of snake oil.

> The author earlier accepted anecdotal evidence of alien abductions as a type of evidence.

> How is the author so sure there is no anecdotal evidence that snake oil works?

The full paragraph reads:

If the story is that all the world’s top doctors and scientists believe snake oil doesn’t work, then say so. “Scientists: Snake Oil Doesn’t Work”. This doesn’t have the same faux objectivity as “No Evidence Snake Oil Works”. It centers the belief in fallible scientists, as opposed to the much more convincing claim that there is literally not a single piece of evidence anywhere in the world that anyone could use in favor of snake oil. Maybe it would sound less authoritative. Breaking an addiction to false certainty is as hard as breaking any other addiction. But the first step is admitting you have a problem.

The author doesn't think there is literally no evidence for snake oil, which is why he argues against the "No Evidence Snake Oil Works" headline. I think that's quite clear if you look at the surrounding context instead of quoting only a fragment.

I don't think I misread it.

There absolutely is evidence that could be used in favor of snake oil. A Google Scholar search easily finds:

"Inhibition of human fibroblast growth in vitro by a snake oil" - https://www.sciencedirect.com/science/article/abs/pii/000712... .

"Snake oil originally was a Chinese medication rich in omega-3 fatty acids which was made from the Chinese water snake. It was believed to decrease the pain of arthritis." - https://link.springer.com/article/10.1007/s12603-019-1202-1 (also, http://mrarmy.pbworks.com/w/file/fetch/101472259/merchants%2... ).

"Traditional keloid remedies have been in use for generations. The Yoruba of South Western Nigeria, for example, use several methods to reduce the symptoms of keloids. Boa constrictor oil from the snake's visceral fat and ointments of shea butter (from nuts of the Vitellaria paradoxa tree), locally produced in Nigeria, are two commonly used traditional medications. Some patients seen in the authors’ clinics in Osogbo and Ibadan, Nigeria who have been using either shea butter or snake fat, reported observable remission of symptoms and swelling." - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3905615/

Thus, there CANNOT BE A "convincing claim that there is literally not a single piece of evidence anywhere in the world that anyone could use in favor of snake oil". (Remember, we're accepting that personal accounts of alien abductions counts as evidence.)

Breaking an addiction to false certainty is indeed hard.

> there CANNOT BE A "convincing claim

I think there can be convincing claims that convince people of falsehoods, in which case the claims are also misleading. If "convincing" were replaced by "convincing, but misleading" or "convincing, but incorrect", would you then agree with the paragraph?

My basic claim is the author did a slight-of-hand by accepting eye-witness claims to alien abduction to be counted as evidence for alien abductions, then rejecting the existence of easily demonstrated evidence that could be used to support the effectiveness snake oil.

I cannot answer your question because I don't understand the author's viewpoint. That's because to me the phrase "no evidence" is not really related to the 'belief in fallible scientists' but to the belief in fallible claims.

This could be because I was trained in science, and the author is trying to make some point about poor science communications by scientists and journalists.

But the author is engaged in some rather poor science communications, by misrepresenting the BMJ paper, by using a mutable definition of what evidence means, and by lumping everything claims from 'zero evidence' to 'no hard evidence' into a single characterization.

Before you respond, take a look at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3676982 titled "Hydroxychloroquine, Parachutes And How to Understand 'The Evidence'". Here's the start:

> It is deliberately misleading to make general claims that hydroxychloroquine doesn’t work for CoViD-19. It is even more misleading to claim that “the evidence” proves that hydroxychloroquine doesn’t work.

> To understand this, consider the use of parachutes.

Doesn't this argument sound familiar?

Trump has a history of making claims without evidence

https://www.usatoday.com/story/news/politics/2017/03/05/trum...

He's been out for about a year, let it go man.

Reply on news.ycombinator.com