Against pandemic research exceptionalism | Hacker News Reader

Against pandemic research exceptionalism

science.sciencemag.org

My normal position is fairly anti-government, but there is a role for an agency here that I wish they'd fill. It'd be fantastic to have a centralised agency organise a panel of experts who have gone though all the known papers and studies, synthesised them then just sat for a week doing a massive public Q&A session. Targeted at people who are really quite clever rather than the usual aiming-low that the mass media goes for.

The volume of research has gotten far too much for me to keep track of; the media are in full swing polluting the channel with irrelevant and frequently political noise. I really do not care about what politicians think about given drugs, and honestly their justifications for why they are doing X or Y are not going to matter to me for a while because politics doesn't happen quickly. It'd be nice to have a centralised, low bandwidth well moderated information source for the smartest 5-10% of people to get good information in response to their questions and uncertainties.

There are great information channels for PhD medicine types & doctors (I hope). There are good sources for people who are at risk of drinking bleach because they heard it might help (try a local news channel). There is a hole for, say, the top 5% smartest people where they really don't have an in to figure out how they might help, because they need information hidden in very technical articles or behind rather black-box modelling as described in the media. But they are otherwise very educated, intelligent, capable, wealthy or what have you. These are coordination problems a government could actually help with.

I agree in principle with your wish for some neutral <unquote>, informed, apolitical, insightful, measured, and cautious commentary that distils the current consensus.

But it sounds like a lot of the concerns you express are specifically around the problems that are currently being experienced by one poorly administered country.

I think elsewhere in the world, a lot of people feel they're already getting this level of information, without the political overload & misdirection.

I reject the notion that Australia is poorly administered. Our COVID-19 cases are extremely low [0] and the government has been doing a fine job compared to most others.

[0] https://www.health.nsw.gov.au/news/Pages/20200503_00.aspx

Aren't the cases low because the temperatures were high when the virus first hit (and are still comparatively high)?

I'm sure Italy would have done better in the summer with 30 degrees Celsius.

Do you have any data to support the assertion that higher temperatures help calm Covid rates? Seeing as Singapore is having an extremely tough time right now, I don’t think you do. This is pure supposition.

It's a well-known correlation with influenza. Yes it may be untrue with Corona Virus, but that remains to be seen. It would be unusual if the correlation didn't hold.

Please stop insisting we have evidence when none exists.

My question was: do we have evidence that warm weather slows Covid outbreaks?

So far, the answer is “no”, and asserting anything else is not an argument, it’s a made up story.

Asserting that there is absolutely no temperature dependence without evidence is pretty much the same thing as asserting there absolutely is without evidence. If there is no data, the best choice is to say that you don't know while coming up with plausible hypotheses. That a virus might share some traits with other closely related viruses is not so uncommon. Extrapolation from prior evidence when no direct evidence is available is one of the main capabilities of the scientific method. This is pretty far from a 'made up story.'

It's a bit of moot point though. Because of the extremely plausible nature of the idea that covid might have some temperature dependence, there are many people investigating this specific claim. With a cursory web search, you can find several papers and pre-prints on the subject, a small majority of which find evidence to support that trend.

Saying that temperature can't affect covid transmission because Singapore is having a tough time is akin to suggesting climate change isn't real because it snowed in April. It is just one data point.

There was a nice article about similar coronavirus. No virus is exactly equal to the other virus, but the comparison can be helpful. https://www.quantamagazine.org/what-can-other-coronaviruses-... (HN discussion: https://news.ycombinator.com/item?id=23019199 | 204 points | days ago | 189 comments)

Singapore is doing fine and has one the lowest death rates in the world.

Nearly 100x less than the global average for case fatalities. For all these people testing positive no one is actually dying, most don't even have symptoms.

hasn't it been shown that vitamin-D deficiency has been shown as an aggravating factor in case mortality with covid? It's not a big extrapolation then to suggest places with warm sunshine would tend to have better outcomes.

What make you sure?

There's no evidence that higher temperatures have an effect on disease transmission.

https://papers.ssrn.com/sol3/Papers.cfm?abstract_id=3551767

This doesn't address the problem, it only moves it. These kinds of panels already exist in many countries on many topics -- I've worked with several. In all cases, the selection of the panel and the positions they take have political motivations. They are never constructed as a random sample of domain experts. Even when trying to achieve that appearance, "domain expert" is defined so as to justify the exclusion of many inconvenient domain experts from consideration. None of which should be surprising, since this is basic human nature.

Indeed, the functional purpose of these kinds of panels, based on what they actually do, is to lend an air of authority to decisions that were made politically. For the participants, they are paid in prestige, which is a valuable currency.

> "My normal position is fairly anti-government, but.... Targeted at people who are really quite clever.... It'd be nice to have a centralised, low bandwidth well moderated information source for the smartest 5-10% of people"

that only reveals ego and information blindness. it's hard to triangulate toward the truth, much less a plan, with that kind of blindness, no matter the composition of expertise. smart people may know more of their little patch of elephant, but make no mistake, they know nothing of the whole elephant. it's so easy to miss massive swaths of the elephant because you're too smart to ask the simplest questions first.

luckily, an easy first-order fix is to not dictate the composition of the audience asking questions. pick at random. then you'll get a truer picture of the bewildering complexity of the elephant and a better sense of where to take that (and a truer picture of the information holes between the panelists to fill for next time).

besides, what you ask for is simply the standard academic seminar at any major research university.

Try r/covid19

> It'd be nice to have a centralised, low bandwidth well moderated information source for the smartest 5-10% of people to get good information in response to their questions and uncertainties.

Any such channel which is truly open to questions immediately gets swamped by political attacks, along with an endless horde who think they're in the smartest 5% yet accuse the scientists of being in the pocket of the lizard people. Malicious agents will be able to mine even slightly nuanced explanations for out of context quotes to show that their political opponents are stupid or evil. If the channel is to be useful at all, it must make statements under uncertainty, but that gives it the chance of being occasionally wrong -- and anything it ever gets wrong will be repeated ad nauseum as an argument to defund it. If the channel is visible and trusted, it also becomes a prime target for politicians looking to shift blame away from themselves.

The reason I'm completely sure this will happen is that the information channel you're asking for already exists. It's called the World Health Organization, and they've been doing informative press conferences this whole time. However, everybody on this site has decided that they're useless because of a single, incorrectly read, out-of-context tweet from 4 months ago.

> It's called the World Health Organization

That's a political organization, not what is being proposed here.

You’re perfectly illustrating what I talked about.

But the objection is perfectly valid. They are a political organization that sometimes places politics over reality (for example, refusing to recognize Taiwan, repeating Chinese reports unchallenged and praising China's transparency, or as another comment pointed out, appointing Robert Mugabe as goodwill ambassador).

That internal challenge should be added to your excellent list of external challenges the proposed organization would face, and the WHO does face: politically motivated people and powerful nations will capture the organization from the inside and abuse any trust it has built up to serve their own ends.

> refusing to recognize Taiwan

So it wouldn't be political if they did recognise Taiwan? No one on Earth besides a few Caribbean islands recognises the ROC and yet somehow the WHO has no credibility for doing the same.

Very much making the GP's point for them without even realising here.

Mugagbe is a perfect illustration of that. They were trying to tackle non-communicable diseases in Zimbabwe, getting him on board and giving him a meaningless title was their plan. Instead of seeing it as a boon for the poor in the country, people decided to make it political and demanded he not get the worthless honor. Either outcome had absolutely zero effect on the people screaming the loudest about it.

Put the politics aside for a moment and ask yourself the question: Would health outcomes for the poorest Zimbabweans be better or worse if Mugabe had kept the title?

The fact is, the Republic of China exists and is independent of the People's Republic of China. They have a military and a government that doesn't answer to Beijing, and have maintained that government in Taipei for 70 years.

That's reality.

Recognizing reality need not be political, but refusing to recognize reality for political reasons definitely is.

But the main point is that politics, not medicine or science, is dictating the WHO's decision here. Would health outcomes for the Taiwanese be better if the WHO included Taiwan?

As for Mugabe, you're arguing that it was good politics by some measure, and it may very well have been, but it also erodes faith in the organization from people who don't think it was good politics.

> Would health outcomes for the poorest Zimbabweans be better or worse if Mugabe had kept the title?

If we assume that outcomes would have been better, that implies that listening to the WHO does improve health, and therefore we should also assume that other people who would have lost faith in the organization would have had worse health outcomes as a result. I don't know how it would all balance out, but again, the point is merely that it's a political decision, not a scientific one.

> Recognizing reality need not be political, but refusing to recognize reality for political reasons definitely is.

Getting China offside means 1 in 5 people on Earth would no longer get WHO advice and state media would undermine them at every turn.

Getting Taiwan on side means appeasing political activists in the West and having little if any measurable positive health outcomes, the Taiwanese are used to this state of affairs and have more than competent doctors and lawmakers, they are one of the smartest countries on Earth.

Despite the closeness and affection between my own country and them, when their foreign minister visits they don't go near parliament or meet elected officials in public, they go to trade shows and meet with industry.

As much as it might pain some to accept, compromises are the best outcome for all here, the WHO is being realistic about the position they are in. Hopefully more sideline commentators can be too.

>Getting China offside means 1 in 5 people on Earth would no longer get WHO advice and state media would undermine them at every turn.

Doesn't sound like a bad thing if you like Chinese people. WHO has been consistently wrong about everything and is pretty much useless.

> Getting China offside means 1 in 5 people on Earth would no longer get WHO advice and state media would undermine them at every turn.

Basically you're saying the same thing I'm saying. Authoritarian nations are allowed to dictate WHO policy in order to keep them "onside".

You're just arguing that it's necessary, and I'm inclined to agree, but it's also the reason the WHO is not an impartial or trustworthy source of information. And it's probably impossible for any such organization to be so at this time.

> However, everybody on this site has decided that they're useless because of a single, incorrectly read, out-of-context tweet from 4 months ago.

Actually, they lost any credibility to me when they wanted to appoint Robert Mugabe as an ambassador years ago.

In more recent times, their position on the use of masks was criticized as confusing[1].

And let's not forget their equally confusing tweet on immunity to SARS-CoV-2 from a week ago or so (which was ultimately retracted). So not just one from 4 months ago.

Of course it's a big organization, so caveats apply. Nevertheless, they didn't do the best job at communication.

[1] https://blogs.bmj.com/bmj/2020/03/11/whos-confusing-guidance...

Normally, this is the role of peer review and top publishing venues. Those places should be putting a big push to rigorously review as many articles as and as fast as possible now.

This would be the CDC. They stopped doing those briefings at the, uh, request of the President.

https://thehill.com/homenews/administration/494187-trump-thr...

A friend of mine runs a relatively popular blog on climate change. In this blog he discusses current research (including his own) in a manner targeted at relatively educated people. Its exactly the sort of thing you want.

He receives hate mail constantly. Major players in political administrations troll his comment section and evade bannings. Its a mess.

I don't know if what you describe is feasible.

As someone who was arguing for speedy reporting a few weeks ago: the longer the pandemic goes on, the more important rigor becomes.

Waiting a short time to get things right today is a a very different matter than a 4-8 weeks ago, where on-the-ground information was completely absent. Speed still really matters, but the longer this goes the more taking our time and doing things right will pay off.

Are you arguing that in the initial stages incorrect information is less harmful than no information?

No, they are not arguing that. I can tell because their post says something else, and because that would be stupid.

In March there wasn’t enough information to decide how strict the social distancing needs to be in order to reduce the spread of the infection. At the same time it was clearly necessary to do something, without knowing in advance how much it would affect the epidemic.

Now we can take a little more time to decide what to do next, and take more focused and effective measures.

There’s a difference between incorrect and incomplete.

Facts, like how many people in a trial of 23 recovered, can be shared and useful early on even though they’re incomplete. That’s different from attempting to draw a strong conclusion about what works from n=23, that might be an incorrect conclusion.

The form that the incomplete data took was part of the problem.

Adding numbers to a spreadsheet is one thing. When the same data takes the form of an NJEM or Lancet paper, it definitely implies--and some cases, outright says--more than "FYI: 37/53 people got better when we did [this]." Part of this is how papers are ordinarily written, but I think the authors (and editors) should have stopped all over the word "improvement" in those manuscripts.

> Numerous trials investigating similar hypotheses risk duplication of effort, and droves of research papers have been rushed to preprint servers, essentially outsourcing peer review to practicing physicians and journalists.

I’m curious about this point. Often we have the opposite problem, that we can’t get anyone to replicate studies with strong looking but potentially specious results. Isn’t perhaps a small amount of duplication of effort a good thing, especially if studies are rushing and letting standards slip a little? Aside from coordinating and not using too much of the funding to investigate a particular hypothesis, what are the other reasons to avoid duplication of effort?

Their point is that there is actually a very large amount of duplicate effort is happening now, as everyone separately goes for the first ideas: this makes it more probable that a lot of effort will be wasted on super-duper-triple confirming that the first thing everyone thought of didn't work, slowing the exploration of more nuanced ideas.

But besides waste of time and resources, the only other problem they mention is an increase of false positives. These naturally become more common as you try the same thing more often unless you correct for it by keeping track of all the other trials.

My problem is that there are too many inconclusive trial. Someone makes a small trial without a control group, so the result is promising but difficult to interpret. Someone else makes a small trial without a control group, so the result is promising but difficult to interpret. Another person makes a small test without a control group, so the result is promising but difficult to interpret. ...

So after a few months, we only have promising inconclusive results.

Not that all the trials are different, so it is difficult to compare them. IIRC There was a recent trial with remdesivir without good results, but they only tried the drug in patients with strong symptoms. So people asked what happens if they use the drug in the beginning of the infection.

Not only research, but a lot of diagnostic companies are trying to take advantage of the gold rush:

Theranos Would Be Thriving in the Covid-19 Pandemic

https://elemental.medium.com/theranos-would-be-thriving-in-t...

Really interesting question here. I wish it went a bit more into specifics of how to navigate the tradeoffs. Maybe one way to put this is that it’s fine to do preliminary studies that motivate and direct the actual rigorous research, but we’re stuck in a place where people are only doing the preliminary studies, a lot, and not getting to the rigorous stage.

> perpetuate the perception that, when it comes to the rigors of science, crisis situations demand exceptions to high standards for quality.

Where are the high quality studies warranting strict lockdowns?

In fact I would be curious if a single medical textbook anywhere in the world mentions this as a possible solution to any epidemic. It seems to be a innovation, coming from politicians more than scientists.

The practice so far, still uncommon, was to quarantine sick people, not force the entire population to stay at home.

What happened to the title? The actual title is "Against Pandemic Research Exceptionalism". If this was intentional, it was a bad choice, because information was lost.

I had wondered if a bandit algorithm would be an effective approach. Can anyone who knows more about it than me comment?

I doubt AI is good enough yet to read through research papers and reliability identify methodological flaws that render the conclusion suspect or invalid. Pretty sure that’s still a job for human experts.

As I understood it you would start choosing treatments at random, as one or more treatments started to outperform the others they are chosen with higher frequency. A sort of on-line trial. No AI involved.

Yes, it would work if human patients were lab rats.

You should realize that in clinical trials human patients are very much like lab rats. They receive a randomized treatment.

Note that the alternative is use the whole humanity as lab rats. Something like:

fake quote> We have the gut feeling that this drug is better. We got good results in animals models with a somewhat related illness. Let's try this new drug with everyone for a few years.

There are multiple phases of the trials. The first use a small number of people in case there is something very wrong. The last use more people, to try to detect subtle problems.

I'm not sure I understand what you are getting at here.

Studies with lab rats have control groups, placebo groups, populations, etc. We don't give all the rats in the world some drug we have a hunch about.

I think we agree.

Quoting your comment:

> human patients are very much like lab rats

Some people consider that offensive because lab rats are discardable. (They even kill all the rats after each study, they are not reused in another study or get free in a rat sanctuary.)

This is a common excuse against double blind randomized trials with a control group. Why are some people getting the "worse" standard drug instead of the new (perhaps) "magical" drug? Why make experiments with them instead of giving the new drug? Why use blind groups and use them as lab rats?

(In every Covid-19 thread, someone propose to use the database of all the known cases and some statistic to simulate a control group (How hard can it be?), and then give everyone in the trial the new drug.)

Expanded version of my comment:

The alternative to using a good clinical trial with all the modern constrains to differentiate between new drugs that are good and new drugs that are bad is to use only some experiments in animals or in vitro. Not every illness affect every animal, so there are some restrictions, or the test in animals must use something that is similar enough.

Imagine that without a good clinical trial in humans, someone approves the new drug. After some time the problems with the new drug may appear. In this case instead of testing the new drug with a small group of people in the trial, you are testing the new drug in the wild with whatever person that is so unlucky to get ill and get the new drug, so you are using the whole humanity as a trial group.

Yes, there are many papers on looking at clinical trials as bandit problems (because they are bandit problems) and applying different bandit algorithms.

You may also want to look at SMAR Trials and adaptive trials.

The name for an "A/B test" is "randomized control trial" (with two arms). There are also trials with more arms, like

* standard treatment + placebo

* standard treatment + some drug

* standard treatment + double amount of drug

and other combinations, like using two drugs in one arm.

Bandit algorithms aren't different only because they have multiple arms. They also follow the least possible amount of harm by updating the conclusions in a Bayesian style during the entire experiment, not only on the end.

They also can stand more arms appearing during the experiment after people know more.

Reply on news.ycombinator.com