Even better would be randomized experimental data that this was better than an alternative... 'It works' can mean lots of different things...
Unfortunately, in the middle of a pandemic, that's not going to happen with enough speed to prevent a lot of people dying.
The perfect can be the enemy of the good enough. We don't have to guess or learn about the mass of an electron; that's been well-established through great lab work. But not all decisions are like that. Many times the hardest decisions are those you make with incomplete and contradictory information ... and not making a decision is, in effect, making a decision. This is one of the things that's so interesting here. If it were simple yes/no, what's science and what's speculation, there wouldn't be much interesting learning or decision-making happening.
The fixation on RCTs has kinda become "a meme" at this point. However it's lives on the line in some situations.
Not everything can be double-blind. Not every experiment can be ethically done with a placebo (or done at all).
Also the placebo effect is real but it is more pronounced in things like back pain or headaches, not so much in broken bones or aggressive viruses.
Also, in most cases the control group get the "standard" treatment, because "do nothing" (taking literally the GP) is too bad. And you want to be sure that the new treatment is better than the "standard" treatment.
[1] A real randomized control group, not an unrelated bunch of guys in another city of France.
Historical data? Past experiments? Sure, they might have biases, but the higher sample size of historical data can help with that.
I think (specific) control groups make sense when you're talking about small effects or very specific cases (for example, the control group will undergo the standard treatment instead of the new treatment)
For example: If there is a small Ebola outbreak in New York and you give the patients hydroxychloroquine and the death rate is 80%, is it good or bad?
How are you sure that this method is not worse than the ventilator method and will kill even more people during the pandemic?
How many death did they get with this method? How many death would they had got with the ventilator method?
From the article:
> The study included all sequentially hospitalized patients between March 1, 2020, and April 4, 2020, inclusive of these dates.
> As of April 4, 2020, for patients requiring mechanical ventilation (n = 1151, 20.2%), 38 (3.3%) were discharged alive, 282 (24.5%) died, and 831 (72.2%) remained in hospital.
If you ignore the 831 patients that were still in the hospital, and consider only the 38 that survived and the 282 that died, you get 282/(282+38)=88%. It is a very short time. What are the odds for people that survive more than 4 days in the hospital?
Did the team this post get the better results using the same accounting method? Did they apply this method only to the people that would need a ventilator or to more people because it is cheaper and easier to setup and maintain? (IIUC they used ventilator for the worse cases anyway.)
> And with numbers like that, I think it's perfectly ethical to try anything reasonable.
Do you want the best possible treatment to each patient, or you prefer that each doctor tries his/her pet theory in production?
It's not that your questions are bad, it's that in practice medical professionals have to operate without answers to them. Doing so involves communication with peers about their own observations of what may work better/worse, such as in this article. Double-blind randomized trials are not the only source of actionable information: Case studies also form a significant pillar of knowledge, especially pending results from more systematic studies.
[0] https://link.springer.com/chapter/10.1007/978-1-4757-2068-6_...
For example one day we have very weird results trying to compare two measurements of the same day about something related with the speed of sound. After a lot of time trying to understand the problem we notice that one measurement was from 9am when the temperature was like 10°C(50°F) and the other was from 1pm when the temperature was like 25°C(77°F). The following days we tried to take all the measurements as close as possible.
The same applies to benchmarks in computers. A few days ago I got an improvement for a program that reduced the runtime from 85s to 59s. Until I noticed that one was with the notebook plugged in an the other with the batteries. I tried again both with the notebook plugged in and the improvement was only from 65s to 59s. Still a nice improvement but not fantastic.
Quoting again part of the original comment I'm repliying
> Learning is not the same as science, and we need randomized trials to actually be sure of anything. Unfortunately, in the middle of a pandemic, that's not going to happen with enough speed to prevent a lot of people dying. The perfect can be the enemy of the good enough.
Without a randomized control group it is very difficult to be sure if a small improvement of the survival rate is real. So you can't learn if the new method or the old method is better.
Case studies of those attempts are exactly the kind of data points that can provide a great deal of information and hypothesis generation for more systematic studies.
As a former biomedical engineering researcher, this pandemic has been profoundly eye-opening in learning about the resistance to advancing the state of the art of medicine in the USA. (I’m not even speaking about the knee jerk reaction towards banning nebulizers in hospitals thanks to a flimsy NEJM letter last month reporting coronavirus aerosol contaminations.)
No! They will definitively not let their patients and colleagues die firs! They will prevent you from killing them!
You might believe in youRe therapy, but the vast amount if novel therapies will have (novel) adverse effects and you will haVe to supply the prove that your therapy will do less harm than doing the conventional thing. We have clinical trials to administer new therapies in a controlled and ethical way to cause the least harm. Just trying things will kill more people than it helps. The vast amount of new ideas do more harm than they help. Look at chloroquine and what a shit show it is, because everybody just tried it willy nilly. The side effect of harming Lupus patients who could not get their medicine alone caused tremendous unintended harm. And that wasn’t even harming because if administering a useless therapy with serious side effects. Move fast, fail fast and fail often aren great principles for developing software but they have no place in developing therapies. (I actually wanted to hurl swear words at you. But that helps nobody. It is true that medical research is overregulated and could be improved Bit there are reasons for that. Throwing all regulations away is just Trumpish behavior.)
Your point about supplying the safety data is exactly where the roadblock is found. I’ll update you when the device is approved by the FDA and voila these objections mysteriously disappear ;)
You know a proper FDA approval takes years, right? Not the emergency do-whatever-you-want approval, a proper approval. By all means, please do post.
Further, if the GP had prior assumptions about these possible treatments being wrong, you commit the same error in the opposite direction with your casual assumption that your approach is right, that the FDA will not only approve the thing, but that the objections will also lose all validity. Even if approved, the objections may still have merit. The objection need not be completely wrong in order for the FDA to decide the balance of risk weighed in favor of approval.
Finally, the idea of using nebulized ethyl alcohol is interesting! I hope it works. It is, however, extremely easy to find information on the dangers of inhaling alcohol fumes. If it is a viable treatment path, patients are likely to be extremely sensitive to dosing levels. Recipients would also, by definition, be high-risk, making the safety issues more uncertain. It is not, on it's face, unduly obstructionist to be highly skeptical of such a treatment without a decent amount of evidence.
Edit: I even partly agree with your sentiment that this is a time when some (very careful) risks may need to be taken that ordinarily wouldn't be. But your comment conveys a strong sense of arrogance, of unassailable certitude. I hope I am wrong in that assessment: Someone looking to take these sorts of risk right now should be approaching them with the utmost humility. This makes your tone very disturbing.
For instance, the banning of nebulization in hospitals for fear of viral aerosolization published in a faulty NEJM letter[1] is preposterously cautious. By this logic, Lysol disinfectant sprays ought to be banned too, even though it’s recommended by the CDC. So this “arrogance” you sense is probably because it is apparent to me I am hungrier to save lives than (many) doctors and hospital administrators who won’t take a chance to even measure the risk to accumulate much needed experimental evidence; even if their patients are willing! Hippocratic Oath is the retort.
Frankly, “no data no dice” does not cut it when confronting such a global humanitarian challenge. The medical profession appears to be just that - licensed occupational employees motivated about retaining their jobs. Hence my snarky expectation that this will be an easier, ie quicker, sale if/when the nebulizer has been rubber stamped by the FDA. No one was ever fired for buying IBM
Front-line doctors simple have a different perspective: There are many prospective treatments that might show promise. Most will not work. Many may have their own catastrophic effects. At the same time, doctors must make immediate decisions on how to treat patients, without the luxury of time to review all prospective experimental treatments, most of which lack sufficient data to make a rapid determination of their risk. And remember their foundational, ancient oath: "First, do no harm". If that oath engenders a certain amount of conservatism in treatment choices, you should not berate the doctors themselves as deficient: Your issue is with the fundamental philosophy of medical practice.
I am glad you are hungry to save lives. But remember humility: Any number of significant failures that cost lives can be found will minimal searching, and undoubtedly most involved were eager to save lives as well. The drug combo FenFen comes to mind: I'm sure the researchers involved were eager to save lives and improve the health of obese people, but they're efforts caused serious heart problems in up to 30% of those taking the drug. Medical history is littered with failures and death. You must remember that. If doctors are conservative in their treatment choices, they have good reason, and should not be condemned for it. You'll win no people to your cause if you vilify those you seek to convince, all you will do is attract people who are already inclined to agree with you.
"The proning and the high-flow nasal cannulas combined have brought patient oxygen levels from around 40% to 80% and 90%, so it’s been fascinating and wonderful to see"
People die, if their oxygen levels are getting too low. This method gets them up again. Sounds like "working" to me.
I have no doubt that proning increases the readings we are getting on pulse ox monitors, sometimes pretty immediately. I’ve seen it, literally first hand, in a number of my patients. But I have no idea if it saves lives. Many of those same patients still got intubated and some died.
Understanding impact on surrogate markers isn’t enough.
That said, large effect sizes in one hospital don’t control for confounding variables.
It’s required to prevent systematic biases like giving the drug that you think might help to the patients with better prognosis (instead of wasting it on those that are very likely to die anyway).
They're not optimal for saving lives.
Science is important, but there are times like this where brute intuitive engineering is the only tool available.
This article isn't not science. It encompasses a key pillar of medical knowledge and advancements: case studies.
RCT is wonderful when we can get them and not always show stopper when not.