Hacker News Reader

Comment by hcks | Hacker News Reader

What point is this article trying to make?

This para, buried deep in the paper, is the point:

A minor caveat to our findings is that the rate of the primary outcome was substantially lower in this study than was anticipated at the time of its conception and design, which potentially underpowered our ability to detect clinically meaningful differences, as well as important interactions. Although randomized participants had similar characteristics compared with those who were screened but did not enroll, they could have been at lower risk of death or major trauma because they jumped from an average altitude of 0.6 m (SD 0.1) on aircraft moving at an average of 0 km/h (SD 0). Clinicians will need to consider this information when extrapolating to their own settings of parachute use

Real studies have an equivalent para far too often.

It's right in the abstract and conclusions though, not really buried.

Yeah well the satire would have worked better if it were buried, although the real joke is 'minor caveat'.

> What point is this article trying to make?

There is no evidence from RCT studies, that parachutes work (this post).

There is no (or only very weak) evidence from RCT studies, that condoms work: https://jech.bmj.com/content/65/2/100

According to the Cochrane Database of Systematic Reviews, there is no evidence from RCT studies, that face masks would protect against respiratory infections: https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD...

The point is that the Cochrane review is probably wrong, and face masks do protect, just like parachutes and condoms. And the more general point is to criticize the whole process, that has made half of the Western medical doctors mask skeptics. The point is to say that there are protective equipment that obviously, measurably works, but for which it is difficult to present evidence from RCT-type studies.

This article predates the Cochrane review and pandemic so I doubt that was their goal.

That is a good point, but I think their point was to bring attention to the flawed philosophy, so that mistakes could be avoided. And surprisingly soon after their 2018 paper, the flawed philosophy lead to such mistakes in a surprisingly relevant, important and timely matter.

Linking their general points from 2018 to specific examples from the recent pandemic, is of course post hoc. But it demonstrates that their notion was not merely theoretical, but important in practice.

It's also parodying the the problem in some RCT to use unrealistic test cases which are supposed to be representative of actual clinical uses (hence testing parachutes as protection for jumping out of landed aircraft).

Studies, including RCTs, are often affected by selection biases introduced (intentionally or unintentionally) by how participants are selected and/or self-select. Even the gold standard RCTs are thus not guaranteed to produce valid (or rather, useful/non-misleading) results.

They thoroughly researched the effectiveness of parachutes on a pool of candidates at various speeds and altitudes. While the initial pool of candidates covered a variety of situations, for some reason, the pool of participants willing to actually participate in the study was heavily biased towards people in an aircraft standing still on the airport, with about 60 cm from the door to the ground.

The bias may have been further increased by the selection criteria where participants deemed mentally unfit by the investigator were excluded, and willingness to jump from an airplane at altitude with a 50% chance of an empty backpack increased the likelihood of exclusion.

The bottom half of the right column on page 4 explains how this is relevant for real world studies.

Note that you missed the actual catch, which sibling comment `eastern` managed to spot.

They state the point at the end, and it's serious point: if you were inclined to use "science" to argue against a certain intervention that some group believes in, you could do a randomized trial of that intervention, but manipulate the conditions so that the intervention is unlikely to help. The study would look right and sound right to some people. It would use all the best practices. And yet it would be a bullshit study.

I suppose an example would be comparing the death rates of people who'd been on a statin drug for a few weeks to those who weren't on the drug at all. You'd probably find no different in morbidity, which might suggest that statins don't do anything.

> manipulate the conditions so that the intervention is unlikely to help

> [The study] would use all the best practices

You've go some major contradiction here, since when is such manipulation "the best practice"?

Well, is giving 50% of people placebo parachutes and having them jump out of a plane at altitude considered best practices?

"Best practices" is always bullshit, man. It's a marketing term, not an engineering or a science term. Any practice is bad or good based on the context and purpose for which it is applied.

That's the spirit in which I used the term. In plain English: you can correctly apply certain practices that are famously powerful and acceptable, while subtly undermining them in a way people might not notice, or would happily pretend not to notice if the study fits with their political position on the matter.

Then your supposedly serious point resolves to a tautology "people can bs if they do bs". Duh! This conveys nothing re. any scientific technique or using that in arguments against some intervention

I am explaining the point of the paper. If you don’t think a parody based on that point is useful, maybe take it up with the take it up with the authors, Duh!

They do a Randomized Controlled Trial on parachutes, but due to ethical reasons the plane is only a few feet off the ground and there's no effect (the parachutes don't have time to deploy).

Are you under the impression there would be an effect on trauma rates if the parachute managed to deploy during a 60cm jump?

No, but what I'm saying is if you watched the study the parachutes would look like they were doing nothing at all, not even flopping the chute out.

I took it as a commentary on relying on scientific methodology when deciding the importance of a paper. Often the fact that there is a RCT study or whatever, a peer reviewed article in a prestigious journal, this is seen alone as sufficient reason to believe a finding. Scientists make this assumption, laypeople make this assumption, pop-sci media certainly do their thing; but this is essentially a counterexample, a scientific article that ostensibly checks all the boxes of rigorous science while being pretty obviously nonsense.

Death panels are hard work and if you haven't noticed you're on one then you're shirking.

(Less snarkily, we're all making resource allocation decisions constantly and on the margin those are life or death decisions).

IMHO, on the margins you're allowed to use common sense. Also: your grandma / your grandpa, your resources applies.

Sort of this: we asked 100 people sitting on a plane to jump out, with or without a parachute. All those who accepted the challenge were sitting on a fake plane standing still at 0.6 metres from the ground. We saw that the use of the parachute made no difference.

There is a well-known joke "nobody has ever done a solid randomized double-blind trial to test whether parachutes actually work."

Now someone has -- and it turns out they don't have any effect!

It's also possible that the paper criticizes some common features of "scientific" studies.

EDIT: I understood the joke of the article, my point is: who are they making a satire of?

Reply on news.ycombinator.com