> "the fact that clinical trials seek people maximally healthy but for the target conditions rather than a representative sample of those with the target condition is a very frequent complaint."
Those aren't quite in contradiction. So, where you start, is with a trial that tries to eliminate meaningful comorbid conditions to allow it to answer the question: "Does this drug work for Disease X?", rather than the question "Does this drug work for Disease X when accompanied by Diseases Y and Z?" There are just far, far too many permutations of X, Y, Z, etc. to allow them all into a study and even pretend to have answered the question about the drug. So, it's not where you start. But, you're absolutely paying attention to patient demographics here, that's part of the point: can we come up with a reasonably homogenous patient pool for step 1? That's not really about being "optimally healthy," it's about homogeneity, except where the latter implies "not a giant variety of comorbid diseases."
But, as I hope I've made clear in the way I reiterated it, that's the initial step. Because, hey, we really do need to answer the question about Drug A for Disease X before we start working on the various permutations of Disease X with other diseases. The latter is usually studied through retrospective analysis rather than RCT. Why?
A given study has only a certain sample size to work with. There's this sort of odd implicit idea that small sample size is merely an issue of study design, but not really: in practice, a research group only has a catchment area of a certain size - they'll only be able to catch so many patients per unit time, and you can't run a study forever (both because you run out of grant funding and because the state of the disease and therapy moves on.) The more specific you get - e.g., people with Disease X and Y and Z, but neither Y and Z to be more than moderately severe - the fewer patients you can potentially recruit, to the point where the final result is statistically meaningless. This isn't really the worst thing ever: the less capable you are of recruiting patients because of their rarity, the less useful such a study would be to the public because of their rarity. As a practical effect, you can't really build a study to control for every meaningful confounding comorbidity. So, we tend to wait for data to accrue and analyse it retrospectively. It's not as good as an RCT, but it's the best we can do under practical constraints.
It's one of those things worth criticizing, in theory, but continues because we don't have the means to do better. Ideally we'd have studies of 20k+ people selecting for significant sub-groups of each permutation of common comorbid conditions. That's just beyond any hope of organizing, running, or paying for.
>>"Anyone who wishes to lend a brain cell is welcome."
>Its pretty misleading to promise this. People who disagree >with the "publish an endless series of statistically >significant differences" paradigm are not going to find it >at all welcoming.
That's simply untrue. I know people working in operations research who are developing novel predictive models for ED patients to predict where they'll need to be sent after workup (and how to prep the appropriate hospital dept. for their arrival, and to estimate when that will be); the VA does shit-tons of operations research. I know people working in the basic sciences who are working on developing ever-more-sophisticated models of the current state of physiology/biochemistry, to allow for increased accuracy of predicting a drug's effectiveness and toxicities before moving on to in vitro testing. And an interesting negative result is about as easily published these days as an interesting positive result (even Easterbrook's big '91 paper on publication bias found that it was for observational and lab studies, but couldn't identify bias in RCT publication.)
The full variety of interesting work to be done in clinical trial design isn't super obvious from the outside. Especially since the folks moaning and whining about it tend to be academics, and a big chunk of good clinical trial work doesn't involve many university academics, and the other cool work (e.g., operations research) is being done in industry and at the VA. But "not super visible to university academics, who are the ones talking about publish or perish" and "non-existent" are very different things.