Yes, and then there is the influence of the drug makers too.
The other point is complexity. The number of variables needed to create a useful model of a jet engine is a lot smaller than the number needed to model an organic system. This means that today we have useful models of jet engines and we can reliably test something with simulations - we may generate loads of valid data and easily apply "top-notch" statistics. The same, unfortunately, is not true for medicine.
I think it's becoming clearer each day that many drug side effects, and general drug efficacy is effected by the genetics of the drug taker. Without being able to tease out these genetic effects, blindly aiming for broad population safety or efficacy levels is always going to be troublesome.
This is separate from the data publication issue that the article focuses on.
Not just expensive but also potentially unethical.
Consider this: post-exposure prophylaxis for HIV. It's used both for sexual and non-sexual exposure to the HIV virus. It's a month-long series of antiretrovirals (the exact protocols used differ). It's effective if you start taking it within 72 hours of exposure.
The drugs themselves have been tested for use by people who are HIV+, but nobody has done clinical trials on using them as post-exposure prophylaxis. We know it works, but we don't know how well because testing would be unethical. You'd have to take half the people who are exposed and give them placebo and put them at risk of getting HIV.
At a certain point, you have to throw the dice and say "given the evidence we have, this is what we are going to do".
(The fact that such judgement calls are being distorted by evidence being intentionally withheld by drug companies is disgraceful though.)
Confidence of effectiveness is very easy to get - you just need to find an effective drug. e.g. when antibiotics were first tried - they are massively effective compared to previous treatments - you get high confidence of effectiveness from just a small sample size.
A reason it is hard for new drugs to get high confidence is because they are only marginally better than existing drugs.
But new therapies have to compete with existing therapies in the market anyway. If a new therapy can't outperform existing ones, what actual value does it have? Perhaps drug companies should think about solving problems that aren't already solved rather than bemoaning their inability to reap massive profits while reinventing the wheel or pushing minor formulation tweaks to renew their patents.
Take any drug, test in on millions of patients, here is your six sigma. What problem did you solve?
Your comment shows me, that not much has changed.