It’s time all clinical trial results are reported
alltrials.net
alltrials.net
Signed and donated.
This is a great effort, and I suggest others help out as well. The amount of fake trials and hidden experiment data that pharmas participate in is absurd. I can't recall the exact incident ('carbocation probably will) but within the last decade there was a rather large scandal where a particular drug was known to have a fatal side effect -- suicidal ideation, I believe [wrong, valvular heart damage - see below] -- and more than 90% of the trial data that pointed to said effect was hidden and concealed. The drug hit the market and was later pulled off the shelves because, you guessed it, people started dying! While I don't believe the reporting of all trials will fix that level of deceit, it's certainly a great step in the right direction. Pushing for public reporting of data concerning meds that people might eventually put into their bodies only makes sense.
Next up: Abolishing sponsored research clinical trials, or at least increasing the transparency of research sponsorship instead of burying the fact that the research PI is actually an employee of Pfizer somewhere deep in the appendices.
It's amazing how we have organizations like Consumer Reports to give us objective reviews of toasters and cars, yet nothing like that exists for medications or medical devices, isn't it?
Edit: So I'm confusing 2 things:
1) Fen-phen was a diet pill pushed to market despite evidence that it caused valvular heart damage. It was pulled from the market. Read: http://www.pbs.org/wgbh/pages/frontline/shows/prescription/h...
2) SSRIs were the drugs that increase suicidality (something commonly known now) and the data confirming that was reportedly hidden by pharmas. Read: http://reason.com/archives/2007/07/10/can-caution-be-too-ris...
[1] = http://www.fda.gov/Drugs/DrugSafety/PostmarketDrugSafetyInfo...
[2] = http://www.slate.com/articles/health_and_science/medical_exa...
[3] = http://www.thelancet.com/journals/lancet/article/PIIS0140-67...
[4] = http://www.consumersunion.org/pub/core_health_care/001651.ht...
The article cites many university studies which show that use of SSRIs actually coincided with a decreased risk of suicidal ideation.
Is there more recent evidence to suggest the British studies were correct?
I hear a lot of anecdotal evidence when SSRIs and other anti-depressants are discussed, which I agree is important because everyone can react differently to them and doctors and patients should be aware of what may occur, but the article you linked to actually states that the clinical evidence is in support of SSRI prescription in all age groups with major depression.
EDIT: I do note that the article mentions the GSK settlement with Spitzer, and regardless of whether the studies hold true, they should not have been covered up unless there was a question to the scientific standards of the trials.
A very large part of the multi-billion dollar a year R&D budgets of the pharmaceutical companies go to pay for clinical trials. How would you replace that money?
Having a vested interest control scientific studies is poor regulatory system design.
Also, drug companies often hire physicians and researchers (often the best ones) to help with drug development far before any trial is run. Would you like to exclude those experts from input on the trial design?
From your comment I get the sense you have a limited understanding of what is really involved in drug development.
However you appear to have retired undefeated as devil's advocate, as if the questions you put are unanswerable.
I would admit that there are more questions and answers that need to be given before we fully implement my HN comment, but that is hardly a refutation of the core idea: that a system free of conflict-of-interest requires independent entities and umpires that are more empowered than the conflicted interests in that system.
A recent Forbes opinion piece has its own take on inserting independent entities into the system [0]:
What we need is a construct with teeth, an established network of industry-independent researchers (not contractors) equipped to evaluate trial design, statistical power, data analysis, and interpretation, not as regulators but as contributors to final research publications. It would be a sort of Congressional Budget Office for clinical trial results.
Admittedly the position outlined in that article does not go as far as a completely independent entity operating all clinical trials for the purposes of regulatory approval, but it's in the right direction.
[0] http://www.forbes.com/sites/emilywillingham/2012/11/26/how-c...
For example, Pfizer was found guilty of promoting Bextra off label, a very serious charge from the viewpoint of the FDA and the Justice Department. Punishment includes banning companies from doing business with CMS (who run Medicare and Medicaid).
Pfizer entered into an agreement with the Justice Department/HHS where a shell company was allowed to plead guilty and be banned from doing business with Medicare. That's right, a shell company that doesn't actually sell any drugs.
http://www.cnn.com/2010/HEALTH/04/02/pfizer.bextra/index.htm...
I'd agree the unwillingness of the gov't to use the powers it has is a problem. But that's a separate and equally important issue that others in this thread have flagged, filed under the label 'regulatory capture'.
It's not a situation that requires fire and brimstone and outrage. It should just be approached with dispassionate understanding: we have a defective system that is not designed to withstand attacks by self-interested system 'hackers'.
The solution is to design a system that neutralises short-term, narrow self-interest.
Publishing every single trial will probably also make more expensive.
That's what you get when you end up with an overly-regulated market for drugs. You cannot publish anything about them without governmental authorization.
I agree with the premise of the petition but a petition is a very weak tool to reach that goal. It will get you nowhere.
I fail to see where there is "under-regulation" in the pharma industry. Try setting up a Pharma business, you will be surprised how hard it has become.
At the end, there's a "things you can do" section, including:
> Agree to participate in a clinical trial only on condition (i) that the study protocol has been registered and made publicly available (ii) that the protocol refers to systematic reviews of existing evidence showing that the trial is justified; and (iii) that you receive a written assurance that the full study results will be published, and sent to all participants who indicate that they wish to receive them.
http://www.testingtreatments.org/tt-main-text/research-for-t...
The goal here should be simple -- all clinical tests for a drug must be registered before they being in order to be considered evidence of a drug's effectiveness. This has to be done on the level of the FDA or other drug-approving organization, and is a simple and effective means of achieving this end.
Studies that are registered but never publish results will be noted and can be used to identify organizations that either routinely trash results through negligence or poor planning (the good faith version) or who refuse to publish negative results (the bad faith version). In either case, this can be taken into consideration when considering whether to green light a drug.
It's not immediately evident, but ClinicalTrials.gov does provide their database in a structured format...probably the most convenient endpoint is this giant sitemap, which they have generated for scraping uses: http://www.clinicaltrials.gov/ct2/about-site/crawling
What would be the effect on pharmaceutical companies from such a requirement (and how do you even enforce it)? Would they simply do fewer trials, to avoid the risk of a bad trial result hurting sales of a product?
Right now pharma companies run tens of thousands of fake trials where they have no intention of ever publishing the data because it's a way for them to legally pay doctors for signing up patients. Basically they just ask the doctor to fill out a one page survey at the end asking how they felt about how the medication worked, but it's garbage data that couldn't even be used for anything. Basically most phase IV trials, with the exception of the small number the government mandates as a condition for drug approval, are just a scam so that pharma companies can bribe doctors and also so they can list their marketing expenses as R&D in their SEC statements.
Also, the vast majority of trials are never published as journal articles even if they're registered. This is because there is no government mandate to actually report the results of any clinical trial, even the registered one. Which is why when determining the safety and efficacy of a drug you should never look at the published studies, because that will always give a wildly inaccurate view in a way that's extremely biased in favor of the drug companies. (This is what the FDA does when deciding wether or not to approve drugs.)
OK....what do you look at then ?
In Irvin Kirsch's book The Emperor's New Drugs he goes over this at length.
Commonly-requested documents: http://www.fda.gov/regulatoryinformation/foi/electronicreadi...
File a request: http://www.fda.gov/RegulatoryInformation/FOI/HowtoMakeaFOIAR...
Structured data about drug reactions ("adverse events") reported to the FDA: http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInforma...
I'm taking an SSRI now, after about ten years off, and have taken them on and off for more than two decades, and they most definitely have a dramatic effect on anxiety and depression - the effects and general mechanism are pretty obvious. Basically, emotional reactivity is reduced as well as the tendency to ruminate. For me, this effect is clear as day, as are why it causes the side effects that it does which is why I typically don't stay on them for very long. While they don't work for everyone, SSRIs have been enormously popular for at least two decades.
So I gather that you believe the entirety of SSRI's popularity, as well as all the good/bad physiological effects many of us taking them find so apparent, are strictly placebo effect?
When you say "I gather that you believe" you seem to be implying that it's an opinion question. But for what it's worth, that's not how it works exactly. Again read The Emperors New Drugs for a good explanation. The book Anatomy of an Epidemic also has some good info in the relevant chapter.
The suggestion, following Irving Kirsch's book, that the SSRIs are broadly "not effective," is a statement about the distribution of outcomes over a population. (It also takes into account the fact that even placebos cause changes in outcomes, and that we consider a drug not to be "effective" if its outcomes do not differ meaningfully from those attributable to placebos.) This is maybe better stated as "not efficacious" because we're talking about efficacy, vs. placebo, in a population.
This is entirely different from the parent's (counterfactual) statement "the SSRI I'm taking has no effect whatsoever," which is an assertion about having any effect (not a relative effect vs. placebo) in a single individual (not a population).
"Not effective"[as a vector operation and vs. a baseline established by placebo] != "has no effect whatsoever"[as a scalar operation and vs. a baseline of no intervention]
Grandparent has most definitely not said that your perception of SSRI effects are "imagined," nor that you are "misguided." But you are arguing against a straw man until you at least acknowledge the distinctions here (which, it must be said, are pretty well explained in the book in question ;)
Is it your opinion that the popularity of SSRIs is misguided, and that their use would be better off discouraged, since they have been shown to be no more "effective" than placebo? That the vast majority of (or all?) patients who report "success" with SSRIs are simply experiencing an effect that could just as easily be delivered via sugar pill?
Wikipedia's paragraph on this here (http://en.wikipedia.org/wiki/Selective_serotonin_reuptake_in...) has a decent summary of the current status. SSRIs have been shown, by two meta-analyses, to have a "clinically significant" effect for very severe depression, and a "statistically significant" effect for moderate depression, which is considered below the threshold of usefulness. The original poster's words were, "not effective for any type of depression". So will you correct this statement as well?
As for a formally written challenge to Kirsch, I could only find this times editorial http://www.nytimes.com/2011/07/10/opinion/sunday/10antidepre... which makes the argument that someone in my position has already suspected, that the studies being cited are flawed - that it is exceedingly difficult to identify bona-fide "mildly or moderately depressed" people to participate in a study, not to mention that measuring depression is a completely subjective affair:
> Often subjects who don’t really have depression are included — and (no surprise) weeks down the road they are not depressed. People may exaggerate their symptoms to get free care or incentive payments offered in trials. Other, perfectly honest subjects participate when they are at their worst and then spontaneously return to their usual, lower, level of depression.
Without an objective means of measuring depression, measuring the effect of drugs objectively is not straightforward.
Kirsch has, like everyone else, approached this study with a definite bias, a career based on raising the prominence of the placebo affect. I'm not sure you can read a few books and declare "the truth" as case closed here, all ideas are subject to challenge. In this case I think there's a lack of interest in challenging this idea as the drug companies are obviously doing quite well (and I am no fan of drug companies) and probably don't want to raise more attention to it. It's an extraordinary claim that the millions of people who insist that SSRIs are very effective for them are being deceived en-masse, and one under-critiqued meta-study might not be enough to prove that claim.
Interesting comment on that article as well:
> "Papers showing that antidepressants work had found their way into print; unfavorable findings had not." The point is, that there are any number of ways a clinical trial can be run such that an "effective" drug will not be able to demonstrate a level of statistical significance. While there is no way you can run a clinical trial so that an "ineffective" drug will demonstrate a level of statistical significance. If you run nine clinical trials that fail to show a drug is active, and one clinical trial that shows that it is active; when presented with this kind of data, most people in the general public would wrongly conclude that the drug is not active. In fact, the correct conclusion is that the drug is active. That is why it is the job of qualified experts at the FDA, and not the public, to examine the information generated from clinical trials and pass judgment as to whether or not a drug has been proved to be "safe and effective".
If it were possible to make an ineffective drug demonstrate a level of statistical significance, then we'd have tons of studies proving the efficacy of homeopathy, wouldn't we ?
That's the medical equivalent of saying, "it works on my machine." That doesn't mean it works for everyone. It doesn't mean that it works for most people, or that there isn't a significant risk of harm to many people. Hell, your perception that it works for you doesn't even mean that it actually works for you (there could be confounding factors for all we know), and it certainly doesn't mean that it's likely to work better than the next-best alternative.
Think about it: If somebody says SSRIs are incorrectly believed to be effective due to publication bias, that pretty much presupposes that there's at least one study that showed that they work. That means that somebody, somewhere had a similar experience as you've had. That doesn't make your experience a sufficient basis for prescribing these drugs on a routine basis.
Clinical trials are like beta tests. The drug might work for some people (or some people might feel that it works), but that's not the only thing worth considering before you ship it to large numbers of people who need treatment.
The fact is there's both positive and negative studies of SSRIs, the debate over their effectiveness remains ongoing, and at the moment these medications are widely accepted and used. Real world experience of these medications among a huge number of practitioners and patients is still ahead of this debate, in my opinion.
Another comment [1] says that there is an FDA regulation to report results within a year [2]. Are you talking about something else, another type of trial or something?
[1] http://news.ycombinator.com/item?id=5099671 [2] http://clinicaltrials.gov/ct2/about-site/results
People forget that publishing a paper is no small feat in and of itself. Extensive data analysis, manuscript writing, reviewing, editing, legal clearance, etc.
Are people really suggesting that a pharma company who conducts a trial and figures out that 1/2 through the trial the drug isn't working should be forced to do a complete data analysis and publish the results? That sounds ridiculous.
(For fake studies, and for the vast majority of trials not being published.)
(http://www.the-scientist.com/?articles.view/articleNo/27383/...)
https://www.pfizer.com/research/research_clinical_trials/fin...
This is just a list of on-going trials, but I believe Pfizer has signed on to publishing their outcomes as well.
http://clinicaltrials.gov/ct2/about-site/results
There are currently about 7936 studies with "Results": http://clinicaltrials.gov/ct2/results?term=&recr=&rs...
Also, several major journals require authors to adhere to data reporting guidelines of CONSORT: http://www.consort-statement.org/home/
On the other hand, it just sounds like more fodder for uneducated people^ to completely misunderstand what's going on, and all the various failings and pain that comes from that. The last thing we need is pundits and newspapers making wild and broken accusations and 'analysis' based on trying to find the linkbait side to every result.
^Which in the case of clinical trials and papers in general is 99.9% of people, including myself.
I think that what you mean is that people with an agenda will point to a study with, say, a positive but insignificant effect, and say "well there is a positive effect" when they should actually say "there was no effect". In that case, I agree, but there's simply no avoiding that except through better education and more thoughtful phrasing in articles.
I can also understand that the curation can be done for good (let's not release this cause we screwed up and so the results are pointless and people might get the wrong idea) and for evil (as MBCook[0] said, "let's only release the one study out of 5 that proves the point which makes us the most money").
I typed the parent comment out quickly, and to be honest I'm still digesting my thoughts when it comes to this (I feel that for the most part digestion of these topics is a life constant).
There are some books and websites that help, but it'd be great if there were more.
Ben Goldacre - Bad Science
Michael Harris - Medical Statistics Made Easy
Richard Smith - The Trouble With Medical Journals
"His tales of drug companies buying the opinion of doctors is not the most alarming of his revelations. Goldacre sets out clearly what is wrong with the way drugs get on to the market.
New drugs are tested by the companies that make them, often in trials designed to make the drug look good, which are then written up and published in medical journals. Unless, that is, the company doesn't like the result of the trial (maybe it shows the drug not working or having severe side-effects), in which case this result might be hidden.
Regulators should have all the data on a drug's effects but they often don't share it, so researchers can't study the data. The book gives examples of regulatory bodies handing over page after page containing blacked-out results to academics trying to collect data from unpublished trials, the excuse for non-disclosure being commercial sensitivity.
Companies pay doctors to extol the virtues of their drugs on the conference circuit (spelling out the sources of information they want doctors to use) and fund patient groups to lobby regulators to approve new drugs."
http://www.ted.com/talks/ben_goldacre_what_doctors_don_t_kno...
http://www.ted.com/talks/ben_goldacre_battling_bad_science.h...
Journalists suffer from the same problems, as well as the need to write something eye catching. But they already write about diet fads and other non-scientific health claims (and people believe in it), so I don't see how it could get any worse by having more trial data public.
http://www.sciencebasedmedicine.org/
regularly. My single favorite link to share in a Hacker News comment is the article "Warning Signs in Experimental Design and Interpretation" by Peter Norvig, director of research at Google, on how to interpret scientific research.
http://norvig.com/experiment-design.html
Check each news story you read for how many of the important issues in interpreting research are NOT discussed in the story.
To me, that doesn't sound like a problem with open access, it sounds like a problem with trusting untrustworthy sources. How many times do you see something you know about being reported on, and them getting it wrong? Why would anyone trust pundits and newspapers?
Now, whether or not that means you should have invisible hands curating content before it reaches the eyes of the public, I'm still not really sure where I stand.
"Overall, about half of all clinical trials have never been published, and we know that trials with negative results about a treatment are much more likely to be brushed under the carpet."
Bonus: By having all trials public, researchers won't waste time researching something that has already been researched (replication aside).
Sometimes in a clinical trial a drug is discovered to not "work" in some patients, for example ones that have a concurrent condition (drug x works fine except if patient also has condition y). The results of that trial may not be published. Trials can be used to fine tune the profile of the patient you ultimately want to prescribe it to. I don't really understand what the problem is with that. Companies in almost all industries conduct research behind close doors.
I worked for big pharma doing clinical trials for 5 years and really disliked it, but I feel there's a serious witchhunt going on. People are being very vocal over something they know little about.
I'm hesitant buy some of the supplements at http://examine.com/ because I don't know how many negative trials go unreported. Some things like creatine have overwhelming evidence in favor of their efficacy, but the evidence of efficacy of other things like beta alanine is less convincing and would be heavily impacted by negative studies withholding their results.
I'd expect at the very least governments ought to compel the release of results from any trials [part]funded by public bodies
That should be the standard trade-off for the ability to treat humans as rats.
And more generally, if you are running experiments with human beings; society has a material interest in both the process and the results.
Imagine we currently had a well-designed, scientific regulatory system. All clinical trial activity was transparent so that regulatory bodies and the public had the complete picture of the efficacy and safety of pharmaceutical products.
Now imagine pharmaceutical companies were lobbying government to allow them to instead filter their clinical trial data, suppressing scientists from publishing negative data, and only publishing positive data to support their products. They launched a site called http://www.sometrials.com, arguing that only publishing some trials reduced regulatory cost and burden, ensuring cheaper drugs for more people. Wink wink.
Then a modified version of your statement would make a lot of sense, imho: I am terrified of the idea that pharmaceutical companies are trying to compel scientists to do anything against their will. Private corporations don't have this right any more than they have the right to read your encrypted email.
Note that the fact that trial results aren't released to the public doesn't mean that they aren't disclosed to the FDA.
I loudly apploud the poster for the having the nuts to put up a great site =)
What does free speech have to do with it?
Free speech means that you communicate what you want, when you want, if you want.
(Quick aside, with two points: First, the right to free speech doesn't guarantee you the means of broadcasting your views. Such a "right" would mean enslaving those tasked with providing those means, violating their actual rights. Second, not all utterances constitute speech: the words a mafia boss uses to order the beating of a recalcitrant shopkeeper or the murder of a rival are not examples of speech.)
You do not, for comparison, have the right to obtain a passport while exercising your right to silence in response to the information requests from the issuing authority.