AllTrials Campaign

About two weeks ago, I got a personal email from Ben Goldacre and Tracey Brown. I and 39,999 other people, that is. It was an update about a petition I signed a while back, demanding higher standards in evidence-based medicine. They've just reached 40,000 signatures, which is really great, but they are aiming for 1 million. This petition is good for medicine, and good for science generally.

The campaign calls for the publication of the results of all clinical trials. Vast amounts of clinical research are carried out, predominantly by drug companies, who have traditionally reserved the right to publish only what they choose. This practice (not entirely limited to drug companies) is harmful and unethical for many reasons. 

If we think about the traditional standards of statistical significance, a systematic causal effect is deemed to have been observed in any case where the results of an experiment are sufficiently unlikely to have arisen (by random variability alone) if there was no effect present. 'Sufficiently unlikely' often means a p-value of 0.05, meaning that if there is no effect to observe, random noise will produce a statistically significant impression of a real effect in an expected 1 out of 20 similar experiments. 

This means that even if there is no true effect, the more individual experiments you do, the greater the chance of sooner or later producing a trial suggesting an interesting positive result. In medical science (as in any other discipline), therefore, partial reporting of outcomes leaves substantial room for distortion of the data and the conclusions they lead to. Quoting from Goldacre's Bad Science website:

The scale of this problem is enormous. It exposes patients to unnecessary harm, because the wrong treatment may be prescribed when the evidence is distorted. It also affects some very expensive drugs. For example, governments around the world have spent billions on a drug called Tamiflu: the UK alone spent £500 million on this drug in 2009, which is 5% of the total £10bn NHS drugs budget, on one drug. But Roche, the drug’s manufacturer, published fewer than half of the clinical trials conducted on it, and continue to withhold vitally important information about these trials from doctors and researchers even today. Some academics now suspect that the drug may be no better than paracetamol.

And quoting from the AllTrials website:

Around half of all clinical trials have not been published; some trials have not even been registered. If action is not taken urgently, information on what was done and what was found in trials could be lost forever, leading to bad treatment decisions, missed opportunities for good medicine, and trials being repeated unnecessarily.

The situation may not always be quite as bad as incorrectly reporting efficacy for a treatment. When cherry-picked data are reported, effect sizes can be exaggerated, so that, for example, a new drug appears to do a better job than a rival product when in fact it performs no better. Even if there is no deliberate cherry picking, data from a larger number of subjects is obviously more informative, so if that data exists, it must be made available in order to maximize the benefits for society.

You might think that this is data that belongs to the drug companies, and they can do what they like with it, but these are trials carried out on real people, who are led to believe that sacrifices they are making are contributing to the advancement of science. Anyway, one couldn't reasonably argue that reporting totally made up data would be ok - there's a justified obligation to tell the truth, and any omission of relevant data constitutes a violation of that obligation just as much as actively lying. 

I wholeheartedly support the AllTrials initiative, which seeks to make it obligatory for all findings from clinical trials to be placed in the public domain. Medical science is literally about life-and-death decisions, so if any discipline demands the full power of scientific method, this is it. Rational decision making, of course, demands high quality inference from the best available data.

Click the logo below to go to the AllTrials website, and consider adding your name to the petition:

By adding our names to a petition like this, we not only make an important statement about how we think medical research ought to be conducted for the better, we add extra weight to the political credibility of science and evidence generally, which is an important additional bonus. If it is seen to be politically unacceptable to allow medical data to be swept under the carpet, impeding the ability of doctors and patients to make informed decisions, then at some point, awareness must grow of the many other areas where decisions are routinely made, based on no data whatsoever. And as awareness grows, so too must incredulity and outrage.

I'm proud to be a small part of a growing movement of people who want to stand up and say 'Look, science is bloody great. You either agree with us, or you are deluded.'

Ben Goldacre, one of the people most involved in this drive to make the drug companies honest, is not only a sterling advocate of evidence-based medicine, but also a committed campaigner for evidence-based... well everything really, and I think he is absolutely right. One of the most right people on the planet, perhaps. If we think about the alternatives to scientific method, with its logical evaluation of empirical experience, all we have is guesswork and superstition, and lets face it, these are fairly stupid things to base important decisions on.

Goldacre's admirable multi-pronged attack on society's failures to apply proper scientific method when it should includes co-authoring this excellent article, Test, learn, adapt: developing public policy with randomized controlled trials, on using RCTs to assess social policies. The extension of scientific method into politics is a logical consequence of any serious commitment to serving society, but bizarrely, politicians have traditionally denied any substantial relevance of science to their discipline. This great and practical paper gently destroys this myth, as well as outlining many of the steps required to make evidence-based politics work. This radio broadcast, also by Goldacre, discusses the same topic. Its a subject very closely allied to my post on moral science, which goes beyond the truism that science is vital for working towards our moral goals, to argue that science is necessary and sufficient for establishing with confidence what our moral goals actually are.

More recently, Goldacre has authored another paper on evidence-based education (downloadable here), extending his righteous agenda into yet another important direction. One of the nice things about this particular direction is that by enabling systematic improvement of education, we can expect better awareness of science among the future members of society, thereby feeding back into the wider program.

These, then are the main reasons we should all sign the AllTrials petition:

• we demand the best possible health care, and we understand that this standard can not be reached if  data from clinical trials is withheld from publication
• we definitely don't want to make it easy for multi-billion dollar drug companies to make money by being dishonest
• we want to send a loud message that there is a growing tide of opinion in favour of evidence and its scientific evaluation, in all areas of human endeavour, to tell our leaders that their scorn for science is a measure of their scorn for society, which they hold at their peril