Trick or Treatment—The Undeniable Facts about Alternative Medicine (Electronic book text)

Turning to medical research, there are numerous treatments that have been tested by meta-analysis. For example, in the 1980s researchers wanted to know if corticosteroid medication could help reduce respiratory problems in premature babies. They designed a trial which involved giving the treatment to pregnant women likely to have premature births and then monitoring the babies born to these mothers. Ideally, the researchers would have conducted one trial in a single hospital with a large number of cases, but it was only possible to identify a few suitable cases each year per hospital, so it would have taken several years to accumulate sufficient data in this manner. Instead, the researchers conducted several trials across several hospitals. The results of each individual trial varied from hospital to hospital, because the numbers of babies in each trial was small and random influences were large. Yet a meta-analysis of all the trials showed with certainty that corticosteroid medication during pregnancy benefited premature babies. This treatment is part of the reason why the number of infant deaths due to respiratory distress syndrome has fallen dramatically — there were 25,000 such deaths in America in the early 1950s and today the number is fewer than 500.

The meta-analysis in the premature baby study was fairly straightforward, because the individual trials were similar to each other and so they could be merged easily. The same is true of the hypothetical example concerning astrology. Unfortunately, conducting a meta-analysis is often a messy business, because the individual trials have generally been conducted in different ways. Trials for the same medication might vary according to the dose given, the period of monitoring, and so on. In Linde’s case, the meta-analysis was particularly problematic. In order to draw a conclusion about the efficacy of homeopathy, Linde was attempting to include homeopathy trials investigating a huge variety of remedies, across a range of potencies, being used to treat a wide range of conditions, such as asthma and minor burns.

Linde trawled through the computer databases, attended numerous homeopathic conferences, contacted researchers in the field and eventually found 186 published trials on homeopathy. He and his colleagues then decided to exclude from his meta-analysis those trials that failed to meet certain basic conditions. For example, in addition to a group of patients being treated with homeopathy and a control group of patients, an acceptable trial had to have a placebo for the control-group patients, or there had to be random allocation of patients to the treatment and control groups. This left eighty-nine trials. What followed was months of careful statistical analysis, so that each trial contributed appropriately to the final result. For example, the result of a very small trial would carry very little weight in the overall conclusion, because the reliability of a trial’s result is closely linked to the number of participants in the trial.

The meta-analysis was eventually published in September 1997 in the Lancet. It was one of the most controversial medical research papers of the year, because its conclusion endorsed exactly what homeopaths had been saying for two centuries. On average, patients receiving homeopathy were much more likely to show signs of improvement than those patients in the control groups receiving placebo. The paper concluded: ‘The results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homeopathy are completely due to placebo.’ In other words, according to the meta-analysis, homeopathy was genuinely effective.

Not surprisingly, Linde’s conclusion was questioned by opponents of homeopathy. Critics argued that his meta-analysis had been too lax, inasmuch as it had included too many trials of relatively poor quality, and they feared that these might have biased the overall conclusion in favour of homeopathy. Homeopaths responded that there had been a quality threshold, which Linde had implemented specifically in order to exclude poor-quality trials. Remember, Linde had included only those trials that were placebo-controlled or randomized. Nevertheless, the critics were still unhappy, as they maintained that the quality threshold had not been set high enough.