Trick or Treatment—The Undeniable Facts about Alternative Medicine (Electronic book text)

Before explaining the meaning of publication bias, it is important to stress that this is not necessarily a form of deliberate fraud, because it is easy to conceive of situations when it can occur due to an unconscious pressure to get a particular result. Imagine a Chinese researcher who conducts an acupuncture trial and achieves a positive result. Acupuncture is a major source of prestige for China, so the researcher quickly and proudly publishes his positive result in a journal. He may even be promoted for his work. A year later he conducts a second similar trial, but on this occasion the result is negative, which is obviously disappointing. The key point is that this second piece of research might never be published for a whole range of possible reasons: maybe the researcher does not see it as a priority, or he thinks that nobody will be interested in reading about a negative result, or he persuades himself that this second trial must have been badly conducted, or he feels that this latest result would offend his peers. Whatever the reason, the researcher ends up having published the positive results of the first trial, while leaving the negative results of the second trial buried in a drawer. This is publication bias.

When this sort of phenomenon is multiplied across China, then we have dozens of published positive trials, and dozens of unpublished negative trials. Therefore, when the WHO conducted a review of the published literature that relied heavily on Chinese research its conclusion was bound to be skewed — such a review could never take into account the unpublished negative trials.

The WHO report was not just biased and misleading; it was also dangerous because it was endorsing acupuncture for a whole range of conditions, some of which were serious, such as coronary heart disease. This begs the question, how and why did the WHO write a report that was so irresponsible?

The WHO has an excellent record when it comes to conventional medicine, but in the area of alternative medicine it seems to prioritize political correctness above truth. In other words, criticism of acupuncture might be perceived as criticism of China, of ancient wisdom and of Eastern culture as a whole. Moreover, usually when expert panels are assembled in order to review scientific research, the protocol is to include experts with informed but diverse opinions. And, crucially, the panel should include critical thinkers who question and challenge any assumptions; otherwise the panel’s deliberations are a meaningless waste of time and money. However, the WHO acupuncture panel did not include a single critic of acupuncture. It was quite simply a group of believers who unsurprisingly were less than objective in their assessment. Most worrying of all, the report was drafted and revised by Dr Zhu-Fan Xie, who was Honorary Director of the Institute of Integrated Medicines in Beijing, which fully endorses the use of acupuncture for a range of disorders. It is generally in appropriate for someone with such a strong conflict of interest to be so closely involved in writing a medical review.

If we cannot trust the WHO to summarize adequately the vast number of clinical trials concerning acupuncture, then to whom do we turn? Fortunately, several academics around the world have made up for the WHO’s failure by providing their own summaries of the research. Thanks to these groups, we can at long last answer the question that has lingered throughout this chapter — is acupuncture effective?

The Cochrane Collaboration

Doctors are confronted each year with hundreds of new results from clinical trials, which might cover everything from re-testing an existing mainstream treatment to initial testing of a controversial alternative therapy. Often there will be several trials focused on the same treatment for the same ailment, and results can be difficult to interpret and sometimes contradictory. With not enough hours in the day to deal with patients, it would be impractical and nonsensical for doctors to read through each research paper and come to their own conclusions. Instead, they rely heavily on those academics who devote themselves to making sense of all this research, and who publish conclusions that help doctors advise patients about the best form of treatment.