In Pursuit of Evidence by Kate Chatfield RSHomAnyone who has been working with homeopathy in the UK, particularly in the past five years, must have noticed a steady increase in both the extent and depth of adverse publicity that homeopathy is receiving. Repeatedly the critics ask the same question: “Where is your evidence?” Consideration of this question first led me to become involved with homeopathy research, but increasingly I find that my background in philosophy compels me to challenge the very concepts of evidence, proof and science that drive the research agenda. What follows is an account of evidence-based medicine, some of the implications for homeopathy, and philosophical challenges for the foundations of our assumptions.
The practice of allopathic medicine has been subject to many trends, driven not only by developing scientific knowledge, but also in reaction to social, political and financial pressures. The introduction of evidence-based medicine (EBM) has been heralded as the most recent revolutionary phase, described as a ‘paradigm shift’ that will change medical practice for years ahead (Guyatt et al 2002).
In 1992 a group of physician-researchers, otherwise known as the Evidence Based Medicine Working Group (EBMWG), published an article urging physicians to base clinical decisions purely on the basis of evidence. In this way unreliable, intuitive judgements would be replaced with rational calculation and the use of research (EBMWG, 1992). Gone are the times when it was acceptable to base clinical decisions upon personal experience, discussions with peers, case reports, information from textbooks or intuition. Now we have a more reliable scientific method for decision-making that involves study and critical assessment of all available research that is relevant to our question.
This notion of EBM is predicated upon the assumption that there is a hierarchy of evidence for medical interventions, such that certain types of evidence are deemed of greater value than others. At the top of this hierarchy are results from randomised controlled trials (RCTs) and systematic reviews or meta-analyses of these trials. A systematic review is a means of examining results from more than one trial to look for trends. A meta-analysis takes this overall view further by performing statistical analysis on the combined results to look at the statistical significance of these trends.
The synthesis of large amounts of clinical trial data into manageable systematic reviews or meta-analyses is supposed to revolutionise medical practice and offer objective and politically transparent criteria for treatment choice and funding decisions. Proponents pride themselves on their attempt to ‘realign medicine with science’, whilst critics object that EBM emphasises exclusively the science of medicine whilst denying completely the art of medical practice, (Miettinen, 2001).
Implications for HomeopathyThe call for an evidence base is not limited to allopathic medicine and there is growing pressure on all forms of complementary and alternative medicine (CAM) to provide the kind of evidence that is deemed of high value from RCTs and reviews of these trials.
The House of Lords, in its report on CAM, suggested that it is desirable to obtain evidence about the efficacy, safety, cost-effectiveness and mechanisms of these therapies, before their use is advocated (House of Lords, 2000).
The requirement for evidence has practical implications for the availability of CAM. NHS healthcare providers require evidence before considering CAM services (Prince of Wales’s Foundation for Integrated Health, 2003).
In 1997, a London health authority decided to stop paying for homeopathic treatment after concluding that there was not enough evidence to support its use. The Lambeth, Southwark and Lewisham Health Authority had been referring more than 500 patients per year to the Royal Homeopathic Hospital in London (Wise, 1997).
Finding the EvidenceThe two main resources for evidence-based practice in the UK are Bandolier (
www.jr2.ox.ac.uk/Bandolier), written by Oxford scientists, and the Cochrane Library (
www.cochrane.co.uk). Both carry easily accessible information about an extensive range of medical conditions and the research results. A brief examination of the Cochrane Library quickly highlights the vast disparity between both the amount and the types of research evidence that are available for allopathic and homeopathic interventions. For example under ‘rheumatoid arthritis’ we find a large amount of research evidence for each particular kind of allopathic intervention reviewed separately: 13 different drugs, different forms of exercise, different forms of electrotherapy and so on. Whereas, across the whole site, homeopathy is only reviewed under four areas in total: asthma, dementia, induction of labour, and the use of Oscillococcinum for prevention and treatment of influenza.
Typically, where homeopathy has been subject to systematic review for a particular condition such as asthma, all kinds of homeopathy are lumped in together. Isopathy, therapeutic prescribing, combination-remedy treatment, individualised prescribing, and so on, are all considered under the label of homeopathy. Hence it is immediately apparent that homeopathy is not reviewed in an equivalent way to allopathy. We would not find a systematic review of allopathy, including each kind of allopathic intervention for the treatment of asthma, because it would be considered meaningless. Whereas there are so few trials of homeopathy for the treatment of asthma that they are all banded together regardless of whether or not they in any way reflect what happens in the real world.
There have of course been many more research trials than are mentioned in either the Cochrane Library or on Bandolier, but research in homeopathy is so badly under-resourced that there is still only a tiny fraction of the amount that would be necessary to make sensible comparisons with trials of allopathic interventions.
Historically the CAM research community has attempted to address the call for evidence and many of its members have undertaken the challenge to provide the gold-standard RCT research that the EBM movement demands. Experience of this has taught us many important lessons including the following:
• That homeopathy, as a whole person treatment, does not lend itself well to testing through the placebo-controlled, randomised trial, and many researchers have begun to look for alternative methodologies that are more suited to researching this complex intervention (Weatherley-Jones et al, 2004).
• That even when trials have produced positive results they are dismissed as flukes, or subject to researcher bias. For example, David Taylor Reilly and colleagues conducted a series of trials, over a number of years, in patients with hay fever, asthma and perennial rhinitis that demonstrated a significant difference between the placebo and homeopathic groups. The important thing about these trials was that the results were reproducible and yet they are still not considered firm evidence. (Reilly et al, 1986; Reilly et al 1994; Taylor et al, 2000).
What We Mean When We Speak of Evidence The current notion of EBM is fundamentally based upon the assumption that evidence can provide us with ‘facts’ about the world and that scientific beliefs stand or fall in the light of this evidence. This is however an over-simplistic understanding of the nature of evidence that philosophically would be described as explicitly ‘positivist’.
Positivists believe that scientific questions can be settled in a purely objective manner and that experimental testing allows the scientist to compare his theory directly with the facts. They claim it is possible and essential for the inquirer to adopt a distant, non-interactive posture such that values and other biasing, and confounding factors are thereby automatically excluded from influencing the outcomes. In this way science provides us with a sure route to the truth.
However, over the past 50 years much of the philosophy of science has been preoccupied with challenging the positivist picture of scientific methodology. The challenges fall broadly into two main areas:
• First, observations are necessarily dependent upon the observer
• Secondly, theories are never determined exclusively by data (underdetermination)
Observations and ObserversThe positivistic understanding of evidence does not take into consideration culture, contexts and the subjects of knowledge production. What an observer sees that is, the visual experience that an observer has when viewing an object, depends in part on past experience, knowledge and expectations. Individual experiences cannot be logically deduced to be identical or even the same. Observation is dependent upon and coloured by our individual senses and our background beliefs and assumptions, so that what we see is never what really is, even under the most highly controlled experimental settings (Kuhn, 1970, and Feyerabend, 1978).
Our acceptance or rejection of evidence is also value-laden, and our natural tendency is to reject evidence that does not fit with currently held theory; hence, positive results from even the highest- standard RCTs in homeopathy will be rejected by those who do not believe in the possibility that ultra-high dilutions can have an effect.
UnderdeterminationOtherwise known as the Dunhem-Quine thesis, this states that any given body of evidence may support numerous, even contradictory theories. Scientific theories are inevitably underdetermined by data, such that what counts as good scientific theory depends largely upon other factors: social and political agendas, preferences, biases and whims – not the evidence itself.
We can find clear examples of underdetermination in homeopathy. Many people have carried out meta-analyses of placebo-controlled homeopathy trials in an attempt to answer the question of whether or not homeopathy is more effective than placebo (NHS Centre for Reviews and Dissemination, 2002)
These meta-analyses have been drawing upon the same data sources, published trials of homeopathy, and yet they come up with conflicting and contradictory results. The same data can be used to support either the claim that homeopathy does have an effect over and above placebo and also that it does not have an effect over and above placebo. Both theories are underdetermined by the available data and hence the data can be used to support either one.
In August 2005 The Lancet published a damning meta-analysis of homeopathy trials, which concluded that their results were consistent with the theory that the clinical effects of homeopathy are placebo effects (Shang et al, 2005). An editorial in the same issue called for ‘the end of homeopathy’. This meta-analysis is an extreme example of underdetermination. The final statistical analysis was performed upon only eight trials of homeopathy and six trials of allopathy. The authors used certain criteria for the selection of ‘highest-quality’ trials and these criteria were based upon their own beliefs about what makes a trial high quality, and nothing to do with what makes a trial sound from the perspective of a homeopath. It would be an easy task to use different criteria for the selection of high-quality trials and generate results that support the exact opposite theory.
Scientific ProofJames Randi, magician and self-proclaimed sceptic, has famously offered $1,000,000 to anyone who can prove that homeopathy is any more than placebo effect; but none of the above constitutes proof.
• Randomised, placebo-controlled trials that demonstrate statistically significant results? We have those (Bell et al, 2004; Yakir et al 2001)
• Meta-analyses that demonstrate effect over and above placebo? We have these (Kleijnen et al, 1991; Linde et al 1997)
• Trials involving children, demonstrating positive results? We have those too (Frei & Thurneysen, 2001; Jacobs et al 2000)
• Trials involving the treatment of animals, demonstrating positive results? Yes we also have these (Merck von et al 1989; Searcy et al 1995)
• How about trials that demonstrate the effects of ultra-high dilutions in the test tube? We even have these (Belon et al, 2004)
But none of the above constitute proof…
So just what would serve as ‘scientific proof’ that homeopathy has a positive effect over and above placebo? Well, in fact, no amount of clinical trials could ever prove this because, as any philosopher of science can tell you, proof can never be attained through scientific endeavour.
A large proportion of our beliefs are obtained by a process of projecting from observed (past or present) events to cases that are either unknown, unobserved or in the future. For example: “Every swan I have ever seen has been white. It is likely that the next swan I see will be white.”
Scientific inquiry proceeds in exactly the same way, projecting from (repeated) observation to universal laws. David Hume was the first philosopher to highlight the problems with this form of reasoning, way back in the 18th century. Hume stated that what leads us to believe that the future will resemble the past is a matter of custom or habit and cannot be rationally justified.
Seventeenth century Europeans only had access to white swans, so even a completely unbiased researcher would have had reason to conclude that all swans were white. When Europeans finally travelled to Australia, they discovered that it is not true that all swans are white. If someone had suggested in the 17th century that some swans might be black, this would have been dismissed as entirely fanciful; no one could have predicted that some swans are black.
Karl Popper, a leading influence in 20th-century philosophy of science, is most famous for his belief that theories can never be proven, however many pieces of evidence that we have to support them. Rather, they can only be disproven. It does not matter how many white swans we observe, we could never prove that all swans are white because we can never be sure that we have seen all swans. However the observation of just one black swan is enough to disprove the theory that all swans are white.
Hence ‘scientific proof’ is a misnomer and the best we can aim for are hypotheses that have not yet been refuted.
We are never going to be able to prove the efficacy of homeopathy through clinical studies. All we can do is add to the number of observations that support the theory that homeopathy has an effect over and above placebo. Even when we have a very large number of these positive trials, they will not constitute proof. Randi’s million dollars are safe.
The High-Dilution ProblemIt’s hardly surprising that homeopathy is accused of being no more than placebo. This is a completely logical conclusion from the perspective of a person who thinks and works within the current prevailing scientific paradigm. Positive results from placebo-controlled trials of homeopathy are doubted, as critics claim that remedies are composed of nothing more than water. To accept that homeopathic remedies have an effect would entail revision of much of our basic scientific conception of chemistry and the body (Sehon & Stanley, 2003).
In spite of this apparent irrationality, homeopathy is the most frequently used form of CAM in five out of 14 surveyed countries in Europe, and among the three most frequently used in 11 out of 14 surveyed countries (NOU 1998). Between one quarter and one fifth of all EU citizens use homeopathic medicines (EU commission statement, 1997).
In 2005 the results from a six-year outcome study at the Bristol Homeopathic Hospital were published (Spence et al 2005). This was an analysis of over 23,000 outpatient consultations from 1997 to 2003. Over 70% of follow-up patients reported clinical improvement following homeopathic treatment, clearly demonstrating the clinical effectiveness of the homeopathic intervention. Whilst this extensive study can tell us that homeopathy was largely effective for this particular group of people, what it cannot demonstrate is that this was not simply placebo effect.
However, if homeopaths can facilitate a placebo-induced healing response in over 70% of people who visit them, many of whom have previously not been helped by various types of allopathic intervention, then surely homeopaths should be highly revered and re-labelled ‘miracle workers’.
Time for a New Paradigm?The history of science teaches us that there have always been things that could not be explained according to currently held scientific opinion. Science is not static, it evolves as understanding increases and is occasionally subject to a complete revolution, bringing with it an entirely new scientific paradigm (Kuhn, 1970). Kuhn describes how, when we are close to this type of paradigm shift, there will be many things that appear to be incommensurable with the prevailing paradigm. These he terms ‘anomalies’ and they cannot be explained within the framework of current scientific opinion. Homeopathy might be considered an example of an anomaly.
It is normal for people who try to work with these anomalies to be marginalised, as the emergence of a new paradigm is painful for those with vested interests in the existing paradigm. However the emergence of a new paradigm that embraces holism, vitalism and homeopathy need not involve the rejection of rationalism. Rather we might expect a re-direction of the rationalistic tradition that encompasses better explanation of known anomalies.
Whilst the practice of homeopathy becomes ever more popular, in spite of its incommensurability with the prevailing scientific paradigm, homeopaths should not be at all surprised by resistance and even attack from those whose beliefs are firmly rooted within that paradigm. Until we have an accepted explanation about how ultra-high dilutions can have an effect, then no small amount of ‘scientific evidence’ through clinical trials will convince the sceptics.
Maybe it is time for researchers in homeopathy to focus upon what is of benefit to the development of homeopathic practice instead of trying to prove that homeopathy has an effect over and above placebo. Just exactly what is it that we do that enables more than 70% of our patients to say that they have felt benefit? Clearly we are doing some things well, but many of our traditions and beliefs in homeopathy have not been subject to rigorous testing, and research has a lot to offer us in this area. If through research we can improve upon that 70% figure, then just imagine the future for homeopathy.
References Bell IR, Lewis II DA, Brooks AJ, Schwartz GE, Lewis SE, Walsh BT, Baldwin CM. (2004) Improved clinical status in fibromyalgia patients treated with individualized homeopathic remedies versus placebo Rheumatology 43:557-582
Belon P, Cumps J, Ennis M, Mannaioni PF, Roberfroid M, Sainte-Laudy J, Wiegant FAC. (2004) Histamine dilutions modulate basophil activation. Inflamm. Res. 53: 181-188
EU commission statement (1997) Resolution on the Commission report to the European Parliament and the Council on the application of Directives 92/73/EEC and 92/74/EEC on homeopathic medicinal products. COM(97)0362 - C4-0484/97
Evidence-Based Medicine Working Group (1992) Evidence-based medicine: A new approach to teaching the practice of medicine JAMA 268 2420-2425
Feyerabend P (1978) Science in a free society. London: New Left Books
Frei H & Thurneysen A (2001) Homeopathy in acute otitis media in children: treatment effect or spontaneous resolution? Br Homeopath J 90: 180–182
Guyatt GH et al (2002) Introduction: The Philosophy of Evidence-Based Medicine. Users’ guide to the medical literature: A manual for evidence-based practice Chicago, AMA Press
House of Lords Select Committee on Science and Technology (2000) Complementary and Alternative Medicine. Session 1999–2000 6th Report.
Jacobs J, Jimenez M, Malthouse S, Chapman E, Crothers D, Masuk M, Jonas WB (2000) Acute Childhood Diarrhoea- A Replication. Journal of Alternative and Complementary Medicine, 6: 131-139.
Kleijnen J, Knipschild P, Ter Riet G. (1991) Clinical trials of homoeopathy. British Medical Journal.302:316-23.
Kuhn TS (1970) The Structure of Scientific Revolutions. [2 ed].The University of Chicago Press: Chicago
Linde K, Clausius N, Ramirez G, et al. (1997) Are the clinical effects of homoeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet 350:834-43.
Merck von CC, Sonnenwald B, Rollwage H. (1989) Investigations of treatment of acute bovine mastitis with homeopathic drugs. Berl Munch Tierarztl Wochenschr 102: 266-72.
Miettinen OS (2001) The modern scientific physician: 1. Can practice be science? CMAJ 165: 441-442NHS Centre for Reviews and Dissemination (2002) Homeopathy. Effective Health Care Bulletin 7(3): 1-12.
Norges offentlige utredninger, NOU 1998 Alternativ medisin 21 (Official report published by the Norwegian Department of Health)
Reilly DT, Taylor MA, McSharry C, Aitchison T (1986) Is homoeopathy a placebo response? Controlled trial of homoeopathic potency with pollen in hay fever as a model. Lancet ii: 881-886
Reilly DT, Taylor MA, Campbell J, Beattie N, McSharry C, Aitchison T, Carter R, Stevenson R (1994) Is evidence for homoeopathy reproducible? Lancet 334: 1601-1606
Searcy R, Reyes O, Guajardo G (1995) Control of subclinical bovine mastitis: utilization of a homoeopathic combination. Br Homoeopathic J 84: 67-70.
Sehon SR & Stanley DE (2003) A philosophical analysis of the evidence-based medicine debate BMC Health Services Research 3:14-24
Shang A, Huwiler-Müntener K, Nartey L, et al (2005) Are the clinical effects of homeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet 366: 726-32
Smith CA (2006) Homeopathy for induction of labour. The Cochrane Database of Systematic Reviews Issue 3
Spence DS, Thompson EA, Barron SJ (2005) Homeopathic treatment for chronic disease: a 6-year, university-hospital outpatient observational study. J Altern Complement Med 11: 793-798
Taylor MA, Reilly D, Llewellyn-Jones RH, McSharry C, Aitchison T, Lancaster T (2000) British Medical Journal 321:471-476
Vickers A (2000) Randomised controlled trial of homeopathy versus placebo in perennial allergic rhinitis with overview of four trial series. British Medical Journal 321: 471-476.
Weatherley-Jones E, Thompson, EA, Thomas KJ (2004) The placebo-controlled trial as a test of complementary and alternative medicine: observations from research experience of individualised homeopathic treatment. Homeopathy 93:186–189.
Wise J (1997) Health authority stops buying homeopathy. British Medical Journal 314:1574, 1997)
Yakir M, Kreitler S, Brzezinski A, Vithoulkas G, Oberbaum M, Bentwich Z. (2001) Effects of homeopathic treatment in women with premenstrual syndrome: a pilot study. Br Homeopath J. 90(3): 148-53.
