This is a guest post by Professor Miriam Solomon. Following Donald Gillies’ earlier post on Miriam’s book, Making Medical Knowledge, in which he clarified the EBM+ position, Miriam has very kindly provided us with her reply:
What is at stake in arguing for EBM+?
I am most grateful to Donald Gillies for his blog post (July 26, 2016) clarifying the position of the EBM+ group. He is right that I presented an inaccurate picture in Making Medical Knowledge (Oxford 2015), conflating the position of the EBM+ group with that of some other critics of the traditional evidence hierarchy. I apologise for this. In this response I would like to clarify what I think is at stake in arguing for the EBM+ position. I am largely—but not completely—sympathetic to the EBM+ position, and I think that my reservations may contribute to useful discussions.
Many critics of evidence-based medicine (EBM) have pointed out that randomized controlled trials, systematic reviews, and meta-analyses—which give the highest quality of evidence in the EBM hierarchy—do not provide sufficient knowledge or guidance for all of medical research and practice. The design and interpretation of high quality clinical trials also requires “mechanistic reasoning” (the term most often used by philosophers) or “pathophysiological reasoning” (the term most often used by scientific researchers and clinicians). Such reasoning is justified when there is evidence for it (“mechanistic evidence” or “evidence for pathophysiological processes”). The EBM+ group is careful to say that what needs to be added to EBM is evidence for mechanisms/pathophysiological processes, and they call this “mechanistic evidence.” Other philosophers of medicine, e.g. Jeremy Howick, call only for “mechanistic reasoning” and sometimes conflate this with evidence of mechanisms, which is a source of confusion.
While the EBM+ group want to expand EBM to include all the relevant evidence and reasoning for medicine, I have argued for keeping EBM as it is while noting that EBM is not a complete epistemology or methodology for medicine. What is at stake in whether or not we expand EBM to EBM+? I think that what is at stake is the defining insight of EBM: that however much we may think that we understand and can intervene with a disease process, we still need high quality clinical trials to assess effectiveness. Moreover, if we can successfully intervene with a disease process (some philosophers of science might call this “robustness”), we have enough knowledge for practice, even if we do not understand how the intervention works, and even if we thought in advance that such a successful intervention was improbable. I argue in Making Medical Knowledge that EBM is a descendent of Greek empiric medicine in this emphasis on clinical success. I also suggest that it would have been clearer to call EBM “Epidemiological Medicine,” emphasizing that the kind of evidence it is interested in is evidence of effectiveness in patient populations (which we might call “epidemiological evidence”). If we had used the term “Epidemiological Medicine,” I think that we would be less concerned by the fact that EBM does not provide sufficient knowledge or guidance for all of medical research and practice. Epidemiological evidence is one kind of evidence, and there are other kinds.
Epidemiological evidence is worth singling out into a special category because it is evidence that is especially relevant for clinical practice. Because there is a long history of failure of well-conceived interventions in the disease process (for example, the use of vertebroplasty for osteoporotic spinal fractures and the use of angiogenesis inhibitors for cancer), epidemiological evidence is necessary. However much basic science evidence there is for the basic mechanisms of the disease process and the proposed therapeutic process, there is no substitute for clinical trials. This is because basic science models are usually idealized, leaving out some of the mechanisms because they are unknown or difficult to take into account.
EBM+ argues that both epidemiological evidence and evidence of mechanisms are necessary to infer causality. I disagree with this, finding epidemiological evidence sufficient. Our disagreement shows in a particular kind of case: when there is evidence of clinical effectiveness but no evidence of mechanism (or even evidence that there is not an appropriate mechanism). A typical such case in the history of medicine would be the discovery that willow bark (which contains the ingredient in aspirin) relieves pain. Willow bark was widely recommended as a treatment for pain millennia before there was any understanding of how it alleviates pain. EBM+, as I understand it, would say that we did not have knowledge that willow bark causes pain relief before we had an idea about the molecular mechanisms of prostaglandin inhibition. I think that the EBM+ position is implausible: in the case of willow bark, knowledge of causation came before knowledge of molecular mechanisms.
EBM+ does not use the example of aspirin. The paradigmatic example for EBM+ is Leibovici’s 2001 study showing that there is an inverse correlation between remote, retroactive, intercessory prayer, and length of stay as well as length of fever in hospitals for patients with bloodstream infections. The EBM+ group think that even if an RCT shows a strong inverse correlation, we would (and should) reject the idea that remote, retroactive, intercessory prayer is causally effective. And they use this example to suggest that we need evidence for a plausible mechanism as well as evidence for correlation to infer causation. I find this example misleadingly extreme, because it suggests retroactive causation (which many regard as impossible), and because we already have so much evidence about the ineffectiveness of intercessory prayer. I think evidence of correlation (epidemiological evidence) can be sufficient (of course, not infallible) evidence for causation, especially since high quality EBM can avoid typical sources of confounding through randomisation and controls (not all, as Worrall has pointed out).
EBM+ has its roots in the Russo-Williamson thesis (RWT) (Russo and Williamson 2007) which claims that causal knowledge requires both knowledge of correlation and knowledge of mechanisms. I think that the RWT asks for too much, if the goal is knowledge sufficient for clinical practice. Even a weak version of the RWT, requiring both knowledge of correlation and knowledge of the possibility of a mechanism, is too strong. If remote, intercessory prayer (not retroactive) was followed by shorter patient stays in a robust manner, I think we should accept it as part of clinical practice. This example is unlikely only because we have long experience of the ineffectiveness of remote intercessory prayer. A more typical example is: when it turned out that saline mist, rather than pure water, was more effective in the treatment of cystic fibrosis, this was accepted despite mechanistic evidence and reasoning predicting the opposite. The correlation was discovered accidentally, through the positive experiences of Australian surfers with cystic fibrosis.
This disagreement over the need for mechanistic evidence for causal claims that underlie medical practice is the important difference between the EBM+ position and my own. We agree that mechanistic evidence and reasoning are indispensable for doing science and practicing medicine; we just disagree about where they are needed. Furthermore, we agree that mechanistic evidence does not belong in the traditional EBM evidence hierarchy. The EBM+ group calls for a “parallel” grading system for evidence of mechanisms, and gives some ideas about how such a grading system would be established. I agree with them that the EBM hierarchy will not suffice when considering empirical data other than clinical trials. My guess is that any such “parallel” grading system will have to consider such a wide variety of types of evidence for mechanisms that it will need to formalize scientific method as a whole.