Is EBM really an illusion? Notes about a Dutch debate on EBM

September 20, 2017 Brendan Clarke

Federica Russo – University of Amsterdam

Over the summer, while most of us were taking a break after a long academic year, in the Netherlands the Raad voor volksgezondheid en samenleving – the Council for health and society distributed a booklet on EBM. It is titled Zonder context geen bewijs. Over de illusie van evidence-based practice in de zorg – No evidence without context. On the illusion of evidence-based practice in health.

Image credit Wikipedia

It is the result of a project run at the Council. It describes the rise of EBM and presents a number of criticisms available in the literature. It then purports to enhance EBM by advocating a move from an evidence-based to a context-based practice. The members of the working group think that the concrete, real context of patients plays an important role in the various aspects of the medical practice, for instance shared decision making. A quality of the book is its accessibility to readers that are not specialised in either medicine or philosophy of medicine, hopefully making it appealing to the public. The text makes quite provocative claims opening the floor for a discussion about where EBM should go next.

Perhaps not surprisingly, the text attracted severe criticism from advocates  of EBM in the Netherlands. For instance, Patrick Bossuyt published a short commentary in the Nederlands Tijdschrift voor Geneeskunde – the Dutch Journal of Medicine. He expresses deep dissatisfaction with the text. He thinks the criticism is misdirected and the positive views thereby expressed are all well known to EBMers, so it is not enlightening in any respect. In particular, the main objection to the text of the Council seems to be that there is no novelty there: medicine is about context, because all decisions about patients already happen in a context.

Having read both texts (yes, I do read Dutch, and I also speak Dutch, or rather a kind of broken Dutch with a distinctive Italian accent), I find myself in agreement and in disagreement with both parties. It is not my intention to reconcile them – something that I rather leave to their initiative – but to use this as an occasion to highlight some matters in need of further research, and rather urgently.

I disagree that the text of the Council does not consider the more recent developments on evidence assessment, for instance the GRADE system; the text of the Council does discuss more elaborated variations of the old and simple evidence hierarchy but, admittedly, not in great detail. And I disagree that EBM is an illusion. It is not an illusion, since it an established scientific approach and since it did advance the practice of medicine in many ways. Also, while I agree that we need to critically assess EBM (just as any other scientific approach), it is also unjust to criticise EBM without contextualising its rise and development. The rise of EBM in the past decades can be in fact seen as a movement of self-reflection and scientific cleaning-up in the medical profession, which tried to re-assess its own knowledge, discarding authority of the ‘old and wise physician in white coat’. I agree that the role of statistics and of RCTs in EBM has been, at times, overstated, and that it is crucial to include the context in the practice.

So it seems that on this point – the importance of context – everyone agrees: the Council, Bossuyt, myself, and possibly many other too. So what is controversial about it? The disagreement seems to be that while the Council thinks that EBM doesn’t take the context of the patient into account, Bossuyt objects that this is precisely what medicine (and therefore EBM too) is about.

To begin with, there is a slight equivocation in both texts: EBM has been primarily about the efficacy of clinical interventions and about the reliability of diagnostic methods. Knowledge thus produced is then used in the practice of medicine, from GPs to specialists.  I take it that, implicitly, the Council is pleading for a more context-based medicine both at the level or research and at the level of primary / specialised care. But these are distinct questions, and in neither text are properly spelled out.

Consider the first aspect first: how to include context into research (especially in designing, running, and assessing RCTs). “Context”, however, is an elusive concept, on which social scientists (and philosophers thereof) have spilled rivers of ink. But precisely for this reason we need to think more about it. Context includes various qualitative aspects of health and disease that are not necessarily captured by the quantitative methodology of RCTs. One such aspect is, for instance, how knowledge of the underlying bio-chemical mechanisms is to be included in the statistical machinery of the RCT. This is something that EBM+ scholars are devoting much effort to. Another aspect is how to use patients’ narratives in a way that they can provide extra clues for researching disease mechanisms. And once we have advanced our medical knowledge in this way, how is this to be integrated in medical practice? After all, anamnesis is precisely about singling out important factors in the patient’s history, or context. Are guidelines of any help here? Are there too many? Should they be different? How? The set of questions just formulated concerns, by and large, discussions that should happen at the methodological and epistemological level. These questions are surely motivated by practical issues – we want to offer better treatments and cure patients – but they ultimately concern how we should best establish which treatments work. The arguments seem now to move to a different terrain: the question about the practice leads us to questions of management. It is a question about health economics to decide how to run the practice, not a question of scientific method. Agreed, these practical questions are not unrelated to the previous, research-based ones. But who is the real target of the text of Council? EBM researchers, or the managers? And from which perspective is Bossyut defending EBM, as a clinical epidemiologist, or as an adviser to the Zorginstituut Nederland? Understanding the real target helps here because there may be different questions and different motivations behind the text of the Council and that of Bossuyt. These need to be clearly on the table, before the discussion is taken any further.

This is not to side a priori, with EBMers or with the Council. EBM, far from being an illusion, has been a real change in medicine. At the same time, however, the quest for evidence has rapidly spread to several domains, also outside clinical reasoning. Perhaps too rapidly, as we haven’t managed to make all the needed distinctions and differences. Questions of scientific method are distinct from question of health management, or from medical practice and conflating them does help improving medicine in any of these respects.

In sum, I take this squabble between the Council and Bossyut as an opportunity to set the terms for collaboration. After all, both parties aim and work for a better health system. I see, in their text, more agreement than disagreement. And yet, precisely where they agree, much work is still needed.