Author: Brendan Clarke

Federica Russo – University of Amsterdam

Over the summer, while most of us were taking a break after a long academic year, in the Netherlands the Raad voor volksgezondheid en samenleving – the Council for health and society distributed a booklet on EBM. It is titled Zonder context geen bewijs. Over de illusie van evidence-based practice in de zorg – No evidence without context. On the illusion of evidence-based practice in health.

Image credit Wikipedia

It is the result of a project run at the Council. It describes the rise of EBM and presents a number of criticisms available in the literature. It then purports to enhance EBM by advocating a move from an evidence-based to a context-based practice. The members of the working group think that the concrete, real context of patients plays an important role in the various aspects of the medical practice, for instance shared decision making. A quality of the book is its accessibility to readers that are not specialised in either medicine or philosophy of medicine, hopefully making it appealing to the public. The text makes quite provocative claims opening the floor for a discussion about where EBM should go next.

Perhaps not surprisingly, the text attracted severe criticism from advocates  of EBM in the Netherlands. For instance, Patrick Bossuyt published a short commentary in the Nederlands Tijdschrift voor Geneeskunde – the Dutch Journal of Medicine. He expresses deep dissatisfaction with the text. He thinks the criticism is misdirected and the positive views thereby expressed are all well known to EBMers, so it is not enlightening in any respect. In particular, the main objection to the text of the Council seems to be that there is no novelty there: medicine is about context, because all decisions about patients already happen in a context.

Having read both texts (yes, I do read Dutch, and I also speak Dutch, or rather a kind of broken Dutch with a distinctive Italian accent), I find myself in agreement and in disagreement with both parties. It is not my intention to reconcile them – something that I rather leave to their initiative – but to use this as an occasion to highlight some matters in need of further research, and rather urgently.

I disagree that the text of the Council does not consider the more recent developments on evidence assessment, for instance the GRADE system; the text of the Council does discuss more elaborated variations of the old and simple evidence hierarchy but, admittedly, not in great detail. And I disagree that EBM is an illusion. It is not an illusion, since it an established scientific approach and since it did advance the practice of medicine in many ways. Also, while I agree that we need to critically assess EBM (just as any other scientific approach), it is also unjust to criticise EBM without contextualising its rise and development. The rise of EBM in the past decades can be in fact seen as a movement of self-reflection and scientific cleaning-up in the medical profession, which tried to re-assess its own knowledge, discarding authority of the ‘old and wise physician in white coat’. I agree that the role of statistics and of RCTs in EBM has been, at times, overstated, and that it is crucial to include the context in the practice.

So it seems that on this point – the importance of context – everyone agrees: the Council, Bossuyt, myself, and possibly many other too. So what is controversial about it? The disagreement seems to be that while the Council thinks that EBM doesn’t take the context of the patient into account, Bossuyt objects that this is precisely what medicine (and therefore EBM too) is about.

To begin with, there is a slight equivocation in both texts: EBM has been primarily about the efficacy of clinical interventions and about the reliability of diagnostic methods. Knowledge thus produced is then used in the practice of medicine, from GPs to specialists.  I take it that, implicitly, the Council is pleading for a more context-based medicine both at the level or research and at the level of primary / specialised care. But these are distinct questions, and in neither text are properly spelled out.

Consider the first aspect first: how to include context into research (especially in designing, running, and assessing RCTs). “Context”, however, is an elusive concept, on which social scientists (and philosophers thereof) have spilled rivers of ink. But precisely for this reason we need to think more about it. Context includes various qualitative aspects of health and disease that are not necessarily captured by the quantitative methodology of RCTs. One such aspect is, for instance, how knowledge of the underlying bio-chemical mechanisms is to be included in the statistical machinery of the RCT. This is something that EBM+ scholars are devoting much effort to. Another aspect is how to use patients’ narratives in a way that they can provide extra clues for researching disease mechanisms. And once we have advanced our medical knowledge in this way, how is this to be integrated in medical practice? After all, anamnesis is precisely about singling out important factors in the patient’s history, or context. Are guidelines of any help here? Are there too many? Should they be different? How? The set of questions just formulated concerns, by and large, discussions that should happen at the methodological and epistemological level. These questions are surely motivated by practical issues – we want to offer better treatments and cure patients – but they ultimately concern how we should best establish which treatments work. The arguments seem now to move to a different terrain: the question about the practice leads us to questions of management. It is a question about health economics to decide how to run the practice, not a question of scientific method. Agreed, these practical questions are not unrelated to the previous, research-based ones. But who is the real target of the text of Council? EBM researchers, or the managers? And from which perspective is Bossyut defending EBM, as a clinical epidemiologist, or as an adviser to the Zorginstituut Nederland? Understanding the real target helps here because there may be different questions and different motivations behind the text of the Council and that of Bossuyt. These need to be clearly on the table, before the discussion is taken any further.

This is not to side a priori, with EBMers or with the Council. EBM, far from being an illusion, has been a real change in medicine. At the same time, however, the quest for evidence has rapidly spread to several domains, also outside clinical reasoning. Perhaps too rapidly, as we haven’t managed to make all the needed distinctions and differences. Questions of scientific method are distinct from question of health management, or from medical practice and conflating them does help improving medicine in any of these respects.

In sum, I take this squabble between the Council and Bossyut as an opportunity to set the terms for collaboration. After all, both parties aim and work for a better health system. I see, in their text, more agreement than disagreement. And yet, precisely where they agree, much work is still needed.


We’ve been thinking about possible responses to the EBM manifesto (EBMM) that was published after this year’s Evidence Live conference. We are enthusiastic about the manifesto as a way of improving the theory and practice of EBM. That said, we think that improvements are possible, particularly in making evidence more helpful in making clinical decisions, as we set out below.


I think that every academic has had the dispiriting experience of writing what they thought was a good paper, only to have great difficulty publishing it. This post is about one of these – which has been hanging about in my “do something with this, someday” file for four years or so.


I was particularly pleased to read the recent BMJ paper by Margaret McCartney et al, which discussed ways that EBM-based guidelines might be made to better serve the need of individual patients. I’ll urge you to read the whole thing (it’s not long at all), but here, I want to develop one of their themes a bit.

Their argument is that guidelines have become transformed into tramlines, or excessively inflexible rules determining the correct treatment for an individual.

They make other arguments too – such as that guidelines “exceed the limitations of the evidence for many people”. Although this raises an subsidiary point about the disparity of power between clinicians and patients, it gets us into epistemically difficult territory too – that guidelines, these paragons of objectivity and reliability – may end up as an expression of expert option rather than evidence. A startling factoid from this discussion – “Only 11% of American cardiology recommendations are based on high levels of evidence, with 48% based on the lowest level of evidence and expert opinion.“)

Back to the tramline argument: the thumbnail sketch here is that guidelines have been diverted from their original purpose of reducing variation in patient care (A Good Thing) to an authoritarian and bureaucratic imposition that threatens patient autonomy and clinical judgement (A Bad Thing). McCartney et al don’t present things in this cartoon-badman way, and it’s not a position that I would straightforwardly endorse, but there’s something to be said for taking a more subtle form of this transition seriously. For example, General Practitioners in the UK are assessed by a series of targets called the Quality and Outcomes Framework (or QOF, to its friends). While these guidelines are optional (i.e. GPs don’t need to enrol on the scheme),they are linked to financial incentives – dubbed ‘pay-for-performance’. It is therefore in the financial interest of GPs to fulfil as many of the QOF indicators as possible in as many of their patients as possible. The indicators themselves are not terribly controversial. For example, they say that GPs should measure and reduce cholesterol in people with heart disease:

The percentage of patients with coronary heart disease whose last measured total cholesterol (measured in the preceding 12 months) is 5 mmol/l or less.

Given the link between pay, and fulfilling this indicator, it’s not hard to paint the QOF as a tool that aims at compliance, rather than one that aims to make individuals healthier. As Richard Lehman recently blogged, the QOF doesn’t seem to make people live longer.

Note that this isn’t an argument against all one-size guidelines. Some of these (flu vaccination is the example cited by the authors) make sense both for individuals and for groups. But sometimes, the best option for an individual might not coincide with the best option for the group. ACE inhibitors work well to control high blood pressure, but cause severe cough in about 20% of patients that take them. For groups that are fond of the sound of their own voice (like, you know, university lecturers) this side-effect is particularly disabling, and an alternative antihypertensive drug might be a better option, even if they are fractionally less effective at reducing BP than an ACE inhibitor.

The point that I take from this paper is that combining patient preferences with clinical evidence is highly desirable, but complicated. The ACE-inhibitor example above appears dead simple, but developing tools to assist decision-making that really takes account of patient preference that is generally fit for purpose is much harder. How can clinical standards be maintained, with as little unwarranted variation in standards of care as possible, while accommodating individual patient needs, values, and preferences?

McCartney et al appear to agree with this. Their recommendations, at the end of the paper, as that “Patient decision aids should be published in tandem with guidelines, but better research is required into how to provide information about choices that is easily and quickly understood”.

Words by Brendan Clarke

Image: Guidelines or tramlines? Credit: Tram Tracks by Mike Brand on Flickr(

I’m speaking at the EBM+ project meeting at UCL on Monday on a topic that I’ve been working on for a couple of months now. Very briefly, the talk is about Wigmore charts, and ways that we might use them to support clinical decision-making.

Wigmore chart
Wigmore chart

Wigmore charts (like the example here that I’ve borrowed from Anderson, Schum and Twining’s excellent 2005 book) were originally designed to support complex legal arguments. Imagine that you are trying to build a complex legal case: trying to convict someone of fraud, say. Wigmore charts are a tool for showing how these complicated legal argument works. Here, the “ultimate probandum” is the legal verdict that you are trying to reach (in this case, something like “x knowingly defrauded y”). The chart shows the steps of the legal argument that support this final verdict, all the way down to the many pieces (often, in court cases, many thousands of them) on which the case is build.

My current thought is that these inferential networks would be useful in medicine too, particularly when dealing with complex decisions about evidence. I think that we might use Wigmore charts, or something similar, as an heuristic (see Chow’s recent BJPS paper for a cracking introduction). But to say more would give the game away.

You can have a look at my slides here – [2mB .pdf].