Month: March 2016

This 9th conference of the Munich-Sydney-Tilburg (MuST) conference series aims at gathering philosophers and scientists of the natural and social sciences in order to examine the theoretical and methodological issues involved in evidence evaluation, statistical inference and causal inference in relation to risk assessment and management in various disciplines, with a special attention to pharmacology. In particular, following questions will be on focus:

How should we collect, evaluate, and use evidence for the purpose of risk management and prevention? What methods should be adopted in causal inference for preventing harm? What kinds of scientific inferences are we allowed to draw from data-mining techniques? What are the relevant decision-theoretic dimensions involved in different kinds of risks, and what kinds of decision rules are more advisable in diverse contexts? What types of uncertainties can we identify when dealing with hazards?

These questions raise methodological concerns related to the data and tools available for risk measurement and modeling, the right kinds of interventions we should adopt in order to prevent or minimize it, and the best ways to gather, evaluate and combine different sources of knowledge. Furthermore, they are intimately connected with epistemological issues in the philosophy of science, and the foundations of statistics and probability.

Pharmacology is a particularly interesting field of investigation in these respects. Together with revolutionary successes, e.g. the discovery of penicillin, the history of pharmacology is also characterized by a series of tragic disasters (from the thalidomide to the rofecoxib case), which showcase the extreme variance of its scientific performance. Furthermore, pharmaceutical decisions are set in a complex environment where scientific uncertainty, conflicts of interests, and regulatory constraints strongly interact. The workshop intends to investigate these phenomena in light of the current methodological and philosophical debate.

This series of annual conferences is a joint undertaking between the Sydney Centre for the Foundations of Science (SCFS), the Tilburg Center for Logic and Philosophy of Science (TiLPS) and, since 2012, the MCMP. For a list of previous conferences, click here.

More information about the MuST9 Conference here

EBM+ Participants in the MuST9 conference

 [stextbox id=”info”]The Reasoner is a monthly digest highlighting exciting new research on reasoning, inference and method broadly construed. It is interdisciplinary, covering research in, e.g., philosophy, logic, AI, statistics, cognitive science, law, psychology, mathematics and the sciences.[/stextbox]

The workshop was organised by Federica Russo and took place in the Universiteitsbibliotheek of the University of Amsterdam.

Evidence evaluation is a core practice of the sciences. In medicine, specifically, the issue has been tackled by developing analytical and conceptual tools such as the evidence hierarchies. The question that lies at the heart of these methods is how to decide what is the best evidence to base our decisions for diagnosis or for treatment upon. Evidence hierarchies aim to rank evidence according to its quality. In many such hierarchies evidence obtained from randomised controlled trials (RCTs) or systematic reviews of a number of RCTs (meta-analyses) occupies top rank, while evidence obtained from expert judgment or from mechanism is relegated to the bottom of the hierarchy.

Evidence hierarchies have attracted large consensus but also criticism. One such criticism has been that these approaches leave out an important element, namely evidence of mechanisms. In this meeting, scholars affiliated to the EBM+ consortium discussed their work in progress with potential interested parties such as scientists and officers based at Academic Medical Centrum (AMC-UvA), Leiden University Medical Centrum (LUMC), and the Dutch Health Institute (ZINL). The workshop focused on how evidence of mechanism may be considered alongside statistical evidence to improve medical practice.
The workshop started with a talk by Jon Williamson (University of Kent). Jon presented the AHRC funded project “Evaluating Evidence in Medicine” and its main research questions.

He argued that there is much use for evidence of mechanism in assessing causality and that, consequently, it should be considered alongside statistical evidence. Next, Michael Wilde (University of Kent) gave a talk on “Evaluating Evidence of Mechanisms in Medicine: A Handbook for Practitioners”. This work in progress authored by various members of the “Evaluating Evidence in Medicine” project should help medical practitioners to assess the quality of evidence of mechanism and determine how to combine it with evidence from statistical trials in a systematic way.

After that Willem Jan Meerding from the Council for Health and Society gave a talk on “The Illusion of Evidence-based Practice”. He drew attention to the fact that current Evidence Based-Medicine has a too narrow focus. He emphasized the importance of considering different types of evidence like evidence of mechanism, correlations and other experience. Patrick Bossuyt (Academic Medical Center) presented joint work with Juanita Heymans (Health Care Institute) on “Collecting Evidence for Reimbursement Decisions”. Patrick recognised the importance of evidence of mechanism in medical practice. However, he also thinks that due to complexity and interdisciplinary, it will be quite challenging to treat evidence of mechanism in a systematic way.

After lunch, Veli-Pekka Parkinnen (University of Kent) presented joint work with Federica Russo (University of Amsterdam) and myself on “Scientific Disagreement and Evidential Pluralism: Lessons from the Studies on Hypercholesterolemia”. He gave an overview of the historical disagreement of cardiologists and epidemiologists on whether high cholesterol causes heart disease. Sophie van Baalen (University of Twente) talked about “Evidence and Judgment: Epistemological responsibility in clinical decision-making”. She also focused on the lower end of the evidence hierarchy—expertise—and considered the role of tacit knowledge in clinical decision-making.

Denny Borsboom’s (University of Amsterdam) talk on “Network Approaches to Psychopathology: Graphical Models, Explanatory Schemes and Statistical Inference
was quite challenging. Rather than considering psychological diseases, like depression, as latent variables, he considers them to be a bundle of symptoms that affect each other in a network. Mike Kelly (Cambridge University) on “Relational and individual conceptions of the causes of health inequalities” distinguished between individual-level and population-level explanation. He argued for the absolute importance of considering a population level cause as an entity on its own rather than just as an aggregation of individual level causes.

The workshop ended with a round table discussion moderated by Federica Russo about possible future cooperation between participants of the workshop.“The Evaluating evidence in medicine: Whence and Wither” workshop was funded by the Arts and Humanity Research Council (AHRC), the Institute for Logic, Language and Computation at the University of Amsterdam (ILLC) and the Amsterdam School for Cultural Analysis (ASCA).

Medical diagnosis aims to assess whether a given patient has a certain disease. This is done by employing disease frequencies in reference classes of individuals that show the same test results. In combining multiple test results we are confronted with the problem of conflicting reference classes.

Let T1 and T2 be possible binary test results and D be a disease. Suppose that we know for i=1, 2 the true positive rate (sensitivity) of test Ti, i.e., P(Ti|D). Suppose further that we know the false positive rate (1-specificity) of the test Ti, i.e., P(Ti| non-D). Finally, suppose that we know the disease base-rate in the population we are interested in P(D). We are interested in the probability of Dc given the individual c has been tested positive in each test.

Under the assumption that we know the conditional test-covariances,

1) P(T1,T2| D)-(P(T1|A)*P(T2| D)) and
2) P(T1, T2|non-D)-(P(T1| non-D)*P(T2|non-D)),
P(D|T1, T2) is uniquely determined.

This value can then be used to determine the probability for the individual c having the disease D. For a table of the values for each combination of test outcomes see Gardner, p. 115.

However, in many cases conditional test-covariances are not known and due to lack of relevant data their estimation is a hard problem. As Hunink et al claim p.203, “To account for conditional dependencies requires data from a group of patients among whom all the test variables are known.”

But exactly such a data set is not given. There is hope here that evidence of mechanism can often help us to estimate the test-covariances (or at least restrict them to lie in a certain interval). How this can be done will be topic of my next post. Stay tuned.


  1. Gardner, Ian A., et al. “Conditional dependence between tests affects the diagnosis and surveillance of animal diseases.” Preventive veterinary medicine45.1 (2000): 107-122.
  2. Hunink, Myriam, et al.’’ Decision making in health and medicine. Integrating evidence and values. Cambridge University Press, 2001.