Don’t undervalue epidemiological and mechanistic studies

September 29, 2020 Jon Williamson

EBM+ supports the idea of evidence synthesis and integration from all pertinent streams of evidence, and particularly an enhanced role of mechanistic studies. It stands against an exclusive focus on randomized controlled trials (RCTs) as the gold standard in evidence synthesis and integration and in causal inference, as depicted in the evidence hierarchy above.

Recently, the RCT-centred approaches originating in clinical medicine, e.g., Cochrane Collaboration and GRADE (Grading of Recommendations Assessment, Development and Evaluation), have tried to impose their clinical methodology for evidence synthesis on public health. Part of their arsenal is an algorithmic downgrading of observational epidemiology, via risk of bias (ROB) tools. These ROB tools are also used to sideline mechanistic studies, which are already downgraded by GRADE due to their “indirectness”.

A prominent controversy concerns the carcinogenicity of processed meat. Soon after WHO’s International Agency for Research on Cancer (IARC) had classified processed meat as “carcinogenic to humans”, Gordon Guyatt, a lead proponent of GRADE, stated in a commentary in the Financial Times that IARC had ignored two large trials and instead “leaned heavily on epidemiological data.” He continued, “Unless relative risks are greater than five, epidemiological studies typically provide only low-quality evidence.” However, the two large trials were uninformative with regards to processed meat and the IARC Working Group of experts concluded that the many high-quality cohort and case-control studies provided sufficient evidence for the carcinogenicity of processed meat – despite relative risks well below two.

In 2019, Guyatt and colleagues followed-up with a series of GRADE meta-analyses and recommendations concluding that the meat “studies provide low- or very-low-certainty evidence according to the GRADE criteria”. The disagreement by the world’s leading experts was immediate and strong. Some 250 experts and organisations signed a public statement by the World Cancer Research Fund “Red and processed meat still pose cancer risk”, and Harvard’s School of Public Health added that the GRADE “recommendations contradict evidence”.

But processed meat is only one prominent case study, and air pollution and lung cancer could be next. In fact, Dr Cox, chair of the US EPA Clean Air Scientific external Advisory Committee, argues that all air pollution epidemiology studies lack adequate control for confounding, and are subject to high risk for potential bias, and therefore, that observational epidemiology studies should only be used in evidence integration if they can demonstrate ‘manipulative causation’ (largely intervention studies showing a direct health benefit of changing air pollution levels).

At least since Sir Bradford Hill developed his viewpoints for assessing causality, observational epidemiology has been the cornerstone of science-informed public health policy. To raise awareness about and counter these problematic developments, concerned scientists have now founded the DE-grading Epidemiology (DEEP) network. Stay tuned!

Kurt Straif
Boston College and ISGlobal, Barcelona, Spain