Conceptual lessons for containing an epidemic of questionable COVID-19 counterfactual comparisons

The COVID-19 pandemic has provided limitless opportunities to compare pandemic policies across countries and over time. When the aim is to assess the comparative success of these policies, the comparison requires thinking counterfactually about ‘what would have been’ in some unrealised hypothetical (counterfactual) scenario. Whether generating modelling projections,1 making data-driven comparisons across countries2 or attributing excess harms,3 causal inference often rests on counterfactual comparisons, even if those comparisons are only implicit. However, in the pandemic, counterfactual analyses that are overly simplistic, uninformative or outright flawed have been an epidemic in their own right. The examples I explore here are not the worst offenders and my aim is not to criticise them but to use them to illustrate cautionary lessons. By exploring the theory of counterfactuals and common problems with their use, we can learn to do better. Slow conceptual thinking is needed even in…

Read the full article ›