Should policy makers trust composite indices? A commentary on the pitfalls of inappropriate indices for policy formation

This paper critically discusses the use and merits of global indices, in particular, the Global Health Security Index or GHSI (Cameron et 2019). The index ranked 195 countries according to their expected preparedness in case of a pandemic or other biological threat. The Covid-19 pandemic provides the background to compare each country's predicted performance from the GHSI with the actual performance. In general, there is an inverted relation between predicted versus actual performance, i.e. the predicted top performers are among those that are the worst hit. Obviously, this reflects poorly on the potential policy uses of the index. This paper analyses the reasons for the poor match between prediction and reality in the index, and mentions six general observations applying to global indices in this respect. The level of abstraction in these global indices builds uncertainties upon uncertainties which potentially removes them from the policy needs on the ground. From this, the question is raised if the policy community might have better tools for decision making. On the basis of data from the INGSA Policy-making Tracker, some simple heuristics are suggested, which may be more useful than a global index.