LSE - Small Logo
LSE - Small Logo

Linda Hantrais

Susanne MacGregor

April 29th, 2021

Containing COVID, part 2 | The problem of unreliable and incompatible evidence

1 comment | 4 shares

Estimated reading time: 4 minutes

Linda Hantrais

Susanne MacGregor

April 29th, 2021

Containing COVID, part 2 | The problem of unreliable and incompatible evidence

1 comment | 4 shares

Estimated reading time: 4 minutes

Calculating and comparing COVID-19 death rates is not a simple matter. Linda Hantrais (LSE) and Susanne MacGregor (LSHTM) explain why, and how these complex processes enabled public figures, journalists and social media users to spread misleading information.

Read the first post in the series

Statisticians became aware early in the pandemic that issues with data collection and reporting methods would make trends in published rates of infections and deaths difficult to interpret, compare and explain. They were quick to point out that raw numbers are not reliable indicators for comparing even absolute levels of infections and deaths across countries. Their warnings went largely unheeded, as the media and political opponents continued to use headline figures to contrast performances across countries, and to blame government leaders for their relative failures to contain the spread of the virus.

According to the World Health Organisation’s definition, issued on 16 April 2020, a COVID-19 death results ‘from a clinically compatible illness, in a probable or confirmed COVID case, unless there is a clear alternative cause of death that cannot be related to COVID disease’. In the same document, WHO advised that, on medical certificates, cause of death should be notified as COVID ‘for ALL decedents where the disease caused, or is assumed to have caused, or contributed to death’, implying in the last case that the disease may have accelerated death from other causes.

Despite WHO recommendations, national datasets used to track the spread of the virus and its lethality remained inconsistent between countries. National registrations of COVID deaths also varied over time within countries according to whether WHO definitions were applied and how they were adapted in different settings − hospitals, care homes or the community − and for national or local monitoring purposes. The most accurate records were considered to be those based on a clinical judgement in a hospital setting confirmed by a positive test as the underlying or probable cause of death, and collated at regional or national level. In the absence of a reliable positive test and other known underlying conditions, certification referred to an expanding list of recognised symptoms.

Miscounting could be attributed to missing data, incompatibility of data from different sources as well as collating and processing delays when laboratory test confirmation was awaited. Under-reporting arose, particularly in the early stages of the pandemic, if official data included only deaths in hospitals confirmed by laboratory tests. When deaths in care homes and the community were added, as in France on 4 April 2020, spikes were recorded in the cumulative statistics. Fluctuations in daily rates could also be attributed to delays in reporting deaths occurring at weekends until the following week.

National datasets used to track the spread of the virus and its lethality remained inconsistent between countries

Conversely, one of the reasons evoked to explain the persistently high death rate in Belgium was that their statistics included all deaths where COVID was the ‘probable’ cause. Over-counting could also result from revisions to definitions. In August 2020, Public Health England controversially changed its definition of deaths to bring it into line with practices in other regions in the UK and elsewhere. Application of the new definition was calculated to have reduced the total count by over 5,000 deaths. But, as noted by Worldometer, no downward adjustment could be made retrospectively to the ‘cumulative count’ in international tables.

In addition, testing was far from being routine worldwide and was known to return false negatives and positives, which explained why the virus could be variously mentioned on a death certificate as the probable, presumptive, suspected or confirmed cause of death. Some countries were testing only symptomatic individuals, whereas others tested asymptomatic cases believed to be at high risk of being infected and infectious. Statistics for the number of mild cases in the community were most prone to under-counting. In countries with a high positive rate of infections, the number of confirmed cases was, therefore, likely to represent only a small fraction of the real number. Discrepancies in reporting practices made comparisons unreliable, not only between countries and regions within them, but also over time as testing became more widely available. A relatively high incidence of cases and deaths per million inhabitants might reflect the efficacy of a testing strategy rather than the true incidence of the disease and its lethality. Observations such as these undermined the value of comparing case fatality rates (the proportion of COVID deaths, compared to the total number of cases diagnosed).

To overcome the seemingly intractable problems of accurately monitoring death rates, public health experts advocated the calculation of all-cause, age-adjusted excess mortality directly or indirectly attributable to the virus. Where available, early evidence, by June 2020, indicated close to a two-fold increase in excess mortality in the countries worst affected by the pandemic. Over a longer period, rates tended to level off, although other collateral damage was reported due to delays in treatment of otherwise preventable or curable diseases. Estimates of the impact on the quality of life demonstrate that significant numbers of years of life were being lost among people who did not die from the disease.

In addition to the lessons that can be drawn about the caution needed in understanding, interpreting and using different data sources to produce reliable evidence with which to inform policy, governments had to consider how best to prevent, contain and counter the spread of false or misleading information. Misinformation came not only from viral posts on social media, but also from comments made by public figures, and statements printed or broadcast by journalists.

This post represents the views of the authors and not those of the COVID-19 blog, nor LSE.

About the author

Linda Hantrais

Linda Hantrais is Emeritus Professor of European Social Policy in the Department of Politics and International Studies at Loughborough University, UK, a Visiting Professor at the LSE, and Chair of the UK Academy of Social Sciences’ International Advisory Group. This blog draws heavily on her publication with Marie-Thérèse Letablier of Comparing and Contrasting the Impact of the COVID-19 Pandemic in the European Union (Routledge, 2021).

Susanne MacGregor

Susanne MacGregor is Honorary Professor of Social Policy at the London School of Hygiene and Tropical Medicine and Emeritus Professor of Sociology and Social Policy at Middlesex University London. Her recent publications include: The Politics of Drugs: Perceptions, Power and Policies (Macmillan, 2017).

Posted In: #LSEThinks | Governance and state capacity | Health and social care

1 Comments