The public image of the response to COVID-19 has been presented primarily through a scientific lens. However, as Alana Couvrette argues, the COVID-19 pandemic has been as intense a period of socio-political experimentation, as scientific. Drawing on examples of experimentation in a range of policy areas, she suggests the findings of these experiments are not only already influencing the emerging post-pandemic world, but potentially social attitudes to policy experimentation itself.
What is the first thing that comes to mind were you to think about the contributions of experiments in addressing the COVID-19 pandemic? What imagery crosses your mind?
Did you, by any chance, think of the work of scientists? Did your mind wander to one donned in a white coat either looking intently through a microscope, carefully dispensing the contents of a pipette into some sort of vial or looking quizzically at a beaker held up to the light? Surely, the development of vaccines that protect against coronavirus received the title of the most important contribution of experiments in responding to the pandemic.
If you arrived at this conclusion, you may not be wrong. After all, what could be more important than the availability of vaccines, signalling the beginning of the end of human suffering from a virus that has so far claimed the lives of over 3 million people. Assigning value to experiments is ultimately a matter of perspective. However, in terms of their impact on our day-to-day lives, experiments conducted outside a laboratory setting have not only played a key role in helping us through the pandemic, but are also shaping the emerging post-pandemic world.
in terms of their impact on our day-to-day lives, experiments conducted outside a laboratory setting have not only played a key role in helping us through the pandemic, but are also shaping the emerging post-pandemic world
Taking a step back: what do we mean by experiments conducted outside a laboratory setting? In a public policy setting, experimentation “is the deliberate use of methods that seek to explore, test, and compare the effects and impacts of policies, interventions and approaches, in order to inform and support decision-making”. As its definition suggests, policy experimentation can be brought to life through a variety of different methodologies, with randomized experiments like randomized controlled trials (RCTs) being one of the most popular forms.
In the field of medical science, randomized experiments have long been considered the ‘gold standard’ for evaluating medical interventions and medicines. This is in large part due to their ability to establish a causal relationship between an intervention and an outcome in a way that minimizes bias inherent with other research designs. The use of randomized experiments is not a new phenomenon, but what is relatively new is their wider application in business – as a way to better evaluate products and services – as well as in the public sector, as a method to assess the effectiveness of government programs and services (known as testing ‘what works’).
So then, how have policy experiments helped to address the COVID-19 pandemic?
Starting with their most notable role, policy experiments have helped to shape a critically important variable in reducing virus transmission: public behaviour. Around the world, governments have relied on randomized experiments to test the effectiveness of their communication campaigns in increasing people’s compliance with public health measures (such as wearing a mask, practicing frequent hand washing, maintaining social distance, etc.) and, in some cases, their willingness to get vaccinated.
For example, the governments of Canada, the United States and the United Kingdom all conducted randomized experiments to test the impact of behaviourally-informed communication materials in encouraging adherence to key COVID-19 prevention behaviours. There’s also evidence of certain governments using the results of past experiments seeking to increase the uptake of other recommended vaccinations to inform COVID-19 vaccination policy. In a time where both individual and collective actions have tremendous consequences on the health of a Nation, it is no surprise that many other governments like Ireland, Australia, Italy, Japan have also recognized policy experimentation as a best practice in situations of crisis management.
Example of behaviourally-informed material developed by the Government of Canada to encourage hand washing.
In a year marked by loss, policy experiments have also worked to improve societal well-being in other ways. Take for example the increase in levels of loneliness and social isolation in older adults compounded by the pandemic. A team of researchers sought to address this challenge through a randomized experiment, which demonstrated that a short-term digital group intervention was effective in improving seniors’ coping abilities and reducing loneliness and depressive symptoms. Another group of researchers worked to mitigate a hidden cost to the pandemic; the stigma associated with the disease for those infected and groups considered as more at risk of infection. Through a randomized experiment they showed that the provision of accurate information from a reliable source helped to reduce stigmatization. In a more light-hearted manner and in an effort to look for ways to party safely despite the coronavirus, government, academia and industry teamed up for ‘RESTART-19’, a study in which ordinary citizens became laboratory subjects. The experiment, which involved over a thousand people participating in a pop-up concert, provided useful insights into how COVID-19 spreads in large-scale events and which health and safety protocols had the greatest impact on reducing virus transmission.
a cultural shift towards experimentation more widely across our societies may even prove to be one of the most lasting aftereffects of the pandemic
Finally, policy experiments have also contributed to forming a vision of a post-pandemic world. Looking to municipalities, in an effort to compensate for the curtailing of public transportation operations, many have turned to increasing urban cycling networks. This rapid implementation of new cycling infrastructure created a quasi-experimental setting that some researchers have been quick to study and offer us a glimpse into how we might rethink city planning to embrace cycling as a core component of urban transportation systems. Besides helping to envision the future, randomized experiments are also being called upon to build it. In the wake of significant disruptions in the ways that we work, COVID-19 provided an impetus to reconsider the idea of a four-day work week, with some countries even pledging government funds to test the idea using a RCT.
Although the practice of policy experimentation during the COVID-19 pandemic may not have been apparent to many, its contributions are undeniable. Yet, despite this, research has shown that people often disapprove of randomized experiments and, in a rather alarming fashion, prefer the universal implementation of untested policies. Moving forward, researchers will have to continue to lead the charge in raising awareness of the risks and ethical implications of implementing untested interventions (whether they be pandemic related or not), challenge the irrational biases that sustain our experimentation aversion and normalize the use of randomized policy experiments in evidence-based practice. In the long-term, a cultural shift towards experimentation more widely across our societies may even prove to be one of the most lasting aftereffects of the pandemic.
Note: This article gives the views of the author and do not necessarily reflect the official policy or position of the author’s employer, the Government of Canada, the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.