In this article, Robert Metcalfe argues that field experiments can be instrumental in helping policymakers understand how to improve the welfare of their citizens. Field experiments represent a relatively new methodological approach capable of measuring the causal links between variables, thereby allowing policymakers to understand the behavioural responses of their citizens to changes in policies.
Do neighborhoods matter to outcomes? Which classroom interventions improve educational attainment? How should we raise money to provide important and valued public goods? Do energy prices affect energy demand? How can we motivate people to become healthier, greener, and more cooperative? These are some of the most challenging questions policymakers face. Academics have been trying to understand and uncover these important relationships for decades.
Many of the empirical tools available to economists to answer these questions do not allow causal relationships to be detected. Field experiments represent a relatively new methodological approach capable of measuring the causal links between variables. By overlaying carefully designed experimental treatments on real people performing tasks common to their daily lives, economists are able to answer interesting and policy-relevant questions that were previously intractable. Manipulation of market environments allows these economists to uncover the hidden motivations behind economic behaviour more generally. A central tenet of field experiments in the policy world is that governments should understand the actual behavioural responses of their citizens to changes in policies or interventions.
Field experiments represent a departure from laboratory experiments. Traditionally, laboratory experiments create experimental settings with tight control over the decision environment of undergraduate students. While these studies also allow researchers to make causal statements, policymakers are often concerned subjects in these experiments may behave differently in settings where they know they are being observed or when they are permitted to sort out of the market.
For example, you might expect a college student to contribute more to charity when she is scrutinized in a professor’s lab than when she can avoid the ask altogether. Field experiments allow researchers to make these causal statements in a setting that is more generalizable to the behaviour policymakers are directly interested in.
To date, policymakers traditionally gather relevant information and data by using focus groups, qualitative evidence, or observational data without a way to identify causal mechanisms. It is quite easy to elicit people’s intentions about how they behave with respect to a new policy or intervention, but there is increasing evidence that people’s intentions are a poor guide to predicting their behaviour.
However, we are starting to see a small change in how governments seek to answer pertinent questions. For instance, the UK tax office (Her Majesty’s Revenue and Customs) now uses field experiments across some of its services to improve the efficacy of scarce taxpayers money. In the US, there are movements toward gathering more evidence from field experiments.
In the corporate world, experimenting is not new. Many of the current large online companies—such as Amazon, Facebook, Google, and Microsoft—are constantly using field experiments matched with big data to improve their products and deliver better services to their customers. More and more companies will use field experiments over time to help them better set prices, tailor advertising, provide a better customer journey to increase welfare, and employ more productive workers.
Governments deliver many important services to their populations, and these services have implicit costs, advertising, information, and education. Not knowing how these four factors impact on the behaviour of their citizens means that governments are potentially misallocating taxpayers’ money. Using field experiments helps policymakers understand how to causally improve the welfare of their citizens.
This article originally appeared at the OUPblog.
Please read our comments policy before commenting.
Note: This article gives the views of the author, and not the position of USApp– American Politics and Policy, nor of the London School of Economics.
Shortened URL for this post: http://bit.ly/1JymVJZ
______________________
Robert Metcalfe – University of Chicago
Robert Metcalfe is a Postdoctoral Research Scholar in Economics at the University of Chicago. He is co-author, with John List, of the paper Field Experiments in the Developed World: An Introduction, recently published in the Oxford Review of Economic Policy.
