In the UK, the election of a new government has seen a renewed focus on research policy and the use of evidence in policymaking. In this repost, David Rose, Mark Burgman and William Sutherland, draw on their experiecnces of working within different government departments, to consider how evidence is currently used by the civil service and to set out an agenda for how evidence and experts could be better integrated into high level decision making.
The UK government’s chief scientific adviser, Sir Patrick Vallance, recently said he was mounting a drive to recruit more scientists and engineers into the civil service. His comments echo the much-discussed call by the prime minister’s chief special adviser, Dominic Cummings, for more data scientists and other “assorted weirdos” to become civil servants.
From our experience working with and in government departments, we know that there are many problems in how the civil service uses evidence to support decision-making. And scientists (and other specialists from the social sciences and humanities), who value sound methodologies and are trained to search for robust evidence, well-designed studies and research questions, could indeed help with that.
more scientists working within current systems won’t be enough. We need a transformation in thinking that goes far beyond what degree civil service recruits studied
But more scientists working within current systems won’t be enough. We need a transformation in thinking that goes far beyond what degree civil service recruits studied. Some of this can be achieved by building on some excellent practice already underway in many parts of the civil service, for example, the increased use of social science expertise and the valuable role played by department chief scientific advisers.
But here are six further ways the civil service could transform its approach to evidence and decision-making.
The process of gathering evidence is more efficient if an issue has been anticipated, but civil servants often work reactively. Academic researchers, on the other hand, regularly carry out “horizon scans” of various issues to identify key forthcoming topics for policy and research.
For example, a recent horizon scan of conservation issues found the increasing use of traditional Asian medicine, the use of artificial wombs, and the impact of the shrinking ozone hole on Antarctic sea ice could all be important issues to consider in 2020.
All government departments could ensure they routinely look ahead by reflecting on existing horizon scans and encourage further scans in missing subject areas. There is a dedicated horizon scanning team in the civil service already, and increasing its activity could benefit all aspects of government.
Consider all the options
When decision-makers are blinkered by convention, a restricted formulation of the problem or narrow self-interest, they may overlook better policy alternatives. Horizon scanning provides a measure of protection against this myopia, but civil servants can also work with outside experts, businesses, charities and other organisations to broaden the scope of scenarios they consider.
Listening to as diverse a suite of perspectives on the problem as possible is particularly important, because research has shown that diverse thinking brings new ideas. This is why Dominic Cummings wants to recruit more “misfits and weirdos”, because a greater variety of thinking could produce more alternative policy solutions.
Collate evidence properly
There can often be problems gathering timely, suitable evidence to inform policymaking. But the civil service also has problems with its institutional culture when it comes to storing and using rigorous evidence. Based on our own secondment experience, we’ve seen that it can be difficult simply to find the evidence that exists in the ether of government IT infrastructure, even when it comes from research commissioned by the civil service. This either slows down decision-making or results in the same evidence-gathering exercises being repeated.
Things that could address this problem include routinely carrying out reviews of all relevant available evidence on a topic (known as subject-wide evidence synthesis), academic partnerships to fill critical knowledge gaps, and simply summarising evidence from previously commissioned research and evidence-gathering exercises. Producing clearly indexed summary paragraphs in accessible language (as done by the Conservation Evidence journal) would make it easier for new staff to find evidence and build institutional memory of previous research, even when staff move on to new jobs.
Evaluate evidence quality
Not all evidence is created equal. Research shows that how a study is designed considerably affects the accuracy of its results. What’s deemed “good” evidence can often be questionable. And “scientific” evidence isn’t necessarily more valuable than other forms, such as experience.
Civil servants may not be skilled at assessing these kinds of issues, which can affect the quality of evidence they use. So more need to be given an understanding of the basic elements of interpreting evidence. This includes bias, chance, pseudoreplication (repeating tests with the same subjects to bulk out your data), data dredging (selecting data that isn’t typical of the overall pattern) and regression to the mean (the fact that atypical values are likely to be followed by less extreme ones).
Use experts properly
Experts can be repositories of a wealth of useful information and are regularly used by government departments. But scientists’ views can be just as influenced by bias as anyone else’s, and this can affect how they collect and interpret data. In fact, research shows that if experts are used to estimate facts or to judge the outcomes of future events, the most highly regarded scientist is rarely any better than a relatively recent graduate.
Instead of relying on highly regarded individuals and the culture of scientific authority that goes with them, civil servants should seek a diverse set of independent estimates. They should collate the data with appropriate statistical procedures and form an overall scientific judgement. To make sure civil servants get their interactions with experts right, they should use skilled in-house science brokers.
Adopt effective decision-making techniques
Transparent decision-making and reducing bias are hallmarks of evidence-based policymaking. Yet a report published in 2018 found that, despite some improvements in transparency, many government departments did not routinely publish the evidence underlying new policies. It is often unclear how decisions are made.
A step change is needed to make the most of proven group decision-making processes designed to limit bias and increase transparency, such as the Delphi technique. And this is one area where civil servants with scientific training really could make a difference, by helping separate facts and values, and orchestrating the discussion of trade-offs between policy options.
These changes wouldn’t be hard to introduce and many civil servants would agree they would help. But they also complain that there isn’t the time to make the changes. Instead of looking for a special fix, the civil service needs to overcome its institutional inertia and make these changes happen.
Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.