The EU referendum campaign and aftermath brought to the fore the ongoing debate about evidence and evidence-based policymaking (EBPM), write Jo Ingold and Mark Monaghan. While Michael Gove’s suggestion that “people in this country have had enough of experts” could be seen as a rebuttal to criticisms of the Leave campaign’s misuse of statistics, it also draws attention not only to what counts as evidence but also to the perceived gap between ‘the academy’, policymakers and the public.
It’s fair to say that research has never monopolised the policy process to the extent that policies are formulated solely, or even primarily, upon evidence. At the same time, research is never entirely absent from the process nor is it always exploited to justify a pre-existing policy stance as those who pronounce that we that we are now in an era of policy based evidence would have us believe. Often the reality lies somewhere in the middle. A number of studies have looked at how evidence may or may not have impacted on the policy decision-making process. Learning from other contexts, or ‘policy transfer’ is one other way of harnessing particular kinds of evidence, focusing on the migration of policies from one jurisdiction to another, whether within or across countries. Studies have begun to move away from theories of direct transfer to consider the processes involved in movement of ideas from one area to another. In effect, they consider the ‘translation’ of evidence and policy.
Our research brings together the evidence-based policymaking and ‘policy as translation’ literatures to try to shed light on the process by which evidence is used in policymaking. Although these literatures have developed separately (and to a large extent remain so) we see both as, at root, being concerned with the same issues, in particular how ideas, evidence and knowledge are integrated into the policymaking process. With EBPM there is a stated desire to formulate policies based on the best available evidence, while ‘policy as translation’ focuses on the ‘travel of ideas’ and views the policy process as fluid, dynamic and continually re-constituting, rather than a linear or rational ‘transfer’ process.
The Department for Work and Pensions case study
In our article we examine how evidence, ideas and knowledge are used (or not used) in the policy process. We draw on a series of focus groups with 75 analysts and policy advisers in the Department for Work and Pensions (DWP). The DWP seemed a good place to explore this issue – under New Labour it saw a huge increase in the number of evaluations and is currently the home of ‘flagship’ (and much-critiqued) government policies such as Universal Credit and the Work Programme. It has also been criticised by the UK Statistics Authority, the Work and Pensions Select Committee and others for its ‘misuse’ of statistics.
In the focus groups we tested with DWP officials the usefulness of a number of models, beginning with ‘linear’ models of EBPM and policy transfer and then exploring the ‘processual’ model of EBPM, which is a synthesis of models of research utilisation and EBPM, with ‘evidence use’ occurring in the process of policy formulation and analysis (Stevens, 2007; Monaghan 2011). For the first time we also modelled the concept of ‘policy translation’ and, following discussion with officials present, developed this into our conceptual model of ‘evidence translation’.
The Evidence Translation Model is intended to be recursive and includes five key dimensions which influence how evidence, ideas and knowledge are used in policy:
- The substantive nature of the policy problem in the context of the Zeitgeist
- Agenda-setting – where evidence is sought (fishing/farming) and what evidence is used
- The filtration processes which shape and mould how evidence is used (flak/strain)
- The policy apparatus for policy design and implementation
- The role of ‘evidence translators’
Evidence Translation Model
Source: Policy & Politics 2016 (44:2)
Our research draws attention to what is perceived to be evidence and at what stage of the policymaking process it is used. A surprising finding was that methodology can serve as an evidence selection mechanism, not specifically in terms of an evidence ‘hierarchy’ (with random controlled trials being the ‘gold standard’) but in relation to how particular methodologies are perceived and understood by key decision-makers. Central to the evidence translation process is the role, skill, knowledge and will of actors as ‘evidence translators’. We argue that agency is crucial at every stage of the recursive evidence translation process.
The model is intended to contribute to an ongoing debate about the mechanisms that impact on the use of evidence, ideas and knowledge in policymaking. How this process can be better facilitated requires further exploration and testing both within (and across) academic and policy communities.
Please note: this blog draws from the authors’ article in Policy & Politics. The project on which the research was based was funded by the Higher Education Innovation Fund.
About the Authors
Jo Ingold is a Lecturer in Human Resource Management and Public Policy at Leeds University Business School.
Mark Monaghan is a Lecturer in Criminology and Social Policy at Loughborough University.
I intended to speak about two provisions which were introduced as part of the Tax Equity as well as Fiscal Obligation Act of 1982 (TEFRA).