The Institute for Government has just published three new reports on the state of policy making in the UK government. They show concerns about the lack of knowledge in Whitehall, the lack of connection to new ideas and about the way in which evaluations are used. These all represent a big opportunity for academics who want to increase their impact, argues Jill Rutter, but they will need to change to meet policy makers’ needs if they are to fill those gaps.

What we found

Our research focussed on interviews with fifty senior civil servants and twenty former ministers, as well as an analysis of over 60 evaluations of government policy. We found that despite consistent attempts to improve policy making over the past fourteen years, both ministers and civil servants were still frustrated at the policy making process. There were two particular areas of interest to the academic community.

We found that both agreed that one persistent area of weakness was on the government’s use of evaluations.  This was rated the lowest on the “qualities” of policy making that the Cabinet Office promulgated in 1999 as part of its Modernising Government white paper by both ministers and civil servants.  People agreed that these were often commissioned – but much less often used and learned from – for a variety of reasons: that the department promoting the policy was also the commissioner of evaluation, so it lacked real independence; that by the time the results of evaluations were available, the policy had already been superseded and the minister who had originally promoted it had moved on. We concluded that the culture of policy making was to be not very interested in learning from past experience.

We also found concerns from ministers that civil servants were often not expert on their subjects, and were not up to date with the latest thinking – whether in academia or in think tanks – in their fields.  Moreover, within the civil service there was concern both that knowledge was not valued sufficiently – and that this would get worse as many experienced people leave Whitehall as administrative budgets shrink.

Making policy better

Our report makes a number of recommendations to ground future policy making in the system within which policy is made – to create a resilient process that is biased in favour of better policy making, rather than let it fall victim to the demands of political expediency. In this, the academic community has a potentially important role to play.

• Recognising the fundamentals

First, we recommend that all policy making is based in a set of seven policy fundamentals. One of these is “open idea generation which also takes account of evaluations of past policy”. Taking this seriously means tapping into ideas emerging from the research community as well as understanding the implications of evaluations. Another fundamental is “rigorous policy design” – which means really understanding how the policy will play out in practice. Many past policies have failed at this stage because policy makers did not understand the characteristics of incomes at the lower end of the income distribution, in the case of tax credits; or the behavioural reactions of people faced with new choices, which are why, for example, licensing law relaxation failed to deliver the expected change in drinking culture. Again, academia has huge potential to contribute understanding here and help policy makers design policies that are more likely to work in practice.

• More rigorous and independent evaluations

We also make two specific recommendations to improve evaluations and to improve knowledge and expertise. We think there needs to be more independence and rigour in evaluations. They are too important to be left to be commissioned and used by the department which leads on the policy.  In our system that stacks the political incentives in favour of bland evaluations with inconvenient results massaged. So we want to give oversight of evaluations to a new Head of Policy Effectiveness in the Cabinet Office – a very senior official, tasked with oversight of the quality of policy making in government. They would also have an independent budget to fund evaluations proposed through an open bidding process – and be able to commission lessons learned exercises in cases of failure.  A key part of their role would be to make sure that policy makers learnt from past experience.

• Accessing knowledge and expertise

But we also want to connect policy making in better to current thinking. The civil service cannot and should not monopolise thinking on policy – but department both need to value internal knowledge and expertise more, but also be well connected into external thinking. We suggest departments should operate a “one degree of separation” rule – so that where they do not have expertise in house, they have the right connections to access it immediately. They should make more use of standing contracts with individual experts or departments, so they can be brought in quickly when their advice is needed, and ministers ought to be able to commission outsiders to challenge civil service advice.

But it is not just policy makers who need to change

If these recommendations are adopted they will build on existing efforts to improve the evidence base for policy making, while making it more likely to withstand real world pressures. They should increase the demand for research know-how and knowledge. But supply – the academic community –  will need to change too. In particular, this means being ready to respond to opportunities to engage with the issues that are preoccupying policy makers. And it will mean more actively cultivating networks in the civil service, political parties and think tanks to increase their awareness and help shape thinking at critical times.

Jill Rutter is Programme Director at the Institute for Government. She joined the Institute as a Whitehall secondee in September 2009 and was co-author of the Institute’s report on arm’s length bodies, Read Before Burning (July 2010). She has also been part of the better policy making project, co-authoring two reports Policy making in the real world and Making Policy Better (both April 2011). Before joining the Institute for Government, Jill was Director of Strategy and Sustainable Development at Defra.