James Lloyd’s recent post “Should academics be expected to change policy? Six reasons why it is unrealistic for research to drive policy change” has received considerable attention in research and policy circles since it was first published two weeks ago. Drawing on their respective experiences with research impact in policymaking, Chris Neff, Paul Smyth and Luke Craven each offer responses to Lloyd’s post. While they each recognise the difficulties inherent in the process, setting up impact as impossible or unrealistic fails to recognise the responsibility academics have to engage. Furthermore, fatalistic discourse devalues the engaged or applied work being done by academics the world over.
Read the original piece here.
Chris Neff, a Lecturer in Public Policy from the University of Sydney, offers his thoughts on why impact is indeed desirable as a public service the university must offer to broader society. He also takes aim at three of Lloyd’s ‘six reasons’ arguing that this line of thinking itself contributes to an environment that stifles impact.
I would note that I am against the corporatisation of academia and requiring research prioritised based on its marketability. My position on whether academics should be expected to change public policy is not about new standards of “impact.” It is more about the responsibility that academics have to the public. We are a privileged few. And coming up with a list of reasons that we can’t expect more of ourselves when the public needs our help is beneath our abilities and our character. As a result, I disagree with James Lloyd’s “Six reasons why it is unrealistic for research to drive policy change.” I will note just three specific criticisms.
First, I do not accept the premise that “some research has no policy relevance.” For something to have no relevance at all is quite a bold statement and frankly, I’m unaware of any research portfolio with zero policy relevance.
Secondly, Lloyd’s statement that “much research supports the status quo” is not quite right. The policy analysis literature notes that there are hierarchies of power that support maintaining the status quo, but that is different from saying the research supports existing policies. One need only look at climate change, austerity measures, institutional inequality, poverty, and education to see policies in dire need of change.
Lastly, Lloyd’s point that “politics almost always trumps evidence” is a reason for more academic-engagement not less. I was a federal lobbyist in the US so I can speak to this directly. Most staff and members of Congress or Parliament would love to have more data, more evidence and more engagement from academia. There is a policy black hole being filled by lobbyists and think tanks that should include academics of every discipline. Policy change is not hard unless we conceive of it in impossible terms. We should expect more, not less of our profession and the bottom line is that people need our help.
Paul Smyth is more accommodating to Lloyd’s characterisation of the interaction between academics and policymakers. All is not, lost, though, he argues. There are many examples of productive collaborations between researchers and policymakers from which we can learn about how best to foster these relationships.
My former joint role with the University of Melbourne and the Brotherhood of St Laurence taught me that while there is no stimulus to research quite like seeing it have impact, significant challenges face those wanting to bridge the two very different worlds of research and action.
As Lloyd’s article concludes, the vocations of policy researcher and policy entrepreneur are really so different I can only chuckle at the image of my erudite university colleagues lost in the corridors of power in the hope of an impact event they might be able to record for their managerial tormentors back in the ivory tower. Tick here if you have written your ‘non technical summary’; and there, if you are ‘logged in a searchable depository’ and so forth.
At the Brotherhood, the two roles have quite different job descriptions and it has been a perennial challenge to get the two collaborating productively. But getting it right is magic. How? In addition to Lloyd’s three pointers to the future I would add working with the policy actors to ensure an environment receptive to researchers. A shared commitment to raising the level of policy knowledge will galvanise researchers far more than anything else.
Luke Craven is most concerned by the negative tone of the James Lloyd’s original piece and argues policy impact is only unrealistic if we, the academic community, are unwilling to adapt to the realities of the policymaking space. Though the piece provides some helpful suggestions for a way forward, it is likely to entrench the belief that academics can’t or shouldn’t attempt to influence policy. The fatalistic and defeatist discourse requires interrogation.
Lloyd’s basic claim is that it is neither realistic nor desirable to expect academics to achieve policy impact. Bold, but should we take his position as correct? Over the course of the week, a whopping fourteen academic colleagues have sent me the post– mostly, just as a link, without comment, as though Lloyd’s position is somehow conclusive or incontrovertible. While, of course, the post contains some productive suggestions, its tone is undeniably fatalistic. The possibility of genuine impact, it suggests, should be put right in the centre of the ‘too hard’ basket.
Already, many academics seem to be using the piece as a shield against calls that they should engage with the policy process. That, and its wide circulation, is deeply concerning. While Lloyd’s intention may well be to offer useful suggestions, the way that the blog is being used contributes to the very problem he seeks to solve. Impact is only “undesirable” if we make it so. Impact is only “unrealistic” if we are unwilling to adapt to the realities of the policymaking space. Lloyd’s position, perhaps unwittingly, is likely to entrench the belief that academics can’t or shouldn’t attempt to influence policy.
I concur with Paul that a policy environment that is receptive to researchers is key. The same is true of the inverse: researchers need to be receptive to the demands and realities of the policymaking process. But, none of that precludes impact. The risk with accepting Lloyd’s position as gospel – and framing the discussion in terms of ‘limits’ – is that we push those receptive possibilities further away.
As Chris, Paul and I have noted, the likely result of Lloyd’s piece is less – not more – impact. The irony is that Lloyd’s own “research” seems to “support the status quo”; that is, environments that are hostile to genuine influence and impact. As Chris noted, we must be attentive to the power relationships that maintain our day-to-day realities. That is of true of the research-policy divide as it is of climate change. The fatalistic and defeatist discourse that underpins the “’too hard’ basket” argument requires interrogation. But that is a story for another day.
More than that, though, to claim that “it is neither realistic or desirable to expect academics to achieve policy impact” devalues the engaged or applied work being done by academics the world over, including many of us here at Power to Persuade. By no means is it without its challenges. We need to be better. But, even now, is undoubtedly of benefit to us as academics, to our colleagues in bureaucracy’s long (and, apparently, impenetrable) corridors, and the broader publics that we both hope to serve.
These responses originally appeared on the Power to Persuade blog and are reposted with permission.
Note: This article gives the views of the author, and not the position of the LSE Impact blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Paul Smyth is an Honorary Professorial Fellow in social policy in the School of Social and Political Sciences at the University of Melbourne. From 2004 to 2013 this professorial role was held jointly with the Australian welfare NGO, the Brotherhood of St Laurence, where he was also the General Manager of its Research and Policy Centre.
Christopher Neff is a Lecturer in Public Policy in the Department of Government and International Relations at the University of Sydney. His research interests include theories of the policy process, policy analysis, the role of policy entrepreneurs, and comparative public policy. More specifically, his research looks at policymaking regarding emotional issues such as LGBTQI politics, mass shootings, and the “politics of shark attacks.”
Luke Craven is a PhD student at the University of Sydney and the Sydney Environment Institute. His interests lie in the application of social and political theory to contemporary policy problems, with a focus on food politics, policy, and system reform. He holds a Bachelor of Arts (Honours) from the University of Sydney, where he won the University Medal for this thesis on migration and social vulnerability in Vanuatu.
Hi Chris, Paul and Luke,
Thanks for taking an interest in my piece, all the way over in Australia!
Let me respond to some of your key points in turn, starting with Chris.
* “coming up with a list of reasons that we can’t expect more of ourselves when the public needs our help is beneath our abilities and our character.”
This misconstrues the argument of the piece severely. The list of points relates to whether it is realistic to expect UK social science researchers to achieve measurable policy change. We can and should expect academics to seek to inform policymaking to the best of their ability, and the piece welcomes the progress of the impact agenda in this regard. But expecting academics to achieve measurable policy change is unrealistic, just as it would be for other professionals in other sectors. It sets the bar very high – indeed, higher than is set for public affairs professionals who work full time to change policy for their employers or clients.
* “I do not accept the premise that “some research has no policy relevance.”…”
There is lots of very good UK social science research that has limited relevance to UK policymaking, for example, theoretical research and research of other countries with only limited read-over to the UK. So, I don’t think this is a bold statement. More widely, the relevance of research to contemporary policymaking and policy debates is inevitably a function of the policy cycle. If you are UK social scientist doing research in a ‘dormant’ policy agenda that the government is not seeking to reform or is not the subject of campaigning by policy stakeholders such as charities, you may find it very hard to achieve policy change using your research. For example, you might have to wait until a change of government in the hope that a different set of politicians takes more of an interest in your area.
* “…statement that “much research supports the status quo” is not quite right…”
There are lots of fields of public policy where we might want to see policy change. However, even where a piece of research has fully found its way on to the policymaker’s desk, it may not result in policy change, but in fact, support the status quo in policy design.
Imagine a study that revealed the UK was in the top 25% of countries for children’s eye-health. The study arguably shows that the UK could and should do better. But, policymakers may look at the cost-effectiveness of the policy interventions required to put the UK in the top 5% of countries for children’s eye-health, and conclude these do not represent good value-for-money, relative to investing more in tackling childhood obesity and children’s dental health, where spending more will yield greater improvements in overall children’s wellbeing, health, and QALYs. In this situation, the study on children’s eye-health has supported and reinforced the policy status quo on children’s eye-health in the UK – it has not achieved policy change, even though it has been incredibly useful – and impactful – for policymakers in evaluating the UK’s children’s public health policy choices.
* “point that “politics almost always trumps evidence” is a reason for more academic-engagement not less.”
I agree. It is a reason to ensure that research is accessible, non-technical summaries are available, and for academics to ensure their research can be used far and wide. But, it does also mean that it is not necessarily realistic to expect an academic to achieve policy change if their research findings point to policy changes that are politically unfeasible.
Moving on to Luke’s contribution:
* Lloyd’s basic claim is that it is neither realistic nor desirable to expect academics to achieve policy impact.
No, this summary of the piece is completely wrong. The basic claim is that it is not realistic to expect UK social scientists to achieve policy change, i.e. discernible measurable changes in policy and practice. The piece clearly welcomes the progress of the impact agenda and supports the principle of evidence-based policymaking.
* “The possibility of genuine impact, it suggests, should be put right in the centre of the ‘too hard’ basket.”
This misconstrues the argument completely. The piece is not about impact, but measurable changes in public policy design and practice which, as the piece notes, are different to impact. Research can have lots of impact, but not result in policy change – as the ‘example’ above seeks to show.
* Already, many academics seem to be using the piece as a shield against calls that they should engage with the policy process.
This is a sweeping statement, and contradicts my experience of the piece receiving great feedback from academics and civil servants who recognize the issues it highlights around academics achieving measurable policy change, whether it is realistic to expect academics to achieve policy change, and the concluding suggestions for next steps in the policy impact agenda.
* Impact is only “undesirable” if we make it so. Impact is only “unrealistic” if we are unwilling to adapt to the realities of the policymaking space.”
Again, the piece is about whether we should expect UK social scientists to achieve measurable changes in policy design. It does not argue that impact is undesirable or unrealistic, and indeed supports evidence-based policymaking and the progress of the impact agenda.
* “a policy environment that is receptive to researchers is key.”
As it happens, the UK policy environment is arguably very receptive to UK social science research and researchers. In my experience, the key barriers are the fact that so much policymaking is concentrated in London, so much UK social science research sits behind journal pay-walls and is inaccessible to civil servants and politicians, and the limited time available to policymakers and civil servants to absorb research, analyse it, and incorporate it into policy decision making. But that is a topic for another blog!
* The risk with accepting Lloyd’s position as gospel – and framing the discussion in terms of ‘limits’ – is that we push those receptive possibilities further away.
As my piece concludes with positive pointers to how we can improve the policy impact of research building on the observation of the difficulties highlighted, I don’t think it is pushing possibilities further away, but setting out the issues in expecting academics to achieve policy change, and potential next steps for the impact agenda in improving the impact of UK social science research.
* “…to claim that “it is neither realistic or desirable to expect academics to achieve policy impact”…”
This is a made-up quote that doesn’t appear in the piece, and completely misrepresents the argument, which is about whether academics can be expected to achieve measurable policy change, not whether they should seek to achieve policy impact. As mentioned, the piece welcomes the progress of the impact agenda, and the principle of evidence-based policymaking.
In conclusion, it’s great that you’ve taken an interest in the piece, but I think some of the points in the original may have been lost in translation. Interestingly, the feedback from people working in public policy is that the original piece states the obvious: that it is not realistic to expect academics to achieve policy change – for the reasons described. In a way, your interpretation is fascinating and illuminates the different perspectives of professionals working inside and outside the policy process.
I’m someone who worked inside English Central government for about ten years at a time of significant social policy development. With earlier experience in research and higher education I am generally well disposed towards the use of academic evidence. My experience, however, of very senior very experienced academics was too often that they didn’t properly consider the realities of the policy making context in how they approached engagement with government, greatly reducing their impact. In the worst cases some took an arrogant approach – almost “we are really clever you must listen to us’ . Civil servants and lobbyists of course are usually much better at politics than academics . So of academics really want to have impact they need to pay strong attention to context, cultivate key relationships and develop smart, situation specific tactics