katellenThe current approach to measuring and assessing research impact favours certain kinds of academics and research topics over others. Kat Smith and Ellen Stewart outline three areas that require further consideration. Academics who are negatively impacted by the current framework might look to suggest adjustments which limit or ameliorate these effects.

Academics working in the UK are increasingly encouraged and incentivized to seek research impact beyond the academy and the consequences of these changes have caused alarm for some. In a new article in the Journal of Social Policy, we outline a range of concerns that have been raised in publications to date, across disciplines, and then present an interview based case study of 52 academics working on health inequalities during the decade in which the UK’s current research impact architecture has evolved.

We assessed these concerns in the context of impact-related guidance from research funders and REF2014 panels. Our findings highlight a range of problems with the current approach to measuring, assessing and rewarding research impact, many of which were identified by researchers who were broadly supportive of the idea of improving the relationship between researchers, policymakers and the public. In this blog, we focus specifically on outlining concerns that seem likely to be concentrated on particular kinds of academics: those with relatively less social capital; those with caring roles; early career researchers; and those undertaking critical and blue skies research.

On academic social capital and caring roles

Several interviewees, particularly those who were at an earlier career stage, suggested that impact reward systems may be unintentionally reifying traditional academic elites. It is, after all, a lot easier to achieve research impact if you are already a senior academic with a strong reputation in policy circles – it’s even easier to do this if you went to school with, or are otherwise personally connected to (e.g. as friends/neighbours/family) senior policy folk (see for example Ball; Ball & Exley). Research on policymaking processes in UK suggest that older, white men continue to dominate structures. Surveys of higher education suggests the situation is similarly imbalanced for chair level academic posts. If our interviewees are correct, then the research impact agenda is reinforcing this, much as Les Back argues, with the impact incentive structures encouraging ‘an arrogant, self-crediting, boastful and narrow’ form of academic work that positions ‘big research stars’ as ‘impact super heroes’.

Image credit: Cirrus fibratus radiatus over ESO’s La Silla Observatory[61] CC BY 4.0 via Wikimedia

Further issues arise when we consider that the timing of key opportunities for ‘impact’ can be particularly difficult for academics with caring responsibilities (evening and weekend networking opportunities, for example).  There are also basic workload issues that are likely to be more constraining for those with caring roles (several interviewees reflected that they did not feel the demands of achieving impact had been matched by commensurate reductions in other workloads). Since we know that women take on a disproportionate amount of caring work, then this is a gender issue.

Early career researchers

Simply by virtue of being earlier in your career, it may be less likely that you are well-connected to decision-makers and other potential research users beyond academia. It may also, as two interviewees suggested, be rather riskier to take on impact-orientated work as an earlier career academic.  The risks relate partly to the fact that time spent on research impact work is necessarily time not spent on other kinds of academics work such as traditional journal article and proposal writing, which, currently at least, most interviewees believed remained more important for career progression. There are also, however, credibility risks of entering public, media and policy domains that invariably stretch beyond one’s own research expertise.  These risks may be lower for academics with established track records of research and strong existing reputations but they are potentially substantial for academics at the start of their research careers.

Critical and ‘blue skies’ researchers

Research on both academic and policy work has highlighted the value of critical and blue skies academic work and few seem to be actively suggesting that it is desirable to restrict this kind of work. Yet, as science studies authors such as Knorr Cetina clearly demonstrate, academics work hard to develop grant applications that have the greatest chance of success and when major sources of academic funding (from the higher education councils, through REF, and UK research councils) are all stressing the importance of research impact, then we can expect that many academics will commit themselves to undertaking work that more obviously facilitates research impact, limiting applications for undertaking critical and blue skies research, a concern picked up in the RAND evaluation of the REF2014 impact system. It may also lead to unhelpful ‘fudging’ exercises, as several of our interviewees described, in which more critical researchers feel pressured to phrase policy recommendations in strategically vague ways, softening perceived criticism and (as one put it) “bend[ing] with the wind in order to get research cited”.

figure 1 impact degree significanceSource: Smith and Stewart (2016). We Need to Talk about Impact: Why Social Policy Academics need to Engage with the UK’s Research Impact Agenda. Journal of Social Policy.

We expect that most of the concerns raised here will be familiar to colleagues who have come across contributions on this issue from Greenhalgh & Farly, Pain et al, Back, or simply from discussions with colleagues in the offices, staff rooms and corridors of the UK’s universities. Here, we feel it is important to reflect that the current architecture is not necessarily a ‘done deal’: academics who might be negatively impacted by the current framework might suggest adjustments which limit or ameliorate these impacts. For example, REF impact case studies should perhaps be subject to some form of equality assessment (as research outputs are). It may be worth considering additional support for achieving, and/or a lower threshold for demonstrating, research impact for earlier career academics and those with substantial caring roles. If we accept that there is a wealth of valuable academic work that could not (for economic, political or practical reasons) be expected to have demonstrable impact in the short to medium term, it may be desirable to make a stronger case for the need for impact-orientated research opportunities to be balanced by research calls/funding that are specifically for critical and blue skies research.

This blogpost is based on the journal article published 16 May 2016: We Need to Talk about Impact: Why Social Policy Academics need to Engage with the UK’s Research Impact Agenda. Journal of Social Policy. DOI: http://dx.doi.org/10.1017/S0047279416000283

Note: This article gives the views of the author, and not the position of the LSE Impact blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.

About the Authors

Kat Smith, PhD is a Reader at the Global Public Health Unit in the School of Social and Political Science at the University of Edinburgh. Her research focuses on analysing policies affecting public health (especially health inequalities) and better understanding the relationships between public health research, policy, advocacy, and lobbying. She brought some of this work together in a book entitled Beyond Evidence-based Policy in Public Health: The Interplay of Ideas, which is part of a new book series, Palgrave Studies in Science, Knowledge, and Policy that she co-edits with Professor Richard Freeman. More recently, she was the lead editor of Health Inequalities: Critical Perspectives, published by Oxford University Press in 2016.

Ellen Stewart is a Postdoctoral Research Fellow in the Usher Institute, University of Edinburgh. Her research explores health policy and politics, with a particular focus on understanding different forms of ‘lay’ and ‘expert’ knowledge within practices of public engagement and evidence-based policy. Her book Publics and their health systems: rethinking participation was published in January 2016.

Print Friendly