Despite the increased importance of demonstrating impact, it remains a concept many academics feel ill-equipped to measure or evidence. Clare Wilkinson reveals how researchers from a broad range of disciplines think about evidencing impact, what obstacles might stand in their way, and how they might be further supported in future. Knowledge around research impact continues to exist in siloes, with those at early-career stages often less clear about it. Researchers are eager to see best-practice examples and information on the types of evidence that could be collected, but are less receptive to support mechanisms that more obviously add to bureaucracy. Interestingly, academics rarely consider how their own research backgrounds, methodologies, and tools may feature in their impact activities.
Research impact, as defined by UK agendas such as REF2014 and Pathways to Impact, has increasingly influenced researchers’ lives over the past ten years or so. And yet, impact is a concept many researchers still feel ill-equipped to measure or evidence. This is despite many academics’ own research backgrounds offering methodologies and tools which could be translated and applied to impact planning and capture.
In a recent case study I explore how researchers from a broad range of research areas think about evidencing impact, what obstacles to impact-gathering might stand in their way, and how they might be further supported in future.
Unsurprisingly the research found myriad potential barriers to gathering research impact, such as uncertainty over how impact is defined, captured, judged, and weighted, or the challenges for researchers in tracing impact back to a specific time-period or individual piece of research. Many of these constraints have been recognised in previous research in this area – or were anticipated when impact was first discussed – but talking to researchers in 2015 about their impact experiences of the REF 2014 data-gathering period revealed a number of lingering concerns.
A further hazard identified by the case study is the inequalities in knowledge around research impact and how this knowledge often exists in siloes. Those researchers most likely to have obvious impact-generating activities were developing quite detailed and extensive experience of impact-capturing; while other researchers (including those at early-career stages) were less clear on the impact agenda’s relevance to them or even whether their research had featured in an impact case study. Encouragingly some researchers did seem to increase in confidence once having experience of authoring an impact case study, but sharing skills and confidence with the “next generation” of researchers likely to have impact remains a possible issue for those supporting impact evidence-gathering.
Image credit: Lego Explorer by JonoTakesPhotos. This work is licensed under a CC BY 2.0 license.
So, how can researchers, across the board, be supported to effectively evidence their impact? Most popular amongst the options given to the 70 or so researchers that participated in this case study were: 1) approaches that offered them more time or funding to gather evidence; 2) opportunities to see best-practice examples; 3) opportunities to learn more about what “impact” means; and 4) the sharing of information on the types of evidence that could be collected.
These are all activities that are able to be more readily supported for REF2021. For instance, all impact case studies are available and fully searchable online, while there are a number of reports and articles that document the ways impact was considered; from best practice guides, to disciplinary differences, and evaluation of the approaches taken. Having access to those materials can help researchers picture the types of evidence they might source; those that are more common (like references to a website), but also those that are less frequently used (such as minutes from a meeting) but which could nonetheless be very relatable to their areas of research.
There were, though, possible support mechanisms for impact evidence-gathering that proved less popular amongst the people I heard from. These include those interventions that could more obviously add to “bureaucracy”. The REF and a detailed research application may sometimes feel to researchers like a large, unwieldy beast, with various bits of paperwork, budgets, and other practical considerations. All this is before researchers even get to the depth or detail of the research itself and so additional factors to consider or demands on their time were generally not popular.
Although more than half of researchers thought mechanisms like impact groups (where researchers share ideas and work together) and systems (such as tracking software) to record impact evidence would be useful, these were far less popular compared with other options given. Similarly, resources like a centralised system to review impact materials or impact peer review processes were not amongst the most favoured options to facilitate support.
Overall, perhaps the most contentious finding related to the role researchers’ own disciplinary backgrounds played in their abilities to evidence impact. Even when researchers had the skills or techniques to gather research impact evidence, the case study suggested that research and impact were being treated as quite distinct phases. Researchers could thus be better mobilised to consider how their own research approaches may feature in their impact activities or how they might be supported by mediatory roles to do that.
Despite the many challenges, this case study suggested a good degree of flexibility, diversity, and a growing appreciation of impact post-REF2014.
This blog post is based on the author’s article, “Evidencing impact: a case study of UK academic perspectives on evidencing research impact”, published in the Journal of Studies in Higher Education (DOI: 10.1080/03075079.2017.1339028).
Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
About the author
Clare Wilkinson is an Associate Professor and Co-Director of the Science Communication Unit, at UWE Bristol. Clare has conducted a range of research on science, health and the media, and co-authored the book Creative Research Communication: Theory and Practice published by Manchester University Press in 2016. Clare can be found on Twitter at @clarewilk4.