Impact has in the past two REF cycles established itself as an integral criteria of research assessment in the UK. However, the kinds of impacts that are valued and the ways in which the ‘reach’ and ‘significance’ of impact are interpreted by institutions can conversely de-value certain kinds of transformative research. Taking this into account, Sonja Marzi and Rachel Pain argue for the importance of impact-in-process and the role research itself can play in producing meaningful changes in people’s lives.

The REF results were recently released to much fanfare… and UK academia is once again poring over our placings in the rankings. This is the second REF to include impact case studies; 6,781 synopses of research were submitted that claimed to produce ‘change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia’. Each involved enormous additional work in organising, hunting down, evidencing and assessing impacts, before evaluation on the basis of ‘reach’ and ‘significance’. The increasing link between impact and funding creates all sorts of inequalities and problems, with researchers expected not only to produce impact-relevant research, but to plan for evidencable changes from the start; and ‘impact bounty hunters’ making sometimes impossible demands for testimony from stakeholders outside Universities.

Once again, too, there is little time to take a breath before anxious planning for next time begins. The introduction of ‘impact’ met a range of reactions, from hyperbolic threats to emigrate to concerns about the types of research that it would support and produce. The latter concern is perhaps most compelling in relation to research conducted jointly with non-academic communities (using approaches including knowledge co-production, collaborative or participatory research). These are often used alongside smaller and less powerful external partners such as community groups, voluntary sector organisations, activists or creative practitioners who define and share impacts.

Process-driven research, which works iteratively with non-academics as full partners at all stages, is less amenable to common measures of ‘reach’ and ‘significance’

Academics involved in participatory co-production research found themselves in a paradoxical position. While longstanding efforts to make research beneficial to communities outside universities seemed to have been recognised, conducting research jointly with those communities had been further devalued by having to measure impact according to metrical criteria. Process-driven research, which works iteratively with non-academics as full partners at all stages, is less amenable to common measures of ‘reach’ and ‘significance’; testimonies from high profile stakeholders, or surveys that quantify change for example. Researchers may enjoy some cachet, given the increasing popularity of co-production, but are often less REF-returnable.

Even after a decade of these critiques, evidencing REF-able impact remains fraught, when conventional neoliberal metrics frame impact within a donor recipient model, the end-product of a unidirectional process where Universities own research and its outcomes. For those who practice participatory co-production, this framing simply doesn’t fit.

We suggest, instead, wider and more diverse conceptions of impact are included in future REFs. In particular, a form of impact we call ‘impact-in-process’. This is built from feminist ideas of ethics of care within research; impact is generated with research participants, and acknowledges the benefits to partners from research processes as well as outcomes. The former are often immediate, small scale, reciprocal transformations, of a sort that feel more vital since the pandemic and cost of living crises. Impact-in-process follows the principles of collaboration, and is created through knowledge exchange, mutual learning and shared ownership of the research. It aims to shift power, increasing skills, capacities, confidence and awareness for both participants and researchers.

An example of co-produced impact-in-process is our recent ESRC-funded project Co-producing knowledge during emergencies and pandemics: developing remote participatory visual methods using smartphones. With 24 women co-researchers in Colombia, we used remote audio-visual methods such as participatory video to explore women’s displacement and conflict trajectories, and their private and public activism during the pandemic.

The project has had tangible outcomes, notably a documentary film which women filmed and co-produced. A number of impact case studies have mobilised this type of output, tracking dissemination, measuring audience responses and assessing impacts on policy and practice.

But like many participatory researchers, we don’t believe these are necessarily the most meaningful impacts that the research produces. In the project evaluation, the women talk almost exclusively about impacts from the research process. For example, they learnt skills that are valuable beyond the project timeline, using smartphones for online meetings and filming, especially important as their professional, social and community networks are now more online. More important still are benefits from working together. When we asked women to reflect on why the project was worth their time, they talked of how doing the film together benefitted them, rather than the film as an outcome. These benefits were social and emotional:

‘I feel very good about the movie because it was something new for me and I had the opportunity to meet other people who had the same displacement or worse than mine. It was a new teaching and we learned many things and we had the opportunity to share and bring out that pain that we carry inside us’ (Female participant, Medellin)

They especially valued the chance to share their experiences of displacement, and their solidarity for each other while resisting violence and inequality in their new urban homes:

‘It has been something new for me because I have learned not to keep what I feel, I never thought I would be able to express so many emotions with this wonderful team. For me it was very important to have learned from all of you. I feel happy because I was able to capture a little of my life so that people see that we can get ahead without stepping over each other’ (Female participant, Medellin)

Regular meetings at which experiences are shared, within carefully facilitated research, reduced isolation, combatted trauma and created a collective understanding of women’s survival. They report heightened confidence, and pride in being documentary directors. At the same time this impact is not one-directional; as researchers we learnt from the participants, transforming our research and lives by listening to what they had to say.

We doubt that these small-scale impacts would ‘count’ towards an ICS as currently construed, except as a footnote to larger and more numerous impacts down the line. But impact-in-process generates some of the most profound effects our research can have. For the next REF, we should therefore question the language of reach and significance – to whom, and for whom?

The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.

Image Credit: Emmanuel Appiah via Unsplash.