Impact case studies will form an important part of all universities’ eventual submissions to the next round of the Research Excellence Framework (REF). Drawing on a linguistic and thematic analysis of 175 impact case studies from REF 2014, Bella Reichard, Mark Reed, Jenn Chubb, Ged Hall, Lucy Jowett, Alisha Peart and Andrea Whittle set out the written characteristics that defined high-quality impact case studies in REF 2014 and suggest how they can inform the creation of better impact case studies as part of the upcoming REF. 

Writing impact case studies for REF2021 is something that many people in the UK HE sector are currently involved in. One way to get inspiration is to check case studies from REF2014, in particular those that were clearly given 4*. We were interested to what extent you can generalise from these examples, and how they differ from low-scoring case studies. In a recent article, we conducted a comprehensive analysis of 175 REF2014 case studies across each of the Main Panels. To do this, we used qualitative and quantitative linguistic analysis of high versus low-scoring cases to provide evidence of what it took for a case study to get a top score in 2014. Although scores were not made public for individual cases, we looked for institutions whose case studies were all given grades in the same range to identify 175 (out of 7,000) high- or low-scoring case studies from a cross-section of disciplines. 

In a nutshell, high-scoring case studies clearly articulated evidence of significant and far-reaching benefits that could be clearly attributed to research conducted at submitting institutions. 

However, new rules are being introduced for the upcoming REF and there are fears that the bar may have been raised. This could mean that sticking to what we understand was ‘best practice’ during REF 2014 may not be enough to reach top scores in 2021, so the lessons from this research should be seen as the minimum required to write a top-scoring case study for REF2021. Also keep in mind that while there are examples of 2* and 3* impact case studies that might have scored higher if they had been better written and evidenced, it is not possible to get top grades unless you actually have significant and far-reaching impact. Here are some key lessons from what we found in our research to help boost the articulation of impact in REF2021.

Key lessons from high-scoring case studies in REF2014

1. Establish links between research (cause) and impact (effect) convincingly 

In REF2014:

• Only 50% of low-scoring case studies clearly linked the underpinning research to claimed impacts (compared to 97% of high-scoring cases). 
• High-scoring case studies were significantly more likely to include phrases that attributed impact to research: “cited in (policy documents)”, “used to (inform)” and “resulted in”. Those phrases indicate a focus on the effect, that is, on what the activity led to. In contrast, low-scoring case studies tended to link backwards, foregrounding the research activity (e.g. “(findings) of the research”). Moreover, they were more likely to include ambiguous or uncertain phrases like “a number of” (implying that it is not known how many) and “an impact on” (implying that the nature of the impact is not known).
• High-scoring case studies made causal relationships more explicit (above average when compared to general English texts, where low-scoring case studies are below average), which makes a text easier to process.

For REF2021:

• In Section 2, describe the key findings from the underpinning research that pertain to your impact. Only include essential contextual material, and avoid unnecessary detail on methods or other findings that were not integral to the impact 

2. Articulate how specific groups have benefited and provide evidence of significance and reach 

In REF2014:

• 84% of high-scoring cases articulated significant and far-reaching benefits, compared to 32% of low-scoring cases, which typically focused on pathway. 
• High-scoring impact case studies contained more phrases that specified reach (e.g. “in England and”, “in the US”), compared to low-scoring case studies that used the more generic term “international”, leaving the reader in doubt about the actual reach. They also included more phrases implicitly indicating the significance of the impact (e.g. “the government’s” or “to the House of Commons”), compared to low-scoring cases that emphasized pathways (e.g. “the event”, “has been disseminated”).
• Phrases containing variations on the word “dissemination” were more common in low-scoring case studies. These case studies focused more on pathways than impacts.

For REF2021:

• Systematically explain why each impact is significant and far-reaching, referring to evidence.
• Dissemination is not impact: even if you have impressive numbers of reads, downloads, views or listens, how do you know if anyone learned anything from it, benefited, or did anything different as a result? Keep asking “what happened next” until you find the benefit. 
• For policy impacts, where possible include evidence that the policy was implemented, enforced and worked on the ground.

3. Make your narrative easy to understand 

In REF2014:

• High-scoring impact case studies scored more highly on a readability measure correlated with reading speed because they included shorter, less complex sentences. The Flesch Reading Ease score, out of 100, was 30.0 on average for 4* and 27.5 on average for 1*/2*. While this is a significant difference (p<0.01), the scores indicate that case studies are generally of “college-graduate” difficulty. 

For REF2021:

• Aim to write for an interested and educated non-specialist audience. Would your colleague down the corridor, working in a completely different corner of your Unit of Assessment, understand your text on first (and fast!) reading?
• Break up the text to make it easier to read, using sub-headings and other structural devices 
• Put evidence in context to explain its significance, e.g. rather than just citing visitor numbers, say if these was the largest number of visitors in any week that year, or explain who visited and why they are significant (perhaps they are from groups who don’t typically visit museums). A small number may be significant in context, but you will need to create the argument to justify this interpretation of your evidence.

Our analysis suggests that impact case studies are a distinctive form of writing. The most successful case studies were written very differently to a normal academic style. This indicates that being the world expert on a topic, writing successfully for academic or even professional audiences, does not make you an expert in writing an impact case study. It is important to take advice on how to evidence and communicate your impacts, and if you don’t have as much support as you would like, we hope our work will give you the pointers you need to get the credit you deserve for your impact.

This post draws on the authors’ co-authored paper: Writing impact case studies: a comparative study of high-scoring and low-scoring case studies from REF2014, published in Palgrave Communications.

Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.

Image Credit: MJ S via Unsplash.