LSE - Small Logo
LSE - Small Logo

Dr Sarah Morton

August 27th, 2020

Assessing research impact – a tale of 7 impact studies

2 comments | 57 shares

Estimated reading time: 10 minutes

Dr Sarah Morton

August 27th, 2020

Assessing research impact – a tale of 7 impact studies

2 comments | 57 shares

Estimated reading time: 10 minutes

How to assess research impact? Sarah Morton draws on her own experience of assessing impact arguing that despite diverse topics, settings and countries, there are patterns of successful impact that can help move our learning forward. She outlines her five top lessons for designing and evaluating impact in a research project. 


 

I am not alone in thinking that creating rich connections between researchers, practitioners, communities and policy-makers alongside a vision for how research can impact an area of work are the core elements of successful research impact. Since 2008 I have been involved in planning for impact, setting out impact pathways and assessing the impact of research on people, policy and practice in various roles at the University of Edinburgh, across national and international research initiatives and more recently through my own business, Matter of Focus.

 

In 2015 I published an empirically based framework for assessing research impact, and I have used it to assess impact in 7 studies including 14 case studies, of research centres and initiatives across 8 countries around the world. These include university-led, NGO and wider research initiatives. The most recent three have been international initiatives looking at children’s issues: impact of the Global Kids Online Programme run by the London School of Economics and UNICEF Innocenti, office of research; the Know Violence in Childhood global learning initiative, and the Multi-Country study of drivers of violence against children (UNICEF and University of Edinburgh).

Working across diverse structures and institutions in detailed case studies across 8 countries has led to deep insights into the business of planning for, creating and assessing the impact of social research. In this blog I pull together some of the insights from that work.

I want to pull these impact studies together for a number of reasons:

  • There are some amazing successful impact examples that we can all learn from
  • I want to close the loop between impact assessment and planning for impact – so that learning can help inform future approaches to Knowledge Exchange and Mobilisation (KE/KMb)
  • The literature tells us that people and relationships are at the heart of impact – looking at impact studies can help see this in action
  • Despite diverse topics, settings and countries, there are patterns of successful impact that can help move our learning forward.

What makes this set of studies interesting is the opportunity they give to draw lessons across a variety of contexts, topics, approaches and initiatives. The method is set out in my paper, but essentially uses a contribution analysis framework to set out and test a pathway to impact following this framework:

 

The Research Contribution Framework (Morton 2015)

 

The most successful data collection methods involved a mix of documentary analysis, follow-up surveys and interviews with research users. Activity reports were important in setting out what had happened and who had been engaged. Where research users indicated changes in policy or practice, these were followed up in the relevant documents to identify evidence.

So what can we learn looking across these impact studies? I think there are five areas for learning:

1.Have a vision and intention for impact

All of the studies have been of research endeavours where impact was an important aim or value of the researchers and funders- perhaps unsurprisingly, people who commission impact studies believe that it is important that research has an impact on the people and communities whose problems and issues it investigates. Putting this intention front and centre and working hard to make it a reality are core features to successful impact. Where there was variable effort or focus on impact across a research portfolio, or where a funder had commissioned a post-hoc impact study rather than embedding it at the start it is much harder to find any evidence of impact, or indeed any clues for where impact may have happened.

 

2.Involve research users throughout

Relationships are at the heart of successful impact practice. The research impact literature shows that involving research users from the start increases the impact of research. This was really apparent in the impact studies, with research user engagement from early in projects or programmes clearly sharpening investigation, warming up potential wider stakeholder groups to engage with findings when they are ready, and helping to hone key messages for different audiences. Joint publishing with academics and non-academics has also been important. Those engaged in the research cycle also benefit from capacity building to understand and work with research.

 

3.Take the time and make the effort

It takes time and effort to craft and execute a successful impact strategy. Building relationships with stakeholders means finding, contacting and nurturing trust. In some cases, this has been a tension with the pressure from funders or institutions for a traditional research timetable and outputs. Knowledge mobilisation strategies mean engaging thoughtfully, and sticking with a project even when things aren’t going to plan. This really needs dedication, resources and a willingness to be open and learn.

 

4.Be honest about failure

Nearly all of the impact studies were focussed on successful impact but there were somewhere there was little or no evidence.  It can be helpful to track and illustrate where there hasn’t been impact, or where engagement strategies have failed. Lack of impact was linked to lack of impact strategy, or to the breakdown of relationships, due to job changes, organisational pressures or lack of capacity to engage from either side.

 

5.Collect data and feedback as you go

The biggest barrier to understanding impact across these case studies was a lack of data. This was usually because people hadn’t collected it when they were in the midst of the projects. It is so much harder to go back and find it later – something I often talk about as a detective job! A few simple feedback mechanisms can easily be systematically threaded through any knowledge mobilisation strategy and then can form the bedrock for building up the story of impact.

 

I hope this inspires you to keep doing the important job of connecting research with people, communities and society. Please get in touch if you want to find out more.

Print Friendly, PDF & Email

About the author

Dr Sarah Morton

Sarah is co-founder and Director of Matter of Focus, a company that supports organisations to effectively use evidence to tell their impact story, and an Honorary Fellow of the University of Edinburgh. Sarah is internationally recognised for her work in developing innovative approaches to ensure that decision-makers have access to the best evidence for taking organisations, policies and practices forward. Sarah has pioneered a participatory approach to using contribution analysis to understand impact, which is the foundation of her current work and the foundation of a software platform to help organisations track impact (OutNav). She was an associate editor of the journal Evidence and Policy for 6 years and has undertaken numerous research impact studies for national international initiatives. She was extensively involved in developing research impact case studies for submission to the UK Research Excellence Framework 2014 @sasmort @matter_of_focus

Posted In: Featured | Impact

2 Comments

This work by LSE Impact of Social Sciences blog is licensed under a Creative Commons Attribution 3.0 Unported.