Grant applications and impact statements for research projects are important points for researchers and assessors to consider the future impact of research and whether the proposed pathways to these goals are feasible. Drawing on evidence from REF2014 impact case studies, Lai Ma provides a framework for achievable impacts that can help both researchers and assessors in determining how their work can lead to different impacts and outcomes.

In a previous post I talked about the lack of guidance for reviewing impact statements in grant applications. For both applicants and reviewers, predicting what kinds of impact a research project may bring in five, ten, or twenty years can be beyond any individual’s expertise and imagination. I suggested that ex ante impact assessment should focus on process-oriented impacts, meaning that impact statements should be about what researchers are planning to do. I argued that the emphasis on process-oriented impact will prompt researchers to actively plan and engage in impact activities–and these plans can be evaluated in a more transparent manner.

However, I couldn’t help but wonder what kinds of impact researchers can realistically achieve within a certain timeframe. In other words, if impact statements are to be reviewed and scored by ‘normal contributions’, it is necessary to know what would be practical, feasible and achievable impacts. So I set out to analyse the UK REF2014 impact case studies to get some answers. The rationale is that these impact case studies are indicative of what kinds of impact are actually possible (they are, hopefully, not fictitious or imaginary ones) and they can be used when setting criteria for ex ante impact assessment. The contents of the case studies were coded using the definitions of output, outcomes, and impact in the Impact Journey logic model. The somewhat surprising finding is that most impact case studies report outcomes rather than impacts.

In fact, even the examples provided by REF2021 Guidance show that impacts consist of outcomes in the logic model. For example, under ‘Impacts on the environment’, it has listed the following: ‘Policy debate on climate change or the environment has been influenced by research’ (Outcomes), ‘New methods, models, monitoring or techniques have been developed that have led to changes or benefits’ (Outcomes) and ‘The management or conservation of natural resources, including energy, water and food, has changed in a developing country’ (Impacts). In other words, impact is inclusive of what are called outcomes and maybe even some outputs in the logic model.

What do these findings mean for impact assessment in grant applications? We consider the following when developing a framework for impact evaluation in grant applications:

1. Is it necessary to ask for long-term impacts? Have long-term impacts been already identified by the funding programme?
2. What are the objectives of a funding programme? Is it about finding solutions to global challenges such as the United Nations Sustainable Development Goals (SDGs)? Or does it aim to encourage collaboration between researchers and industry or non-governmental organisations?
3. Do funding programmes expect researchers to propose or influence policy changes? Do researchers have the competencies to provide evidence for policy?
4. How are researchers supported to achieve impacts? What kinds of expertise, resources, and infrastructure are needed for their success?

The framework identifies four types of funding programmes and suggests different criteria for evaluating impact statements, including some evaluation templates that can be refined and adapted for specific funding programmes. In a nutshell, (1) For funding programmes that support basic and curiosity-driven research, it is unnecessary to predict actual societal impacts that a research project may generate because the purpose is to provide intellectual space and resources for exploring questions that are deemed important and challenging by the research community; (2) For funding programmes that address global and societal challenges and/or national priorities, it would be appropriate for applicants to articulate which global or societal challenges the research projects will address, and where possible, provide examples of policies and practices the research project may change or influence; (3) For funding programmes that encourage collaboration between academic research and non-governmental organisations (NGOs), there are immediate societal challenges and issues to be resolved, hence the evaluation of impact statement in these instances should focus on the expected outcomes of the collaboration; (4) For funding programmes that promote public engagement and science communication, it would be appropriate to assess the planned activities, the expected outcomes as well as existing and potential collaborators and participants in the intermediate term.

Impact statements in grant applications are instrumental for showcasing the alignment between a proposed research study and the aims and objectives of a funding programme. They can also inculcate a research culture in which the pursuit of knowledge is in, with and for society. I hope the framework is useful for better evaluation of impact statements in grant applications.

The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.

Image Credit: N. via Unsplash.