Research Impact Assessments are regularly regarded as a tiresome part of the research process. However, Annette Boaz and Stephen Hanney find that taking a systems approach to health research demonstrates the value of assessing impact. Drawing on examples from a new review for the WHO Health Evidence Network, they highlight the role that impact assessments play in securing research funding and strengthening the health system.
Research is understood to be crucial to health system improvements. However, securing finance to fund research and build capacity, and ensuring that research is used to improve health systems, remain challenging.
Whilst for many researchers, the need to provide a research impact assessment is a burden, our new review for the WHO of policies to strengthen National Health Research Systems (NHRSs), highlights the value and importance of including societal research impact assessment in strategies to achieve system strengthening.
How can a systems approach help strengthen health research?
A systems approach to health recognises the value of having strategies that combine policies to promote the various health research functions and components that constitute a system. These include agenda-setting, monitoring and evaluation (M&E), securing finance, building capacity, and the production and use of research findings.
In a systems approach, the need for research impact assessments as part of the M&E component becomes clear as a pivotal element combining other functions in order to facilitate key aims:
1.Securing Funding: Assessing impacts can provide important evidence to help support efforts both to justify past funding, and secure future funding to build capacity and conduct research.
2.Achieving Impacts: When health researchers know that their work could be assessed partly according to the impact it makes on improving health systems, they have extra incentives and justifications for spending time enhancing the likelihood of such impacts arising. This includes through engaging stakeholders in agenda-setting and in the production and dissemination of research. Analysis of 36 assessments of health research programmes shows such engagement activities contribute to research meeting the needs of health systems, thus resulting in greater impact.
Assessing impacts can provide important evidence to help support efforts both to justify past funding, and secure future funding
Systems Approaches in Action
1.Impact Assessments help secure funding
Back in 1993, the English Department of Health (DH) had realised it might soon have to justify recent major increases it had received in funding to build the health research system. It wanted to be able to demonstrate that the funding of research resulted not only in publications and research training, but also in impacts on health policy and practice that strengthened the health system, and provided benefits to the economy. The English DH, therefore, asked the Health Economics Research Group (HERG), Brunel University, to devise ways of assessing the payback, or wider impact, from health research.
The HERG developed the Payback Framework with its multidimensional categories of impact that included informing health policies and improving health and health equity. The framework also showed how impact could be assessed, in particular through the use of targeted case studies. (This case study approach to research impact assessment also subsequently informed the Higher Education Funding Council for England’s approach to the Research Excellence Framework.)
When health researchers know that their work could be assessed partly according to the impact it makes on improving health systems, they have extra incentives and justifications for spending time enhancing the likelihood of such impacts arising.
Even when NHRSs are seen as being successful in improving health systems, shrewd NHRS leaders recognise that to maintain support it can be valuable to continue demonstrating how their research is contributing to improved healthcare. Health research systems that have promoted impact assessment at various times in their development include those in England, Ireland and Rwanda.
At the global level, in 2013, WHO devoted its main annual report, the World Health Report, to showing that achieving Universal Health Coverage would be boosted if research systems were strengthened and every nation was a producer as well as user of research. Research impact assessment was seen a key element in achieving this goal through providing evidence of the value of funding research: “Adding to the impetus to do more research is a growing body of evidence on the returns on investment.” (p.46)
2.Impact assessments and improving health systems
The World Health Report went on to describe the importance of impact assessment for improving health systems: “To encourage a shared responsibility among researchers for reaching universal [health] coverage, performance measures could be adjusted….Incentives should make reference…also to measures of influence on policy and practice”. (p. 135).
While an increasing number of health, and other, research systems, are introducing research impact assessment, doing so throws up many challenges, including in relation to time-lags, questions of attribution and contribution, and the appropriate unit of assessment. In order to optimise the societal and economic impact of research, an international group of research evaluation experts (at first primarily from the health research systems in Alberta, Catalonia and England) addressed these and other challenges by creating the International School on Research Impact Assessment (ISRIA). It provided a series of training events to build research impact assessment capacity across continents.
While data collection for impact assessment can seem burdensome, there is growing evidence, illustrated in the examples above, that such evaluation has a vital role to play in a well-functioning health research system
An international summit, In the Trenches: Implementation to Impact held in Banff, Alberta, Canada, in June 2019 was the second in a series of events intended to build on the success of the ISRIA, and take a wider perspective on impact assessment within research systems. Considered as a whole, the Summit’s presentations and discussions covered the range of functions of research systems, and key points from each are described in a proceedings’ paper that is organised according to the systems framework applied in the WHO review. This paper further illustrates the value of adopting a systems approach to strengthening research, and the key role of impact assessment within it.
In Alberta itself, a partnership between clinicians, researchers and others has been making concerted efforts to embed the research programme into the Alberta Health Services, in order to identify and address areas where service provision could be improved through research. In one of the presentations, O’Connor and Graham explain that their assessment of the socio-economic returns from the embedded research programme shows how “having a shared vision of impact sustained the partnership over the years.”
From evidence to evaluation
Generating the evidence to help improve health and health systems is often a major motivation for many researchers. While data collection for impact assessment can seem burdensome, there is growing evidence, illustrated in the examples above, that such evaluation has a vital role to play in a well-functioning health research system, especially in helping secure funding, and incentivising and justifying efforts devoted to achieving impacts that strengthen the health system.
Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
Image credit: JOSHUA COLEMAN on Unsplash
