The Policy Institute at King’s College London, along with colleagues in the digital humanities department, teamed up with technology company Digital Science to build a searchable database and produce a rich analysis of the impact case studies for the 2014 Research Excellence Framework. Saba Hinrichs and Jonathan Grant introduce the key findings of the analysis and explain how the resource provides numerous avenues for future research.
For the first time, Higher Education Institutions (HEIs) were required to submit impact case studies as part of the 2014 Research Excellence Framework (REF). In total, 6,679 non-redacted case studies were submitted, and yesterday, we published a report of the results of our text mining analysis of these data.
The case studies are now available to read online in a searchable database developed by Digital Science, providing a rich resource that has enabled us to demonstrate that UK research has thousands of different applications worldwide. The analysis of the case studies, led by the Policy Institute and department of digital humanities at King’s College London, used text mining techniques leading to the identification of 60 impact topics or areas where research influences society, such as medical ethics, climate change, clinical guidance and women, gender and minorities. Automated text mining was also supplemented with ‘deep mines’, where more than 1000 case studies were read to provide a deeper picture of the data – looking at specific questions such as ‘what is the impact and value of research on clinical practice and health gain?’ and ‘what has been the impact of research on BRIC countries?’.
Search the database or download all the data in JSON format
The results of this analysis are fascinating and also discussed in Research Fortnight and on the HEFCE blog. The take home message is that the benefits from research are multi-impactful. For example, the case studies submitted for the unit of assessment on psychology, psychiatry and neuroscience showed that these disciplines made a contribution to 49 of the 60 impact topics. These included obvious applications such as in mental health, but also in impact topics including transport, schools and education and crime and justice. We also discovered that the case studies contained more than 3,700 individual pathways to impact presenting a real challenge to anyone interested in producing impact metrics.
Many of the case studies were able to provide a clear illustration of the contribution that universities make to society, in a way that has not been revealed before. For example, one case study reported on research showing that the painkiller co-proxamol was the most common drug used for suicides in the UK. This finding led to its withdrawal, and has been estimated to have led to approximately 600 fewer deaths by 2012 in the UK alone.
Our analysis has also shown that impact of UK universities is truly global with the Details of the Impact sections of the case studies mentioning every country in the world – with the US being the most frequently mentioned, followed by Australia, Canada and Germany.
The global reach of impacts arising from research undertaken in UK HEIs. Source: The nature, scale and beneficiaries of research impact (2015)
The case studies provide a rich resource demonstrating the breadth and depth of research impact. They can also help us to change perceptions. For example, the largest impact topic was informing government policy, which was associated with 1,233 case studies. The word ‘policy’ was mentioned in 3,206 case studies. This is reassuring, given the pre-REF scepticism about whether case studies could capture and articulate impacts on public policy. It was suspected that researchers mostly influenced policy through personal contacts and under-the-radar advisory channels, rather than through specific research that could be described in a case study.
Our analysis is really just the start. There are limitations to using these case studies as research material, such as the universally positive sentiment in their language and the fact that researchers could carefully select which case studies to use and the number of identical or near-identical submissions. However, this rich resource provides numerous avenues for future research – to enable us to really dig deeper into the global impact of UK University research.
This piece first appeared on the PolicyWonkers blog and is reposted with permission.
Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Dr Saba Hinrichs is a new Senior Research Fellow at the Policy Institute at King’s College London, working primarily within health policy and the ‘science of science’ work within the Policy Research in Science & Medicine research unit. Her background is in Engineering and Health Policy and her interests are in advancing health research and delivery systems, and the need to engage with policymakers to bring evidence, ideas, and innovation into practice for this purpose. Her expertise is in the use of systems engineering principles for research, policy design, impact assessment, and complex evaluations of healthcare services.
Professor Jonathan Grant is Director of the Policy Institute at King’s College London. His main research interests are on R&D policy and the use of research and evidence in policy and decision making. The Policy Institute is responsible for driving the College’s vision to develop and enhance its visibility, impact and outreach with the policymaking community in the UK and internationally. The aim of the Institute is to help secure, maximise and accelerate the translation of academic research to the benefit of policy and practice.