The Research Excellence Framework’s purpose is to assess world-class research. However, it overlooks the work of many groups who make possible high-quality research submissions. James Baker illustrates this through the case of the Research Software Engineer, without whom much high-quality research would not be possible, but whose contributions do not get counted by the REF. The Hidden REF celebrates all those who, like the Research Software Engineer, contribute to the research process. 

This June the EPSRC announced the third round of their Research Software Engineer Fellowships. The scheme, in the words of the EPSRC, “aims to recognise the contribution of RSEs who are driving the development of high-quality research software and demonstrating leadership”, and based on this the call makes a number of claims about the benefits of RSEs to both UK HEIs and UK HE as a whole. I’m delighted with this investment in Research Software Engineer capacity. And whilst as a historian, I’m probably not your average RSE enthusiast, I’m sold on the value of RSEs to high-quality research. The problem I want to explore is that the Research Excellence Framework, the process we have for assessing high-quality research, does not recognise RSEs.

The role of the RSE in the research process

According to the call, “Research Software Engineers are key members of computational research teams”. They don’t only benefit those teams, but they also “provide the critical skills required to develop, support and evolve software to tackle [..] research challenges”. And they do this through their ability to “combine professional software engineering expertise with the ability to rapidly gain a solid working understanding of a research area”.

As we know from a 2014 Software Sustainability Institute survey, 69% of academics reported that it would not be practical to do their research without software. And whilst there may be clusters of research software use in particular disciplines, it is clear that the skills RSEs foster are vital to researchers submitting ‘high-quality research’ to all four REF panels, and most of the 34 units of assessment.

These benefits chime with my own experience of RSEs (in some cases, even before they knew they were RSEs), with interactions that have indelibly shaped the kind of researcher I am and the kinds of research that I do.

All the things that I know because of RSEs have made my publications better, and continue to make my publications better. They make them more REFable

Thanks to RSEs, I know not only how to computationally analyse my data, but also how to ask the right questions to better understand what computational processes are doing to my research.

Thanks to RSEs, I know not only how to apply version control to my data and (hacky) code, but also why versioned data and code are the vital building blocks of reproducible research.

Thanks to RSEs, I know not only how to publish my research data, but also how to do so in a manner that is responsible, that documents the activities and knowledges that produced those data.

Thanks to RSEs, I know not only how to collaborate with software engineers, but also what makes a software engineer a genuine partner in research, such that the research takes directions it would not have without their collaboration.

And thanks to RSEs, I know not only how to write about the software that I use, but also how to cite it properly in my research, to give credit to software engineers where credit is due.

The hidden RSE and the hidden REF

All the things that I know because of RSEs have made my publications better, and continue to make my publications better. They make them more REFable. But once those papers, and the papers of anyone else whose benefited from the rise of the RSE, enter the REF2021 the vital contributions made by RSEs – even those who benefit from honest and equitable co-authorship (big shout out here to the ‘Living with Machines’ team and their recent paper) – will become lost in the scores, the aggregations, the rankings. This is, of course, true for many individuals and groups involved in research. And it is one of the motivations behind The hidden REF, a competition that recognises all research outputs and every role that makes research possible.

Once those papers, and the papers of anyone else whose benefited from the rise of the RSE, enter the REF2021 the vital contributions made by RSEs …will become lost in the scores, the aggregations, the rankings

In the context of The hidden REF, RSEs make for an interesting case because – as the EPSRC call suggests – of all the groups the REF obscures, RSEs are far from undervalued right now. Instead, from a position of modest strength, there is some (well-founded) resistance within the RSE community to being counted in exercises like the REF2021, not least because at some level it doesn’t fit with who RSEs want to be. But, as D’Ignazio and Klein tell us, what gets counted counts, and without wishing to force metrics onto the RSE community, if we want their work to be valued in the long term, after schemes like the EPSRC RSE Fellowship have inevitably run their course, then the contribution of RSEs to high-quality research needs to be recognized in how we celebrate research.

The hidden REF will celebrate everyone that contributes to research. We believe that if we recognise everyone who is vital to research, we will create the right environment in which to advance it. For more information The hidden REF see hidden-ref.org.

Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below

Image credit:  Ryoji Iwata on Unsplash