LSE - Small Logo
LSE - Small Logo

Richard Watermeyer

Lawrie Phipps

January 28th, 2025

Using GenAI for the REF is a no-brainer

3 comments | 10 shares

Estimated reading time: 6 minutes

Richard Watermeyer

Lawrie Phipps

January 28th, 2025

Using GenAI for the REF is a no-brainer

3 comments | 10 shares

Estimated reading time: 6 minutes

There is a growing interest in how Generative AI can be used to support and streamline research assessment processes. Richard Watermeyer and Lawrie Phipps argue the standardisation and formulaic nature of REF assessment, alongside its cost, make it a prime candidate for where generative AI could relieve academic drudge work.   


The academic research community is gearing itself up for yet another instalment of the Research Excellence Framework (REF). For the likely few uninitiated readers of this blog and for those in need of reminder, the REF is the means by which the UK Government distributes somewhere in the region of £2billion of annual quality-related research funds to higher education providers able to evidence research ‘excellence’. Occurring roughly every 6-8 years, the last one being in 2021 and the next in 2029, the REF is a hugely time intensive and costly performance-based research funding system, which despite its periodic implementation is, as a wellspring of income and prestige,  an ubiquitous feature of research life in UK universities.

Although being some four years off the next REF, universities across the UK are already awhirl in developing their submissions, of which continuous internal peer-review (of research outputs and impact case studies) forms a core component. The work is long and arduous, resource demanding and correspondingly expensive. Calculations put the average university spend on the last REF at £3m, with a total sector cost of £471m. Of the latter outlay, £24m was spent on panellists – 900 academic and 220 research users – who were responsible for the peer-review of 185,594 research outputs (152,367 of which were journal articles and 28,699 were books) and a further 6,871 impact case studies.

Calculations put the average university spend on the last REF at £3m, with a total sector cost of £471m.

Estimates of peer-reviewing academic journal articles sit somewhere between two to five hours. Even as seasoned (and rapid) reviewers the time demands on REF2021 panellists would have been huge. Yet, the 185,594 research outputs submitted to the REF, as best pickings, are just one sliver of the UK’s research output generated in a REF cycle. While there is no exact figure of total research outputs produced by academic researchers in the last REF cycle, we might consider individual institutional output for the same period. Thus, if we take the University of Bristol, which produced 55,131 research outputs between 1 January 2014 to 31 December 2020, we get a sense both of the volume of research produced in UK universities and the time, therefore, necessarily committed by them in curating the strongest collection of outputs for formal review by REF panels.

It seems reasonable to expect that the time and financial costs of REF2029 won’t differ too much from REF2021, despite some suggestion of economising. It will be just as well then to anticipate another half a billion pounds spend and incalculable hours given over, or be that lost. Consider the displacement effects to core academic tasks and potential impact on careers, to institutional preparations and the formal undertaking of REF panels. Given the current financial stress of many UK universities; the prospect of substantial sector-wide retrenchment of academic staff; and an excessive work burden already suffered by many academics, this seems difficult to square. Yet there would appear little chance of the sector being alleviated of the REF’s financial and human burden. Despite, various appeals for its abandonment, the new Labour government has thus not acted against its roll out in 2029. So, what then in terms of easing the strain; financial and human? What other avenues of recourse are available or might be considered?

Could REF2029 see the inauguration of GenAI tools into the work of its panels?

It seems impossible to ignore the perhaps inevitable techno-solution; could generative artificial intelligence (GenAI) deliver the academic community from the REF, while saving the REF itself (from economic re-rationalisation post-2029)? Could such tools be operationalised for either fully undertaking or at least supporting the peer-review of research outputs (most of which, in the case of journal articles especially, have already undergone expert appraisal), or be purposed for the generation and evaluation of impact case studies? To what extent might this already be happening? What is the legitimacy of and extent to which those with executive responsibility for REF governance, both institutionally and nationally, are licensing or sanctioning the use of GenAI in the preparation and (formative and summative) assessment of REF submissions? Could REF2029 see the inauguration of GenAI tools into the work of its panels?

In terms of facilitating a much faster, less arduous, better calibrated and ostensibly far less costly evaluation process, the answer seems glaringly obvious. Academics, submitting institutions, and REF panellists would surely enjoy a significant unburdening of time and cost. Academics would be freed of a not only profligate but harmful administrative burden. Institutions might redirect millions of pounds earmarked for the REF into other much more necessary and urgent forms of resourcing. Money saved through a GenAI infused REF, might for instance reduce the impact on universities of a recent national insurance hike; might keep academics in jobs and departments from closure.


GenAI tools may be misappropriated by institutions in terms of leveraging unfair advantage and in further exacerbating a culture of gamesmanship already endemic to the REF

Yet there are reasons why GenAI in REF processes might be resisted, which have to do with the potential harms of outsourcing peer-review as a human activity to machine intelligence, and in corollary, therefore, concerns of bias, inaccuracy and even de-intellectualisation. There may also be quite plausible concerns of how GenAI tools may be misappropriated by institutions in terms of leveraging unfair advantage and in further exacerbating a culture of gamesmanship already endemic to the REF. GenAI might for instance offer competitive advantage for (those institutions that use it explicitly or tacitly in) selecting, against norm-referenced criteria, the most competitive research outputs and impacts for submission. Yet GenAI might also conceivably, with its ongoing optimisation, provide greater rigour and accuracy to what can be habitually subjective determinations of research excellence, particularly in the context of impact evaluation.

Maybe the REF is just the kind of academic drudgery that GenAI’s use may be reasonably justified for and arguably with less concern of ethical breach or even human betrayal. Could it even provide greater credibility to claims of public accountability for the money spent on UK research? Where so much of the REF’s work may be experienced as an unjustifiable tax on universities – at a point of peak financial vulnerability – and unnecessary incursion on researchers’ ever depleting time the human cost of being without GenAI is surely greater, perhaps on this rare occasion, than with..?


The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.

Image Credit: Zoya Yasmine, Better Images of AI, The Two Cultures (CC-BY 4.0)


Print Friendly, PDF & Email

About the author

Richard Watermeyer

Richard Watermeyer is professor of higher education and codirector of the Centre for Higher Education Transformations at the University of Bristol, UK.

Lawrie Phipps

Lawrie Phipps is the senior research lead at JISC, UK.

Posted In: AI Data and Society | Featured | REF2029 | Research evaluation

3 Comments