The Policy Institute at King’s College London, along with colleagues in the digital humanities department, teamed up with technology company Digital Science to build a searchable database and produce a rich analysis of the impact case studies for the 2014 Research Excellence Framework. Saba Hinrichs and Jonathan Grant introduce the key findings of the analysis and explain how the resource […]
Publication of the results of the 2014 Research Excellence Framework evaluation of the quality of work undertaken in all UK universities last December attracted much attention, as league tables of university and department standings were constructed and estimates of the financial consequences of the achieved grades were assessed. Soon after that, a book was published savagely criticising the peer […]
Impact doesn’t have to be a dirty word – staying positive about the promotion of scientific excellence.
The research funding landscape looks bleak in many areas at present, but that’s all the more reason to focus on success stories, argues Ben McCluskey. Universities are doing great work to bring jobs and money into the regions they serve, but they should be supported by a framework based on national cooperation, not competition.
In light of the incredible research […]
After the REF dust settled on 2014, what will come next for the higher education sector? Will Davies breaks down audit culture and describes neoliberalism as the ‘disenchantment of politics by economics’ where processes traditionally left in the realm of politics must now be reconfigured in calculative, economically rational terms. When audit becomes married to rapidly shrinking budgets, and […]
Against REFonomics: Quantification cannot satisfy the demands of rationality, equity and tolerability.
Academics are assured by government ministers and institutional heads that research assessment is designed on their behalf. Liz Morrish looks at whether the assessment tools created have extended their reach and left academics exposed. At its best, the REF distorts research agendas and priorities. However, a graver hazard is that a new selective and competitive academic will be formed, whose […]
In the wake of the REF, LSE launches impact website to demonstrate how research can make a difference.
Love it or loathe it, impact is fast becoming the buzz word in UK academia. To coincide with the release of the REF2014 results on 18 December 2014, which will demonstrate how well (or otherwise) UK academia is creating impact from its research beyond the academy, LSE is joining the growing number of UK higher education institutions to showcase […]
Head of Research Policy at the Higher Education Funding Council for England, Steven Hill, presents an overview of the work HEFCE are currently commissioning which they are hoping will build a robust evidence base for research assessment. He argues that attention on the costs, benefits, problems and solutions of the REF are an obvious starting point, but it is also important that the […]
Predicting the results of the REF using departmental h-index: A look at biology, chemistry, physics, and sociology.
Can metrics be used instead of peer review for REF-type assessments? With the stakes so high, any replacement would have to be extremely accurate. Olesya Mryglod, Ralph Kenna, Yurij Holovatch and Bertrand Berche looked at two metric candidates, including the departmental h-index, and four subject areas: biology, chemistry, physics and sociology. The correlations are significant, but comparisons with RAE indicate that […]
Collaborative ‘science of science’ needed to ensure research and education make a difference to practice.
Zoë Sheppard, Vanora Hundley, Edwin van Teijlingen and Paul Thompson of Bournemouth University present the challenges of impact in healthcare recently discussed at a symposium held by the Centre of Postgraduate Medical Research and Education at Bournemouth University. Given the imminent results of the Research Excellence Framework 2014, the summarised findings and issues raised on the implementation of impact point to further collaborations needed on […]
Time to abandon the gold standard? Peer review for the REF falls far short of internationally accepted standards.
The REF2014 results are set to be published next month. Alongside ongoing reviews of research assessment, Derek Sayer points to the many contradictions of the REF. Metrics may have problems, but a process that gives such extraordinary gatekeeping power to individual panel members is far worse. Ultimately, measuring research quality is fraught with difficulty. Perhaps we should instead be asking which […]
Perceptions and ‘impacts’ of the REF: Key aim for next round should be to explore apprehension and minimise anxieties.
Discussions around the REF have tended to be negative, but academics appear to have experienced the framework in a number of different ways. To understand the variety of themes and important issues, Tony Murphy and Daniel Sage undertook a media analysis that points to the range of concerns academics have around the REF and its processes. They argue there is much […]
Five Minutes with Trish Greenhalgh: “We need to be clear that research impact isn’t a single dimension.”
Trish Greenhalgh is currently Dean for Research Impact at Barts and The London School of Medicine and Dentistry. In discussion with Managing Editor Sierra Williams she delves into the nature of academic impact and the remit of her appointment. She finds that many academics still have a naïve and overly rationalistic view of how their work might link with policy. Drawing on the […]
Research impact on policy-making is often understood in instrumentalist terms, but more often plays symbolic role.
The idea that research should have an impact on policy is premised on an instrumentalist, or problem-solving theory of research utilisation: namely, that research is valued by policy-makers as a means of adjusting their outputs. Yet Christina Boswell’s research has shown that expert knowledge is just as likely to be valued for its symbolic function: as a means of […]
The rejection of metrics for the REF does not take account of existing problems of determining research quality.
Amidst heavy scepticism over the role of metrics in research assessments, Martin Smith wonders whether the flaws of the current system have been fully recognised. There is no system of research assessment that is perfect and peer review may well be a better, although problematic, measure of quality than metrics. But the REF has become disproportionate. The question that arises […]
Research assessment, altmetrics and tools for determining impact: Reading list for #HEFCEmetrics review launch.
HEFCE have announced they are arranging an independent review of the role of metrics in research assessment and management. The Impact blog welcomes this review and will look to encourage wider discussion and debate on how research is currently assessed and how it could be in years to come. Over the last two years we’ve featured a number of […]
HEFCE announces Open Access policy for the next REF in the UK: Why this Open Access policy will be a game-changer.
With the final consultation period now over, the Open Access policy for the next REF has been released. Alma Swan looks at the rollout which requires the deposit of articles into repositories and finds this is pragmatic but good policymaking. With that simple requirement, the culture in British universities can be shifted towards open access. Swan also notes areas where […]
Achieving Impact in Research aims to address the importance of understanding and achieving impact for the purposes of gaining research funding and reporting achieved impact for the Research Excellence Framework (REF).The book includes contributions from researchers and researcher developers who feel that impact is ill-defined and poorly understood despite its prevalence in policy documents, websites and institutional activities. Catherine Easton finds that this a […]
A Bayesian approach to the REF: finding the right data on journal articles and citations to inform decision-making.
Now that the REF submission window has closed, a small panel of academics are tasked with rating thousands of academic submissions, which will result in university departments being ranked and public money being distributed. Given the enormity of the task and the scarcity of the resources devoted to it, Daniel Sgroi discusses a straightforward procedure that might help, based on the […]
As the REF submission period ends, mismatched publishing incentives signal challenging times ahead in academia.
Academics are frequently subject to new types of evaluations. November marks the end of the submission process for the UK funding council’s evaluation, the Research Excellence Framework (REF). John Hudson discusses some of the shortcomings of the REF and the methods individual papers are ranked. New evaluations and requirements change the incentives of economists and can affect their research – sometimes […]
The Matthew effect and REF 2014: Funding disparities between UK universities may cause greater strains over time.
As the submission deadline for REF2014 draws nearer, there is a need to reflect on how the subsequent allocation of funding will affect the UK research environment. Dorothy Bishop argues that the rumoured funding formula would dramatically increase the gulf between the elite and other institutions and that if funding continues to be concentrated only in elite institutions with a high […]