• Permalink Gallery

    The REF’s focus on linear and direct impact is problematic and silences certain types of research

The REF’s focus on linear and direct impact is problematic and silences certain types of research

In the last Research Excellence Framework (REF), the new element of research impact was understood in very linear and direct terms. Aoileann Ní Mhurchú, Laura McLeod, Stephanie Collins and Gabriel Siles-Brügge consider how accepted definitions of impact may have had the effect of silencing certain types of research. Research and impact should be seen as a two-way street, where […]

Print Friendly
  • Permalink Gallery

    The starting pistol has been fired – now is the time to heed the drive towards open access books

The starting pistol has been fired – now is the time to heed the drive towards open access books

The Consultation on the Second Research Excellence Framework (REF) revealed funding bodies’ intention to extend open access policy to also include monographs by the time of the third REF in the mid-2020s. Despite this being some time away, Martin Eve argues that preparations must begin now. The economic challenges of publishing open access monographs are clear, so time should […]

Print Friendly
  • Permalink Gallery

    The importance of being REF-able: academic writing under pressure from a culture of counting

The importance of being REF-able: academic writing under pressure from a culture of counting

Writing is crucial to an academic’s role of producing, shaping and distributing knowledge. However, academic writing itself is increasingly being shaped by the contemporary university’s managerial practices and evaluation frameworks. Sharon McCulloch describes how her research on academics’ writing practices has revealed tensions around the ways in which managerial practices interact with academics’ individual career goals, disciplinary values and […]

Print Friendly
  • Permalink Gallery

    The organisational contexts in which research with impact is produced: lessons from REF2014

The organisational contexts in which research with impact is produced: lessons from REF2014

What are the organisational contexts in which ‘impactful’ research is produced? Following an empirical analysis of a selection of REF2014 impact case studies, Neil Kellard and Martyna Śliwa discuss the links between impact scores and a variety of important contextual factors. In what might be seen as a challenge to the established hierarchy of HEIs, high scores for research […]

Print Friendly
November 24th, 2016|Impact, REF 2014|1 Comment|
  • Permalink Gallery

    The uneven impacts of research impact: Adjustments needed to address the imbalance of the current impact framework.

The uneven impacts of research impact: Adjustments needed to address the imbalance of the current impact framework.

The current approach to measuring and assessing research impact favours certain kinds of academics and research topics over others. Kat Smith and Ellen Stewart outline three areas that require further consideration. Academics who are negatively impacted by the current framework might look to suggest adjustments which limit or ameliorate these effects.

Academics working in the UK are increasingly encouraged and […]

Print Friendly
July 25th, 2016|Impact, REF 2014|2 Comments|
  • Permalink Gallery

    Peer review and bibliometric indicators just don’t match up according to re-analysis of Italian research evaluation.

Peer review and bibliometric indicators just don’t match up according to re-analysis of Italian research evaluation.

The Italian research evaluation agency undertook an extensive analysis to compare the results of peer review and bibliometric indicators for research evaluation. Their findings suggested both indicators produced similar results. Researchers Alberto Baccini and Giuseppe De Nicolao re-examine these results and find notable disagreements between the two techniques of evaluation in the sample and outline below the major shortcoming in the Italian Agency’s […]

Print Friendly
  • Permalink Gallery

    Mining the REF impact case studies for lessons on leadership, governance and management in Higher Education.

Mining the REF impact case studies for lessons on leadership, governance and management in Higher Education.

The Leadership Foundation for Higher Education’s Director of Research, Professor Fiona Ross CBE commissioned Dr Elizabeth Morrow to mine the 2014 REF impact case studies to learn more about leadership, governance and management (LGM) research. The case study data suggests new possibilities for inter-professional collaboration, and also elucidates different types of impact, which may not necessarily be sequential. Furthermore, given the […]

Print Friendly
  • Permalink OLYMPUS DIGITAL CAMERAGallery

    What impact evidence was used in REF 2014? Disciplinary differences in how researchers demonstrate and assess impact

What impact evidence was used in REF 2014? Disciplinary differences in how researchers demonstrate and assess impact

A new report produced by the Digital Science team explores the types of evidence used to demonstrate impact in REF2014 and pulls together guidance from leading professionals on good practice. Here Tamar Loach and Martin Szomszor present a broad look at the the types of evidence in use in the REF impact case studies and reflect on the association between use of […]

Print Friendly
  • Permalink Gallery

    Evaluating research assessment: Metrics-based analysis exposes implicit bias in REF2014 results.

Evaluating research assessment: Metrics-based analysis exposes implicit bias in REF2014 results.

The recent UK research assessment exercise, REF2014, attempted to be as fair and transparent as possible. However, Alan Dix, a member of the computing sub-panel, reports how a post-hoc analysis of public domain REF data reveals substantial implicit and emergent bias in terms of discipline sub-areas (theoretical vs applied), institutions (Russell Group vs post-1992), and gender. While metrics are […]

Print Friendly
  • Permalink Gallery

    The impact of academia on Parliament: 45 percent of Parliament-focused impact case studies were from social sciences

The impact of academia on Parliament: 45 percent of Parliament-focused impact case studies were from social sciences

How does academic research feed into the parliamentary process? Analysing the impact case studies of the 2014 REF, Caroline Kenny draws out potential lessons on how Parliament is currently engaging with academics, and how it might in the future. Impact case studies referring to Parliament were found in all four of the main subject panels, but 45 per cent […]

Print Friendly
  • Permalink Gallery

    Five reasons the Teaching Excellence Framework is bad news for higher education

Five reasons the Teaching Excellence Framework is bad news for higher education

Jessica Patterson argues the announcement of proposals for a Teaching Excellence Framework (TEF) is bad news indeed for those concerned about a more rapid pace to the marketisation of higher education. Here she outlines five key concerns against a teaching assessment framework given the wider context of greater casualisation and stratification in the workforce and a ‘value for money’ approach […]

Print Friendly
  • Permalink Gallery

    Women’s working lives in the managerial university and the pernicious effects of the ‘normal’ academic career.

Women’s working lives in the managerial university and the pernicious effects of the ‘normal’ academic career.

University faculties need to be able to demonstrate to young people, male and female, that women can be just as inspiring teachers and researchers, and be able to live as enjoyable a domestic life as their male counterparts. Angela McRobbie reflects on how the ideal career track in the academy, suffused with constant benchmarking around ‘excellence’ and the REF’s logic […]

Print Friendly
  • Permalink Gallery

    Was the REF a waste of time? Strong relationship between grant income and quality-related funding allocation.

Was the REF a waste of time? Strong relationship between grant income and quality-related funding allocation.

If the funding allocated to universities on the basis of the REF is correlated to the amount of grant income universities already receive, what is the point of the output assessment process? Jon Clayden explores the relationship between grant income generated and REF-related QR funding and finds a strong correlation between the two, suggesting that the double-counting exercise is surely not the best we […]

Print Friendly
  • Permalink Gallery

    Why did REF2014 cost three times as much as the RAE? Hint: It’s not just because of the added impact element.

Why did REF2014 cost three times as much as the RAE? Hint: It’s not just because of the added impact element.

The benefits of any research assessment framework should ideally outweigh the costs and burden incurred by universities and staff. Derek Sayer argues there should be cause for concern now that recent analysis shows the 2014 REF bill was three times as much as the last UK assessment exercise. The costly increase in staff time was driven by the increased importance […]

Print Friendly
  • Permalink Gallery

    Rather than narrow our definition of impact, we should use metrics to explore richness and diversity of outcomes.

Rather than narrow our definition of impact, we should use metrics to explore richness and diversity of outcomes.

Impact is multi-dimensional, the routes by which impact occur are different across disciplines and sectors, and impact changes over time. Jane Tinkler argues that if institutions like HEFCE specify a narrow set of impact metrics, more harm than good would come to universities forced to limit their understanding of how research is making a difference. But qualitative and quantitative indicators continue […]

Print Friendly
  • Permalink Gallery

    The Management of Metrics: Globally agreed, unique identifiers for academic staff are a step in the right direction.

The Management of Metrics: Globally agreed, unique identifiers for academic staff are a step in the right direction.

The Metric Tide report calls for research managers and administrators to champion the use of responsible metrics within their institutions. Simon Kerridge looks at greater detail at specific institutional actions. Signing up to initiatives such as the San Francisco Declaration on Research Assessment (DORA) is a good start. Furthermore, by mandating unique and disambiguated identifiers for academic staff, like ORCID iDs, […]

Print Friendly
This work by LSE Impact of Social Sciences blog is licensed under a Creative Commons Attribution 3.0 Unported.