• Permalink Gallery

    Rather than promoting economic value, evaluation can be reclaimed by universities to combat its misuse and negative impacts

Rather than promoting economic value, evaluation can be reclaimed by universities to combat its misuse and negative impacts

To critics across higher education, evaluation frameworks such as the REF and TEF represent mechanisms of control, the generation of a “target and terror” culture. Deirdre Duffy explains how the REF and TEF resonate most closely with impact evaluation, a form of evaluation that can prove useful for a simple cost-benefit analysis but can also be problematic as it […]

Print Friendly
  • Permalink Gallery

    Credit for research outputs should go to the originating institution but with a transitional arrangement for this REF cycle

Credit for research outputs should go to the originating institution but with a transitional arrangement for this REF cycle

One of the most contentious aspects of the Stern review of the 2014 REF was the recommendation that research outputs should not be portable in future exercises. The subsequent consultation revealed a significant minority to be in support of this, echoing Stern’s concerns that current rules distort investment incentives and encourage rent-seeking. However, a majority opposed this recommendation as […]

Print Friendly
  • Permalink Gallery

    If academics are serious about research impact they need to learn from monitoring, evaluation and learning teams

If academics are serious about research impact they need to learn from monitoring, evaluation and learning teams

The impact of academic research, particularly on policy and the private sector, is an increasingly important component of research assessment exercises and funding distribution. However, Duncan Green argues that the way many researchers think about their impact continues to be pretty rudimentary. A lack of understanding of who key decision-makers are, a less-than-agile response to real-world events, and difficulties […]

Print Friendly
  • Permalink Gallery

    “This device is licensed”: the material and immaterial bureaucracy of academic research

“This device is licensed”: the material and immaterial bureaucracy of academic research

Derek Dunne draws attention to the hidden bureaucratic labour that is increasingly a part of academic life. Rather than see this as the “white noise” to be tuned out of everyday working practices, he calls for us to question the forms that are put in front of us demanding our acquiescence, whilst also locating potential sites of resistance.

 

“This device is licensed.”

I […]

Print Friendly
  • Permalink Gallery

    The REF’s focus on linear and direct impact is problematic and silences certain types of research

The REF’s focus on linear and direct impact is problematic and silences certain types of research

In the last Research Excellence Framework (REF), the new element of research impact was understood in very linear and direct terms. Aoileann Ní Mhurchú, Laura McLeod, Stephanie Collins and Gabriel Siles-Brügge consider how accepted definitions of impact may have had the effect of silencing certain types of research. Research and impact should be seen as a two-way street, where […]

Print Friendly
  • Permalink Gallery

    Making your Marx in research: Reflections on impact and the efficacy of case studies using the work of Karl Marx.

Making your Marx in research: Reflections on impact and the efficacy of case studies using the work of Karl Marx.

Drawing from a recent study on how impact occurs in the social sciences, Sioned Pearce looks at some specific issues with the case study approach to understanding impact. Viewed alongside the life and works of Karl Marx, the REF’s approach to impact measurement can be seen as highly problematic. Marx’s work was an accumulation of a lifetime of intellectual thought, the […]

Print Friendly
  • Permalink Gallery

    What makes research excellent? Digging into the measures aimed at quantifying and promoting research excellence.

What makes research excellent? Digging into the measures aimed at quantifying and promoting research excellence.

“Research excellence” is a tricky concept in theory and arguably trickier to capture in practice. Toni Pustovrh shares findings from a recent study which looks at how research is currently quantified and evaluated in Slovenia. In-depth interviews with scientists reveal a variety of views on the concept and the current mechanisms in place. The analysis suggests that neither a predominantly peer-review based […]

Print Friendly

Making space for the academic book of the future

Yesterday Steven Hill spoke at the University Press Redux conference in Liverpool on the role of policy in shaping the academic book of the future. This post is a summary of the argument of his talk. While there is much potential for innovation, the uptake seems slow. Even when we see the benefits of innovation, the change in process […]

Print Friendly
  • Permalink Gallery

    Ancient Cultures of Conceit Reloaded? A comparative look at the rise of metrics in higher education.

Ancient Cultures of Conceit Reloaded? A comparative look at the rise of metrics in higher education.

When considering the power of metrics and audit culture in higher education, are we at risk of romanticising the past? Have academics ever really worked in an environment free from ‘measurement’? Roger Burrows draws on his own recollection of the 1986 Research Selectivity Exercise (RSE), scholarly work on academic labour and fictional portrayals of academic life, which all demonstrate the substantial expansion of the role […]

Print Friendly
  • Permalink Gallery

    Top ten tips for universities seeking to implement Open Access

Top ten tips for universities seeking to implement Open Access

With funders requiring open access and researchers increasingly aware of it, now is the time for universities to make significant headway in providing a coherent plan for encouraging wider open access adoption. Neil Jacobs from Jisc provides an overview of what actions have been taken around the sector and outlines ten specific areas that institutions should consider further in order […]

Print Friendly
  • Permalink Gallery

    Was the REF a waste of time? Strong relationship between grant income and quality-related funding allocation.

Was the REF a waste of time? Strong relationship between grant income and quality-related funding allocation.

If the funding allocated to universities on the basis of the REF is correlated to the amount of grant income universities already receive, what is the point of the output assessment process? Jon Clayden explores the relationship between grant income generated and REF-related QR funding and finds a strong correlation between the two, suggesting that the double-counting exercise is surely not the best we […]

Print Friendly
  • Permalink Gallery

    Rather than narrow our definition of impact, we should use metrics to explore richness and diversity of outcomes.

Rather than narrow our definition of impact, we should use metrics to explore richness and diversity of outcomes.

Impact is multi-dimensional, the routes by which impact occur are different across disciplines and sectors, and impact changes over time. Jane Tinkler argues that if institutions like HEFCE specify a narrow set of impact metrics, more harm than good would come to universities forced to limit their understanding of how research is making a difference. But qualitative and quantitative indicators continue […]

Print Friendly
  • Permalink Gallery

    Using REF results to make simple comparisons is not necessarily responsible. Careful interpretation needed.

Using REF results to make simple comparisons is not necessarily responsible. Careful interpretation needed.

What are the implications of the HEFCEmetrics review for the next REF? Is is easy to forget that the REF is already all about metrics of research performance. Steven Hill, Head of Research Policy at HEFCE, reiterates that like any use of metrics, we need to take great care in how we use and interpret the results of the REF.

This is […]

Print Friendly
  • Permalink Gallery

    The Management of Metrics: Globally agreed, unique identifiers for academic staff are a step in the right direction.

The Management of Metrics: Globally agreed, unique identifiers for academic staff are a step in the right direction.

The Metric Tide report calls for research managers and administrators to champion the use of responsible metrics within their institutions. Simon Kerridge looks at greater detail at specific institutional actions. Signing up to initiatives such as the San Francisco Declaration on Research Assessment (DORA) is a good start. Furthermore, by mandating unique and disambiguated identifiers for academic staff, like ORCID iDs, […]

Print Friendly
  • Permalink Gallery

    The Academic Book of the Future: exploring academic practices and expectations for the monograph.

The Academic Book of the Future: exploring academic practices and expectations for the monograph.

What does the future hold for academic books? Rebecca Lyons introduces The Academic Book of the Future, a two-year project funded by the AHRC in collaboration with the British Library in which a cross-disciplinary team from University College London and King’s College London explores how scholarly work in the Arts and Humanities will be produced, read, shared, and preserved […]

Print Friendly
  • Permalink Gallery

    Time to abandon the gold standard? Peer review for the REF falls far short of internationally accepted standards.

Time to abandon the gold standard? Peer review for the REF falls far short of internationally accepted standards.

The REF2014 results are set to be published next month. Alongside ongoing reviews of research assessment, Derek Sayer points to the many contradictions of the REF. Metrics may have problems, but a process that gives such extraordinary gatekeeping power to individual panel members is far worse. Ultimately, measuring research quality is fraught with difficulty. Perhaps we should instead be asking which […]

Print Friendly
  • Permalink Gallery

    Evaluation systems need not be perfect: University research assessment and the ongoing quest for simplicity.

Evaluation systems need not be perfect: University research assessment and the ongoing quest for simplicity.

In order to get a perfect assessment method, are we at risk of developing systems that are ever more complex and time-consuming? Dorothy Bishop looks at the differences between readily available measures to award research funding and the highly complicated RAE formula. An evaluation system need not be perfect – it just needs to be ‘good enough’ to provide a […]

Print Friendly
  • Permalink Gallery

    Five recommendations for using alternative metrics in the future UK Research Excellence Framework

Five recommendations for using alternative metrics in the future UK Research Excellence Framework

Although many are excited by the possibilities for using alternative metrics to supplement research assessment, others are concerned about the ease with which the figures can be gamed. It is clear that there is already gaming within traditional citation impact metrics in peer reviewed journals and without quality control mechanisms, social media metrics would be susceptible to the same. Mike […]

Print Friendly
  • Permalink Gallery

    The rejection of metrics for the REF does not take account of existing problems of determining research quality.

The rejection of metrics for the REF does not take account of existing problems of determining research quality.

Amidst heavy scepticism over the role of metrics in research assessments, Martin Smith wonders whether the flaws of the current system have been fully recognised. There is no system of research assessment that is perfect and peer review may well be a better, although problematic, measure of quality than metrics. But the REF has become disproportionate. The question that arises […]

Print Friendly
This work by LSE Impact of Social Sciences blog is licensed under a Creative Commons Attribution 3.0 Unported.