• Permalink Gallery

    Beyond Impact Factors: an Academy of Management report on measuring scholarly impact

Beyond Impact Factors: an Academy of Management report on measuring scholarly impact

What constitutes scholarly impact? And which stakeholders have importance for research? Usha Haley shares findings of a recent Academy of Management report that sought answers to these questions by surveying its 20,000 members and conducting a selection of in-depth interviews with prominent figures. A majority of respondents indicated journal rankings did not reflect scholarly impact, yet publications in top-tier […]

Print Friendly
  • Permalink Gallery

    Where are we with responsible metrics? And where might we go next? Reflections from two recent events

Where are we with responsible metrics? And where might we go next? Reflections from two recent events

Widespread scepticism and concern among researchers, universities, representative bodies and learned societies about the broader use of metrics in research assessment and management has led to concerted efforts to promote the “responsible use” of such metrics. But how effectively are UK higher education institutions engaging with this agenda? Lizzie Gadd reflects on two recent responsible metrics-themed events. While it […]

Print Friendly
  • Permalink Gallery

    From invisibility to impact: radically different measures are needed to capture the true impact of research

From invisibility to impact: radically different measures are needed to capture the true impact of research

Academics are increasingly expected to produce directly applicable solutions to hard-to-solve “real-world” problems such as poverty, development, and environmental degradation. However, conventional assessments of science have not yet been adequately adapted to capture the diverse effects of this type of problem-centred research. Examining a prominent recent example of multidisciplinary research on consumption, environment and sustainability in Ireland, Henrike Rau, […]

Print Friendly
  • Permalink Gallery

    Analysing Altmetric data on research citations in policy literature – the case of the University of Sheffield

Analysing Altmetric data on research citations in policy literature – the case of the University of Sheffield

One of the sources of attention Altmetric.com tracks is the number of times research outputs have been cited in policy literature. Andy Tattersall and Chris Carroll explored the case of the University of Sheffield and what the data says about the impact of its research on national and international policy. The percentage of outputs with at least one policy […]

Print Friendly
  • Permalink Gallery

    Quantity does matter as citation impact increases with productivity

Quantity does matter as citation impact increases with productivity

Many scholars are encouraged to focus on the quality not the quantity of their publications, the rationale being that becoming too focused on productivity risks reducing the quality of one’s work. But is this, in fact, the case? Peter van den Besselaar and Ulf Sandström have studied a large sample of researchers and found that, while results vary by […]

Print Friendly
  • Permalink Gallery

    Why has no other European country adopted the Research Excellence Framework?

Why has no other European country adopted the Research Excellence Framework?

Most European countries have followed the UK’s lead in developing performance-based research funding systems (PRFS) for their universities. However, what these countries have not done is adopt the same system, the Research Excellence Framework being its most recent iteration. Instead, many use indicators of institutional performance for funding decisions rather than panel evaluation and peer review. Gunnar Sivertsen has […]

Print Friendly

2017 in review: top posts of the year

As 2017 nears its end and before our focus is drawn to whatever the new year might have in store, now is the perfect time to look back and reflect on the last twelve months on the Impact Blog. Editor Kieran Booluck reports on another year in which our readership has grown, and also shares a selection of the […]

Print Friendly

2017 in review: round-up of our top posts on metrics

Mendeley reader counts offer early evidence of the scholarly impact of academic articles
Although the use of citation counts as indicators of scholarly impact has well-documented limitations, it does offer insight into what articles are read and valued. However, one major disadvantage of citation counts is that they are slow to accumulate. Mike Thelwall has examined reader counts from Mendeley and found them […]

Print Friendly
  • Permalink Gallery

    Where are the rising stars of research working? Towards a momentum-based look at research excellence

Where are the rising stars of research working? Towards a momentum-based look at research excellence

Traditional university rankings and leaderboards are largely an indicator of past performance of academic staff, some of whom conducted the research for which they are most famous elsewhere. Paul X. McCarthy has analysed bibliometric data to see which research institutions are accelerating fastest in terms of output and impact. The same data also offers a glimpse into the future, […]

Print Friendly
  • Permalink Gallery

    Metrics, recognition, and rewards: it’s time to incentivise the behaviours that are good for research and researchers

Metrics, recognition, and rewards: it’s time to incentivise the behaviours that are good for research and researchers

Researchers have repeatedly voiced their dissatisfaction with how the journals they publish in are used as a proxy for the evaluation of their work. However, those who wish to break free of this model fear negative consequences for their future funding and careers. Rebecca Lawrence emphasises the importance of addressing researchers’ recognition and reward structures, arguing it is time […]

Print Friendly
  • Permalink Gallery

    Better information on teaching is required to redress the balance with research

Better information on teaching is required to redress the balance with research

How universities allocate resources – and how academics allocate their own time – between research and teaching is a perennial problem in higher education. The labour market for research is intensely competitive and truly global; while the market for academics focused on teaching is notable by its lack of competition. An obvious result is that academics’ promotion prospects depend […]

Print Friendly
  • Permalink Gallery

    Does research evaluation in the sciences have a gender problem? What do altmetrics tell us?

Does research evaluation in the sciences have a gender problem? What do altmetrics tell us?

Many measures used for research evaulation, such as citations or research output, are hindered by an implicit gender bias. Stacy Konkiel examines whether or not altmetrics, which track how research is discussed, shared, reviewed, and reused by other researchers and the public, might be better suited to help understand the influence of research in a more gender-balanced way. Findings […]

Print Friendly
  • Permalink Gallery

    Reshaping the tenure and promotion process so that it becomes a catalyst for innovative and invigorating scholarship

Reshaping the tenure and promotion process so that it becomes a catalyst for innovative and invigorating scholarship

The metrics used to identify excellence, and on which current tenure and promotion decisions are based, have become a barrier to more exciting and innovative scholarship. Christopher P. Long suggests an overhaul of tenure and promotion practices, advocating a holistic approach in which structured mentoring plays a key role and values-based metrics that will empower faculty to tell more […]

Print Friendly
  • Permalink Gallery

    The methodology used for the Times Higher Education World University Rankings’ citations metric can distort benchmarking

The methodology used for the Times Higher Education World University Rankings’ citations metric can distort benchmarking

The Times Higher Education World University Rankings can influence an institution’s reputation and even its future revenues. However, Avtar Natt argues that the methodology used to calculate its citation metrics can have the effect of distorting benchmarking exercises. The fractional counting approach applied to only a select number of papers with high author numbers has led to a situation […]

Print Friendly
  • Permalink Gallery

    Research assessments based on journal rankings systematically marginalise knowledge from certain regions and subjects

Research assessments based on journal rankings systematically marginalise knowledge from certain regions and subjects

Many research evaluation systems continue to take a narrow view of excellence, judging the value of work based on the journal in which it is published. Recent research by Diego Chavarro, Ismael Ràfols and colleagues shows how such systems underestimate and prove detrimental to the production of research relevant to important social, economic, and environmental issues. These systems also reflect the […]

Print Friendly

Clickbait and impact: how academia has been hacked

It has become increasingly clear that prevailing academic incentive structures have a potentially damaging and distorting effect on the nature of academic debates. Portia Roelofs and Max Gallien use the example of a controversial recent journal publication to illustrate how deliberately provocative articles have the capacity to hack academia, to privilege clicks and attention over rigour in research. This […]

Print Friendly
  • Permalink Gallery

    Collaborative researcher behaviour has not (yet) increased in response to incentive regimes’ performance measures

Collaborative researcher behaviour has not (yet) increased in response to incentive regimes’ performance measures

A somewhat cynical view of researcher motivations suggests that, when faced with new quantitative performance measures as part of their local incentive regimes, researchers will quickly modify their behaviours in an effort to optimise their own performance. Charlotte Wien, Bertil F. Dorch and Asger Væring Larsen set about testing this notion, looking more closely at how their own Danish […]

Print Friendly
  • Permalink Gallery

    Rather than promoting economic value, evaluation can be reclaimed by universities to combat its misuse and negative impacts

Rather than promoting economic value, evaluation can be reclaimed by universities to combat its misuse and negative impacts

To critics across higher education, evaluation frameworks such as the REF and TEF represent mechanisms of control, the generation of a “target and terror” culture. Deirdre Duffy explains how the REF and TEF resonate most closely with impact evaluation, a form of evaluation that can prove useful for a simple cost-benefit analysis but can also be problematic as it […]

Print Friendly
This work by LSE Impact of Social Sciences blog is licensed under a Creative Commons Attribution 3.0 Unported.