• Permalink Gallery

    Making visible the impact of researchers working in languages other than English: developing the PLOTE index

Making visible the impact of researchers working in languages other than English: developing the PLOTE index

As outlined in the Leiden Manifesto, if impact is understood in terms of citations to international publications, a bias is created against research which is regionally focused and engaged with local society problems. This is particularly critical for researchers working in contexts with languages other than English. Peter Dahler-Larsen has developed the PLOTE index, a new indicator which hopes […]

Print Friendly
  • Permalink Gallery

    Six principles for assessing scientists for hiring, promotion, and tenure

Six principles for assessing scientists for hiring, promotion, and tenure

The negative consequences of relying too heavily on metrics to assess research quality are well known, potentially fostering practices harmful to scientific research such as p-hacking, salami science, or selective reporting. The “flourish or perish” culture defined by these metrics in turn drives the system of career advancement in academia, a system that empirical evidence has shown to be […]

Print Friendly
  • Permalink Gallery

    Against metrics: how measuring performance by numbers backfires

Against metrics: how measuring performance by numbers backfires

A proliferation of companies, government agencies, and higher education institutions are in the grip of what Jerry Z. Muller has termed “metric fixation”. But by tying rewards to metrics, organisations risk incentivising gaming and encouraging behaviours that may be at odds with their larger purpose. The culture of short-termism engendered by metrics also impedes innovation and stifles the entrepreneurial […]

Print Friendly
  • Permalink Gallery

    The academic papers researchers regard as significant are not those that are highly cited

The academic papers researchers regard as significant are not those that are highly cited

For many years, academia has relied on citation count as the main way to measure the impact or importance of research, informing metrics such as the Impact Factor and the h-index. But how well do these metrics actually align with researchers’ subjective evaluation of impact and significance? Rachel Borchardt and Matthew R. Hartings report on a study that compares […]

Print Friendly
  • Permalink Gallery

    Beyond Impact Factors: an Academy of Management report on measuring scholarly impact

Beyond Impact Factors: an Academy of Management report on measuring scholarly impact

What constitutes scholarly impact? And which stakeholders have importance for research? Usha Haley shares findings of a recent Academy of Management report that sought answers to these questions by surveying its 20,000 members and conducting a selection of in-depth interviews with prominent figures. A majority of respondents indicated journal rankings did not reflect scholarly impact, yet publications in top-tier […]

Print Friendly
  • Permalink Gallery

    Where are we with responsible metrics? And where might we go next? Reflections from two recent events

Where are we with responsible metrics? And where might we go next? Reflections from two recent events

Widespread scepticism and concern among researchers, universities, representative bodies and learned societies about the broader use of metrics in research assessment and management has led to concerted efforts to promote the “responsible use” of such metrics. But how effectively are UK higher education institutions engaging with this agenda? Lizzie Gadd reflects on two recent responsible metrics-themed events. While it […]

Print Friendly
  • Permalink Gallery

    From invisibility to impact: radically different measures are needed to capture the true impact of research

From invisibility to impact: radically different measures are needed to capture the true impact of research

Academics are increasingly expected to produce directly applicable solutions to hard-to-solve “real-world” problems such as poverty, development, and environmental degradation. However, conventional assessments of science have not yet been adequately adapted to capture the diverse effects of this type of problem-centred research. Examining a prominent recent example of multidisciplinary research on consumption, environment and sustainability in Ireland, Henrike Rau, […]

Print Friendly
  • Permalink Gallery

    Analysing Altmetric data on research citations in policy literature – the case of the University of Sheffield

Analysing Altmetric data on research citations in policy literature – the case of the University of Sheffield

One of the sources of attention Altmetric.com tracks is the number of times research outputs have been cited in policy literature. Andy Tattersall and Chris Carroll explored the case of the University of Sheffield and what the data says about the impact of its research on national and international policy. The percentage of outputs with at least one policy […]

Print Friendly
  • Permalink Gallery

    Quantity does matter as citation impact increases with productivity

Quantity does matter as citation impact increases with productivity

Many scholars are encouraged to focus on the quality not the quantity of their publications, the rationale being that becoming too focused on productivity risks reducing the quality of one’s work. But is this, in fact, the case? Peter van den Besselaar and Ulf Sandström have studied a large sample of researchers and found that, while results vary by […]

Print Friendly
  • Permalink Gallery

    Why has no other European country adopted the Research Excellence Framework?

Why has no other European country adopted the Research Excellence Framework?

Most European countries have followed the UK’s lead in developing performance-based research funding systems (PRFS) for their universities. However, what these countries have not done is adopt the same system, the Research Excellence Framework being its most recent iteration. Instead, many use indicators of institutional performance for funding decisions rather than panel evaluation and peer review. Gunnar Sivertsen has […]

Print Friendly

2017 in review: top posts of the year

As 2017 nears its end and before our focus is drawn to whatever the new year might have in store, now is the perfect time to look back and reflect on the last twelve months on the Impact Blog. Editor Kieran Booluck reports on another year in which our readership has grown, and also shares a selection of the […]

Print Friendly

2017 in review: round-up of our top posts on metrics

Mendeley reader counts offer early evidence of the scholarly impact of academic articles
Although the use of citation counts as indicators of scholarly impact has well-documented limitations, it does offer insight into what articles are read and valued. However, one major disadvantage of citation counts is that they are slow to accumulate. Mike Thelwall has examined reader counts from Mendeley and found them […]

Print Friendly
  • Permalink Gallery

    Where are the rising stars of research working? Towards a momentum-based look at research excellence

Where are the rising stars of research working? Towards a momentum-based look at research excellence

Traditional university rankings and leaderboards are largely an indicator of past performance of academic staff, some of whom conducted the research for which they are most famous elsewhere. Paul X. McCarthy has analysed bibliometric data to see which research institutions are accelerating fastest in terms of output and impact. The same data also offers a glimpse into the future, […]

Print Friendly
  • Permalink Gallery

    Metrics, recognition, and rewards: it’s time to incentivise the behaviours that are good for research and researchers

Metrics, recognition, and rewards: it’s time to incentivise the behaviours that are good for research and researchers

Researchers have repeatedly voiced their dissatisfaction with how the journals they publish in are used as a proxy for the evaluation of their work. However, those who wish to break free of this model fear negative consequences for their future funding and careers. Rebecca Lawrence emphasises the importance of addressing researchers’ recognition and reward structures, arguing it is time […]

Print Friendly
  • Permalink Gallery

    Better information on teaching is required to redress the balance with research

Better information on teaching is required to redress the balance with research

How universities allocate resources – and how academics allocate their own time – between research and teaching is a perennial problem in higher education. The labour market for research is intensely competitive and truly global; while the market for academics focused on teaching is notable by its lack of competition. An obvious result is that academics’ promotion prospects depend […]

Print Friendly
  • Permalink Gallery

    Does research evaluation in the sciences have a gender problem? What do altmetrics tell us?

Does research evaluation in the sciences have a gender problem? What do altmetrics tell us?

Many measures used for research evaulation, such as citations or research output, are hindered by an implicit gender bias. Stacy Konkiel examines whether or not altmetrics, which track how research is discussed, shared, reviewed, and reused by other researchers and the public, might be better suited to help understand the influence of research in a more gender-balanced way. Findings […]

Print Friendly
  • Permalink Gallery

    Reshaping the tenure and promotion process so that it becomes a catalyst for innovative and invigorating scholarship

Reshaping the tenure and promotion process so that it becomes a catalyst for innovative and invigorating scholarship

The metrics used to identify excellence, and on which current tenure and promotion decisions are based, have become a barrier to more exciting and innovative scholarship. Christopher P. Long suggests an overhaul of tenure and promotion practices, advocating a holistic approach in which structured mentoring plays a key role and values-based metrics that will empower faculty to tell more […]

Print Friendly
  • Permalink Gallery

    The methodology used for the Times Higher Education World University Rankings’ citations metric can distort benchmarking

The methodology used for the Times Higher Education World University Rankings’ citations metric can distort benchmarking

The Times Higher Education World University Rankings can influence an institution’s reputation and even its future revenues. However, Avtar Natt argues that the methodology used to calculate its citation metrics can have the effect of distorting benchmarking exercises. The fractional counting approach applied to only a select number of papers with high author numbers has led to a situation […]

Print Friendly
  • Permalink Gallery

    Research assessments based on journal rankings systematically marginalise knowledge from certain regions and subjects

Research assessments based on journal rankings systematically marginalise knowledge from certain regions and subjects

Many research evaluation systems continue to take a narrow view of excellence, judging the value of work based on the journal in which it is published. Recent research by Diego Chavarro, Ismael Ràfols and colleagues shows how such systems underestimate and prove detrimental to the production of research relevant to important social, economic, and environmental issues. These systems also reflect the […]

Print Friendly
This work by LSE Impact of Social Sciences blog is licensed under a Creative Commons Attribution 3.0 Unported.