Researchers repeatedly cite career advancement as a key incentive for their practices and behaviours. This is critical to understanding the pace of change in scholarly communications, as those researchers inclined to innovate or experiment with new forms of research outputs, methodologies, or communication styles risk being penalised by the evaluation system used by many research institutions that are slow to adapt to the modern research environment. Sarah Slowe, Gareth Cole, Jon Tennant and Charlie Rapple are gathering data on current promotion and hiring guidelines used throughout Europe and will analyse how these compare to researchers’ attitudes of “publish or perish” and the impact factor as the key determining factor for career advancement. A number of recommendations will follow from this analysis, with the ultimate aim of fostering a more informed evaluation, promotion, and recruitment system for researchers.

Recently, the Scholarly Communications Lab at Simon Fraser University in Vancouver, led by Juan Pablo Alperin, released the first results of a study investigating review, promotion, and tenure criteria in North American research institutes. This project is among the first to provide any large-scale empirical evidence behind these guidelines (you can find the data here). Such research is critical in understanding factors like the adoption of open access and “open scientific” practices, the reality of the grip of journal brands and impact factors, and a range of other incentives in scholarly communications.

One key finding is that just 5% of the guideline documents mentioned open access at all, or anything to do with the mechanisms behind it, and many of those seemed to equate it with the murky world of predatory publishing. This seems strange for a world which is rapidly moving towards an open access future.

Inspired by the work of Juan and his colleagues, the four of us recently submitted a proposal to the competitive Scholarly Communication Institute’s TriangleSCI event, which this year is on the theme of “Overcoming Risk”. Our proposal was focused around conducting similar research in Europe, to help inform discussions and policy developments in this arena across the region. However, sadly we did not make the cut, and were left thinking about what to do with the proposal. Well, we thought, why not release it to the wild and see what sort of feedback and response we get from people? The LSE Impact Blog seems as a good a place to do that as any, and, well, here we are. And here is our pitch.

Career advancement through promotion, recruitment, grant awards, and research evaluation procedures is repeatedly mentioned as one of the key incentives for researchers and their practices and behaviours. Nearly every conversation about changes in scholarly communication ultimately ends up at this point; in particular, highlighting research institutions that are slow to adapt to changes in modern research environments, and how the present system penalises those who innovate or adopt novel research communication styles, methodologies, and reporting outside of traditional disciplinary structures.

This has, reportedly, led to those pioneering new routes to achieving goals to feel undervalued or to see these as an “extra” or “add-on” interest rather than as part of their core academic role, as well as stifling experimentation with new forms of scholarly communication. These procedures are often reviewed annually at institutional level, but changes being (as we so far understand) incremental and relatively minor, with more fundamental reviews carried out rarely.

Image credit: Christopher Burns, via Unsplash (licensed under a CC0 1.0 license).

In a recent study, Moore et al., (2016) argued that “it is clear that evaluative cultures are operating without even internal consensus beyond a few broad categories of performance”. This highlighted that, while there is a key push towards excellence, how that is measured or demonstrated is rarely defined. However, there is a general perception among research communities that one of the principle factors involved in this is that of journal brands or ranks, and their relative impact factors. How much this perception is grounded in reality is, however, largely unknown, due to a lack of quantitative evidence. Preliminary results from North America indicated that only 20% of institutes actually mentioned impact factors in their guidelines – much less than we would expect given the ubiquity with which it is often assumed.

There is also a perceived risk for institutions in moving away from a “standard” position. For institutions recruiting globally, and in competition for researchers with other equivalent institutions, there is an understanding that promotions criteria are broadly similar and thus unlikely to deter a potential researcher from accepting a position.

We want to conduct a research project to provide the data to confirm or refute the assumption that promotions criteria are comparable across the sector, and provide the basis for recommendations to help us move to a new status quo. While we recognise not everyone conforms to such criteria and guidelines, these nonetheless provide a useful indicator of what might be the cause of such behaviours, and whether or not practices are reflective of such guidelines.

Thus, our proposal is to address the following key question: what models or practices could be developed to help incentivise and reward innovation and diverse forms of scholarly communication and public engagement, while reducing the risk to those who are seen to be diverging from traditional modes of professional practices and advancement?

For this, we will seek to provide a rigorous evidence base which will allow us to:

• Evaluate current promotion and hiring guidelines throughout Europe
• Identify key aspects related to scholarly communication
• Identify how similar this is to researcher attitudes of “publish or perish” and the impact factor being the key determining factor for career advancement
• Provide recommendations for concrete steps to take us beyond the impact factor and traditional outputs
• Ultimately, create a more informed system for how researchers are evaluated, recruited, and promoted.

As the basis for this, our aim is to firstly collect and review a range of hiring and promotion documents from across European higher education institutes and conduct a semantic analysis of the different traits and aspects related to scholarly communication included in them. This will lead us to identify what the current incentives are and the effect they have on scholarly innovation. Our intention is to replicate similar research currently being conducted at Simon Fraser University and provide a European counterpart to that project. We will also endeavour to provide insight into the unintended exclusion resulting from a narrow view of “what counts” as a research output for promotion, and how this is reflected in the architecture of institutional repositories, which often only cater to certain research output types.

In developing a more evidence-informed understanding of the current situation and processes, we can begin to identify best practices in scholarly communication, and align these with best practices in career advancement for those involved in administering and reviewing such procedures. This can, in turn, help us to develop a more intimate understanding of research cultures around career progression, and the relationship these have with evolving academic practices. Ultimately, our aim would be to translate these into systemic behavioural changes that lead to a greater valuing of a diverse range of processes and outputs in the research landscape.

Call for action

We believe that such an important topic deserves cross-stakeholder engagement. This is why we, as people who represent a range of difference perspectives and experiences from across scholarly communications, invite you to be involved. It does not matter if you are a publisher, student, researcher, librarian, or policymaker – all it takes to be involved is a desire to help understand and reform the research evaluation system in Europe.

If you would like to find out more, or join our little team, please get in touch! You can contact us at osc-erepe@kent.ac.uk.

Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.

About the authors

Sarah Slowe is Head of the Office for Scholarly Communication at the University of Kent, working across departments to deliver comprehensive research support. This includes working to continue to improve the current provision for open access and research data management, and working with researchers to communicate their research to the desired audience in the most effective ways. Sarah tweets @HeadUniKentOSC, and her ORCID iD is 0000-0002-4951-8834.

Gareth Cole is Research Data Manager at Loughborough University. He is responsible for establishing a research data support service, including advising on data management plans and digital preservation, and managing the Loughborough Data Repository.  Gareth also advises researchers on funder and publisher policies and works closely with open access colleagues to advise researchers on OA routes. He tweets @DrGarethCole and his ORCID iD is 0000-0001-7493-0137.

Jon Tennant is an independent researcher and scientific communication consultant. He is the founder of the Open Science MOOC and the preprint server, paleorXiv. Jon is also the author of numerous kids’ books and for scientific news outlets, Executive Editor of the new journal Geoscience Communication, and Community Editor for the PLOS Paleo network. He completed his PhD at Imperial College London and his research looks at deep time evolutionary patterns in groups like dinosaurs and crocodiles. Jon tweets @protohedgehog and his ORCID iD is 0000-0001-7794-0218.

Charlie Rapple is Co-founder of Kudos, helping researchers and their publishers and institutions to accelerate the impact of published research by providing a platform and framework for more strategically planning, managing, measuring, and reporting on research dissemination efforts. Charlie is a communications expert with 20 years’ experience in the scholarly communications sector, including in leadership roles for UKSG, editorial board positions for the UKSG Insights and Learned Publishing journals, and blogging for the Scholarly Kitchen. She tweets @charlierapple and her ORCID iD is 0000-0001-7933-1029.