As part of Peer Review Week, Rebecca Furlong and Eleanor-Rose Papas draw on their in-depth knowledge of peer review to offer Early Career Researchers (ECRs) advice on how to develop the impact of their research and profile through peer review, both as the reviewed and the reviewer.
Readers can find more LSE Impact blogposts on Peer Review and other posts from Peer Review Week 2023 via the links.
Peer review is, at heart, a process of validation. It builds trust by ensuring that what you are reading has been checked by someone with the right expertise. It is also valuable as a developmental process, reviewers suggest changes that will improve the quality of the final article, which after publication benefits the wider academic community. Yet, peer review is not perfect, it is a human process and humans make mistakes. Given peer review’s importance, the training of peer reviewers, the development of the best-practice rules they should follow, and the recognition of their contribution are critical.
How do you learn to peer review?
Learning to review often works like an apprenticeship. A senior academic (most likely a PhD or early career researcher’s supervisor), would be asked to review a paper. The supervisor then invites their student to participate, takes them through the process or, with the editor’s permission, asks them to take over as reviewer. In this way, they learn what a reviewer should look for and how their comments and concerns should be expressed.
This informal process has been the main way to learn for many years, but it is not equitable or consistent. Only some ECRs are taught to review by someone who is already experienced in peer review. The knowledge gained is likely based for better or worse on the supervisor’s own experience of peer review. Similarly, early career researchers can also learn by reading the reports that they receive on their own research and evaluating how helpful and constructive they were. Yet, again whilst some reports will be extremely constructive, even when the comments are negative, others are less so.
There are other ways to learn. The Committee On Publication Ethics (COPE) provides a clear explanation of the principles and practice of ethical peer review. There are research-focused groups that offer training courses to help people understand what a good peer review report looks like, including the free training provided by amongst other publishers, Taylor & Francis. These courses explore how to evaluate different types of research as a reviewer, as well as outlining which aspects of the report don’t need evaluation (eg. copyediting).
Open peer review reports are also a valuable resource, for individual researchers and teams, as they can be considered in context alongside the article. They can provide reference points for reviewers who may be unsure how to assess certain article types, illustrating how to highlight specific types of concerns, or demonstrating the level of detail that would be most beneficial to the authors.
It can be daunting to write a peer review report openly, however the principles remain the same – the report should be constructive, highlight both the strengths and weaknesses of the manuscript, and have a polite tone. One of the concerns with reviewing openly can be that there are added pressures when reviewing for authors at different career stages. This is a valid concern, but it is also present in other peer review models. Further, anonymisation is not infallible. If someone becomes aware that a power imbalance might affect their ability to review an article objectively, then they should withdraw from the process, regardless of how open the peer review will be. Whether operating in an open or anonymised peer review system, and whether acting as a reviewer or an author, it is vital that those taking part are professional and respectful of the others involved; peer review is a collaborative process.
Recognising and rewarding peer review
No matter their level of experience or training, when a researcher contributes to peer review, it’s important that this is recognised and rewarded as scholarship. When an editor invites someone to peer review, they are saying – implicitly or explicitly – “this individual knows this field and can tell us whether this work is valid or not.” This demonstration of someone’s authority within their field should be recognised, so that it can be appreciated by grant and tenure committees. Peer review therefore needs to be explicitly recognised as something that is an important part of the scholarly ecosystem.
Historically, peer review has been treated as a service between academics that should not be financially compensated. At Taylor & Francis reviewer surveys, have consistently shown that peer review is carried out without the expectation of reward. However, changing business models and strains on the peer review system, have led for calls for a paid approach. This is something we have explored at Taylor and Franics through trials, experimenting with different kinds of rewards such as, free journal access and discounts on publishing services, to more direct compensation as part of Research Square’s Research Quality Evaluation Service. This pilot is still running, but tentative early findings point to researchers continuing to most value the ability to record and gain recognition for reviewing activity.
Publishers however face a conundrum in calling attention to peer review activities. While researchers gain recognition from other parts of their role, the long-standing commitment from publishers to anonymise peer reviews for journals operating traditional single- or double-anonymous peer review makes recognition difficult. Many journals do publish ‘thank you lists’ at the end of the year, printing researchers’ names. Others provide certificates that can be shared. Web of Science and ORCID also have ways of sharing peer review activities without compromising anonymity.
Alternatively, open peer review, such as F1000’s model (where the review and reviewer’s name are published), can be vital in bringing researchers’ effort into view, and therefore demonstrating the contribution that peer reviewers make. Reviewer reports are hidden behind the scenes in more traditional models, and there’s no way to showcase the work and effort that academics put into writing peer review reports.
Looking towards greater diversity in peer review
Publishers cannot work alone in improving the quality and recognition of peer review and the scholarly ecosystem as a whole needs to demonstrate that it sees the worth of this service. Peer review skills are essential and should be trained and nurtured as early as possible, and peer review should be officially considered a fundamental part of one’s role as a researcher. Giving early career researchers the skills and support to take part, recognising and rewarding them for it, and making the process as transparent as possible, all matter in ensuring that peer review remains a key part of the validation of scholarly work.
This emphasis and recognition of peer review is complimented by parallel movements to redefine reward and recognition in academia. For instance, the moves taken to recognise open research practices, as part of Horizon Europe and the OSTP Nelson Memo as well as efforts to reform research assessment to make it less dependent on journal impact factors, such as DORA and more recently CoARA. As part of this trend, peer review is beginning to be recognised as academic scholarship in its own right, with increased focus on training and recognition for ECRs from both publishers and institutions. Open peer review is also opening up new approaches to facilitating and driving this recognition. As the red-thread of integrity that runs from basic research to public communication, research and publishing need to work together to ensure the next generation of confident, collaborative, capable peer reviewers.
The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.
Image Credit: Aditya Wardhana via Unsplash.
