In case you missed it the past week has been peer review week. This year’s theme has been the intersection of innovation and technology.
In recognition of this theme we published two pieces on the LSE Impact Blog, the first post by Adrian Barnett, focused on the very human element of doubt that occurs when peer reviewers choose how to review papers and proposes that randomisation can function as an innovative means of acknowledging this uncertainty in peer review and purging hidden bias from these processes.
The second post by Simone Ragavooloo explores the rapid growth in scientific publications that is occurring as a result of the use of generative AI tools and the implications this has for peer review. Which, she argues necessitates the need for more AI peer review tools in order to free up reviewers expert judgement for more critical tasks.
However, as peer review week is only once a year, we thought we would return to some of the best posts on peer review published over the last few years on the blog. You can find all of our posts on peer review here, but this selection brings together posts that emphasise both the technological and social innovations and practices that underpin peer review. Enjoy!
Lack of experimentation has stalled the debate on open peer review
Open peer review is often discussed more in theory than practice. Drawing on evidence from a recent systematic review of open peer review studies, Tony Ross-Hellauer and Serge P.J.M. Horbach find many persistent questions around open peer review remain poorly examined and call for a more experimental approach to open peer review practices.
Self-organising peer review for preprints – A future paradigm for scholarly publishing
Preprints – rapidly published non peer reviewed research articles – are becoming an increasingly common fixture in scholarly communication. However, without being peer reviewed they serve a limited function, as they are often not recognised as high quality research publications. In this post Wang Ling Feng discusses how the development of preprint servers as self-organising peer review platforms could be the future of scholarly publication.
To ensure the quality of peer reviewed research introduce randomness
Journals play an important role in signalling the quality of academic research. This quality is often linked to measures such as the journal impact factor. However, these measures often obscure the overall quality of research papers in a journal. In this post, Margit Osterloh and Bruno Frey argue that the overall quality and originality of published academic research can be improved by introducing randomness into the peer review process.
Can generative AI add anything to academic peer review?
Generative AI applications promise efficiency and can benefit the peer review process. But given their shortcomings and our limited knowledge of their innerworkings, Mohammad Hosseini and Serge P.J.M. Horbach argue they should not be used independently nor indiscriminately across all settings. Focusing on recent developments, they suggest the grant peer review process is among contexts that generative AI should be used very carefully, if at all.
The best peer review reports are at least 947 words
Based on an analysis of the relationship between peer review reports and subsequent citations, Abdelghani Maddi argues that longer and hence more constructive and engaged peer review reports are closely associated with papers that are more cited.
Peer review is not just quality control, it is part of the social infrastructure of research
The purpose of peer review is often portrayed as being a simple ‘objective’ test of the soundness or quality of a research paper. However, it also performs other functions primarily through linking and developing relationships between networks of researchers. In this post, Flaminio Squazzoni explores these interconnections and argues that to understand peer review as simply an exercise in quality control is to be blind to the historical, political and social dimensions of peer review.
There are four schools of thought on reforming peer review – can they co-exist?
Outlining their recent research into the different interests and commitments of groups looking to reform and improve scientific peer review, Ludo Waltman, Wolfgang Kaltenbrunner, Stephen Pinfield, and Helen Buckley Woods identify four schools of thought on the subject. Discussing their different aims and objectives, they highlight commonalities between them and also key areas in which they diverge. They suggest that in understanding these positions, it opens space for the purposeful inclusion of more varied forms of peer review for research.
For Epistemic Respect – Against Reviewer 2
Despite the efforts of journals and editors to the contrary, the well-known academic folk-devil, Reviewer 2 continues to make the lives of researchers miserable. Gorgi Krlev and Andre Spicer draw on a recent encounter with reviewer 2 and the subsequent twitterstorm over what counts as epistemic respect. They explain what epistemic respect means, what needs to change in academia to uphold it, and why this is more than a question of good manners, but is essential to an inclusive and innovative research system.
Is peer review bad for your mental health?
Amidst fears of a mental health crisis in higher education, to what extent is the peer review process a contributing factor? It’s a process fraught with uncertainty, as authors try to forge something constructive from often mixed feedback or occasionally downright unhelpful comments. Helen Kara stresses the importance of being aware of the effects of uncertainty and taking steps to reduce its impact. Focus on what you can control, prepare for different outcomes, acknowledge how you’re feeling, and make sure to practise self-care.
One of the first megajournals, PLOS ONE, has played a significant role in changing scholarly communication and in particular peer review, by placing an emphasis on soundness, as opposed to novelty, in published research. Drawing on a study of peer review reports from PLOS ONE recently published as an open-access book, Martin Paul Eve, Daniel Paul O’Donnell, Cameron Neylon, Sam Moore, Robert Gadie, Victoria Odeniyi, and Shahina Parvin¸ assess PLOS ONE’s impact on the culture of peer review and what it can tell us about efforts to change academic culture more broadly.
The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.
Image Credit: Alicia97 on Shutterstock.