Academic research has been beset by a number of disturbing problems in recent years; from the reproducibility crisis and long publication delays, right through to article retractions and admissions of researcher misconduct. This has led to increasing public and media scepticism as to the quality and integrity of research. Peer review remains the gold standard for ensuring that quality and integrity but is reliant on voluntary, unrewarded contributions. Andrew Preston and Tom Culley argue that formally recognising and rewarding peer review efforts would be a great first step towards restoring balance to a research ecosystem that offers experts disproportionate rewards for publishing while offering no formal incentives to the same experts relied upon to filter out the false, fraudulent or misleading.
The research enterprise is plagued by a series of disturbing problems. A reproducibility crisis, significant delays in publishing and disseminating peer reviewed findings, a surge in the volume of retractions and admissions of fraudulent or questionable research practices, to name a few. All of these issues are, quite rightly, leading to increasing public and media scepticism as to the quality and integrity of research.
This matters because, as Ferric Fang, a professor at the University of Washington School of Medicine and Editor-in-Chief of the journal Infection and Immunity rightly points out: “Science has the potential to address some of the most important problems in society and for that to happen, scientists have to be trusted by society and they have to be able to trust each other’s work.”
In other words, if the system is not working in a credible and efficient manner, we put both people and our potential for progress at risk.
Image credit: Medals by Patrik Nguyen. This work is licensed under a CC BY-SA 2.0 license.
At the heart of this problem is an opportunity. In the era of “fake news” and distrust in reporting, evidence-based decisions are arguably more important than ever. If we can solve some of the issues with the current system, science and research can and will provide massive gains for all of humanity.
Peer review: the pillar of research quality and integrity
Peer review is at the heart of the research ecosystem. It is the gold standard for ensuring the quality and integrity of published research. Peer review plays the essential role of validating findings, qualifying their importance and filtering out misleading or fraudulent work. Thorough peer review can be relied on to mitigate most of the issues in research, and in fact improve the quality of research papers before they are published.
So why do problems exist? The answer is complex, but one thing is very clear. The way the research establishment treats peer review, the primary mechanism relied upon to prevent flawed or incomplete research being disseminated, has not changed in 350 years. The result is a system that is overburdened and under-developed.
The economics of the research ecosystem
Research is overrun by a debilitating “publish or perish” culture. Researchers are, for the most part, rewarded with funding and career progression based on their publication and citation records. This compels researchers to publish as much as possible in journals renowned for delivering citations. Add to this the increasing competition for shrinking government funds and you have an environment that fosters temptation to fudge results, ignore inconvenient data, slice and self-select submissions into as many “novel” publications as possible and generally undermine the truth-seeking goals of research. Thankfully the system has a mechanism for safeguarding the quality and integrity of published research, namely: peer review.
Peer review is a voluntary exercise. Researchers are expected to review for the intrinsic incentive of “giving back” and a quid-pro-quo understanding that others will review their work. This approach may have been appropriate before the internet when there was no way to accurately measure anonymous peer review contributions and when scientists worked in tight communities. But the research ecosystem has changed remarkably. We now have the internet, globalisation, interdisciplinary research and mega-publishers. These developments drive the pressure to publish higher, placing increasing stress on a voluntary and unrewarded peer review process.
To summarise, the system almost exclusively rewards experts for publishing at nearly any cost. The same system then offers no formal incentives to the same experts relied upon to filter out the false, fraudulent or misleading submissions through peer review.
Bringing balance back to the system
There is no silver bullet that will resolve all the issues, but there are simple steps that can be taken to bring balance back to the system. An obvious starting point is to formally recognise and reward peer review efforts so it stands a chance against the disproportionate rewards for publishing.
Recognition for review is not a new concept. For years a number of academic publishers and journals have published the names of their reviewers on an annual basis. Others have begun to provide certificates to reviewers acknowledging their valued contributions. Researchers can then reference these efforts in their performance evaluations as proof of their service. This approach, while better than nothing, does have its shortcomings:
- Academic publishers and journals are not the ones making funding and career advancement decisions for researchers or institutions. A certificate is nice, but if funders and institutions don’t give it any credence when evaluating performance or allocating funds, it’s no more than a wall decoration.
- The fragmented and imprecise nature of the recognition (i.e. no standardised measure or centralised hub for tracking outputs) does not provide a basis for funders, world ranking bodies or institutions to incorporate peer review contributions into their evaluation methodologies, as there is no way to benchmark the peer review outputs.
- Publisher acknowledgement by itself means the system is still relying on the good nature of individual researchers to contribute to the peer review system. Altruism and a thank you is a tough bet when pitted against the very tangible rewards for publications and citations.
Some have argued that publishers should start paying reviewers or offer in-kind benefits. After all, in monetary terms, publishers benefit the most directly from the scholarly publishing system. The jury is still out as to whether this will help, but a few points to keep in mind regarding a publisher payment solution include:
- Previous psychological studies suggest that offering a monetary reward for a piece of work can “crowd out” the intrinsic, non-financial incentives for performing a task well. If paying referees without measures to control for the quality of reviews led to bad peer review, it would defeat the purpose of the intervention.
- A Wiley survey of over 3,000 reviewers suggests researchers prefer formal recognition to cash or in-kind payment for reviewing. The same survey found that “reviewers strongly believe that reviewing is inadequately acknowledged at present and should carry more weight in their institutions’ evaluation process.”
- Researchers value career advancement and institutions (not publishers) make career advancement decisions.
To square the incentives ledger, we need to look to institutions, world ranking bodies and funders. These parties hold either the purse strings or the decision-making power to influence the actions of researchers. So how can these players more formally recognise review to bring balance back to the system and what tools do they need to do it?
Quite simply, institutions could give greater weight to peer review contributions in funding distribution and career advancement decisions. If there was a clear understanding that being an active peer reviewer would help further your research career, then experts would put a greater emphasis on their reviewing habits and research would benefit.
If funders factored in peer review contributions and performance when determining funding recipients, then institutions and individuals would have greater reason to contribute to the peer review process.
World ranking bodies
Like researchers, institutions also care about their standing and esteem on the world stage. If world ranking bodies such as THE World University Rankings and QS World Rankings gave proportionate weighting to the peer review contributions and performance of institutions, then institutions would have greater reason to reward the individuals tasked with peer reviewing.
More formal weighting for peer review contributions also makes sense, because peer review is actually a great measure of one’s expertise and standing in the field. Being asked to peer review is external validation that academic editors deem a researcher equipped to scrutinise and make recommendations on the latest research findings.
Researchers will do what they have to in order to advance their careers and secure funding. If institutions and funders make it clear that peer review is a pathway to progression, tenure and funding, researchers will make reviewing a priority.
In order for peer review to be formally acknowledged, benchmarks are necessary. There needs to be a clear understanding of the norms of peer review output and quality across the myriad research disciplines in order to assign any relative weighting to an individual’s review record. This is where the research enterprise can utilise the new data tools available to track, verify and report all the different kinds of peer review contributions. These tools already exist and researchers are using them. It’s time the institutions that rely on peer review got on board too.
This blog post originally appeared in the “SpotOn Report: What might peer review look like in 2030?” from BioMed Central and Digital Science and is published under a CC BY 4.0 license. The report can be found on Figshare.
For more information on the work Publons are doing to recognise peer review efforts, please visit the Publons website.
Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
About the authors
Andrew Preston is CEO and Co-Founder of Publons.
Tom Culley is Marketing Director at Publons.
You say that a big problem in science is “This compels researchers to publish as much as possible in journals renowned for delivering citations.” Isn’t it strange that you then join the one company that makes a living based on selling a.o. journal impact factors and by that fosters a hypercompetetive model that wastes energy, slows publication and impedes accessibility of research for society?
You state in your introduction that peer review has not changed for 350 years. I think that is a rather simplified image. Actually there has been a lot of change in most aspects of peer review. The peer review as known and as used today is a relatively new phenomenon. Just watch this great animation to get a glimpse of the complex history of peer review:
Or have a look at this peer review timeline I created last year: https://doi.org/10.6084/m9.figshare.4210287.v1 Among others it shows that Nature only introduced peer review in 1967, almost 100 years after launch of the journal.
I really like this statement: “The system offers no formal incentives to experts relied upon to filter out the false, fraudulent or misleading submissions through peer review..” In http://www.peerviewer.com, transparency and giving incentives as a credit to peer reviewer’s work is of out most importance, where peer review experts being hired by the journal publishers to disclose their identity and their scholarly experience.