Preprints are research reports have that have not yet been peer reviewed and accepted for publication. They have increased rapidly during the COVID-19 pandemic. However, high profile discredited studies have led to concerns that speed has been prioritized over the quality and credibility of evidence. Joeri Tijdink, Mario Malicki, Lex Bouter and Gowri Gopalakrishna argue that all stakeholders of the science system have a responsibility in improving the quality and credibility of pre-prints. They outline 5 steps by which this can be achieved.
Research has become the dominant driver of health policy decisions during the COVID 19 pandemic. It has guided us in and out of lockdowns. It has helped us understand the novel pathogen and devise methods to circumvent its impact through treatment, behavioural change and maybe soon with a vaccine. Research can and clearly has saved lives. However, research in this pandemic has also misinformed policy makers and a part of that misinformation came from preprints.
What are preprints?
Preprints are research reports that have not yet been peer reviewed and accepted for publication in a scientific journal. Such research reports, shared through preprint servers, enable researchers to rapidly share their results without any delay. They enable scholarly communications to be shared quickly and freely and also serve to claim priority. Posting research reports on preprint servers also means that useful feedback can be collected faster and it can foster collaborations between researchers.
Although the first preprint servers are several decades old, during the last 5 years more than 30 new preprint servers have emerged, as a result of it strongly being promoted in the open science movement. This has accelerated even more during the COVID 19 pandemic. In the past months alone, the biomedical preprint server, MedRXiv, has seen a 400% increase in preprints being published compared to the same period in 2019. While this means that research is being produced and shared at an ever-increasing speed, it can also be detrimental if that research is not conducted at the highest quality standards.
Are preprints a credible source?
During the ongoing pandemic we have seen preprints rapidly gain media attention and top the Altmetrics charts. Some of the more notable ones include a study linking HIV and COVID-19. The authors, however, rapidly retracted the study two days after it received backlash from the scientific community. Another study on seroprevalence in Santa Clara county USA was revised by its authors less than two weeks after it appeared as a result of substantial criticisms on its methods and the misuse of potential policy implications to fuel certain political agendas.
Some researchers feel the rapid, open critique by peers that these preprints received in the absence of formal peer review, can be considered to be a self-correcting mechanism of preprints. However, we currently do not know how extensively preprints are scrutinized and if the “self–correcting mechanism” we have seen with COVID-19 preprints is an exception. It is also not known how often authors of preprints address the feedback they receive. The largest and longest running preprint server, arXiv does not have a moderated commenting section, while the bioRxiv and medRxiv moderate their comments sections for offensive and non-pertinent comments. Hitherto, most of the commenting and scrutiny of preprints seems to happen on Twitter, especially for COVID-19 preprints. So far only about 10% of all preprints receive comments. Hence, this feature is still under-utilized.
we currently do not know how extensively preprints are scrutinized and if the “self–correcting mechanism” we have seen with COVID-19 preprints is an exception
So are preprints in general a credible source of information if they do not undergo some form of peer review? We believe they are. The small amount of studies looking at the issue, seem to indicate that around 70% of preprints are eventually published in scientific journals, with only minimal differences between the preprint and their published version. However, research, no matter if it’s a journal publication or shared though preprint servers, should and must always adhere to the highest standards of methodological rigor and research integrity in order to minimize the risk of misinforming fellow researchers, policy makers and the general public. The implication is that in the absence of formal peer review, both servers and the authors of preprints should adhere to responsible research practices when publishing in preprints.
5 Ways to foster responsible preprint practices
To help ensure that preprints are a reliable source of information, we propose several ways preprint servers and authors of preprints can safeguard the credibility of their preprints, granted that some servers already incorporate some of our recommendations. These recommendations were collated based on the authors’ research expertise in preprints and open science modalities.
Below we highlight 5 key recommendations that we consider as the most practical and concrete in order to improve preprint quality. A full list of recommendations is available on the Open Science Framework.
For preprint servers
1 .Provide clear guidance to authors on how to conduct, report, share, and update their preprints (and data sets).
2. Link preprints to versions of research reports (e.g. by providing references or links) and clearly indicate when a preprint represents a submitted or the published version of a research report (1).
For authors of preprints
3. Apply the same responsible research practices to publishing preprints as you would to your journal publications. If the potential impact of results requires a speedy publication, clearly indicate that results are preliminary and state when the final study report and detailed methods will be available.
4 .Embody the spirit of the Mertonian norms of communality and disorganized skepticism by being an active reviewer of preprints in your area of expertise. Strive to provide at least as many preprint reviews as you expect to receive from others.
5. When communicating with the media about your preprints or critiquing other authors’ preprints, do so responsibly. Be explicit about strengths and limitations of your preprints.
Conclusion
In the ongoing pandemic, there is no doubt we face numerous challenges that may compromise methodological rigor and research integrity of the research being pushed out. The increasing number of preprints and their use in shaping public health policy make improvements in their quality urgent and important. This is a joint responsibility of all stakeholders of the science system. In this report, we outlined a few implementable steps which we believe can be effective in improving the quality and credibility of preprints.
Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below
photo credit: AbsolutVision via Unsplash.
You are comparing preprints to a non-existent perfection. The problems you identify – mostly illustrated by anecdote – apply almost equally to standard publications, because peer review is a hugely imperfect guarantee of quality. Furthermore, at least preprints accelerate dissemination of science and declare their preliminary status. In contrast, publication often takes a long time and the unmerited “seal of approval” can do a lot of harm. Journals certainly do not facilitate post-publication review or corrections of their publications.