LSE - Small Logo
LSE - Small Logo

Joeri Tijdink

Mario Malicki

Gowri Gopalakrishna

Lex Bouter

September 23rd, 2020

Are preprints a problem? 5 ways to improve the quality and credibility of preprints

1 comment | 51 shares

Estimated reading time: 7 minutes

Joeri Tijdink

Mario Malicki

Gowri Gopalakrishna

Lex Bouter

September 23rd, 2020

Are preprints a problem? 5 ways to improve the quality and credibility of preprints

1 comment | 51 shares

Estimated reading time: 7 minutes

Preprints are research reports have that have not yet been peer reviewed and accepted for publication. They have increased rapidly during the COVID-19 pandemic. However, high profile discredited studies have led to concerns that speed has been prioritized over the quality and credibility of evidence. Joeri Tijdink, Mario Malicki, Lex Bouter and Gowri Gopalakrishna argue that all stakeholders of the science system have a responsibility in improving the quality and credibility of pre-prints. They outline 5 steps by which this can be achieved.


Research has become the dominant driver of health policy decisions during the COVID 19 pandemic. It has guided us in and out of lockdowns. It has helped us understand the novel pathogen and devise methods to circumvent its impact through treatment, behavioural change and maybe soon with a vaccine. Research can and clearly has saved lives. However, research in this pandemic has also misinformed policy makers and a part of that misinformation came from preprints.

What are preprints?

Preprints are research reports that have not yet been peer reviewed and accepted for publication in a scientific journal. Such research reports, shared through preprint servers, enable researchers to rapidly share their results without any delay. They enable scholarly communications to be shared quickly and freely and also serve to claim priority. Posting research reports on preprint servers also means that useful feedback can be collected faster and it can foster collaborations between researchers.

Although the first preprint servers are several decades old, during the last 5 years more than 30 new preprint servers have emerged, as a result of it strongly being promoted in the open science movement.  This has accelerated even more during the COVID 19 pandemic. In the past months alone, the biomedical preprint server, MedRXiv, has seen a 400% increase in preprints being published compared to the same period in 2019. While this means that research is being produced and shared at an ever-increasing speed, it can also be detrimental if that research is not conducted at the highest quality standards.

Are preprints a credible source?

During the ongoing pandemic we have seen preprints rapidly gain media attention and top the Altmetrics charts. Some of the more notable ones include a study linking HIV and COVID-19. The authors, however, rapidly retracted the study two days after it received backlash from the scientific community. Another study on seroprevalence in Santa Clara county USA was revised by its authors less than two weeks after it appeared as a result of substantial criticisms on its methods and  the misuse of potential policy implications to fuel certain political agendas.

Some researchers feel the rapid, open critique by peers that these preprints received in the absence of formal peer review, can be considered to be a self-correcting mechanism of preprints. However, we currently do not know how extensively preprints are scrutinized and if the “self–correcting mechanism” we have seen with COVID-19 preprints is an exception. It is also not known how often authors of preprints address the feedback they receive. The largest and longest running preprint server, arXiv does not have a moderated commenting section, while the bioRxiv and medRxiv moderate their comments sections for offensive and non-pertinent comments. Hitherto, most of the commenting and scrutiny of preprints seems to happen on Twitter, especially for COVID-19 preprints. So far only about 10% of all preprints receive comments. Hence, this feature is still under-utilized.

we currently do not know how extensively preprints are scrutinized and if the “self–correcting mechanism” we have seen with COVID-19 preprints is an exception

So are preprints in general a credible source of information if they do not undergo some form of peer review? We believe they are. The small amount of studies looking at the issue, seem to indicate that around 70% of preprints are eventually published in scientific journals, with only minimal differences between the preprint and their published version. However, research, no matter if it’s a journal publication or shared though preprint servers, should and must always adhere to the highest standards of methodological rigor and research integrity in order to minimize the risk of misinforming fellow researchers, policy makers and the general public. The implication is that in the absence of formal peer review, both servers and the authors of preprints should adhere to responsible research practices when publishing in preprints.

5 Ways to foster responsible preprint practices

To help ensure that preprints are a reliable source of information, we propose several ways preprint servers and authors of preprints can safeguard the credibility of their preprints, granted that some servers already incorporate some of our recommendations. These recommendations were collated based on the authors’ research expertise in preprints and open science modalities.

Below we highlight 5 key recommendations that we consider as the most practical and concrete in order to improve preprint quality. A full list of recommendations is available on the Open Science Framework.

For preprint servers

1 .Provide clear guidance to authors on how to conduct, report, share, and update their preprints (and data sets).

2. Link preprints to versions of research reports (e.g. by providing references or links) and clearly indicate when a preprint represents a submitted or the published version of a research report (1).

For authors of preprints

3. Apply the same responsible research practices to publishing preprints as you would to your journal publications. If the potential impact of results requires a speedy publication, clearly indicate that results are preliminary and state when the final study report and detailed methods will be available.

4 .Embody the spirit of the Mertonian norms of communality and disorganized skepticism by being an active reviewer of preprints in your area of expertise. Strive to provide at least as many preprint reviews as you expect to receive from others.

5. When communicating with the media about your preprints or critiquing other authors’ preprints, do so responsibly. Be explicit about strengths and limitations of your preprints.

 

Conclusion

In the ongoing pandemic, there is no doubt we face numerous challenges that may compromise methodological rigor and research integrity of the research being pushed out. The increasing number of preprints and their use in shaping public health policy make improvements in their quality urgent and important. This is a joint responsibility of all stakeholders of the science system. In this report, we outlined a few implementable steps which we believe can be effective in improving the quality and credibility of preprints.


Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below

photo credit: AbsolutVision on Unsplash

Print Friendly, PDF & Email

About the author

Joeri Tijdink

Joeri Tijdink is a psychiatrist and assistant professor at the Amsterdam University Medical Center and VU Amsterdam in the Netherlands. After he obtained his PhD with his thesis entitled Publish & Perish, he is involved in several research projects studying research integrity. His research focuses on research integrity, research culture, and mental health in academia and also on ways to foster responsible research practices for researchers and within research institutions. He is the author of the self-help guide Scholar on the sofa; How to survive in academia.

Mario Malicki

Mario Malicki is a Co-Editor-in-Chief of Research Integrity and Peer Review journal with background in medicine. His PhD was in Integrity of Scientific Publications, and in previous research he focused on authorship issues, publication bias, journals’ instructions to authors and peer review. He is currently at Meta-Research Innovation Center at Stanford (METRICS) where he focuses on meta-research of preprints.

Gowri Gopalakrishna

Gowri Gopalakrishna is an epidemiologist by training with a PhD in Epidemiology. Some of her notable contributions include her role in the control and containment of the SARS outbreak in Singapore in 2003. In her current role, she works closely with a core team to design, implement and eventually analyse results of one of the world’s largest surveys on responsible research practices in The Netherlands: The Dutch National Survey on Research Integrity.

Lex Bouter

Lex Bouter has a tenured chair in Methodology and Integrity at the Department of Epidemiology and Data Science of the Amsterdam University Medical Centers and the Department of Philosophy of the Faculty of Humanities of the Vrije Universiteit. He is the founding chair of the World Conferences on Research Integrity Foundation. Personal page

Posted In: COVID 19 | Open science

1 Comments

This work by LSE Impact of Social Sciences blog is licensed under a Creative Commons Attribution 3.0 Unported.