Online publication provides us with new freedom to update, amend and extend the research article as we know it. Daniel Shanahan presents a vision of the evolution of the article beyond the limits of the printed page. Creating a living document for a single research project, updated in real time, would lead to it being evaluated based on the question it asks, and the methods it uses, rather than on serendipitous “good” results.
The research article as we know it has been around, in one form or another, for over 300 years. And for 300 years, it has scarcely changed. Amid growing discussion around a ‘reproducibility crisis’ in science and calls for greater transparency in research publications, is it time to move beyond the now-obsolete print model and truly embrace the freedom that online publication offers us, moving towards an evolving document?
Born out of the exchange of letters on scientific topics and results, the research article was introduced as a way of documenting what was done and sharing the outcome, as well as attributing credit to the authors. But time marches on. Restraints that were simply common sense when research articles had to be printed and posted to eager readers seem frankly absurd following the advent of the Internet. Word limits and additional charges for colour images remain widespread, and even the “IMRAD” (Introduction, Methods, Results and Discussion) format reflects the assumption that people are reading the article from a printed page.
Moreover, these restraints can actually undermine the validity of the science itself. This historical requirement to report discoveries in a concise manner, adhering to sometimes harsh word limits, has led authors to focus primarily on what they considered to be of greatest interest – the results – leading to a totally misleading narrative of the processes and thoughts that actually went into getting them. This over-emphasis on results is the root cause of many of the problems that currently plague the scientific literature, such as publication bias, significance chasing, “HARKing” (hypothesizing after the results are known), lack of data sharing and replication, and low statistical power.
Image credit: William Caxton showing specimens of his printing to King Edward IV and his Queen (Wikimedia Public Domain)
These concerns are nothing new – the research article has been around for over 300 years, after all – and have led to calls for disseminating the research outcomes and methods before conducting a study, such as through trial registration, or the prospective publication of study protocols. However, this leads to multiple publications, across multiple journals, often published many years apart.
Using the example of clinical trials, a single study can result in a study protocol, the traditional results paper (or papers), as well as secondary analyses, statistical analysis plans and eventually systematic reviews, among others. Researchers need access to all of these if they are to get a true picture of what was done, and whether the science was valid. And, as any systematic reviewer will tell you, actually tracking down all the publications relating to a single piece of research can be like looking for a needle in a haystack.
There are initiatives in the works to help address this, but these are all just solutions to a problem we ourselves have created.
Reincarnating the research article
So why do we continue to perpetuate a 300-year-old anachronism? Online publication provides us with a freedom that was not seen in the print era, with the ability to update, amend and extend the document, as well as link directly to other articles and data. Rather than the current situation, where each “stage” in the research process leads to a separate publication, can the research article become a living document, with a single article for a single piece of research?
It is a powerful concept. Researchers could publish the full, detailed methods prior to conducting the study, which could then be extended to include the statistical analysis plan, as it becomes available. Once they have completed the study, they can update the article to include the results and analyses performed, without having to rewrite the methods and risk self-plagiarism.
Figure 1: Workflow for a living document of a randomized controlled trial
Source: Shanahan (2015) A living document: reincarnating the research article. Trials 16:151.
While the article would evolve over time, any additions to the article that could impact on the validity of the science would, of course, require peer review. In these cases, the article could be frozen into a discrete version, with the reviewer reports associated with it, as is already seen in post-publication peer review models, such as F1000Research or ScienceOpen. Readers citing the document would simply need to include the date it was accessed, thereby uniquely identifying the version of the document referred to.
As this would be an online resource, rather than a printed one, the document could also link to the raw data and any code used to obtain the results, in addition to any summary results reported to support the written interpretation. The dramatic decrease in data storage costs and the emergence of literate programming environments, such as Arvados, make it possible to enable reproducibility of data analysis with versioned scripts and tools. Researchers could deposit the data, tools and scripts they used to analyse the data, allowing readers to determine how robust the visualisations and statistics embedded in the paper are.
Creating an evolving document for a single research project would lead to it being evaluated based on the question it asks, and the methods it uses, rather than on serendipitous “good” results. It would also make it simple to assess selective reporting of both analyses and outcomes, as all the necessary information would be reported in the same place. By underpinning the results and interpretations with the original data and analysis tools, there would be huge benefits for reproducibility of research, as well as for those conducting meta-analyses. Current technology means that this form of publication is theoretically already possible, although that’s not to say this isn’t without complications.
At the very least, a continuously-evolving document would undermine existing methods of evaluating the impact of an article, particularly short-term metrics like the Impact Factor, which rely on the date of publication. Encouraging and facilitating reproduction also raises the issue of how to combine original research articles with follow-up replication or analyses by a different group of authors.
However, with the ongoing drive towards transparency and reproducibility of research, and the growing recognition of the damage that the current overemphasis on results is doing to science, it is no longer acceptable to continue to perpetuate a centuries-old absurdity.
This piece is based on a commentary article in Trials (2015) A living document: reincarnating the research article.
Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Daniel Shanahan is a firm advocate for transparency and reproducibility in research. He joined BioMed Central in 2013 as Associate Publisher, driving open science and research transparency strategies and initiatives across the company. In his role, Daniel has been involved in several initiatives aiming to improve the quality of published evidence. He is Chair of the Threaded Publications working group, and is also participating in strategic efforts to encourage the wider adoption of reporting guidelines, and to improve policies to combat publication bias and selective reporting, among others.