Online publication provides us with new freedom to update, amend and extend the research article as we know it. Daniel Shanahan presents a vision of the evolution of the article beyond the limits of the printed page. Creating a living document for a single research project, updated in real time, would lead to it being evaluated based on the question it asks, and the methods it uses, rather than on serendipitous “good” results.
The research article as we know it has been around, in one form or another, for over 300 years. And for 300 years, it has scarcely changed. Amid growing discussion around a ‘reproducibility crisis’ in science and calls for greater transparency in research publications, is it time to move beyond the now-obsolete print model and truly embrace the freedom that online publication offers us, moving towards an evolving document?
Born out of the exchange of letters on scientific topics and results, the research article was introduced as a way of documenting what was done and sharing the outcome, as well as attributing credit to the authors. But time marches on. Restraints that were simply common sense when research articles had to be printed and posted to eager readers seem frankly absurd following the advent of the Internet. Word limits and additional charges for colour images remain widespread, and even the “IMRAD” (Introduction, Methods, Results and Discussion) format reflects the assumption that people are reading the article from a printed page.
Moreover, these restraints can actually undermine the validity of the science itself. This historical requirement to report discoveries in a concise manner, adhering to sometimes harsh word limits, has led authors to focus primarily on what they considered to be of greatest interest – the results – leading to a totally misleading narrative of the processes and thoughts that actually went into getting them. This over-emphasis on results is the root cause of many of the problems that currently plague the scientific literature, such as publication bias, significance chasing, “HARKing” (hypothesizing after the results are known), lack of data sharing and replication, and low statistical power.
Image credit: William Caxton showing specimens of his printing to King Edward IV and his Queen (Wikimedia Public Domain)
These concerns are nothing new – the research article has been around for over 300 years, after all – and have led to calls for disseminating the research outcomes and methods before conducting a study, such as through trial registration, or the prospective publication of study protocols. However, this leads to multiple publications, across multiple journals, often published many years apart.
Using the example of clinical trials, a single study can result in a study protocol, the traditional results paper (or papers), as well as secondary analyses, statistical analysis plans and eventually systematic reviews, among others. Researchers need access to all of these if they are to get a true picture of what was done, and whether the science was valid. And, as any systematic reviewer will tell you, actually tracking down all the publications relating to a single piece of research can be like looking for a needle in a haystack.
There are initiatives in the works to help address this, but these are all just solutions to a problem we ourselves have created.
Reincarnating the research article
So why do we continue to perpetuate a 300-year-old anachronism? Online publication provides us with a freedom that was not seen in the print era, with the ability to update, amend and extend the document, as well as link directly to other articles and data. Rather than the current situation, where each “stage” in the research process leads to a separate publication, can the research article become a living document, with a single article for a single piece of research?
It is a powerful concept. Researchers could publish the full, detailed methods prior to conducting the study, which could then be extended to include the statistical analysis plan, as it becomes available. Once they have completed the study, they can update the article to include the results and analyses performed, without having to rewrite the methods and risk self-plagiarism.
Figure 1: Workflow for a living document of a randomized controlled trial
Source: Shanahan (2015) A living document: reincarnating the research article. Trials 16:151.
While the article would evolve over time, any additions to the article that could impact on the validity of the science would, of course, require peer review. In these cases, the article could be frozen into a discrete version, with the reviewer reports associated with it, as is already seen in post-publication peer review models, such as F1000Research or ScienceOpen. Readers citing the document would simply need to include the date it was accessed, thereby uniquely identifying the version of the document referred to.
Beyond print
As this would be an online resource, rather than a printed one, the document could also link to the raw data and any code used to obtain the results, in addition to any summary results reported to support the written interpretation. The dramatic decrease in data storage costs and the emergence of literate programming environments, such as Arvados, make it possible to enable reproducibility of data analysis with versioned scripts and tools. Researchers could deposit the data, tools and scripts they used to analyse the data, allowing readers to determine how robust the visualisations and statistics embedded in the paper are.
Creating an evolving document for a single research project would lead to it being evaluated based on the question it asks, and the methods it uses, rather than on serendipitous “good” results. It would also make it simple to assess selective reporting of both analyses and outcomes, as all the necessary information would be reported in the same place. By underpinning the results and interpretations with the original data and analysis tools, there would be huge benefits for reproducibility of research, as well as for those conducting meta-analyses. Current technology means that this form of publication is theoretically already possible, although that’s not to say this isn’t without complications.
At the very least, a continuously-evolving document would undermine existing methods of evaluating the impact of an article, particularly short-term metrics like the Impact Factor, which rely on the date of publication. Encouraging and facilitating reproduction also raises the issue of how to combine original research articles with follow-up replication or analyses by a different group of authors.
However, with the ongoing drive towards transparency and reproducibility of research, and the growing recognition of the damage that the current overemphasis on results is doing to science, it is no longer acceptable to continue to perpetuate a centuries-old absurdity.
This piece is based on a commentary article in Trials (2015) A living document: reincarnating the research article.
Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Daniel Shanahan is a firm advocate for transparency and reproducibility in research. He joined BioMed Central in 2013 as Associate Publisher, driving open science and research transparency strategies and initiatives across the company. In his role, Daniel has been involved in several initiatives aiming to improve the quality of published evidence. He is Chair of the Threaded Publications working group, and is also participating in strategic efforts to encourage the wider adoption of reporting guidelines, and to improve policies to combat publication bias and selective reporting, among others.
I am not providing feedback:
Publication is for the Field of Knowledge. Methods for publishing usually represent more about publishing than about the Field of Knowledge. Method is about the Field of the Method. Field is not only about the Method for the Field. Evolution is Allowed. Evolution is not granted artificially. Evolution allows Evolution. One chooses to write a book another chooses to write an article and then there are those who file patents. In Evolution choices are choices, constraints are constraints, bottlenecks are bottlenecks, methods are methods, flow is flow, field is field. In Evolution a Publication can serve “Choice”, “Constraint”, “Bottleneck”, “Method”, “Flow”, “Field” and groups of same. Sense exists, Sense can allow the Publication to serve “Choice”, “Constraint”, “Bottleneck”, “Method”, “Flow”, “Field” and groups of same. Similarly, Ph.D. can serve the Field when the Field is served. Serving only articles, proposals, patents is not what serving the Field is about. Sense allows Ph.D. to be Doctor of Philosophy not Doctor of Articles, not Doctor of Proposals, not Doctor of Patents. Research is a part of Evolution. Thank You
The arguments here are intriguing, and I think that there are ways we can improve the reporting of scientific research with modern tools. Indeed, there are high-profile and publicly-important studies that could benefit greatly by pre-publishing their methods and then conducting the study so that when the results are gathered there is no possibility of changing the approach if and when they don’t get the result that they intend. I am planning one such public experiment, and the flow chart given above could help with making a larger part of the scientific community (and the public) a part of the peer-review process before the experiment has completed.
However, there are many studies where this is simply not possible nor practically feasible. When doing exploratory research, you do not often have static methods that you are using, and are constantly adjusting and tweaking those methods, and duplicating unexpected results to make sure that they are real. Going into a research project you can certainly start out with certain methods you intend to use, but results along the way will shape what methods you use next. While I suppose that you could constantly update what methods you are using – for me that often changes every day. Each tweak to your PCR conditions is a new experiment. When you are wading out into new territory you have to be very careful about what information is released, and I think it is better in those cases to keep the research under wraps until you have a better idea of what you have.
I agree that we need to cut loose from the limitations of the paper-based structure. Here’s a paper we wrote recently on the subject. Maybe it is of interst to someone.
https://www.researchgate.net/publication/251400880_Making_journal_articles_live_turning_academic_writing_into_scientific_dialog
Writing is a 6000 year old technology, yet still not an anachronism. I also note that this post is in the form of, guess what, an article. The research article is a simple thing, with a simple logic. There is nothing anachronistic about this logical form. If you want more than this you need to find a way to pay for it. Moreover, having researchers produce more than this simple form takes their time away from the research.
That each article should be rewritten over and over again for the rest of an author’s life is the real absurdity here. Journal articles are not diaries. Journals are diaries.