Many observers have drawn the logical conclusion that the increased exposure and visibility afforded by open access leads to improved citation performance of open access journals. Yang Li, Chaojiang Wu, Erjia Yan and Kai Li report on research examining the perceived open access advantage, paying particular attention to journals which have “flipped” to open access from a subscription model. Findings reveal that the estimated overall effect of open access is positive, with significant improvements to journals’ citation metrics. However, the degree to which a journal may improve varies according to its research field, publisher and quality profile.
As various models of open access have become more established, the number of open access journals has increased markedly over the past two decades. Our recent study assessed the degree to which open access journals benefit from a citation advantage, analysing a large sample of open access journals representing a wide range of knowledge domains. Our unique dataset also contains a large pool of subscription journals which enabled us to find close matches for the open access journals for better causal inferences. Taking advantage of the large pool of controls, we employed a difference-in-difference identification strategy, a commonly used econometric technique for identifying causal relationships, to estimate the open access effect more convincingly than most other current studies, which are more descriptive than confirmative.
More specifically, to gain insights into the performance of open access journals in a scientific framework, we compared the citation behaviour of open access journals to subscription journals over five consecutive years, from 2011-2015. We paid special attention to those journals that “flip” to an open access model from a subscription model. The treatment group included 244 flipped journals between 2011 and 2014 (the year 2015 is excluded because of the need for least one period to observe the open access effect). The control group, on the other hand, includes 12,983 subscription journals drawn from the same research areas and ranks, as categorised by Scopus. The CiteScore metric, which uses a three-year citation window, provided a robust proxy for the citation performance of journals. The estimated overall effect of open access is positive, meaning that becoming an open access journal will significantly improve a journal’s CiteScore by 0.147, on average. This result is not surprising because, in general, open access increases the exposure of journals to research community, leading to more citation opportunities.
However, it is not enough to just look at the average effect of open access on journal citation scores. Journals vary widely according to characteristics such as publisher, research field, and rank. Different journals may exhibit diverse citation behaviours after becoming open access. So we examined the effect of open access by restricting our analysis over subsamples determined by journal publisher, research field, and rank. First, we divided journals in our dataset by publisher, classifying Springer, SAGE, Elsevier, Wiley-Blackwell, and Taylor & Francis as the so-called “Big Five” publishers, since they are the five largest publishers of scholarly journals. The effect of open access on journals from the Big Five publishers is 0.309, whereas the effect on journals from other publishers is 0.0742. This result is perhaps easy to understand as the Big Five publishers are usually able to provide quality assurance through their professional peer review process, encouraging more researchers to submit after becoming open access, and seeing a subsequent boost in journal CiteScores.
We also investigated the open access effect across different research areas. Journals in our dataset were manually categorised into six broad domains: biology, engineering, maths and computer science, medicine, science, and social science. Unsurprisingly, we found that journals in different disciplines faced different treatment effects after becoming open access. There was strong evidence for the significance of positive effects for journals in biology, medicine, and science, where the open access effect led to CiteScore increases of 0.400, 0.191, and 0.105 respectively. However, the effect of open access is insignificant or barely significant for journals in maths and computer science, social science, and engineering.
Finally, our investigation analysed the effect of open access across different quality ranks, as determined by the four quartiles. The open access effect appears most significant for those journals ranked in Quartiles 2, 3, and 4, with CiteScore growth of 0.206, 0.181, and 0.146 respectively. However, the open access effect on the top ten per cent and Quartile 1 journals is not significant. This result is in accordance with the so-called “long tail” theory; i.e. high-ranking journals realise less benefit from open access because researchers will always cite such journals in their fields, regardless of their access model. Lower-ranked journals, in contrast, have greater potential for CiteScore growth after switching to open access.
Given these findings, is open access a good strategy for those journals in fields, publishers, or ranks not found to exhibit a significant citation advantage? Further analysis has shown journals of high quality, as measured by their historical-high CiteScores, and not defined by their fields or prestigious publishers or ranks, may benefit from open access models more than other journals. Put another way, quality is an important factor for an open access strategy to take effect. Subscription journals may benefit from open access if they are promising journals, with potential for growth in their CiteScores.
This blog post is based on the authors’ article, “Will open access increase journal CiteScores? An empirical investigation over multiple disciplines”, published in PLoS ONE (DOI: 10.1371/journal.pone.0201885).
Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
About the authors
Yang Li is a PhD candidate at the LeBow College of Business, Drexel University. His current research interests include user-generated content mining, causal inference, and social network analysis, with particular attention to the integration of machine learning and econometrics in empirical studies.
Chaojiang Wu is an assistant professor in the LeBow College of Business at Drexel University. He obtained his PhD from the University of Cincinnati. His research develops quantitative methods in analysing complex data for better decisions and also applies such methods to applications in business and other communities. His work has been published in leading journals such as Journal of Financial and Quantitative Analysis, IISE Transactions, Statistics and Computing, Computational Statistics and Data Analysis, Journal of Informetrics, and Scientometrics, among others. His ORCID is 0000-0002-0047-9037.
Erjia Yan is an assistant professor at College of Computing and Informatics at Drexel University. He received his PhD in information science from Indiana University. His research interests focus on informetrics and scientometrics, scholarly data mining and analysis, and knowledge diffusion studies. His ORCID is 0000-0002-0365-9340.
Kai Li is a PhD Candidate of the Information Science programme at Drexel University. His research interests focus on how digital methods and tools are represented in scientific publications. His ORCID is 0000-0002-7264-365X.