Open access is built on the promise that more people are able to access and use research. Putting this to the test, Chun-Kai (Karl) Huang, Cameron Neylon and Lucy Montgomery explore where citations come from for 19 million research outputs and find that open access is indeed working as it is meant to – allowing more researchers from more places to use research.
Open access (OA) advocates and researchers have spent decades thinking about how we can tell whether OA really works as it is meant to: ensuring that more people get to use research. In order to answer this question, researchers have often turned to citation data: asking whether OA publications are cited more often than closed publications, the ‘OA citation advantage’. We think it is more meaningful to consider whether OA outputs are cited more ‘widely’, i.e., cited by more researchers from more places.
In our paper, recently published in Scientometrics, we explored where the citations for 19 million individual publications came from, and made comparisons between the diversity of the groups citing OA and non-OA outputs. We found that two articles with the same number of citations can have very different levels of diversity in the institutions, countries, sub-regions, regions, and fields of research they are cited from. See Fig.1.
We assigned citation diversity scores to each research output included in our study. The higher the score, the greater the diversity of citing actors.
Fig.1: Output A and Output B both have two citations. However, Output A is cited by institutions from two different countries, while all citing institutions for Output B are from Country Z. Hence, Output A has a higher level of citation diversity by country. Output C and Output D both have five citations. However, Output C has both more citing institutions, and these institutions are from more countries. This implies Output C has a higher level of citation diversity by country, i.e., higher score in diversity measures (Gini-Simpson and Shannon indexes).
We examined 420 million citations over a ten-year period for 19 million research outputs. The data reveals that not only do OA outputs receive more citations, but those citations also come from more geographically diverse locations, as well as more diverse fields of research (Fig.2). The downward slope in diversity scores over time reflects the fact that earlier outputs have had more time to gather citations (hence more possibility of wider citing actors), but the pattern remains consistent: OA outputs are cited more often and have higher diversity scores than closed outputs.
Green OA, where an output is made available for access via a platform other than the publisher, shows the strongest diversity effect. We attribute this to Green OA having the advantage of more diverse access points (note that in this work our Gold and Green categories are not mutually exclusive). The stronger diversity effect for Green OA raises questions about the OA pathway most likely to produce the greatest positive impact – but this (long debated) question needs to be unpacked through further research!
Fig.2: The mean Shannon scores are compared across the OA categories, with the scores calculated based on the grouping of citing affiliation links by countries, sub-regions, and regions, and citing outputs by fields of research. For the first three cases, all OA categories consistently outperform CLOSED outputs. OA outputs also outperform CLOSED outputs for the fields of research in more recent years. This is likely a result of evolving research practices and data quality levels. We also note that the scores should not be compared across different citing actor types as they measure different ways of grouping citing actor links (hence different scales).
One of the challenges of using very large data sets to understand the effects of OA at global scale is ensuring that patterns in the data reflect real differences in the diversity of the groups using research. In order to test the statistical significance of the effect we observed we also looked for cases where the diversity score for OA was not higher. In fact we found very few! What we did find were consistent positive shifts in the distributions of citation diversity scores from closed to OA outputs. You can check through our efforts to find exceptions, as well as the exceptions that we did find, in an extended supplementary report.
In addition to finding that citation diversity is higher for OA research, we also found evidence that the biggest increases in citation to OA research are coming from researchers in low- and middle-income countries. This is consistent with the hopes of the OA movement: that opening research will help to address the barriers to impact associated with paywalls.
Researchers located in Northern Europe enjoy the biggest increases in their citation numbers when their work is made available OA
However, increased use by researchers who cannot afford to pay for access doesn’t tell the whole story. The data also points to continued North-South divides in the research that is being cited most often. Researchers located in Northern Europe enjoy the biggest increases in their citation numbers when their work is made available OA; as well as the biggest increases in the number of times their work is cited by researchers from Sub-Saharan Africa (see Fig.3). This hints at the need for future OA advocacy to focus on work to ensure a truly inclusive and equitable global research system.
Fig.3: Changes in citations to and from selected sub-regions. a. The three graphs resemble selected citation links to outputs by the sub-regions: Northern Europe, Sub-Saharan Africa, and Eastern Asia, respectively. Within each graph, the percentage change in total citations from the three selected sub-regions (for 2019) are shown. A value above zero indicates a positive effect for OA. While both Northern Europe and Sub-Saharan Africa benefit from OA outputs, there are differences in the results. Eastern Asia is one of the exceptions resulting from less comprehensive coverage by Western bibliographic systems. b. An alternative measure is used to track differences in mean citations between OA and CLOSED outputs—percentage ratios. The results are provided for all years included in the study. A value above 100 indicates a positive effect for OA.
This study adds a new perspective to our understanding of how citations can be used to explore the effects of OA. It also provides new evidence at global scale of the benefits of OA as a mechanism for widening the use of research and increasing the diversity of the communities that benefit from it. Finally, some two decades after the first open access declarations, it is gratifying to see OA has at least in part made good on its promise of sharing the benefits of access to research more evenly across the globe.
This post draws on the authors’ paper, Open access research outputs receive more diverse citations, published in Scientometrics.
The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.
Image Credit: All images reproduced with permission of the authors.
