The gender gap in citations between male and female researchers is well documented. However, the reasons for this gap are less certain and widely contested. Discussing findings from a mixed methods analysis of research publications from Norway, Lin Zhang and Gunnar Sivertsen find that whilst papers authored by female researchers are less cited, they are more frequently engaged with by readers. Through closer analysis of the abstracts of these papers, they argue that these papers more often involve projects aimed at societal progress, which overall are less valued by academics and receive fewer citations.
This research project started with an observation that demanded an explanation. After measuring the citation impact of almost 27,000 scientific publications with female and male 1st authors among almost 12,000 researchers at Norway’s four major universities, we found a clear gender gap in citation (Norway was chosen because it has a national database with reliable information about the gender, age, positions, affiliations, and publications of all researchers). That publications by female researchers are less cited was in itself not surprising, as this has been widely observed and discussed. However, with the addition of an alternative indicator of impact, abstract views, we found something more surprising, publications by female academics attracted more interest from readers, yet still they remained less cited.
An obvious possible explanation for this phenomenon was disciplinary, might female researchers favour fields of research that were more socially oriented and accessible to larger audiences, for instance nursing research over mathematics? This explanation was not helpful. We found the same gender difference within almost all fields of research. The publications by female 1st authors had attracted less citations and more readers. Why?
After reading one hundred abstracts from the publications in our data, randomly selected from the whole spectrum of disciplines, we arrived at a hypothesis: Male researchers more often engage in research aimed at scientific progress, while female researchers more often engage in research that alongside scientific progress aims at societal progress.
Normally in our field of research, quantitative science studies, we would have used automatic methods for content classification in large sets of publications, but the aims of research are not easy to identify using topical words, shared references, or standard phrases. However, the genre of the abstract requires that the aims of research are made explicit, most often at the beginning or at the end.
We selected almost 1,200 abstracts from our data of publications that were either very highly cited, frequently viewed as abstracts, or both. Around one third was in each of the three categories. Then, we read all the selected abstracts, independently and blinded from the previous analysis. We categorised the publications as mainly aimed at scientific progress, mainly aimed at societal progress, or both. Then we compared with the previous results: Publications classified as aimed at societal progress were clearly among those with most interest among readers, and they more frequently had female 1st authors. Publications classified as aimed at scientific progress were more frequently among the highly cited ones and among those with male 1st authors.
Again, our results partly confirmed observations since the seventies: Basic research is more cited than applied research. However, rather than basic/applied, we prefer the distinction between scientific and societal progress, because, it is easier to apply when reading abstracts from all fields of research to find specific expressions of aims within the wide definition of impact in the REF: “an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.”
What was not known before, is that citation impact versus interest among readers is related to the aims of research, and that there is a gender difference here. We were still curious to understand why female and male researchers to some degree prioritise different aims of research. Is it about values and motivations?
What was not known before, is that citation impact versus interest among readers is related to the aims of research, and that there is a gender difference here.
To answer this question, we could use the results from a survey that was conducted in 2017-2018 among researchers in cardiology, economics, and physics in five European countries: Denmark, the Netherlands, Norway, Sweden, and the United Kingdom. We collected more than two and a half thousand responses to questions about their research. One question was about motivations for engaging in research and another about what kind of research they considered the best. The answers to both questions clearly indicated that scientific progress is highly valued among all researchers, male and female. However, we found clear indications that female researchers are more motivated than male researchers to also engage in research aimed at societal progress. In addition, female researchers more often rated contributing to societal progress as a characteristic of the best research. This points towards our conclusion:
- Male researchers more often value and engage in research mainly aimed at scientific progress. They are more cited within the scientific literature.
- Female researchers more often value and engage in research that also aims at societal progress. They gain more interest among readers.
Neither of these points exclude other possible explanations for the same differences. Our concern now is that our findings have implications for evaluation, funding policies and practices within research systems. A critical discussion of how societal engagement versus citation impact is valued, and how funding criteria reflect gender differences, is warranted.
This blog post is based on the authors’ article, Gender differences in the aims and impacts of research, published in Scientometrics.
Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
Image Credit: Adapted from Ussama Azam via Unsplash.
Fascinating post. As a policy person in the US focusing on advancing Next Generation Evidence, which seeks to improve use of data and evidence to improve core societal outcomes, I am interested in understanding more about who is engaged in thinking about societal progress. This is helpful.
Nice to see that our study is helpful also in a policy context beyond research. This inspires us to proceed with developing methods for identifying and measuring research oriented towards societal progress.
I have done a bit of research tracking citations of academic-led Open Access journals that are challenging the big publishers. The gender dimension is indeed extremely hard to track, and in most countries, as you say, author gender is not always obvious or multi-authors have different genders. in some disciplines and research teams, ‘first author’ is simply alphabetically determined. The other problem is background or ‘shadow’ citation where authors of an article quickly dispense with work in the field in the first para of the lit review section , citing a long list of papers or books without engaging at all with the material cited. A true measure of quality can only be gauged by reading individual articles and picking out the perhaps two or three studies that are approvingly used to build on, critiqued, or that are otherwise regarded as central to the analysis. And not negative citations either, where an article is cited but then dismissed as irrelevant or wrong, in the text of the article being scrutinised. So I think a really good citation analysis would go ‘deeper’ and probably have to be qualitative. There may indeed be some gendered trends in the latter ‘deep citations’ that are approvingly cited and built upon in an article but that would be extremely hard to analyse, even before getting to author gender [in countries lacking that data, unlike Norway]. An example is Tuck and Yang, a key article in decolonisation studies. It was published in a fairly minor OA journal in its day, that has now ceased publication. https://clas.osu.edu/sites/clas.osu.edu/files/Tuck%20and%20Yang%202012%20Decolonization%20is%20not%20a%20metaphor.pdf Of its c 4600 citations in 9 yrs, massive for the humanities, some authors engage deeply with the argument [roughly, that decolonisation is more than changing our pedagogy and society, it is also about returning land and resources to their original owners]. Others place the paper in a long list of ‘existing work’ on decolonisation and give it no more attention after that. And the paper was written by 2 people, identifying as a woman and man, with no indication that the 1st female author was lead author or alphabetical 1st author. How would deep and shadow citations figure in bibliometrics like yours?
We agree that a qualitative approach to citations is warranted if the purpose is to study the type of impact an influential publication has in new publications. This time, we observed the quantifiable citation impact and interest among readers of several thousand papers. Then we chose other qualitative methods (text analysis, survey) to explain the observed gender differences. We would not have got that far without quantification in the first place. This quantification was not an attempt to measure quality.
Only 2,4 percent of the 27,000 articles we studied are in the humanities. (There is limited coverage of the humanities in Web of Science.) Most of the articles we studied have three or more authors, and among them, only five percent have authors presented in alphabetical order. So, we think it make sense to see first authors as taking most of the responsibility for the specific project. This is the assumption in many similar studies as well, but we agree that it is contestable.
Have you looked at trends over time? And at possible bias arising from citation management software? I ask because of the now long-term trend in some countries for married female authors to identify themselves with a hyphenated name–the first half being the woman’s family name and the second half being the married name. And multigenerational hyphenated names have always been common in some cultures. I have used software that makes citation of such names tedious or impossible. But, of course, abstract views are easy no matter what the format of the authors’ names.
Thank you for your interest and your questions. We recognize the problem with gender disambiguation in data sources that only provide author names. We solved it by using the national Norwegian database of researchers and publications in which bibliographic data is linked to real persons, not just author names. We know the gender, age, position, affiliation and field of research of every author studied – independently of the name and the information given in Web of Science and PlumX. Indeed, it would be a good idea to study trends over time using the same rich data. For this and other reasons, we are thinking of expanding this study.