Altmetrics are generally seen as indicators for online engagement and attention. However, taking the field of political science as an example, Gustav Meibauer, Kiran Phull, Audrey Alejandro and Gokhan Ciflikli use altmetrics to analyse the dynamics of knowledge production in the field. Finding that altmetrics show a highly hierarchical and gendered spread of attention to work in political science, they ask how and why these metrics can be used responsibly.
Academic knowledge production, exchange, and dissemination takes place through ever-diversifying and traceable digital channels. This is widely considered to be a benefit, as it works to democratise scholarship and evaluation. Digital sharing has transformed the environment within which disciplinary debates emerge and disseminate, creating possibilities for increased transparency, policy engagement, widening recognition, and enabling real-time crowdsourced peer review.
Altmetrics (alternative metrics) are indicators of digital attention that aim to measure these kinds of online interactions. They are usually a composite of metrics, including interactions (e.g., clicks, views, and downloads), capture (e.g., bookmarks, saves, and favourites), mentions (e.g., posts, comments, reviews, and attributions), and social media reactions (e.g., likes, shares, and tweets), in addition to citations and rankings. They may correlate with traditional ways of measuring research, e.g. citations or references. Yet, despite their growing use in research evaluation or funding allocation, altmetrics have received minimal attention outside the field of scientometrics.
This is interesting, as altmetrics can hold a useful mirror to the dynamics of disciplinary knowledge production. In this respect, as part of a recent study, we used altmetrics, to investigate unequal knowledge production dynamics in political science. We focus on gender*, allowing us to build on scholarship that investigates how gendered hierarchies emerge and are reproduced in the discipline, e.g., focusing on professional presence and representation of women, publication and citation practices, and pedagogy and teaching. We also wanted to explore what exactly altmetrics do, to and for the discipline, when it comes to gendered dynamics of knowledge production? To answer this question, we built and analysed a novel dataset combining information on author-gender and Altmetric.com Attention Scores for all articles published in 65 top peer-reviewed political science journals between 2013-2019.
Gender dynamics in altmetrics scores
We find that these Altmetric scores are by and large similar to known gender differences in publication and citation practices. Mean scores are highest on average for mixed-gender authored items (30.54). Exclusively female- authored research generates, on average, the lowest scores (19.23) as compared to exclusively male-authored research (24.49). Publications authored by women thus garner, on average, a 27% lower score than those authored by men.
Scores also increase over time. While some publications may have the same scores, the relative weight or importance of those scores vis-à-vis others heavily interacts with a recency factor. Older items tend to have lower average scores, largely as a result of changing levels of online attention and the growing usage of academic social media. Though Twitter emerged in 2006, “academic Twitter” as a tool for scholarly networking, dissemination, and outreach really began to gain traction around 2013. Overall, we find that the AAS parallels the networked dynamics of Twitter – where female political scientists have fewer followers on average and networked sharing tends to privilege male-authored research.
Fig.1: Gendered distribution of Altmetric Attention Scores per publication year
Male-authorship from academic “superstars” to null scores
A closer look reveals that these patterns are shaped by the overwhelming presence of high-scoring male disciplinary “superstars”, whose research attracts disproportionate online attention and generates viral sharing. The “viral hits” of research in political science remain overwhelming dominated by men: of the top 100 highest-scoring publications in the field, 67 are exclusively male-authored.
Fig.2: Top 20 Political Science Altmetric Attention Scores (2019), color-coded in terms of author gender – the greener, the more male-authored; the redder, the more female-authored
At the same time, male authors also dominate the research that receives little to no online attention (publications with scores of zero) – which matters more than one might think. If we average out the outliers, female-authored publications actually tend to garner higher scores than their male counterparts.
Fig.3: Distribution of Altmetric Attention Score ‘0’
High scores for male-authored research may reproduce the outsized influence that seniority could have in disciplinary and wider political networks. A situation indicative of a profession, especially in its higher rungs, that remains relatively homogenous and slow to change. Though women have moved to overtake men in terms of university entrants and in attaining Political Science degrees up to the PhD-level, the academic career ladder through to full professor continues to fail them and their experience of disciplinary spaces remains substantially different to that of their male colleagues. While a slowly changing field is corroborated by an increasingly diverse academic Twittersphere (e.g., with networks like #WomenAlsoKnowStuff), virality in online research likewise continues to evade women.
Measurements that reflect male academic “superstardom” or success through “virality” can work to reinforce a research environment (whether through funding, opportunities, career progression, etc.) that continues to privilege elite male scholarship. For example, extremes and outliers may generate better recall and attention. That “academic superstars” (whether on Twitter or within university faculties) are more likely to be male may thus reinforce a latent belief that high profile academics are the best bet a department or funding agency can make, even if they not representative of the discipline as a whole.
Measures of quality or inequality?
Bibliometric indicators are not neutral tools, but capable of influencing and producing norms, behaviours, and practices. They influence how quality, innovative, and cutting-edge research is valued – and what is rewarded as such. The use of performance metrics in research management highlights how measurement can generate disciplinary inequalities. These metrics then become “engines of anxiety” that promote particular standards of excellence and accountability. If simply taken for granted, altmetrics can contribute to these inequities.
What should political scientists (and academia broadly) do with this information? One avenue would be to “pay it forward” by supporting female and junior scholars in navigating digital academia and building networks, and by raising awareness of the gendered dynamics of online research-sharing. Another avenue would be to reject the use of altmetrics altogether, working instead towards rewarding intellectual labour differently, and creating more supportive online communities. And yet, altmetrics remain attractive tools that are likely here to stay. It strikes us as urgent, therefore, that professional associations, universities, and departments develop formal recommendations as to what constitute suitable uses and misuses of indicators. Similar to pedagogical initiatives that aim to tackle gender bias in scholarship, we encourage scholars to critically reflect on how altmetrics and social sharing impact their own research experiences, and to inform future scholars of the gendered dynamics inherent to digital scholarship. In this sense, if used critically and contextually, as we aimed to do in this study, altmetrics can be used as indicators for academic inequalities. Indicators which can be used to prompt changes to, and new ways of thinking about, disciplinary power structures as a whole.
This post draws on the authors’ article, Alternative metrics, traditional problems? Assessing gender dynamics in the altmetrics of political science, published in European Political Science.
*For coding purposes, we use the term “gender” rather than “sex” to denote categorizations of male/female authorship because it allows us to build associations with gender roles and norms within academia. We recognize that this may be viewed as problematic because it denies the inclusion of non-binary categories and can lead to assigning gender/sex erroneously.
The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.
Image Credit: LSE Impact Blog via Canva.
