Researchers have long been encouraged to use Twitter. But does researchers’ presence on Twitter influence citations to their papers? José Luis Ortega explored to what extent the participation of scholars on Twitter can influence the tweeting of their articles and found that although the relationship between tweets and citations is poor, actively participating on Twitter is a powerful way of promoting and disseminating academic outputs, potentially indirectly influencing the scholarly impact and improving prospects of increased citations.
Many altmetric studies – meaning altmetrics in general rather than Altmetric.com specifically – have wrongly taken the conceptual framework of bibliometrics as a model to understand the meaning of alternative metrics. For example, it has been assumed that the mention of a research paper on Twitter is comparable to a bibliographic citation, taking for granted that a tweet can be an appreciation index and Twitter a kind of citation index (Eysenbach, 2011; Shuai et al., 2012). However, the processes that generate a citation are very different to a tweet. To have an article cited one must first conduct a study, write a paper and publish it in an indexed journal; whereas the mention of a paper on Twitter requires one only to write a short message. This difference could cause the greater intervention of authors in the mentions of their own papers, at least relative to the case of citations (i.e. instances of self-citation).
Starting from that hypothesis, my recent research explores to what extent the participation of scholars on Twitter can influence the tweeting of their articles, and, by extension, the likelihood of those articles being cited. To that end, 4,166 articles from 76 Twitter users and 124 non-Twitter users were analysed. These data were crawled using PlumX Analytics, which counts the number of tweets a document receives, whereas Scopus was used to extract citation numbers. Finally, a manual search was done to distinguish authors with a handle on Twitter from others not registered to that social network.
Image credit: twitter-sparrow 092 by Josey. This work is licensed under a CC BY 2.0 license.
Results showed that papers from Twitter users could be on average 33% more tweeted than documents of non-Twitter users (Twitter users mean = 2.33 tweets per paper; non-Twitter users mean = 1.75). Obviously, this result is important not because participating on Twitter improves the mention of academic papers, but because the number of mentions could be greatly influenced by the authors of those papers. This introduces the possibility of manipulation of tweets metrics and puts in doubt the suitability of Twitter as source for research evaluation.
Logically, merely having a Twitter account is not enough. So, specifically, what author activity on Twitter most influences the mention of their papers? To answer that question, number of tweets, followers and followings of 76 Twitter users were extracted and time-normalized. Next, a linear regression analysis was performed to detect the variable that most influences the mention of articles. Results showed that the number of followers explains 34% (R2=.341) of the tweets received by their publications, claiming that a 1% increase in followers would generate 0.31% of new tweets. This result demonstrates that the number of followers is an important factor for the spreading of messages on Twitter. However, the effect of this variable is small because an author needs three times more followers to gain only one mention more.
Nevertheless, the most interesting question in this study is to clarify the relationship between tweets and citations. Does authors’ presence on Twitter have some influence on the next citations to their papers? Results show that there is no statistical difference (p value = 0.144), and therefore to be or not to be on Twitter does not affect the number of citations (non-Twitter users mean = 1.77; Twitter users mean = 2.00). This once more evidences the poor relationship between tweets and citations and puts in doubt suggestions that mentions in social networks can be considered an early proxy of research impact. However, when the regression was applied to detect if some activity parameters on Twitter are related to citations, a weak yet significant relationship was found between followers and citations.
Thus, followers explain only 17% (R2=.171) of the number of citations, meaning a 1% increase of followers could produce 0.24% of citations. Although this result could suggest that participation on Twitter has some influence on the citation of articles, it should be interpreted in a different way, introducing the concept of dissemination. As we have seen, Twitter followers act as information speakers by retweeting articles. Authors with a large number of followers can reach a much wider audience, increasing the likelihood that their papers are cited in future. This new interpretation highlights the importance of dissemination in the citation of articles and suggests that part of research impact could be explained by the intensity in which a paper is spread. Likewise, the more media used to promote an academic result, the greater the likelihood of it being read and later cited.
In conclusion, actively participating on Twitter is a powerful way of promoting and diffusing our academic outputs. This allows us to maintain a wide network of followers that amplify our message and reach a larger audience. Indirectly, such broad dissemination could influence the scholarly impact, slightly improving the prospect of increased citations. Therefore, Twitter cannot be viewed as a citation index, but as an information-spreading network; and the tweets of articles should not be considered an impact indicator, but a measure of research dissemination. However, these conclusions set out a disturbing fact about the use of citations for research evaluation. If dissemination could be a factor for citation success, then to what extent are citations a reflection of research quality and novelty? Resolving this doubt would be an appealing challenge for those studying bibliometrics.
This blog post is based on the author’s article, ‘To be or not to be on Twitter, and its relationship with the tweeting and citation of research papers’, published in Scientometrics (DOI: 10.1007/s11192-016-2113-0).
Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
About the author
José Luis Ortega is a web researcher partner of the Cybermetrics Lab at the Spanish National Research Council (CSIC). He has published more than 40 research papers about web metrics (link analysis, altmetrics, etc.), information consumption, web usage mining and academic search engines (Google Scholar, Microsoft Academic Search). Recently, he has released the monograph Social Network Sites for Scientists: A Quantitative Survey where he analyses the most relevant academic social networks (ResearchGate, Academia.edu, Mendeley, etc.) using webometric techniques. His ORCID iD is 0000-0001-9857-1511.