The time-lag associated with citations and journal publishing means that such strategies are almost useless as a means of identifying relevant papers from current literature. Martin Fenner writes that social media, and Twitter in particular, stands to change all that providing almost instant, relevant recommendations: your own ‘personalised’ journal.
‘Can Tweets predict citations?’ asked Gunther Eysenbach in a recent paper that analyzed tweets about academic papers published in the Journal of Medical Internet Research (JMIR). He analyzed a total of 4208 tweets citing 286 distinct JMIR articles. The main conclusion of the paper is reflected in the title: Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact.
For this analysis a subset of 1573 tweets about 55 articles (published between March 2009 and February 2010) was classified into 12 highly tweeted papers (in the top 25th percentile of each issue) and 43 less tweeted papers. Eysenbach correlated the number of tweets with the number of citations in Google Scholar or Scopus analyzed 17-29 months later. Nine out of 12 of the highly tweeted articles were also highly cited, compared to 3 out of 43 of less-tweeted articles that were highly cited for a rate ratio of 0.75/0.07 = 10.75 (95% confidence interval, 3.4–33.6). The paper itself has already been tweeted 878 times, and, according to its research, will most likely become highly cited.
There are some limitations to the study – a small sample size, the possible bias by automated tweets from JMIR or the analysis of a journal with a very internet-savvy readership and therefore not representative for scholarly journals in general – but that does not mean that we should not take another look at the effectiveness of citations as a measure of academic work. Citations have a serious limitation as a metric of scholarly impact: they simply take too long. We can identify highly cited papers after two – five years, but after this time the original paper probably no longer is the most relevant text on its topic. This makes citations almost useless as a strategy to help researchers identify relevant papers from the current literature.
The traditional strategy for scanning newly published papers until now has been:
a) to pick papers based on the journal they were published in,
b) search strategies based on keywords, or
c) personal recommendations.
Journals continue to be an important filter for relevant literature, but they are obviously not targeted to your personal interests. Search strategies can be targeted to topics or people relevant to your work, but keyword-based searches can’t really distinguish between a good and bad paper about the same topic. Until email and social media became commonplace, personal recommendations were limited to a fairly small group of people (your colleagues at work and your peers you met at conferences).
Twitter – and to a lesser extend other social media – is changing all that. Tweets are immediate, 60 per cent of the tweets about JMIR papers were sent the day the paper was published, or the day afterwards. Tweets are personalized, as you see only the tweets of the people you follow in your Twitter stream (unless of course you do a keyword search). Tweets linking to scholarly papers have become popular, the 286 papers in the JMIR dataset were tweeted 14 times on average (for most other journals the numbers are obviously lower). Tweets can contain more than the paper title and link, or own recent qualitative analysis of 467 tweets about scholarly papers in the CrowdoMeter project showed that 10 per cent of tweets included a positive statement about the paper (while 1 per cent were negative), and 27 per cent of tweets highlighted conclusions from the paper.
Tracking social media citations of scholarly papers is part of the altmetrics movement. What is still missing are better tools that integrate social media with scholarly content, in particular personalized recommendations based on the content you are interested in (your Mendeley or CiteULike library are a good approximation) and the people you follow on Twitter and other social media.
These tools will come in many shapes and forms, but should also integrate into the activity stream of your favorite social media, and should include “personalized journals” targeted to your very specific interests and academic connections. We also need more research like the Eysenbach paper about what it means if someone is linking to a scholarly paper via social media. But I’m positive about one thing: a few years from now the “personalized journal” will have replaced the traditional journal as the primary means to discover new scholarly papers with impact to our work.