By providing real-time information, altmetrics are shifting how research impact is understood. Jason Priem and Heather Piwowar outline the launch of ImpactStory, a new webapp aiming to provide a broader picture of impact to help scholars understand more about the audience and reach of their research.
The Web is awesome for lots of reasons, but for scholars it comes down to two:
- It lets us quickly create, disseminate, and consume completely new genres of scholarly products–things like blog posts, datasets, and software.
- It lets us publish, find, read, and value products on their own terms, replacing the heuristic of ranking based on a prestige of the publishing container.
In short, the Web lets us move beyond the article and beyond the Impact Factor. But, strangely…we haven’t. Or perhaps it’s not so strange when we consider that the scholarly reward system we’ve inherited is in fact built squarely on these two features. To move up or get funded, you need Traditional Articles in Traditional Journals.
How do we fix it?
The funny thing is, lots of people–both scholars and evaluators–would like to see that change. We get emails all the time from folks on funding, tenure and promotion committees asking, “how can I encourage these new products, broader impacts, and new publication venues?” But while the desire is there, the data isn’t.
Altmetrics is an effort to help fix that. By measuring activity around scholarly products in online tools and environments, we can capture unprecedented amounts of data showing all sorts of uses of all sorts of products by all sorts of people. We built a tool called total-impact to gather and display this data; it spits out reports like this:
And here we see another problem: data alone is not enough. The Wall Of Numbers approach to altmetrics tells much while revealing little. What we need are data-driven stories to show how new products and venues are making impacts among scholars and the broader public.
ImpactStory is a relaunched version of total-impact. It’s a free, open-source webapp we’ve built (thanks to a generous grant by the Sloan Foundation and others) to help researchers tell these data-driven stories about their broader impacts. To use ImpactStory, start by pointing it to the scholarly products you’ve made: articles from Google Scholar Profiles, software on GitHub, presentations on SlideShare, and datasets on Dryad (and we’ve got more importers on the way).
Then we search over a dozen Web APIs to learn where your stuff is making an impact. Instead of the Wall Of Numbers, we categorize your impacts along two dimensions: audience (scholars or the public) and type of engagement with research (view, discuss, save, cite, and recommend).
In each dimension, we figure your percentile score compared to a baseline; in the case of articles, the baseline is “articles indexed in Web of Science that year.” If your 2009 paper has 17 Mendeley readers, for example, that puts you in the 87th-98th percentile of all WoS-indexed articles published in 2009 (we report percentiles as a range expressing the 95% confidence interval). Since it’s above the 75th percentile, we also give it a “highly saved by scholars” badge. Scanning the badges helps you get a sense of your collection’s overall strengths, while also letting you easily spot success stories.
We’re excited for folks to try out ImpactStory, and excited to get feedback; it’s a beta release, and want to listen to the community as we prioritize new features. Working together, we can build something that helps reseachers tell data driven stories that push us beyond the Impact Factor and beyond the article.
Note: This article gives the views of the author(s), and not the position of the Impact of Social Sciences blog, nor of the London School of Economics.
About the authors:
Jason Priem is a 3rd-year doctoral student at the University of North Carolina at Chapel Hill, studying how the Web is revolutionizing scholarly communication. Jason has been a leader in the altmetrics movement, investigating new measures of scholarly impact on the social Web, and also helps to lead the open-source ImpactStory project. He tweets at @jasonpriem.
Heather Piwowar is a postdoc research associate, funded by the NSF-funded DataONE cyberinfrastructure project. She works with the Dryad team at NESCent, studying data sharing and reuse behaviour. She is based remotely in beautiful Vancouver Canada, with a home base in the Biodiversity building in the Department of Zoology at the University of British Columbia. Heather’s research focuses on studying the patterns, prevalence and impact of data sharing and reuse behaviour of “small science” post-publication datasets. Her doctoral research focused on biomedical data, particularly gene expression microarray datasets.