As many academics and researchers quickly come to understand, finding out how your work is being used by others can be a tricky business. Using a combination of the three easily available tools – Publish or Perish, Google, and Web of Science – may provide the best way for academics to measure various areas of impact.
In the past academics and researchers have had relatively few tools at hand for finding out which bits of their work are appreciated and used by other academics. Our advice to researchers wanting to find out how their work is being used by other academics is to use a combination of the three best available tools, which are:
- ISI Web of Science, which is most useful for senior academics with a slate of published work already in high impact journals; and
- Google Book Search and Google Scholar for people working in disciplines where books and other non-journal academic outputs are important;
- Harzing’s ‘Publish or Perish’ software, which is a tweaked version of Google Scholar that delivers rapid feedback and covers more and more diverse sources than anything else.
Conventional citation systems like ISI Web of Science and Scopus have some severe limitations that need to be kept in mind by social scientists, as they cover only around 30 to 40 per cent of journals and books in social sciences published worldwide. Most bibliometric experts acknowledge that the usefulness of these systems declines sharply if they include fewer than three quarters to two thirds of all journal articles world-wide. The conventional systems also have a heavy bias in coverage towards American and English-language journals and tends to deliver rankings and statistics that are weighted heavily towards success in the US ‘market’, compared with the rest of the world. The ISI system does not cover references in books, which poses serious difficulties for accurately measuring citations within ‘softer’ social science fields and humanities where books remain very important. The older systems completely exclude references in working papers or conference papers, which are important for social scientists as publishing a journal articles can take several years.
Internet-based citation systems like Google Scholar and Google Books, on the other hand, voraciously and automatically record all citations. In particular they include:
- all ‘black’ literature in journal articles or books that has been definitively and formally published, plus
- less conventional ‘grey’ literature, such as working papers, conference papers, seminar discussions or teaching materials that has been issued in a less formal or definitive form – often, of course, including versions of material that is later formally published.
This inclusiveness makes Google Scholar far more up-to-date in its picture of academic debates and controversies in each discipline. It also gives users much more immediate information about the work being found and often gives full-text access to it if the material is not in a published book or behind a journal pay wall. Google Books is a system that is primarily designed to make available a range of different online views of a book’s contents to potential readers. Its development and citations-counting capacity will have very substantial consequences for how academic research develops, especially in the most book-based disciplines, such as the humanities and ‘softer’ social sciences. However, there are limitations with online systems as well, such as the secrecy of algorithms used and the inclusion of questionable academic material or duplicate materials, which has implications for accurately counting the number of outputs and citations.
Finally, there are now simplified and tweaked forms of accessing Google Scholar such as Harzing’s ‘Public or Perish’ software. This is a valuable programme that combats many of the problems of interpreting Google Scholar outputs and allows academics to easily check their own or others’ performance. It presents academic outputs quickly and computes excellent citation statistics about each author’s work, including an overall ‘times cited’ score and times cited per year since publication.
Academics will have to experiment with each type of citation-tracking system and see which works best for them in their particular discipline. More detailed comparative data on each system is presented in the handbook.