Using Google to gauge impact: the Nobel Prize in Economics

Winning a Nobel Prize really is the ultimate demonstration of academic impact, but why would the public seem more interested in one of two joint winners? Rebecca Mann traces the public’s desire for information on Roth and Shapley, this year’s winners of the Nobel Prize in Economics, and looks at why the desire for information on Roth is much higher than his partner.

The role and impact of prizes in research is hotly debated. There is no doubt that the award of some prizes, such as the Nobel Prizes, John Bates Clark medal and MacArthur Fellowships confer substantial prestige on the recipients and their institutions. It is also clear that the intense media coverage associated with such awards substantially raises the profile of winners, such that academics who were previously well-known only within their own field or university achieve household recognition with the general public, if only for a short period after the award.

However, quantifying the degree to which prizes have an effect on the research profile and impact of an academic with any accuracy is clearly difficult. This is because there is no ‘counter-factual’ against which to compare what an academic’s citations and stature may have been without the award, the effect of which cannot then be disentangled from positive pre-existing trends.

To explore whether Google can be used to measure the impact of an academic prize on the public profile of an academic, we conduct a fun quasi-experiment (of highly dubious robustness) and briefly explore some of the factors which may influence the relative size of the effects that we find.

As is now well known, the Royal Swedish Academy of Sciences awarded the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel for 2012 (‘Nobel Prize’) to Alvin E. Roth of Harvard University and Lloyd S. Shapley University of California, Los Angeles for their contribution to the field of market design and economic engineering.

We noted the number of Google ‘hits’ that the full names of both Roth and Shapely generated in a web search within the first 60 seconds of the announcement of the prize on 15^th October 2012. We then conducted the same search for both academics at irregular intervals over the next 24 hours to observe how the number of results increased after the announcement and compared how the results associated with Roth and Shapley changed in that period. Helpfully, both academics have relatively distinctive names so that we can attribute the bulk of any increase to them rather than other similarly-named people.

As would be expected, the online impact associated with the award of a Nobel Prize, as measured by an increase in Google search results, is enormous. At midday (GMT) when the prize was announced, a search for Alvin Roth returned 662,000 results. Just under one hour later, that number had grown to 5.9 million hits. The number of results continued to increase, albeit more slowly, over the next 24 hours, with 6.38 million at 5pm and 16.5 million one full day later. This final figure represents a 25-fold increase on the initial number of results just one day after the announcement.

It is interesting to note that the results trajectory for Lloyd Shapley looks somewhat different. When the results were announced, a search for ‘Lloyd Shapley’ generated 143,000 results. One hour after the announcement, Shapley’s results had only increased to 150,000 and at 5pm, Google returned 179,000 hits. Overnight, however, Shapley’s results surged with a count of 7.12 million hits at midday the next day. This final figure represents a 50-fold increase on the initial number of results.

So although Shapley performed, relative to his initial level of visibility, twice as well as Roth, what factors could explain the fact that Roth gained more than twice as many hits (8.8 million) as Shapley? Three potential explanations relating to demographic, spatial/institutional and visibility/accessibility related factors could explain the differing levels of digital attention that Shapley and Roth appear to have gained from the announcement of the Nobel Prize

First, Shapley at 89 is considerably older than Roth, 60. Although he holds a position at UCLA as an Emeritus Professor, the bulk of his research appears to have been published in the sixties and seventies which means it could be more difficult to access than more recent research (his UCLA website, for example, was last updated in 1997). It may also mean that he is less likely to give as many interviews or generate the type of media content which drives Google results. Roth, for example, updated his blog within hours of the announcement, in a fashion (‘Blog may be delayed today….’) and his post-announcement interview was widely reported and tweeted (in particular, his proposed bacchanalian celebration plans, consisting of ‘coffee’).

There may also be institutional factors driving the disparity in results. Roth, at Harvard Business School, may benefit from superior resources promoting his success compared to Shapley at UCLA. In terms of social and professional networks, Shapley may also be less ‘plugged in’ than Roth who is actively publishing, teaching and attending conferences and academic events. In spatial terms, the East Coast of the US may have denser links between universities, think-tanks, government agencies and the media, which might influence the pattern effects of how the news of the prize spread across commentators and media consumers.

Finally, it is possible that higher public and academic ‘visibility’ of each academic prior to the prize could be relevant. Roth has a popular blog which he updates daily, a detailed personal website and CV, and a Google Scholar profile. He offers economic consultancy services and has given radio interviews. He has also co-authored papers with well-known economists including Richard Thaler and Gary Becker.

These factors which make the work and story of Roth more accessible to the press and interested readers might partially account for the higher Google results he has generated after the prize.  Such measures may not be emphasised in the traditional roles in academia, but as has been often noted on this blog, taking these measures can be useful in increasing ‘impact’, as measured by Google results and more traditional criteria. Some support is lent to this hypothesis by the fact that few people appear to have directly accessed the research for which the prize was given: in the six hour period after the award, only 4 and 2 new downloads of Roth and Shapley’s papers respectively were made from the online Social Science Research Network (SSRN). Over 24 hours, that could be increased to 26 and 10.

Of course, Google results are not the perfect measure of impact. The number of actual clicks or views of articles about Roth and/or Shapley would give a better measure of the public’s interest and awareness of the research for which they were awarded the prize. The Google search algorithm may behave in unexpected ways and many of the results are likely to be duplicates – in particular, the stunning jump in Shapley’s results overnight is difficult to explain.

Is it plausible that demographic, institutional and visibility factors could account for any slice of the disparity between Roth and Shapley’s post Nobel fame? Can web search results be used in a robust way to advance the discussion about academic impact? Perhaps a quantitative approach may miss the obvious reason why Roth has enjoyed such a stellar rise in public consciousness as suggested by our Google search: better awareness of his work on the economics of eating horses and dwarf tossing.

Note: This article gives the views of the author(s), and not the position of the Impact of Social Sciences blog, nor of the London School of Economics.

About the author:
Rebecca Mann is a researcher with the LSE Public Policy Group. She holds a Master of Public Administration from the LSE and degrees in Law and Economics from the University of Sydney. Previously, she was a lawyer at Herbert Smith LLP in London and an adviser in the Innovation & Strategy Unit at the Department of Justice in Melbourne, Australia.