Proving dissemination is only one half of your impact story: Twitter provides proof of real-time engagement with the public

Getting a research story into the broadcast media can be an important opportunity to show impact. Alistair Brown explains how to use Twitter to monitor real time responses to a radio story, in order to assess the effect on a public community.

Especially in arts and humanities disciplines which may not have readily quantifiable economic benefits, the movement of research from academic to public discourse may be a key component of an impact statement. For REF 2014, impact is defined as “an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.” Researchers are increasingly encouraged to share their research “beyond academia” as a way of demonstrating their contribution to society at large. Research insights may, in the first instance at least, be interesting to people in their own right even if not more instrumentally significant.

Such was the case with a journal paper by Dr Helen O’Connell of Durham University’s Department of English Studies, which drew on pamphlets found in the National Library of Ireland to show how tea-drinking in Ireland in the early 1800s was viewed as being as bad for women as drinking whisky (for men) and was associated with insurrectionary behaviour. The potent and politically troubling combination of tea and whisky made this an obviously engaging story for the public. The University’s media office arranged for an interview with Helen O’Connell and the historian Lucy Worsley on BBC Radio 4’s Today programme on 5^th December 2012, and issued a press release which was picked up by other media outlets over the course of the day. The impact in the sense of reaching “beyond academia” was undoubted.

However, REF impact involves an assessment of “significance” as well as “reach,” so the mere fact that research has been disseminated to a wide audience does not constitute an impact by itself; one has also to show the effect it has on those to whom it is disseminated. For this reason, citing the fact that a researcher has appeared on a primetime radio show with several million potential listeners might be one element of an impact statement, but one needs also to evidence that the audience has actively listened to what was being put out, and that it has affected, changed or benefitted them in some way.

Given that (phone-ins excepted) radio broadcasts are a one-way communication medium, online social media – especially the inherently spontaneous Twitter – perhaps the only viable way of gathering such feedback. These reactions may be of particular value since the immediacy of a live broadcast encourages people to offer a personal reflection or judgement on the report. In responding to this particular radio report, it was noticeable that “Twitterers” did not simply post a link and headline as they might have done in reaction to a story on a static webpage, since there was as yet no obvious source to link to (excepting the University’s formal press release). Almost every tweet was instead a direct and individual consequence of what they were hearing. Many of these tweets registered enjoyment, interest, or urged other Twitter followers to “listen again” to the programme. Such quotes could be cited as evidence of impact.

When a piece of academic work receives broadcast media coverage, then, it is useful to have a strategy in place to gather emerging responses, and it is also far easier to do this as it happens rather than retrospectively. The principal reason for this is that Twitter posts older than 7 days are not readily available through public search. Although there are tools that allow for historical searches (such as a Google Custom Search for Twitter or Topsy) these are not wholly reliable and are dependent upon the much-criticised intricacies of Twitter’s search API. For example, a search of Topsy for the keywords “tea AND Ireland” shows only 1 hit responding to the Today programme, as opposed to the 24 tweets with these same keywords that were identified in real time responses.

Another problem is that Twitter simply lists tweets as having been published a number of days ago rather than with a definitive timestamp. Although external tools like Topsy allow one to search for tweets within a particular time period on a given day, as mentioned above this does not return every result. This is problematic because as a story begins to break on websites, searching for the same keywords within the mass of data from a given period will turn up mainly tweeted and retweeted links, with very little in the way of personal response to the live broadcast. Finding a tangible reaction in amongst mere duplication would be extremely laborious after the event.

Retrospective searching of Twitter is thus not really a viable option. By contrast, it was very easy to identify responses happening in real time. In order to monitor Twitter reactions, I used the browser extension Tweetdeck. Searches can be configured very easily in this: simply type the search terms, and then select “Add Column.” All results for this search will now stream down the viewing pane in real time. For this particular story, I added searches for “tea AND whisky” and “Ireland AND tea”. I also added a search for “@bbcradio4”. This would pick up any responses directly to the programme; naturally this included reactions to all items run that morning, but by watching in real time it was easy to spot the most relevant ones around the particular time slot for the tea story, again something that would be harder to do afterwards. I also ran a keyword search for “tea” which (especially at 9.00 in the morning!) naturally gave a vast number of returns. However, this was useful because I noted that some relevant responses were about “tea” and the variant spelling “whiskey.” Having identified this – something again only possible when monitoring live – I then added “tea AND whiskey” as a further search term. As the items came in, they were copied and pasted into a separate Word document.

About 30 minutes after the programme, Twitter responses dried up, and gradually responses to online articles (which were now being produced from the morning press release) took over the search terms. The vast majority of these lacked any personal reactive voice but were merely retweeted links. Whilst it would be useful to follow the comments beneath individual online articles for further evidence of impact, by this later stage Twitter is of less value. However, Twitter is extremely valuable as a way of assessing individual responses to broadcast media; in order to harness such reactions, real time monitoring seems to be the best option.

Note: This article gives the views of the author(s), and not the position of the Impact of Social Sciences blog, nor of the London School of Economics.

About the author:
Alistair Brown is a Postdoctoral Teaching Assistant in the Department of English Studies at Durham University, where he edits the blog Research in English At Durham (READ): http://readdurhamenglish.wordpress.com/ He is also an Associate Lecturer at the Open University, teaching courses on the Arts and Literature. He tweets as @alibrown18 and blogs at www.thepequod.org.uk.