LSE - Small Logo
LSE - Small Logo

Blog Admin

October 16th, 2015

Looking Forward: Social data informs us about society, but also about the forces that will come to shape the future.

1 comment

Estimated reading time: 5 minutes

Blog Admin

October 16th, 2015

Looking Forward: Social data informs us about society, but also about the forces that will come to shape the future.

1 comment

Estimated reading time: 5 minutes

anneFrom online engagement software to leveraging search data for prediction, social data analysis is at the forefront of groundbreaking research. Anne Burns explores the topics recently discussed by academics and industry leaders. Academia needs to be aware of these discussions in order to provide a critical response to them and to assist in developing ethical and sustainable forms of practice. But academia needs also to utilise social data in order to gain useful insights into contemporary social issues.

The Social Data group on Meetup recently convened at Twitter in London for a session entitled ‘Understanding the Global Pulse using Social Data’. Following on from Farida Vis’ account of their first meet up at Twitter in April, this second event gave an overview of how social data is being approached by both industry and academia.

Social data, specifically data arising from social media, is becoming an increasingly important field for research across multiple sectors, from marketing and publishing to government and academia. Publications by Ipsos MORI, Comscore and Demos demonstrate the ways in which online public attitudes and behaviours are being analysed in order to understand not just what is happening now, but how to anticipate the future and facilitate behavioural change.

Francesco D’Orazio (@abc3d) – Social media image influence

Opening the session was Francesco D’Orazio, VP of Product at Pulsar, and member of the Visual Social Media Lab. He spoke about a new project, which looks at the shift in public discourse on Twitter that occurred in the wake of the death of three-year old Syrian refugee Aylan Kurdi, whose body washed up on Bodrum beach in Turkey in early September. Two photographs were widely shared, including on social media, changing how these issues were discussed. As D’Orazio’s graph shows, below, the photograph’s emergence caused a sizable spike in the use of the term ‘refugee’, demonstrating that the image influenced public discussion of the wider crisis. Further results of this study will be shared by the Visual Social Media Lab towards the end of October, as part of a project on The Iconic Image on Social Media.

social data immigration

Azeem Azhar (@azeem) – Social media analytics and machine learning

Azeem Azhar, the founder of PeerIndex, discussed the growing application of AI in social media analytics, in a talk entitled: “From counting likes to teaching HAL: The future of social data”. He began by giving an overview of his background in social media research, relating to the development of tools for determining relative levels of expertise online. Azhar’s vision for the future of social data had four components: through these analytics components, social data would become invaluable, everywhere, every larger in scale, and increasingly the domain of bots and AI. Azhar showed us two examples of AI – a rather creepy robot, and a chat bot app called XiaoIce – in which the ‘intelligence’ is the result of information aggregated from online sources. Some of the results of this are a little uncanny; almost human, but not quite.

Using an example of machine learning in the form of the site Crystal Knows, Azhar demonstrated the connection between ascertaining personality types from online data, and developing ways to communicate with them (e.g. in the form of advertising).

social media analytics

Although this knowledge of how to approach a potential customer is of obvious use, the value of social media can be thought of using Gartner’s analytic value escalator (above), in which knowing what happened, and why, is supplanted by an ability to use datasets to predict what might happen in the future. In this field, the site Dataminr uses Twitter conversations to predict what was going to trend, and to ascertain any relationship between online discussion and factors such as future stock market performance. The future AI bot will therefore be doubly uncanny, in that it will know not just how to respond to us, but also anticipate what we might wish to discuss in advance.

Chris Austin – Online engagement tracking

As the head of analytics and insight at The Guardian, Austin’s role also involved mapping a kind of ‘value escalator’, progressing from simply considering what has happened with regards to online engagement with the newspaper’s content, towards understanding why, and what this indicates for future approaches.

I was particularly struck by his diagram that mapped out several pathways for progressing through a project (below), navigating between the extremes of chaos / structure and unknown / facts. His favoured approach entailed an interconnection between logic and data, which although somewhat chaotic at times, yielded an end result that was stable and strong. I’m very much in favour of this kind of transparency, in which we get to see the process of working out what to do with data, as well as its results. As any social media researcher will know, there are often periods of outright chaos, in which the data is doing or showing something unusual. Austin’s approach encourages us to go along with the chaos, and view it as not an indicator that something is wrong, but as an integral part of the process that works to test our ideas and to validate them.

social data problem solving

This discussion was then followed by an overview of the Guardian’s in-house monitoring software, Ophan, by Chris Moran (see screenshot below). The Ophan dashboard can present an enormous amount of data regarding individual stories and their performance, in terms of page views, engagement time, sites of referral, and geographical location of viewers.

social data guardian

Moran described how this kind of tracking, through social data, is used for planning future activities, in which the ‘story of the story’ indicates how to improve viewer engagement. As Moran argues, this isn’t just a question of counting page views, but rather observing which sites viewers are referred from, and how a conversation develops around what tweets feature links to the post.

Alasdair Rae (@undertheraedar) – Search trends

Alasdair Rae is a senior lecturer at the University of Sheffield, in the department of Urban Studies and Planning. His talk – ‘Searching for knowledge – what can search data tell us about future trends?’ – followed on from Azhar’s discussion of analysis that yields predictive results, in that he traced the correlation between searches on Rightmove – a real estate search site –and the volume of properties sold during a given period.

To begin with, Rae gave an interesting overview of cases where correlations are found to be of little use or relevance. Using examples from Google Correlate (including the rather ingenious example of using ‘Google correlate’ itself as a search term), Rae showed the danger of overstating the importance of statistical patterns. He also demonstrated that search terms do not always entail what we might expect, in that searches for party leaders in the run up to the recent UK election did not necessary indicate who someone was going to vote for, but could simply indicate an interest (both positive and negative) in that person.

Having made this cautionary point, Rae then discussed some connections which are useful, such as that between searches for high value items such as cars, and the health of the economy more generally. Citing a number of papers (two of which are linked to below), Rae outlined some of the ways in which search patterns on Rightmove can indicate future housing market trends. He concluded with a graphic that combined user-defined search areas, in ways that showed users closely following certain features, such as the main roads into and around towns such as St. Albans. Here, the social data replicates and reconfirms the trends in the housing market, centring on desirable areas, such as the centre of London, which appears to glow white hot. Changes in these search areas, like changes in the other forms of data we were shown this evening, is a reflection of what is happening on a wider scale in the economy and in people’s tastes.

social data housing


This meet up demonstrated how the commercial processes and benefits of analysing various forms of online social data can be adapted and examined within an academic context. Particularly within the field of social data studies – in which research is focused on the information generated through a multiplicity of online interactions – academics need to keep abreast of the innovations happening within industry. As this meet up showed, social data tells us not just something about society, but also about the forces that can come to shape society in future, in that predictive technologies demonstrate a desire to anticipate the market, and the consumer, in advance. Academia needs to be aware of these discussions, in order to provide a critical response to them, and to assist in developing ethical and sustainable forms of practice. But academia needs also to utilise social data, in order to gain useful insights into contemporary social issues, such as the migrant / refugee crisis discussed by D’Orazio. Events such as this one are therefore important for academics, as they are an opportunity to consider potential research techniques and topics, and a chance to collaborate with industry to develop joint approaches to the use of social data.


Choi, H., & Varian, H. (2012). Predicting the present with Google trends. Economic Record, 88 (1), pp. 2–9.

Rae, A. (2015). Online Housing Search and the Geography of Submarkets. Housing Studies, 30 (3), pp. 453­–472.

This piece is cross-posted on the Big Boulder Initiative blog.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.

About the Author

Anne Burns is a Research Associate at the University of Sheffield’s Visual Social Media lab, and is conducting an ethnography for the Picturing the Social project, that will explore the practices and forms of social media photography. Anne writes a weekly research blog, through which she will be sharing some of the findings from the ethnography. This can be found here. Anne has recently obtained her PhD from the Loughborough University School of Art. Her PhD focused on a connection between the discussion of women’s photographic practices and social discipline. Principally, Anne analysed how the devaluing of certain types of photograph (such as selfies) or behaviour (such as the pouting ‘duckface’) within popular discourse is used to classify and marginalize women. Her PhD blog, which discusses photography, social media and feminism, can be found here

Print Friendly, PDF & Email

About the author

Blog Admin

Posted In: Big data | Data science | Knowledge transfer | Social Media