For some academics being asked to write a research blogpost can feel like shouting into the void, another addition to a constantly expanding mass of online content. However, the network of connections that can spring out of these engagements and their potential to influence the wider world are significant, and they are also increasingly traceable. In this post Kat Hart uses the Overton database to assess the policy influence of the LSE Blogs platform and the way in which academic blogging can inform public and policy debates.

As policy engagement increasingly features on the UK higher education agenda, there has been a parallel urgency to understand and evidence the different possible pathways to influencing policy.

The policy development process is infamously nebulous (at least from a research administration point of view), stubbornly resistant to simple models or clear, trackable developmental stages, and with norms and approaches differing from organisation to organisation. As a result, it’s tricky to draw broad conclusions from academic–policy engagement data without a lot of accompanying qualitative analysis.

But, whilst time-consuming, the applications of such insights have been wide-ranging – both in illuminating the value of impact and engagement activities, but also in revealing new and sometimes unexpected opportunities for, and examples of, policy influence.

Recognising that we’re always going to need human expertise providing the qualitative analysis, the challenge is in scaling this up to make the process more efficient and reproducible for those seeking to maximise their ability to influence policy and demonstrate value. At Overton, we’ve been thinking of ways to address this challenge, by collecting large numbers of policy documents, making them searchable and extracting useful metadata such as scholarly and policy citations.

The evidence-based policy movement in the UK increasingly recognises the role of intermediaries and knowledge brokers in their ability to platform research in a policy salient way. Knowledge brokers aren’t necessarily all NGOs and think tanks: LSE Blogs positions itself as a knowledge exchange platform and there’s much evidence suggesting that it succeeds at this.

As Overton tracks over 25,000 posts from LSE Blogs, of which ~1,500 (6%) are cited in ~1,780 policy documents (broadly defined as documents written by or primarily for policymakers), it therefore makes for an interesting case study exploring how we might explore different pathways to policy in a data-driven way.

Characteristics of the policy documents that cite LSE Blogs

The ~1,780 policy documents that cite posts on LSE Blogs are published by policy organisations in 50 countries. As well as a strong presence in the UK, figure 1 shows that LSE Blogs has a large number of citations from policy organisations in North America and Australia.

Figure 1: Policy documents citing LSE Blogs, mapped by the location of the policy organisation that published them (citation data 2011-2022). Source: Overton.io

We can dig further to see which organisations most commonly cite LSE Blogs material (figure 2). In the UK the top citing organisations are Parliament (both in submitted evidence & final reports from Select Committees and in research briefings), the Government, the Institute of Development Studies, IPPR and the Constitution Unit. Outside the UK, the EU Publications Office, the EU Parliamentary Research Service and UNESCO top the table, as well as the Analysis & Policy Observatory, a policy aggregator focused on Australian and New Zealand policy organisations.

Figure 2: Top 10 organisations citing LSE Blogs, by the number of policy documents containing citations. Source: Overton.io

Looking at the type of policy documents citing LSE Blogs, we found that 92% of the policy citations were within a document classified as a ‘publication’ (which usually means a longer form report as opposed to blogs, transcripts, working papers, etc.). Understanding the types of documents citing a given source can be useful in determining the nature of policy influence and impact, as well as how it may be perceived by policymakers as a source of information.

Can we tell if LSE Blogs is being cited more over time? If we plot the number of posts from LSE Blogs published each year next to the number of citations from policy documents from that year (figure 3) we can see there’s been a slow downward trend in the number of posts produced each year, next to a slow upward trend in the number of policy documents citing LSE Blogs.

This suggests two likely explanations, firstly that the awareness of LSE Blogs has been growing within relevant policymaker circles and it is being seen as a good source of information in policy development. Second, that as time goes on there is more and more blog material for policymakers to cite, as there is usually a lag between research being produced and then cited in policy.

Figure 3: Trend over time: the number of LSE Blogs published and the number of policy citations of LSE Blogs, 2015-2021. Source: Overton.io

Linking backwards to scholarly research and institutions

Research cited within the LSE Blogs is not always from LSE itself and highlights the platform’s role as a space for convening social science research. Looking at the ~1,500 posts that have gone on to be cited by other policy documents, we can see that the University of Oxford and LSE are most commonly referenced within the blog, followed by scholarly outputs from other UK and US universities (figure 4).

Figure 4: Top 10 Institutions cited within LSE Blogs which are then cited within policy documents. Source: Overton.io

Over 140 researchers are cited or mentioned within this subset of LSE Blog posts. For an institution that is interested in assessing and improving its policy influence, understanding where its researchers are being mentioned in highly cited policy outputs can be a useful starting point to know where to boost training and capacity and to look for opportunities for partnerships and collaborations.

Deepening our understanding of the policymaking process

The role of knowledge brokers in the policymaking process has often been difficult to track, partly due to the complexity of the journey from ‘idea’ to ‘reality’, but also partly due to the opacity of the relationships between policy actors. However, by using policy document citation data we’ve been able to gain a deeper insight into the role particular policy actors are playing within the policymaking landscape.

The example of LSE Blogs also illuminates an intriguing development in the policymaking process: the emergence of knowledge brokerage blogs as a distinct new policy influencing pathway, against the backdrop of ‘traditional’ brokerage pathways such as think tank publications and interest group lobbying. Given that our exploration of LSE Blogs has shown that such blogs can have an influential role in the process, it is interesting to consider why this format might be appealing to policymakers. One potential reason is that blog articles tend to be shorter and more concise and tailored to a policymaker audience, which makes it easier to digest key messages from the underlying research. Perhaps also the more conversational, personal tone invites a more informal dialogue on areas of research than more formal publications and scholarly papers.

Whatever the reasons, the increasing proliferation of policy and impact data will open up exciting new areas of exploration for both public policy scholars looking to understand the landscape and the wider academe who are interested in tracking and reporting on their own policy impact.

If you’d be interested in exploring an idea or discussing our policy citation data, please feel free to get in touch with us by email at support@overton.io

The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.

Image Credit: In text images reproduced with permission of the author, featured image David Bruno Silva via Unsplash.