For research precisely designed to inform policy and practice, ensuring it has the desired impact is crucial. But tracking impact across many countries and diverse contexts can be difficult. Sonia Livingstone and Mariya Stoilova describe how the Global Kids Online project has built an impact toolkit which draws on recent empirical research with over 12,000 children. The ambition is to inform national and international policy and legislation around digital technologies, with a focus on safeguarding children’s rights. The impact toolkit contains accessible summaries of the approach, definitions, and key steps; an impact planning and monitoring framework, complete with templates and an exemplar framework; and tools for engaging key stakeholders, including guidelines on using evidence to inform policymaking, presenting findings to children, and communication strategies.
Ensuring that research has impact is the name of the game these days, and when you’re engaged in research that’s precisely designed to inform policy and practice, you want to make sure that it has the impact desired. But when the research encompasses many countries, meeting the demands of diverse contexts is tricky. And when the fields of expertise to be impacted are also diverse, the challenge is compounded.
The Global Kids Online project, a collaboration between the LSE Department of Media and Communications, UNICEF’s Office of Research – Innocenti, and the EU Kids Online network, has spent the last year building an impact toolkit which draws on its recent empirical research with over 12,000 children and 8,000 of their carers, conducted by our partners in Argentina, Brazil, Bulgaria, Chile, Ghana, Montenegro, the Philippines, Serbia, South Africa, and Uruguay. The ambition is to inform national and international policy and legislation around digital technologies, with a focus on safeguarding children’s rights as children increasingly go online around the world. As our Brazilian partner said:
“In Brazil, there is a gap between the research agenda and policy agenda and it is up to us to bridge this gap. Our responsibility is to produce reliable data that is useful to policymakers.” (Alexandre Barbosa, Cetic.br)
Working collaboratively means drawing on the expertise of our country partners and learning from everyone’s success stories and difficulties; identifying joint priorities and advocacy activities that capitalise on the added value of centralised capacity-building and convening activities; and extending the wider value of the research and impact model to others working with or beyond the network. In reflecting on how we built the impact toolkit for Global Kids Online, we hope this also has wider value for other researchers facing related challenges. So, how did we tackle the task?
What do we mean by impact?
There are many ways in which research impact can be defined but essentially it means finding ways to contribute to desired social changes by drawing on the findings and insights of the research and working with stakeholders to apply these in relevant contexts. Given the collaboration between LSE and UNICEF, we began with the LSE definition that impact is:
“any change brought about in the economy or society as a result of the research or expertise.”
To explain what kinds of changes can be expected, we turned to UNICEF’s Office of Research – Innocenti, which prioritises academic, conceptual, capacity-building, and instrumental impact; and to these we added collective impact as particularly important for multi-country research.
Types of impact
- Conceptual: influencing and reframing debate and dialogue among key stakeholders.
- Capacity-building: building competence at individual, organisational, or systemic levels for multiple purposes (teaching, advocacy, policy development, engaging in new practices, etc.).
- Instrumental: contribution to changes in behaviour, policies, programmes, and practice.
- Collective: brokering new partnerships, networks, or strategic alliances within and between countries aiming to develop joint commitments and common agendas and to foster longer-term social change.
- Academic: contributing to the long-term scientific evidence base.
We figured researchers already know how to make an academic contribution, so the toolkit focuses on the other contributions. We are aware that long-term impact can be difficult to capture but we believe it is possible to capture “intermediate outcomes” or “pathways to impact” that signpost plausible longer-term impacts. Hence our focus is on identifying plausible pathways to research impact in the short and medium term, although we hope this will contribute to longer-term societal impacts, often beyond the lifetime of the programme.
The impact framework
To combine the multi-stakeholder efforts of the network and address different country priorities, we drafted an impact planning and monitoring framework which was carefully tested by seven of our country partners and then adjusted based on their feedback and discussion. The framework provides a way to assess the uptake and impact of Global Kids Online work in the short to medium term. It aims to be a systematic but adaptable living tool that can be revised and adjusted throughout the research process according to country-defined research and impact priorities.
It includes two main parts:
- Planning: this tool invites the research team to consider the efforts needed in relation to necessary inputs, planned activities and outputs, anticipated awareness and reaction, required engagement and participation, and desired impact.
- Monitoring: this tool helps in tracking and recording the actual impact achieved (intended and unintended), inviting the research team to identify the types of impact that have occurred, demonstrate how they have been verified, and reflect on lessons learned.
This can be implemented by individual countries (or partners) and then combined to assess the overall impact of various country efforts, aided by being framed by a common strategic vision to facilitated opportunities for longer-term scaling-up and lesson-learning.
The underpinning theory
Our approach is underpinned by the Theory of Change:
“an outcomes-based approach which applies critical thinking to the design, implementation and evaluation of initiatives and programmes intended to support change in their contexts.” (Isabel Vogel)
We also referred to the Research Contribution Framework developed by Sarah Morton which identifies a pathway to impact that sets out a process of engagement, activity, and change that creates impact as a result of the research engagement and use. And we set out a series of guiding principles in our toolkit – read our Getting Started with Impact document. As our Bulgarian partner put it:
“Our strategy is to identify the current issues, then gather evidence and better understand the nature of the problems, to finally develop the means for addressing these issues. We work with a range of stakeholders, such as children, teachers, government agencies, and partner organisations to achieve the best possible results.” (Georgi Apostolov, Bulgarian Safer Internet Centre)
How did we work?
Global Kids Online implements a partnership approach in which the benefits of central coordination of resources, expertise and tools are united with a distributed approach to evidence-gathering and impact, contributing to the development of evidence-based policy and practice – locally and globally. The network draws together international experts in child rights, child protection, internet and mobile technologies and governance, cross-national survey and ethnographic methods, applied and policy-relevant research, and area specialists from the global north and the global south.
A multi-stakeholder, cross-national kick-off seminar established the overarching research and policy challenges to be addressed by the work of the network and identified key priorities for the collaboration. A follow-up two-day training workshop focusing particularly on knowledge exchange and impact strategies was organised once the project was under way.
The impact tools
The impact tools we developed are available as part of an open-access, Creative Commons-licensed research toolkit for all those who share Global Kids Online’s agenda. The tools are a work in progress, and so far include:
Types of impact tools
- Getting started: an accessible summary of the approach, definitions, key steps, and links to all tools.
- Impact planning and monitoring framework: provides a way to plan, track, and evaluate impact throughout the project lifecycle. Includes a blank framework and a model framework filled in with examples.
- Tools for engaging key stakeholders: resources covering: using evidence to inform policymaking; examples of good practice from country partners; presenting findings to children; and communication tools.
Evaluating the impact
Global Kids Online is already attracting considerable interest from states, the private sector, and NGOs, and the research in our partner countries is already informing advocacy efforts and the development of national policy and practice. However, due to the scale of the project and the different stages of the research cycle of the partners, the task of efficiently tracking and monitoring the impact from our research is complex. Individual countries promote their findings and collaborate with local stakeholders but often have little time to track the ways in which their research is taken up, used, and reused in policy and practice settings, and struggle to link research processes or outputs to wider changes.
To optimise the added value of cross-national comparisons and seek evidence for the collective impact of the project, as well as to identify broader lessons (both positive and negative) to enable refinement of our approach as it is rolled out in additional countries, the network steering group is now commissioning an independent assessment of the country and collective impact of Global Kids Online’s work.
For more information on Global Kids Online, and to access the research toolkit and other resources, please visit http://globalkidsonline.net/.
Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
About the authors
Sonia Livingstone OBE is Professor of Social Psychology in the Department of Media and Communications (@MediaLSE) at LSE. Taking a comparative, critical and contextual approach, her research examines how the changing conditions of mediation are reshaping everyday practices and possibilities for action. You can read her blog or follow her on Twitter @Livingstone_S
Mariya Stoilova is a Post-doctoral Research Officer in the Department of Media and Communications. With a strong comparative and multi-method focus, her research explores children’s use of digital technologies, online privacy and commercial use of data, social inequalities, and family support. Follow Mariya on Twitter @Mariya_Stoilova