Theories of Change have become a popular mechanism for research funders to build impact into the design phase of research projects. Laurens Hessels, Fedes van Rijn, Stefan de Jong, Wendy Reijmerink, Jeroen van Houwelingen, Allison McIntosh, Gerald Jan Ellen and Gerben Koers weigh the pros and cons of Theory of Change models and suggest how they can be better supported by research funders and institutions.

Drawing inspiration from experiences in developmental cooperation, research funders are increasingly embracing the use of Theory of Change (ToC) frameworks or related concepts in funding applications for challenge-driven or impact-oriented research projects. For instance, the Dutch research council NWO mandates the inclusion of an “Impact Plan” and Science Foundation Ireland (SFI) requires researchers to prepare an “Impact Pathway” in funding proposals aimed at addressing societal challenges.

In essence, a Theory of Change is a roadmap outlining a sequence of events expected to lead to a desired outcome. The intention behind using ToC approaches in funding is to assist consortia in identifying relevant stakeholders, developing a shared vision regarding the project’s precise contribution, and facilitating formative evaluations. Making a good ToC is easier said than done, however. Considering the rapid adoption of ToC approaches in research funding, it is timely to assess the experiences of both funding applicants and funding agencies with this “new” approach aimed at enhancing research impact. To this end, we conducted an interactive workshop as a Special Session at the Eu-SPRI conference, with about 30 participants, a mix of researchers and funding agencies.

Strengths

Making impact and assumptions explicit:

A Theory of Change exercise prompts individuals to think beyond research outputs and consider the broader implications of their work. It compels researchers to articulate the intended impacts and establish the causal links between their research and desired outcomes. In this way a ToC helps make underlying assumptions explicit. The process challenges researchers to identify key audiences and determine effective communication channels. In co-creation settings, a ToC provides a structure for this overall design process.

A Theory of Change exercise prompts individuals to think beyond research outputs and consider the broader implications of their work. It compels researchers to articulate the intended impacts and establish the causal links between their research and desired outcomes.

A tool for improved collaboration:

A ToC exercise enhances collaboration by its collective nature. It facilitates the development of a common language and shared vision within a team or consortium, integrating contributions and perspectives from various stakeholders. Additionally, the process of developing a ToC can build trust among team members or consortium partners. Furthermore, a ToC helps define meaningful process indicators that facilitate progress evaluation and learning for all involved partners, promoting accountability and continuous improvement.

Potential for increased efficiency:

Despite the perception of being time-intensive, developing a ToC can save time in research collaborations. By establishing a common understanding and clear goals, a ToC can enhance efficiency. It minimises potential misdirection or misalignment of partners by providing a framework for action – in particular, on the steps that ensure research moves from output to impact.

Limitations

Insufficient skills:

Currently, a majority of researchers have limited experience and understanding of the Theory of Change approach and many struggle to apply it adequately. The term ‘theory’ often is a source of confusion, as it carries a different meaning across scientific disciplines. Additionally, running a productive ToC workshop requires advanced communication and facilitation skills which are not necessarily a core academic skill.

Lack of time:

The ToC approach is labour-intensive, making it challenging to allocate sufficient time for conducting a thorough exercise. Moreover, convincing societal stakeholders to participate in the design stages of project development can be difficult when the benefits are uncertain.

Consensus amidst vested interests and diversity:

Developing a comprehensive ToC requires an open mind and flexibility. However, both researchers and stakeholders, may struggle to let go of their vested interests and preferences. Complex projects face challenges in reaching consensus on desired outcomes and the best approaches to achieve them. Furthermore, a ToC exercise can divert attention away from political conflicts and overly prioritise consensus. There is also a risk of oversimplification, neglecting multiple parallel causal mechanisms and the dynamic nature of research processes, leading to the creation of unrealistic assumptions and failing to account for inherent unpredictability.

convincing societal stakeholders to participate in the design stages of project development can be difficult when the benefits are uncertain.

Mitigating limitations and amplifying strengths

To ensure a useful Theory of Change exercise, it is crucial to establish a shared understanding of its purpose and scope. Participants should clarify whether it is a design tool, a framework for monitoring and/or a means to demonstrate the project’s value to funders. Often a ToC is used for more than one purpose, in which case it is even more important to be clear how it will be used.

Improving awareness and skills related to the ToC approach can be achieved through dedicated training and mentorship. Research funders can play a supportive role by offering guidance to applicants on how utilising a ToC can enhance their chances of securing funding. Multiple examples already exist, like the e-learning offered by NWO or the in-person training provided as part of SFI challenge-based funding mechanisms.

Engaging professional facilitators can also help overcome the current lack of experience and time within the research community. Even experienced participants can benefit from the guidance of an independent facilitator to ensure an unbiased and effective process. For example, some NWO funding instruments use independent facilitators to run ‘sandpit’ procedures for creating large research proposals with complex consortia.

To avoid oversimplification, it is essential for the ToC to be evidence-informed whenever possible, to promote a more robust understanding of the project’s potential impact. At the same time, the efforts made to design a ToC should remain proportional to the project itself.

Finally, workshop participants recommended working with “living documents” to prevent fixation on a particular model. This approach, which is indeed encouraged by several research funders, allows for ongoing refinement and adaptation of the ToC as new insights emerge, ensuring its relevance and applicability over time.

Theories of Change should only be used when appropriate, given the goals of a project or a funding instrument.

The Theory of Change approach has strong merits, but creating a ToC is a challenging task. It requires an analysis of relevant societal challenges, anticipation of research impacts, reflection on the relationship between research activities and their societal context, and communication with different stakeholders. This implies that the availability of time and relevant competencies are critical factors for the success of a ToC. We advise research funders working with ToCs to invest sufficient resources in the development of the required skills and to support their research communities in securing sufficient time for making a ToC.

In addition, Theories of Change should only be used when appropriate, given the goals of a project or a funding instrument. Funding instruments aiming to support research of which the societal impact is very uncertain should carefully consider to what extent a ToC should be compulsory for all applications. In these cases a ToC may create an unfair competition between proposals with clear-cut stakeholders and proposals which cannot define the most relevant audiences yet. The efforts invested in a ToC should also be proportional in relation to the overall goals. It should support the goals of a research project or program, and not be a goal in itself.

Finally, our workshop participants indicated that Theories of Change also has great potential to support research beyond the design stage, by providing guidance for the monitoring and evaluation of research projects as part of an ongoing process.

The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.

Image Credit: Adapted from Hannah Morris via Unsplash.