As part of the Development Management Consultancy Project, postgraduate students Rodrigo Chenú, Maria Trobat I Castelltort, Ximena Chacon-Herrera, Jiha Kim, and Louma Kraytem share their experience working with the International Organization for Migration (IOM) in Jordan to design an impact evaluation guideline for measuring the Multi-Purpose Cash Assistance (MPCA) programme’s causal effect on refugees’ well-being in Jordan.

What is the project’s background?

In our consultancy project, we had the opportunity to work with the International Organization for Migration in Jordan. As an intergovernmental organisation, IOM Jordan provides humanitarian emergency assistance to forcibly displaced people in Jordan. With the Syrian crisis escalating, Jordan has become the world’s second-largest host for refugees per capita, and cash assistance interventions have emerged as the preferred way to support refugees. Since 2018, IOM Jordan has implemented the MPCA programme, an unconditional eight-month cash assistance intervention to help refugees meet their basic needs and mitigate their vulnerabilities. However, there has been a downward trend in donor funds and increasing pressure to provide evidence on the effectiveness of cash assistance interventions on refugees’ well-being in recent years. Therefore, our consultancy project aimed to design an impact evaluation to measure the causal effect of the MPCA programme on refugees’ well-being in Jordan. In line with this objective, the main deliverables were a guideline to implement the impact evaluation, a post-intervention survey to gather outcomes data for the empirical analysis, and a framework to interpret the results.

How do we tackle our challenges?

From the outset, our team faced several challenges. The first obstacle was the limited knowledge and experience in impact evaluation design for humanitarian settings. To address this gap, we acquired skills in impact evaluation and quantitative research methods from courses at the LSE Department of International Development, MITx, and materials about impact evaluation design. We also met with our academic coach to properly understand the scope and complexities of the project. A second obstacle was our limited understanding of emergency humanitarian assistance in the refugee context. Thus, we conducted a literature review on cash assistance interventions in countries of the Middle East and North Africa (MENA) region. We also overcame two significant constraints to ensure successful project delivery: limited time to provide a rigorous impact evaluation design and lack of data needed to determine which research method best suited the programme’s characteristics.

While awaiting the data to determine the methodology for the impact evaluation design, we decided to move forward with the literature review and survey design. We started by reviewing academic articles and reports on previous impact evaluations, focusing on unconditional cash assistance interventions in countries such as Lebanon, Turkey, Kenya, and Colombia. We also researched the context, delivery process, and assistance packages of the MPCA programme in Jordan. Simultaneously, we explored, analysed, and identified the well-being dimensions and questions to construct the post-intervention survey. For this purpose, we concentrated on three key questionaries: the Impact Evaluation of the MPCA on the Well-Being of Syrian Refugees in Lebanon 2020 by the American University of Beirut, the Survey of Syrian Refugees and Host Communities in Jordan 2015-2016 by the World Bank, and the Vulnerability Assessment Framework in Jordan 2023 by the United Nations Refugee Agency (UNHCR).

How do we craft our consultancy report?

After conducting the literature review and identifying the best-suited methodologies to evaluate the MPCA programme, we proposed two options to the client: a Randomised Controlled Trial (RCT) and a Regression Discontinuity Design (RDD). While the RCT involved randomly selecting cash assistance recipients, the RDD relied on a vulnerability score as the eligibility criteria for determining the recipients. IOM Jordan chose the RDD as the preferred methodology because the MPCA programme focuses on the most vulnerable households. The RDD exploits the MPCA programme targeting rules by comparing recipient households just above an eligibility cut-off with non-recipients just below the cut-off. Although we did not receive the required data for the impact evaluation design, we drew on the literature review to deliver an easy-to-adapt RDD when programme data becomes available. In other words, we provided guidelines for using the programme participants’ data to define the eligibility criteria, conduct the sampling process, implement the empirical design, and analyse the results. We also provided a step-by-step guideline with empirical examples and practical applications in Stata to facilitate the RDD implementation.

Beyond the impact evaluation design, our main contribution was designing a 146-question multi-subject survey to collect primary data on refugee well-being outcomes. Along with the survey, we provided guidelines for its preparation and implementation, as well as a framework for constructing, measuring, and interpreting results. These well-being outcomes were aligned with eleven dimensions proposed by Amartya Sen’s capabilities approach (1994), which was the guiding framework for our work. The survey’s dimensions cover not only socio-economic outcomes such as expenditures and financial situation but also an expanded well-being perspective, including education, housing and WASH (water, sanitation, and hygiene), food security, coping strategies, employment, health, household decision-making, community relationships and overall subjective well-being. Finally, we delivered a theory of change to illustrate how the programme’s causal effect on the beneficiaries’ well-being will be measured by implementing the designed survey based on the eleven well-being dimensions.

Concluding thoughts

This hands-on project in collaboration with IOM Jordan was a fascinating and challenging experience, with unforeseen obstacles and changes in the project’s scope. However, these challenges provided insightful learning opportunities and contributed to our understanding of aid delivery and project implementation in complex settings. Our team navigated the changing circumstances and overcame the challenges by working together with a flexible and open mindset. Completing our report was possible thanks to the support and active collaboration of Gabrielle Bravo, Erma Mulabdic and Afaf Sito from IOM Jordan. Last but not least, the guidance provided by our academic coach, Professor Stephanie Levy, was essential in helping us sort out all the obstacles that emerged during this six-month consultancy project.

The views expressed in this post are those of the author and do not reflect those of the International Development LSE blog or the London School of Economics and Political Science.

Featured image credit: Syrian refugees in Lebanon, IOM.