Lockdowns and social distancing measures have severe implications for data collection, research and evaluation services that many academics and donors from across the world commission for their research projects in South Asia. Here Prerna Mukharya (Outline India) explains the challenges owing to the national lockdown in India and how it affects her organisation’s ability to provide essential information for impact assessment projects under conditions of social distancing.
As we all retreat to the safety of our homes, the coronavirus epidemic has sent businesses scrambling into survival mode. Much like every industry, data collection and evaluation services have been severely disrupted by COVID-19.
At Outline India, an organisation that works closely with policymakers, multilaterals, academics, and not for profits interested in launching social programs or monitoring ongoing initiatives for efficacy, scalability and replicability, we too have been impacted by the crisis. As the ears and eyes on the ground for our partners, we are at risk of continuing our work, in its erstwhile format.
Data collection services, such as ours, have had to make tough decisions about halting research and evaluations as the crisis unfurled. Before the global epidemic outbreak, we had three field missions in full swing. Projects commissioned by investigators and academics from different institutions across the United States and Japan, however, all had to be suspended overnight. We spent the next four weeks trying to further curtail field-related work.
One such project was designed to evaluate the improvement in the standard of living for farmers as a result of various government interventions. The evaluation project focused on workshops to educate farmers on modern agricultural practices. However, it is becoming increasingly apparent that COVID-19 will change many of these interventions. Farmers are likely to be unable to procure certain seeds since supply chains have been disrupted. No workshops therefore will be organised, and evaluating the program in its initial format will not be possible under the current conditions.
The second field mission was in western India and focused on working closely in public schools to study learning outcomes. We had a field staff of over 40 data collectors, five supervisors, two senior coordinators, and four researchers who planned the entire mission. We had planned to visit over 100 schools spread across three districts. An intensive, week-long field training exercise was near ending when the news of cases of COVID-19 in India broke. Had the study continued, it would have involved talking to government officials, village heads, school headmasters, young children, and mothers. Our interaction would have been varied and widespread. We had spent about four months meticulously working on the evaluation design, seeking relevant permissions in community and stakeholder mapping and piloting the tool.
As the Indian government announced a nation-wide lockdown, our researchers (many working in rural India) had to be pulled out. We issued urgent directives to our field staff to return home. In India, our field staff are like foot soldiers who traverse India’s heartlands alongside our researchers to collect data to figure why, for instance, something is working well and why a solution isn’t scaling as we’d intended.
Much like startups and grassroots organisations, our fleet of data collectors work part-time. Having worked across 9,000+ villages in India and Nepal, we take pride in our lean model that optimises costs. The bulk of our work however rests on the backs of our front runners, who now stand compromised.
Telephone surveys and other technologies for measuring impact
The crisis puts into the spotlight the very methods we have been relying on for years to measure impact and to reach our audience, most of whom are not online (these include pregnant women, adolescents and farmers), and we do not envisage one-on-one field data collection happening in India for at least 3-4 months.
Over the past fortnight, we have been exploring CATI telephonic surveys as a possible remedy to social distancing measures. There are problems however when it comes to phone interviews: you can never truly verify the identity of a respondent, as well as a survey longer than 15 minutes causing fatigue in the respondent; the last thing people want to do right now is to answer a 40 minute-long survey over the phone. Most research studies however require that we go deep into the minds of our respondents to understand their behaviour.
In a country such as India acquiring phone numbers is a battle in itself. Loyalty to phone numbers is next to none, and phones and talk time are controlled by male members of the rural household, meaning we would have much less exposure on the phone to women. Despite cheap Chinese smartphones flooding our telecommunications market over the last decade, they remain out of reach financially for families across rural India, our primary targets for social interventions.
Our other possible solution to support research projects over the coming months has been to focus on a tweaked version of our ‘tech tool’ named Track Your Metrics. During lockdown, our team and I have been working on amending this tool so that it is now available via a weblink without the need for the application to be downloaded. It will also be available for free in certain cases. We have also recently been in talks with the government regarding how this tool could help the not-for-profit sector make inroads into India’s deepest corners while people’s movement is restricted.
Track Your Metrics will collect feedback from volunteers and staff, including beneficiaries, to understand their physical, financial and mental well-being. A program head will simply need to log on to the platform, select from a list of preexisting surveys, and then send the link via WhatsApp to their subjects, or ask them to use the application. We believe engagement, and a sense of connection in these times can go a long way in terms of how we feel about returning to the ‘new normal’.
We are also setting up feedback surveys for different government and corporate bodies to gain quick insights (akin to opinion polls) about the crisis.
As India adapts to the ‘new normal’ and old research projects stand altered while new ones are created, we will continue to play our essential role in helping gain key data insights from those at the bottom of the pyramid for partners across the world.
Watch Prerna Mukharya’s TedX Talk Social Impact through Data.
This article gives the views of the author, and not the position of the South Asia @ LSE blog, nor of the London School of Economics. Featured Image: Prerna Mukharya conducting a set of interviews during a research project in India. Credit: Prerna Mukharya
Hi Prerna, thanks for this article. Had a couple of follow-up questions. Collecting data through Track Your Metrics requires the respondents to have smartphones with working internet. As you’ve mentioned already, a significant %age of the population do not have access to smartphones. Doesn’t the data collected through a web-based tool risk suggesting a skewed reflection of ground realities? If yes, then how do you mitigate that? Also, would love to know your thoughts on how, in this pandemic induced physical distancing world, the experiences of people without smartphones can be captured?
Thanks Pranjali. We have enabled telephonic surveys to mitigate the bias to a certain degree. For still others, we are connecting with local not-for-profits who serve as intermediaries between funders and respondents.
On the question of social distancing- there will be a redistribution of household level resources to a certain degree. People not online, will perhaps make a more concerted effort to connect via phones/ go online as movement becomes restricted. However, this redistribution will also be affected by changes in income, which in turn will produce externalities of its own.