LSE - Small Logo
LSE - Small Logo

Philipp Chapkovski

Max Schaub

April 6th, 2022

Do Russians tell the truth when they say they support the war in Ukraine? Evidence from a list experiment

27 comments | 221 shares

Estimated reading time: 10 minutes

Philipp Chapkovski

Max Schaub

April 6th, 2022

Do Russians tell the truth when they say they support the war in Ukraine? Evidence from a list experiment

27 comments | 221 shares

Estimated reading time: 10 minutes

Survey evidence indicates a majority of Russian citizens support their country’s military action in Ukraine. But does this give an accurate picture of public opinion? Using an innovative list experiment, Philipp Chapkovski and Max Schaub demonstrate that a significant percentage of Russian citizens are likely to be hiding their true views about the conflict.

In war, the saying goes, truth is often the first casualty. So when several polls showed that up to 80 percent of Russians supported the war against Ukraine, many doubted these figures, including on this blog. Is it possible that support for the brutal invasion is really so high – or is it just that individuals are afraid to share their true opinions? While it is well known that fear of repression can lead to preference falsification, i.e. people publicly supporting positions they privately don’t share, showing that this mechanism is at work is not easy. After all, people are unlikely to say whether they are hiding their true preferences or not if they are hesitant to reveal these preferences in the first place.

Why is it important to know the true level of public support for the invasion? One aim of the Western sanctions is to weaken Vladimir Putin’s regime, and at least some observers hope that this will ultimately end Putin’s reign. And while authoritarian regimes are usually undermined by actors from within the elite, popular support for regime change also matters. Without popular support, a coup is doomed, often with fatal consequences for its leaders. The level of popular support for, or disapproval of the war, therefore, is an important determinant for how realistic the hope for regime change is.

A list experiment

As a way to find out if people reported their attitudes towards the war truthfully, we conducted an online survey and a list experiment. In the list experiment, respondents were asked whether they personally supported none, one, two, three, or four of the following things (shown in a random order): 1) monetary monthly transfers for poor Russian families; 2) legalisation of same-sex marriage in Russia; 3) state measures to prevent abortion; and 4) the actions of the Russian armed forces in Ukraine.

Respondents were explicitly asked to only indicate how many of the items they supported, not which one(s). This means that no one will ever know if a given respondent supports the war (item 4). The list experiment therefore solves the problem of preference falsification – it allows respondents to tell the truth. This method has been used to reveal true levels of racism in the United States or to quantify the occurrence of sexual violence during war, to name but two examples.

However, if we cannot interpret an individual respondent’s answers, how, then, do we measure support for the invasion? The answer is simple yet elegant. The above four-item list was only presented to half of the respondents. The other half received a three-item list, in which the fourth item (support for the invasion) was left out (see Figure 1 below). Who was shown the three-item list, and who was shown the four-item list was determined randomly. The difference in the average number of items supported, therefore, is an answer to our question of how many Russians support the invasion.

Figure 1: Screenshots of list experiment in treatment (left) and control condition (right)

Note: Respondents were either shown the question on the left (with the ‘Actions of the Russian armed forces in Ukraine’ option included) or the question on the right. Data collected on 4 April 2022.

We conducted the list experiment among a sample of 3,000 Russians whom we recruited on the online platform Toloka. Toloka is the Russian equivalent of the US-based Amazon MTurk platform, which has frequently been used by political scientists for conducting experiments. The respondents on the platform are not a perfect mirror image of Russian society, of course. They tend to be younger, more urban, and better educated (Table 1). As such, our respondents are likely more liberal than the population average, meaning that our estimates may represent a lower bound of support for the war.

Table 1: Summary statistics of respondent sample vs Russian population

Note: Compiled by the authors.

The goals of our analysis are twofold: first, we want to find out the true level of support for the invasion among our respondents. This answer is provided by our list experiment. Second, we want to determine whether people falsify their preferences. This answer is provided by comparing the rate of support as determined by the list experiment to the answer to a direct question. After replying to the three-item list, respondents in the control condition were asked “Do you personally support the actions of the Russian armed forces in Ukraine?” That is, we used the wording from the list experiment to formulate a direct question assessing the support for the invasion. This allows us to compare the two different ways of asking and, hence, to quantify the amount of preference falsification among our respondents.

Results

So how much do Russians support the invasion, and do people tell the truth when asked? The bar plots in Figure 2 below show the level of support as assessed by both the direct question and the list experiment. When asked directly, 68% of our respondents stated that they personally supported the war – a two-thirds majority. However, when using the list experiment to assess the support for the invasion, this share drops significantly, to 53%. In other words, when allowed to reveal their true private preferences, only a simple majority of respondents supports the war. This difference of 15 percentage points is highly statistically significant, meaning that it is extremely unlikely to be the result of chance. Russians, at least those in our sample, clearly hide their true attitudes towards the war.

Figure 2: Support for the Russian invasion of Ukraine

Note: Bars show averages, vertical lines show 95% confidence intervals.

In order to illustrate the extent to which the falsification of preferences is driven by fear, we can split the sample by those generally willing to take risks vs those typically hesitant to do so. The risk question is regularly used by social scientists, and an important predictor for economic and other behaviours.

As shown in Figure 3, risk also plays a role for whether individuals reveal their true attitudes towards the invasion. Among the half of the respondents who are more willing to take risks, it makes little difference whether the question about the invasion is asked directly or by means of the list experiment. In contrast, among those who are more risk-averse, preference falsification is rampant: the share of respondents ostensibly supporting the invasion is 1.5 times higher when asked directly (66%) than when asked by means of the list experiment (46%).

Figure 3: Support for the Russian invasion of Ukraine, by level of risk aversion

Note: Bars show averages, vertical lines show 95% confidence intervals.

Do Russians tell the full truth when asked about their support for the war? Based on our experiment, we can safely conclude that they do not. On the one hand, this is good news. It means that the prospect for regime change may not be completely implausible. Given that private preferences for the war are significantly lower than those shown in public opinion polls, if and when the regime starts to unravel, new leaders could pursue peace without fearing a popular backlash.

However, being against the war is not the same as being against Putin, whose high levels of support might well be real, as shown by another list experiment. What is more, the fact that a large number of Russians support the invasion even when given the option to reveal their true private preferences is extremely concerning. After all, our estimates come from a population sample that presumably is more liberal than average. Yet even among this group of people – and despite the fact that the brutality of the war is becoming more evident by the day – the Russian leadership ostensibly can count on the genuine support of a substantial part of the population.

The data for this study was collected on 4 April 2022. The data, otree script, and code used for producing the figures can be found on Harvard’s Dataverse


Note: This article gives the views of the authors, not the position of EUROPP – European Politics and Policy or the London School of Economics. Featured image credit: Max Titov on Unsplash


About the author

Philipp Chapkovski

Philipp Chapkovski is a Research Fellow at WZB, Berlin.

Max Schaub

Max Schaub is an Assistant Professor of Political Science at the University of Hamburg.

Posted In: Politics | Russia-Ukraine War

27 Comments