Do voting advice applications work? Drawing on a new field experiment during the 2024 European Parliament election, Joris Frese, Simon Hix and Romain Lachat find that while there is limited evidence voting advice applications boost turnout levels, their recommendations do appear to have a meaningful impact on the choices made by voters.
Voting advice applications (VAAs) are now common features of elections in many democracies, in particular in multi-party settings. Supporters of these applications argue they encourage voters to engage more in the electoral process, and so increase the likelihood that people turnout to vote. They also argue they inform voters about which party they are closest to, and so enable voters to make better choices in elections.
Existing research, though, has found mixed effects from voting advice applications on either voter turnout or vote choice. Some observational studies suggest they can lead to higher turnout and vote switching. However, these studies are limited by powerful selection effects, as the people who choose to complete a voting advice application are likely already better informed and more politically engaged in the first place (and may differ on a wide range of other attributes too).
Recent experimental studies have tried to address this limitation by randomly assigning encouragement to complete a voting advice application to a particular treatment group. However, these experimental studies have tended to find limited or no effects from the use of voting advice applications. Nevertheless, almost all the existing experiments suffer from small sample sizes, and so have been under-powered to detect what are likely to be small effects anyway.
A field experiment
To improve on these existing experimental designs, we conducted the largest VAA field experiment to date (N=6,501), in three countries – Germany, Italy, and France – ahead of the June 2024 European Parliament election.
Beyond sample size and statistical power, we improve on previous VAA experiments in two important ways: (1) while previous experiments merely included a link to a voting advice application in a survey, we embed a real-world VAA in our survey and were able to analyse the full dataset of VAA responses given by our survey respondents; and (2) instead of measuring vote intentions as outcomes, we conducted a two-wave panel survey, conducted before and after the European Parliament election, to measure actual voting behaviour in the election.
The voting advice application we used was EuroMPmatch, which is slightly different to some existing VAAs, as it used real party positions on significant issues rather than posited positions in party manifestos or expert judgements about where parties stand. EuroMPmatch worked by asking a respondent to express their opinion on 20 issues that the Members of the European Parliament had voted on in the 2019-24 European Parliament, and then reported how closely the respondent was “matched” (in percentage agreement terms) with each of the national parties in their member state as well as each of the European political groups.
Key findings
Our main findings were as follows. First, we found that completing the EuroMPmatch voting advice application had no statistically significant positive effect on voter turnout in the 2024 European Parliament election. Specifically, averaged across the three countries, we did not find any statistical difference in the self-reported rate of turnout in the post-election survey between our treatment group of respondents (who were encouraged to complete the VAA) and our control group of respondents (who were not encouraged to complete the VAA).
Looking at the three countries separately, we found that in France and Germany completing the VAA had no effect on turnout. Nevertheless, in Italy, respondents who were encouraged to complete the VAA were 3-6 percentage points less likely to turnout to vote in the election than those who were not encouraged to complete the VAA. This might suggest that the VAA led some Italian voters to be more confused about their electoral choices, but further research will be needed to test whether this was in fact the case.
Second, however, we did find that completing the EuroMPmatch VAA influenced which political parties respondents voted for in the election. Specifically, amongst the respondents who completed the VAA, people were more likely to vote for the national political party or European political group that received the highest percentage “match” in the VAA. While it is expected that policy preferences are related to vote choice even in the absence of VAA usage, we find that the large jump in increased vote shares right around the top spot (as documented in Figure 1) goes beyond this expected linear relationship by up to 7 percentage points, suggesting that it is indeed caused by the VAA recommendation itself.
Figure 1: Voting advice application recommendations and voter choice
Note: The figure shows the proportion of votes (y-axis) for parties barely above and below the agreement percentage needed to become the top-recommendation in the voting advice application (x-axis). The large jump in votes shown at the threshold (0 on the x-axis) suggests the voting advice application recommendation influenced which political parties respondents voted for in the election.
In sum, our findings suggest that voting advice applications, while not increasing voter mobilisation, do enable voters to make better informed choices in elections – causing more of them to vote for the party that they are ideologically most aligned with. This makes voting advice applications especially useful for second-order elections, such as those for the European Parliament (or regional elections), because many voters are naturally less informed about the policies at stake compared to national elections.
For more information, see the authors’ accompanying working paper.
Note: This article gives the views of the authors, not the position of EUROPP – European Politics and Policy or the London School of Economics. Featured image credit: Alexandros Michailidis / Shutterstock.com
