Voting advice applications are typically used to inform voters ahead of elections, but can these tools also help us to understand where parties are located within the political space? Drawing on a new study, Frederico Ferreira da Silva, Andres Reiljan, Lorenzo Cicchi, Alexander H. Trechsel and Diego Garzia illustrate the potential for voting advice applications to act as a data source for studying political parties and their positions.

Voting Advice Applications (VAAs) are non-partisan, independent platforms designed to inform and assist citizens during electoral campaigns. They help to navigate the policy proposals of competing political parties and candidates, with the goal of finding the best fit between an individual’s policy preferences and their options at the ballot box.

These popular tools are now present in over 40 countries worldwide and are used by significant numbers of the electorate in many European countries. For example, in countries such as the Netherlands, Belgium, or Finland, it is common for more than a third of all voters to use VAAs during election campaigns, while the German VAA ‘Wahl-O-Mat’ has recorded over 20 million users this year.

Besides the most immediate goal of assisting and informing voters, can this instrument also help us to better understand where parties are located in the political space? Using rigorous methods, VAA designers determine the positions of parties with the aid of a few dozen carefully selected political statements. This party placement data can be a useful additional source to study where political parties stand on diverse policy issues. But how reliable is VAA-generated data compared to other established party positioning methods such as expert surveys or the analysis of party manifestos? In a recent study, we sought to provide an answer.

Measuring the reliability of VAA data

To determine the validity and reliability of VAA data to study parties’ positions, we triangulated VAA estimates with the two most frequently used party positioning data sources: the Chapel Hill Expert Survey (CHES) and the Comparative Manifesto Project (CMP). As for the VAA data, we relied on the recently released EU Profiler/euandi trend file (2009-2019), a dataset providing data on positions of over 400 individual political parties across 28 EU member states in the context of the European Parliament elections of 2009, 2014 and 2019, collected through the pan-European VAAs EU Profiler/euandi. The unique cross-country scope of this VAA, covering three EP election cycles, enables significantly extended comparative and longitudinal analyses that go beyond previous comparative efforts.

Furthermore, this VAA uses the so-called iterative method of party positioning, considered a benchmark among designers. Through this method, parties are first invited to position themselves on the different policy statements. In a second step, this input is considered in interaction with coding from the respective country expert teams when determining the final party position (if parties do not reply, the position is determined solely by the coders). As such, this is a highly elaborate party placement technique, as it combines expert assessments, textual analyses of a broader set of documents than just party manifestos, and the input of parties themselves. For these reasons, we consider it the most adequate data source to compare with CHES and CMP across countries and over time.

We compared these three data sources grouping the policy statements on three dimensions which have been widely demonstrated to structure political competition in the European political space: the socioeconomic Left-Right dimension; the Green-Alternative-Libertarian and Traditional-Authoritarian-Nationalist dimension (GAL-TAN); and the Pro/Anti-EU integration dimension. Our analysis points towards a strong convergence between the euandi and the CHES estimates on all three dimensions and over the three elections analysed.

The Pearson correlation coefficients between euandi and CHES data are, on average, around .75 for every dimension. The euandi vs. CMP comparison also yields sizeable (even if lower) correlation coefficients, ranging from .47 to .58. Similar coefficients are obtained when comparing CHES and CMP, suggesting that the specificities of the CMP method – mostly focused on determining salience rather than position – may play an important role in this regard.

Overall, these results substantiate the validity and reliability of party position estimates derived from the VAAs under consideration, which correlate at a high level with the two most frequently used methods for party positioning. Importantly, the euandi estimates were also compared to other data sources as a robustness check. For example, using Euromanifesto data instead of CMP as a data source for manifesto analysis renders similar results. We also find a strong correlation between euandi estimates and party positions based on mass public opinion data from the European Election Studies.

Explaining differences

Finally, we have conducted an exploratory analysis into the factors explaining the (dis)similarities between the three core methods, by means of regression analyses. Regardless of the multiple factors considered, only one is significantly related to differences in estimates between the euandi and CHES data sets: the difference in measurements is significantly reduced among parties participating in the euandi self-placement procedure.

Besides substantiating the added value of the iterative method used in euandi, this important finding suggests that other VAAs not employing the same method may exhibit lower levels of convergence with expert survey data. Regarding the comparison between euandi and CMP, remaining divergences in estimates are more frequent among newer parties, for which it is arguably harder to retrieve documentation and infer party positions. The differences are also greater for Southern European political parties. In general, the results from both models show little evidence of a systematic source of bias in the estimates between datasets.

The advantages of VAAs

Our study shows that VAAs offer a complementary, and reliable method to estimate party positions in a multidimensional European political space, performing akin to the most prominent expert survey (CHES) and manifesto data (CMP) sources. Scholars can thus rely on VAAs as one additional data source to study political parties and their positions in the political space.

Yet VAAs are potentially more than a mere addition to the toolkit in search of a gold standard of party positioning. Some unique features of VAAs may also constitute competitive advantages vis-à-vis other methods. For example, the use of detailed policy positions grounded on carefully defined criteria offers a more fine-grained measure of parties’ stances on concrete, salient policy items. VAAs also enable research on policy congruence through comparisons of parties’ policy positions and the policy preferences expressed by voters in a common political space – a relevant instrument to study representative deficits.

For more information, see the authors’ accompanying paper in the Journal of European Public Policy

Note: This article gives the views of the authors, not the position of EUROPP – European Politics and Policy or the London School of Economics. Featured image credit: Arnaud Jaegers on Unsplash