This is the second of a series of articles about Voting Advice Applications. Read the first one here.
VAAs are digital devices that try to help citizens think about how they might decide to vote in an election. They might be websites, apps or any other online format that you could access via a desktop, laptop, tablet or mobile connected device. They are a niche digital product but they also represent a bigger question for journalism in the Internet age: how can media organisations use the new technology to help the citizen cope with information overload to make better choices for themselves and for society? VAAs are difficult to get right. There are the technical aspects but there are also editorial problems about balance, accuracy and fairness. Even if they work, will they make a difference?
This series of articles will not answer all those questions. But as more people try to use VAAs we hope that this research will give some pointers towards best practice and innovation and help facilitate a debate about their use. The research was carried out by Kamila Varadzinová from Czech Radio as part of a Polis/EBU (European Broadcasting Union) Fellowship. Find out more about the fellowship here.
Designing a VAA
The selection of questions for VAAs is crucial for establishing their reliability and usefulness and is one of the more contentious aspects of the design process. Critics of the applications have often argued that depending on the number and type of questions a potential voter answers, the quality and type of advice can be manipulated. They have also questioned whether specific parties may be advantaged or disadvantaged with the inclusion or exclusion of specific statements.
Evidence indicates that these concerns may be validated, that some parties may have an advantage given the selection of certain issues to be included. However, this does not mean that VAAs are an unworthy endeavour, rather, it shows the importance of the selection process for building reliable VAAs.
During the typical selection process, academics, journalists and members of the public collaborate together and try to select a broad spectrum of statements that cover the most important policy areas.
The following are the main problems that are encountered during the selection process and recommendations I have compiled via research and discussion with experts.
A study by Wagner and Ruusuvirta (2012) showed that in general terms, VAAs reflect party positions well, but that accuracy depends on the number of questions asked.The selection of questions within a given topic is often not broad enough to detect differences of opinions between parties. In many instances, VAAs fail to distinguish differences in party positions on particular issues.One example was the 2008 Austrian Politikkabine, where on the issue of immigration, all parties apart from the FPÖ were shown to have identical policy positions. The only VAA that really managed to separate parties on immigration policy was the VAA Swiss smartvote which asked 11 questions on the topic. Apparently, in large party systems, more questions are needed to effectively distinguish parties from each other.
The main recommendation for designers of VAAs is that it is useful to have more questions when it comes to the key policy areas in order to get a nuanced picture of both how potential voters and parties view different aspects of an issue and to best match voters with the party that is most similar.
Of course VAAs are limited regarding how many questions can actually be included, but some VAA providers (like Czech Volebni kalkulacka) do offer both short and long versions.
Statement selection, which is the issues chosen to be presented as part of the application, has an impact on the advice given to users. The statements chosen can benefit particular parties, as shown by research by Lefevere and Walgrave (2014) They conclude that because VAAs tend to be structured on a left-right model of voting, parties with extreme “left-right positions are advantaged when selection includes more left-right statements” (2). They also show that parties for which particular issues are salient are “disadvantaged when more statements on these issues are included” (Lefevere and Walgrave 2014, 2).Moreover they found that including more statements increases a VAA’s accuracy as more statements on the left–right dimension makes VAAs better able to connect voters with parties in a meaningful way.
A recommendation for this problem could be a pre-test on voters before launching the VAA. In the case that some statements advantage or disadvantage some parties, VAA builders can exclude them.
Studies show that choosing statements is a balancing act without clear guidelines. The challenge for VAA builders is therefore to find a combination of broad and current topics that articulate policy positions and do not advantage one party over another. Users of VAAs must be made aware of the fallibility of these applications; they do not provide a perfect match between parties and users, rather just a comparison of certain issues and certain times.
Lefevere, Jonas and Stefaan Walgrave. 2014. A perfect match? The impact of statement selection on voting advice applications’ ability to match voters and parties, Electoral Studies.
Wagner, Markus and Outi Ruusuvirta. 2012. Matching voters to parties: Voting advice applications and models of party choice, Acta Politica, 47(4): 400-422.
Fellowship supported by the European Broadcasting Union