The failure to predict the magnitude of the Conservative victories in 2015 (big) and 2017 (small) dealt a severe blow to the polling industry in the UK, which asked citizens who they intended to vote for. Should polls be forsaken, or is there an alternative to vote intentions? Drawing on a new study, Andreas Murr, Mary Stegmaier and Michael S. Lewis-Beck provide evidence that polling on vote expectations rather than vote intentions is the answer.
As the UK faces the prospects of early elections, many citizens question the accuracy of public opinion polls, which have missed the mark in recent elections. Typically, the media report vote intention polls, which ask “If there were a general election tomorrow, which party would you vote for?” We argue that replacing the vote intention question with a vote expectation question – such as “Who do you think will win the next general election?” – will increase the accuracy of election prediction.
Why should vote expectations predict better than vote intentions?
The primary reason that vote expectations predict better than vote intentions is because they include information from respondents’ social network. The expectation question ‘who will win?’ gives people a chance to report what they have heard from family, friends, and co-workers. It is logical that questions about vote expectations should more accurately predict election results, since such items capture information about the respondent and his/her social network, whereas vote intentions only concern the respondent.
If vote expectations are more informative than vote intentions, are they indeed better at predicting election results? To answer this question, we evaluated eight prediction models for British general elections – seven models based on voter intentions, and our model, which is based on voter expectations – using monthly survey data and election results from 1950 to 2017.
We chose the seven most common vote intention models, which fall into two groups, depending on how they translate vote intentions into seat share predictions. They either directly translate vote intentions into seat shares via a regression model, or the translation is indirect via an assumed ‘uniform national swing’. Our vote expectation model is similar to the first group of vote intention models in that it forecasts seat shares directly, by regressing seat shares on past seat shares and vote expectations.
How can we tell which of these models predicts more accurately?
The most convincing test of election forecasts occurs when the forecasts are made before the fact, across several elections. Alas, we would have to wait many years, if not decades, before we would see the combined results of such a test. Instead, we settle for the next best thing: we go back in time and predict each election before it took place. We begin with the 1987 election: we estimate the models using monthly survey data and election results from 1950 to 1983. We use the estimated models to generate forecasts for the 1987 election. We evaluate the forecasts’ accuracy using both the proportion of forecasts that correctly identified the party with the most seats (higher is better), and the mean absolute error in seats shares (lower is better). Then we add the 1987 election to the estimation sample, re-estimate the model, and generate a new set of forecasts for 1992. We proceed in this way, adding an additional election, re-estimating models, generating out-of-sample forecasts for each subsequent election, finally reaching the 2017 election.
Do vote expectations predict election results better than vote intentions?
Our results show how much we gain by replacing vote intentions with vote expectations (Table 1). Focusing on all elections, voter expectations improve upon the best voter intention model by seven points in terms of correctly predicting the winner party and by one point in terms of mean absolute error in parties’ seat shares. Focusing on elections with constant constituency boundaries, voter expectations improve upon the best voter intention model by 15 points in terms of correctly predicting the winning party and by one point in terms of mean absolute error in parties’ seat shares. Of course, the gains are even larger if we compare voter expectations with the worst voter intention models.
Taken together, these findings suggest replacing, or at least complementing, the vote intention question with the vote expectation question in political surveys. When the vote intention question is asked, respondents convey information only about themselves; when the expectation question is asked, they convey information about themselves and their social network. Most important, vote expectations predict election results better than vote intentions. If we had our way, every election poll would include the vote expectation question. Doing so would lead to better election forecasts and better media coverage of the election campaign.
Andreas Murr is Associate Professor of Quantitative Political Science in the Department of Politics and International Studies at the University of Warwick. His research focuses on election forecasting, the voting behaviour of immigrants, and the selection of party leaders.
Mary Stegmaier is interim vice provost for international programs and an associate professor in the Truman School of Public Affairs at the University of Missouri. Her research focuses on voting behaviour, elections, forecasting, and political representation in the U.S. and abroad.
Michael S. Lewis-Beck is F. Wendell Miller Distinguished Professor of Political Science at the University of Iowa. He has authored or co-authored over 290 articles and books, including Economics and Elections, The American Voter Revisited, French Presidential Elections, Forecasting Elections, The Austrian Voter and Applied Regression.
All articles posted on this blog give the views of the author(s), and not the position of LSE British Politics and Policy, nor of the London School of Economics and Political Science. Featured image credit: Pixabay (Public Domain).