Each month, the team from electionforecast.co.uk compare new constituency polls as they are released to their estimates of what polls would show in those constituencies. This allows for an assessment of the accuracy of their modeling approach. More information about the overall model can be found here. The results presented in this post use the constituency polls released by Lord Ashcroft on 4th March 2015.
For the past few months, we have been comparing Lord Ashcroft’s constituency polls as they are released to our estimates of what such polls would show from the previous day. These comparisons offer a test of our model’s ability to accurately predict what polls would show in constituencies where there has not yet been a poll.
In March, Lord Ashcroft expanded the range of his constituency polls. Previously, these polls have focused on highly marginal seats. This month, the constituency polls include four seats that the Conservatives are defending from Labour, and where the margin of victory in 2010 was between 8.8% and 10.6%. Additionally, Lord Ashcroft also included another batch of polls from Scotland. Previously, Scottish constituency polls have been focused in areas where the independence “Yes” vote was strong. This month, the Scottish polls are from a range of constituencies where there has been less support for the SNP in recent years.
These constituency polls have proved the toughest yet for our model to predict. In the graphs below, we plot our estimates from 3rd March for these constituencies on the x-axis and the Ashcroft estimates on the y-axis. Note that, as for previous months, our model uses the data before the application of the “spiral of silence” adjustment which reallocates survey respondents failing to give a preference to the parties they voted for at the last election.
Both the polls and our estimates include uncertainty; in the figures below we break these results out by constituency and plot ellipses capturing the joint margin of error around each point. Whenever these ellipses cross the dashed 45-degree line, we can say that our predicted vote share range for a given party, in a given constituency, coincides with the range implied by Lord Ashcroft’s polls.
The plots reveal that our model has performed less well in predicting this batch of polls than it has in previous months. While we accurately recover the current state of play in many constituencies, the model also makes several prediction errors, notably in Dumfries and Galloway, and Edinburgh South West.
To measure the magnitude of our prediction errors more rigorously, we compute a Euclidean distance between our estimates and Ashcroft’s polls. Averaging across all constituencies, the mean distance between our estimates and Ashcroft’s polls is 8.6 percentage points. Given the size of the polls and the uncertainty we were calculating for our estimates, we would have expected it to be 6.7 percentage points. This suggests that, for constituencies such as these, we are somewhat overconfident in our estimates.
We can also compare the performance of our model to a simpler forecasting model such as uniform swing. As we have mentioned before, there is no single correct method for calculating uniform swing, and particularly in this batch of polls where the constituencies cover England, Scotland, and Wales. Electoral dynamics in 2015 are likely to vary dramatically across the different countries in the UK, and so we calculate three versions of national swing (one for each country) to give UNS a good chance to perform as well as our own model.
Here, we calculate the England-, Scotland-, and Wales-only swings from our estimates of the current national polls, calculate the average swing implied by those, and then apply that average swing to each constituency’s 2010 results. We can then calculate the mean Euclidean distance between the UNS predictions and the Ashcroft results for each country. We compare the country-level mean Euclidean errors of the UNS model and our forecasts in the table below.
This table shows that our model outperforms the country-level UNS model in Scotland and Wales as does very slightly worse in England. On the other hand, it is actually in England that both our model and UNS are most predictive, uniform swing applied within Scotland and Wales perform very poorly and we do much better there, if not as well as we do in England.
In sum, compared to previous waves of Ashcroft polls, we did not do quite as well this time. Our estimates were a bit more overconfident than in previous waves and we did not outperform UNS in the four England constituencies. We have now updated the forecasts for these constituencies, as well as other, similar constituencies, by incorporating these latest polls into the model. Lord Ashcroft has promised to revisit key marginal constituencies over the next month, and we will, of course, include new polling information as it is released. With only 62 days until the election, our model is increasing the weight we put on the polls in forming our election forecast. Historically it is during the period after the dissolution of Parliament that the polls move the most rapidly from day to day, making the final weeks of the campaign especially exciting for those of us forecasting the election.
Jack Blumenau is a PhD candidate in Government at the London School of Economics.
Chris Hanretty is a Reader in Politics at the University of East Anglia.
Benjamin Lauderdale is an Associate Professor in Methodology at the London School of Economics.
Nick Vivyan is a Lecturer in Quantitative Social Research at the Durham University.