In this post, Steve Fisher and Jonathan Jones explain their election forecasting model which powers the results presented at Electionsetc.com. Here they explain their model and the assumptions that underpin it and provide some predictions for May 7th. They predict a that there will be a seriously hung parliament, that there is a 70% chance that the Conservatives will be the largest party, and that the Liberal Democrats could yet become kingmakers again.
The latest ElectionsEtc.com forecast is for a seriously hung parliament. The Conservatives, Liberal Democrats, DUP and UKIP combined would have 325 seats; a majority given Sinn Fein do not take their seats. But at the same time Labour, Lib Dems and SNP would have 338. In such circumstances the Lib Dems could arguably go either way. They have warned against either the SNP or UKIP holding the balance of power. With this configuration the Lib Dems hold the balance of power but their choice would be between government on the left dependent on the SNP and government on the right dependent on UKIP.
Ours is also a probabilistic forecast and there is still a lot of uncertainty. The probability of such a finely-balanced outcome where Liberal Democrats are kingmakers is, at 7%, fairly small.
The probabilities for various different combinations of parties having a majority are below, but first we explain how we produce them. The methodology has evolved a bit over time and may change again but currently the steps are roughly as follows.
1. Calculate averages of recent Britain-wide and Scotland-only vote intention polls
For GB polls we use an average of various different methods of averaging. The idea is to look for consistency and robustness across different methods. This includes checking how things change after excluding outliers, excluding particular pollsters one-by-one, weighting for past performance or not, and varying how far back we go and how many polls per pollster we use. The aim is to get a polling average that treats the pollsters as relatively but not completely equal and averages over enough polls that sampling variation cancels out. For much of the time this also has the effect of smoothing over small short-lived blips.
Scottish polls are fewer and further between so we use the whatscotlandthinks.org method of taking the average of the last four polls.
2. Use regression analysis of historical votes and polls to forecast how GB vote intention will change from a given number of days before the election, and to estimate prediction intervals for those changes.
Several months before the election one of the main issues is to estimate the extent of swing back for parties that have gone up or down in the polls since the last election. The graph below shows how the Labour forecast has remained relatively steady over the past year and a half, since Labour support in the polls has dropped at roughly the rate one would expect from previous election cycles. However, the Tories and Lib Dems both failed to make any recovery in the 18 months before the campaign and so their forecasts have dropped.
Some swing back is still expected for those parties where the polls are still showing big changes since the last election, including the Liberal Democrats, UKIP, SNP and Scottish Labour. But for Labour and the Conservatives at the national level, the main reason the forecast shares differ from the current polling average is because the polls have tended to overestimate Labour and underestimate the Conservatives, both by about a point and a half.
But even with our current forecast of a two-point Conservative win, the historical record of late campaign changes and polling error is sufficiently varied that a lot else is possible. As the implied prediction intervals (ranges of probable outcomes) in the graph below show, we cannot be sure which of the two main parties will emerge ahead in the share of the vote.
3. Use the forecast vote shares and uncertainty estimates, and between party correlations in the opinion polls, to simulate hypothetical election results.
For the technically minded we use a multivariate normal distribution with variances for each party estimated by pooling the forecast standard errors from the previous step.
Since parties do not go up or down independently we use the average correlations between changes in party shares in successive polls to inform the covariances for the simulations. So in a hypothetical election where UKIP does particularly well it is more likely that the Conservatives especially will do badly. There is also a big negative correlation between Conservative and Labour performance, which serves to widen the range of possible outcomes in the simulations.
4. Use Ashcroft constituency polls and individual-level data kindly provided by YouGov to identify constituencies where parties are doing particularly well or particularly badly, and apply adjustments to the hypothetical results accordingly.
The most important factors within England and Wales are to do with incumbency.
Those Conservative MPs who took their seat from an MP from another party in 2010 are doing a couple of points better than other Tory candidates. This seems to be an instance of the classic “sophomore surge”, which is common in the US and also seemed to help many first-term Labour MPs hold on in 2001 despite a swing to the Conservatives.
Also, incumbency effects for Liberal Democrat MPs seem to be strengthening, by about 7 points above and beyond the personal vote bonus they got in 2010. But this is against a backdrop whereby the party is falling more where it started stronger, not least because there are many seats where they are starting with fewer votes than uniform swing suggests they should lose. But the net effect of all this for Lib Dem seats is not much different from uniform swing.
Labour are the chief beneficiaries of the decline in Lib Dem support but this means they are advancing more in places where the Lib Dems did well last time than they are in their target seats. Moreover in Scotland Labour are falling further the higher they started. Again, this is partly because there are places where they did not start high enough to fall in line with the national average. This means the SNP are forecast to do better and Labour worse than would be expected from uniform swing with our Scottish forecast share of the vote.
5. Convert forecast seat shares into probabilities for different parties winning in each constituency in each hypothetical election.
For this we apply the method used for the exit poll, explained here. With so many hypothetical elections, this step does not make much difference. But it does bring down the SNP forecast from 53 to 51 out of 59 Scottish seats. There are more opportunities for the SNP to under than over perform a such a high forecast, gains from above average swings are likely to be more than offset by the consequences of below average swings.
6. Set subjective probabilities for a very small number of particularly unusual seats where the constituency variation models are problematic.
There are not many of these, but they are especially important for UKIP. Some of the Ashcroft polls in UKIP target seats are relatively old and the scale and pattern of change for UKIP is so dramatic that it is hard to predict what will happen in a few relatively unusual seats where UKIP are running intensive campaigns. So the estimate of the total number of UKIP seats is effectively a guess based on various polling and non-polling sources of information.
7. Analyse the distribution of seats forecasts across the hypothetical elections to calculate probabilities for key events (e.g. hung parliament, Con largest party etc.)
For any single simulated hypothetical election the forecast number of seats for a party is just the sum of the probabilities that the party will win each seat. The distribution of these forecast seat totals across the 10,000 simulated hypothetical elections tells us the relative chances of different events. E.g. if 5,000 of them were to have the Tories ahead and 5,000 had Labour ahead then there would be a 50:50 chance of the Tories being the largest party.
We can also look to see which combinations of parties will have a majority (more than 323 seats given that Sinn Fein do not take theirs). The pie chart in the figure below shows how complex this gets even when we use strong simplifying assumptions as to which grouping is most likely to control the government when several different combinations are possible. Whether it would be a coalition, confidence-and-supply or some other kind of agreement is not addressed here; that is less important for policy than which parties are involved.
Starting from the top of the pie chart and working clockwise: there is still a 9% chance of a Tory majority. If they are just short of a majority they could call on the DUP, or if that is not enough then the Liberal Democrats or perhaps both if needs be.
Conversely, starting from the top of the pie chart and working anti-clockwise shows the relative chances of different Labour led governments. As with the Tories, Labour would also call on the DUP and Lib Dems first if those parties were sufficient to yield a majority. This assumes that the SNP will be harder for Labour to do a deal with, especially given their differences over Trident. But if it were necessary to appeal to the SNP and SNP support was sufficient to give Labour a majority then a Lab-SNP deal would probably form without any other parties being involved. Experience from other countries suggests that a majority-commanding group of parties is more likely to control the government the fewer parties there are in the group was well as greater ideologically similarity between them.
Finally, in the gold section at the bottom pie chart there is a 7% chance of the Conservatives being the largest party and a Con-LD-DUP-UKIP grouping having a majority. However, the Lib Dems could also form a majority with Labour and the SNP, so we call this the “Lib Dem kingmakers” scenario. This includes our central forecast: Con-LD-DUP-UKIP with 325 MPs, Lab-LD-SNP with 338.
There is a roughly 70% chance of the Tories being the largest party (on votes or seats). However, as a result of the limited number and size of potential governing partners for the Tories, there is only a 46% chance of a Conservative-led government, or 53% including the Lib Dem kingmakers scenario.
For more details of our current forecast, and for updates, please go to ElectionsEtc.com.
Notes: Figures accurate as of April 22nd, 2015. Stephen Fisher is not a member of any political party and does not receive money from any person or organisation that might create a conflict of interest. Jonathan Jones is a member of the Liberal Democrats. Please see our acknowledgements here.
Steve Fisher is an Associate Professor in Political Sociology and the Fellow and Tutor in Politics at Trinity College, University of Oxford.
Jonathan Jones was a researcher at The Spectator until 2013.