Sir Robert Worcester is a Visiting Professor of Government at LSE and an Honorary Fellow. He founded MORI in 1969.
The blogs are at it again, even letter writers to newspapers. A headline in the Standard the other day caught my eye. “Election polling is too uncertain.” It would, wouldn’t it.
From the writer’s first sentence, “With regard to the variable picture on support for the parties in different opinion polls…”, with respect, undermines the rest of his argument. The media’s fixation on the lead, rather than the share of support for each party is the principle cause of his confusion. With the exception of the two ‘new boys on the block’, Angus Reid and Opinium, since the beginning of the year there have been some 59 polls (counting the daily onslaught from YouGov as one every two days). The Conservative share of voting intention has been 38 per cent plus or minus the usual three percent margin of error in all 59, for a 100 per cent ‘Stability Index” rating. See embedded spreadsheet below (you can zoom in or show the table in full screen using the buttons below the embedded spreadsheet):
General Election Voting Trends January – 5 April 2010
The figures for Labour are somewhat less than perfectly within the +/- 3 per cent: only 97 per cent consistent. The figures for the Liberal Democrats is somewhat better: 98 per cent.
The April Indices are even better so far: four out of four at 100 per cent, Tories 38 per cent, Labour 30 per cent, LibDems 20 per cent and Others 12 per cent.
The conclusion of the letter writer, a distinguished academic whose name I shall not mention here, is that “Our (sic) polls, built for an earlier period, are now hopelessly out of date”. He asks more of polls than the laws of statistics and the vagaries of the public interviewed over several days and at different times, can ever deliver, as it is an impossible task. They are what they are, the best approximation of the state of public opinion at the time they were taken, better than anyone’s guess, as they are done by honest people, doing their best with the tools of their trade, the public, the science of sampling, and the art of asking questions.
You can think of it this way. If we were to send out our interviewers to a hundred constituencies, carefully selected to give a random representation of all the British constituencies, and ask each interviewer to stand at the highest point in the constituency and at precisely 12 noon enter the temperature from an accurately tested thermometer into their computer and send it into the central computer to be aggregated, you would have the average temperature of the country to within a fraction of a degree.
Then if the next day you asked them to exactly replicate this and send the next day’s result in, would you expect the same reading? No, of course not, everyone will understand that. So why is it that anyone fails to understand that such a mercurial thing as public opinion cannot be expected to be any more accurate than the temperature?
Of course correctly assessing popular vote shares for small parties is especially crucial when a minority/hung/coalition/balanced parliament looms. Dare we push the envelope further and also pray for accurate seat projections for small parties? Take it from a Canadian who suffers through election after election knowing a majority government is unlikely, but not knowing in advance whether smaller parties are strong enough to provide stability. And don’t even ask about the ability of Canadian-based political observers to offer accurate seat projections. As there is much room for improvement in all countries, why not have a conference and discuss the possibilities?
Well, Bob is being kind here in not mentioning me by name, but I don’t mind coming clean that I did tell the Standard (who made it into a letter) that our current opinion polls were “built for an earlier period, but are now hopelessly out of date. Until we can get polls that get all the parties right, no one is going to believe them – and they are right not to do so.”
Undeniably our opinion polls evolved at a time when we had only two or two-and-a-half parties, but now we have five or six or seven parties that matter, a lot, in every region of the UK. In the June 2009 European Parliament elections, the third, fourth, fifth and smaller parties got 54 per cent support between them, but still the polls and the newspapers concentrated on just the Labour and Tory ratings in view. In the 2010 Westminster election the Other parties will be lucky to get 10 or 11 per cent combined, but exactly how they do is still going to be crucial for how much the top 2 parties get – with the Conservatives damaged mainly by UKIP and BNP support and Labour suffering from the Greens.
Bob takes two tacks in his response above, but with the greatest of respect they do not match up. The first is to stress that the polls are capturing the main parties right, plus or minus 3 per cent. But in modern multi-party competition conditions, where the Tory versus Labour race is a fight about how to carve up just over two thirds of the vote, + or – 3 per cent is the whole election – a kind of annual weather forecast that tells us conditions will be “variable”.
Bob’s second tack is that in asking for polls that get all the top 6 or 7 parties support right I am asking “more of polls than the laws of statistics and the vagaries of the public interviewed over several days and at different times, can ever deliver, as it is an impossible task. They are what they are, the best approximation of the state of public opinion at the time they were taken”.
My response here is that problems in methods should be spurs to innovation and not just accepted with fatalism. We will have perhaps as many as 130 not very useful polls over the two months leading up to polling day – why can’t some of this considerable expenditure of money, energy and expertise be mobilized to get fewer, better polls?
For instance, polls that are repeating fairly regularly could aggregate the picture that they are getting of smaller parties’ support and that would be immediately helpful. I don’t think it would be feasible for YouGov, who are just trekking around within a single large panel on a daily basis, but it’d be very useful if phone polls could do this.
When you have a problem like that facing the polling industry at present, you only have two options – live in denial; or fix it. But to fix something, you first need to admit where the problems are and then look long and hard at how doing new things can address them.
Whilst I absolutely agree with your analysis, the spread for the Tories has actually been 35-42. The couple of rogue 42s in YouGov/ComRes polls early in the year actually mean that the stability index would be closer to 97%. A minor point that doesn’t detract from the thrust of what you are saying.