Could the outcome of a second Scottish independence referendum depend on the precise wording of the ballot question? Rob Ford, Rob Johns, and John Garry discuss three likely wordings and their potential implications.
Scotland’s status in the union is on the agenda once again. Another dominant victory for the ruling Scottish National Party in May’s Scottish Parliament elections was quickly followed by First Minister Nicola Sturgeon announcing on 7 September that she aimed to seek a referendum by 2023, saying the question of Scotland’s future ‘cannot be avoided, nor postponed until the die is already cast’. But does it matter how the question of Scotland’s future is asked? Could the verdict the Scots deliver in a second independence referendum hinge on the precise wording of the ballot question – and, in turn, on how the voters think about the choice before them?
What question might be asked?
The wording of the ballot question was a matter of dispute ahead of the referendum in 2014, with the SNP government initially arguing that Scots should be asked ‘Do you agree that Scotland should be an independent country?’ (Yes/No)
The Electoral Commission criticised this as a potentially leading question, and in the end the Scottish government put forward the alternative ‘yes/no’ question – ‘Should Scotland be an independent country?’ (Yes/No) – that was eventually used.
The subsequent 2016 EU referendum provided Scottish voters, like those elsewhere in the UK, with an alternative way to frame a constitutional question. Voters were not given a yes/no question but instead asked whether they wanted to ‘Leave’ or ‘Remain’ in the EU. The same formulation can be used to ask: ‘Should Scotland remain in the United Kingdom or leave the United Kingdom?’ (Remain/Leave)
All three versions of the referendum question are at least theoretical possibilities for any future Scottish independence referendum. While the likeliest outcome is a rerun of the 2014 question, the pro-independence side may press again for the ‘do you agree’ formulation. Meanwhile the Leave/Remain wording has support from the unionist camp: not only as a supposedly more neutral question but also as it might enable them to profit from the widespread (EU) Remain sympathies in Scotland.
These disagreements suggest that each side believes the choice of question is consequential. Are they right?
Does the question matter?
Survey researchers have long known that the wording of a question can influence how it is answered. Questions which begin ‘do you agree’ encourage acquiescence bias, increasing agreement, particularly among people without strong starting views on the matter at hand. A similar dynamic can apply to yes/no questions, with at least some people drawn towards the apparently more positive, assertive ‘yes’ option.
The effects of a Remain/Leave wording are harder to predict. It could change both the way people think and the way they feel about the options in front of them. Take those – clearly the largest group in the Scottish electorate in 2021 (Figure 1) – who support both independence and EU membership. Many of them are not just pro-EU but identify strongly as Remainers. Indeed, some were only converted to the cause of independence by the Leave victory in the EU referendum. How comfortably could they vote for an independence option labelled ‘Leave’? Now take those who opposed independence in 2014 but voted Remain in 2016. Describing the anti-independence option as ‘Remain’ sit comfortably with their identity but could also remind them that, in fact, ‘Remaining’ in the UK means staying out of the EU. In either case, those who feel most strongly about the EU issue may be more influenced by a Remain/Leave wording.
Source: Authors’ analysis of data from the Scottish Election Study, 2021.
Of course, many Scots feel more strongly about the independence question than about the EU. This is an issue which has dominated Scottish politics for nearly a decade and on which a majority have very firm views, views unlikely to be shifted by nuances of the question. This likely limits the overall effect of referendum wording. On the other hand, with most recent independence polling just a margin of error from 50:50, even a small effect could be enough to tip the balance.
Testing question wordings
Does the wording make a difference? With the assistance of the British Election Study (BES), we ran an experiment to find out. In May/June 2021, Scottish respondents to the BES internet panel (2,837 in total) were randomly assigned to one of the three questions described above and repeated in Figure 2.
In turns out the choice of question does matter, at least a little, as Figure 3 illustrates. Support for independence is highest in the ‘do you agree’ formulation originally favoured by the SNP, and lowest in the ‘Leave/Remain’ formulation favoured by many unionists, with the 2014 ‘Yes/No’ question falling in between the two. It seems the instincts of both nationalists and unionists about wordings are correct – each side’s preferred question is associated with higher support for their preferred outcome, and the difference between those two is just about statistically significant (as indicated by the fact that the confidence intervals around them are not clearly overlapping).
The differences are certainly politically significant. On the Remain/Leave formulation, the independence side dips below its vote share in 2014. On the ‘do you agree’ formulation, it tops 50%. Of course, exactly whether and how wording tips the balance depends on the overall state of public opinion at the time. But the results highlight an important structural feature of referendums as binary choices with much at stake. If voters are evenly divided, as they are currently on independence, then even seemingly innocuous matters such as the question options could change the course of a nation’s history.
To finish, we should acknowledge that experiments like these are better at capturing the likely direction of wording effects than their likely scale. Our experiment simply presented respondents with a question whereas, if a second referendum campaign is anything like as long and hotly contested as the first, then voters will already have given the question extensive thought before they are confronted with the ballot question.
Two points can be made in response, though. One is that the effects seen in Figure 3 occurred despite a decade of arguing about independence. Clearly some people can still be swayed. The other is that the question does not appear for the first time on the ballot paper. Rather, it can define the whole debate. And the terms ‘Remain’ and ‘Leave’ are so potent that, if they are the frame for ‘indyref2’, things could look very different from the Yes/No battle of ‘indyref1’.
______________________
Rob Ford is Professor of Political Science at the University of Manchester.
Rob Johns is Professor in Politics at the University of Essex.
John Garry is Professor of Political Behaviour at Queen’s University Belfast.
Interesting. The key, I suspect, is not so much the wording of the question per se but how the options are framed in the long run up to any future referendum. While the question wording may play a part, any ‘bias’ in question would have to contend with this framing. The wording in the 1979 devolution referendum was, ‘Do you want the provisions of the Scotland Act, 1978 put into effect?’ but, frankly, that question did not really matter and may even have surprised people expecting some reference to ‘devolution’. In this sense, the campaign defines the question on the ballot paper. What mattered was what had gone before in the campaign.
Nonetheless, given the state of public opinion and because independence has now been so thoroughly debated/discussed/argued about, any slight advantage (as wording might have) could make enough of a difference to alter the result. That is why it is unlikely that each side would object to any change that might give even the slightest advantage to the other side. But what this shows is that framing will be important (and, I imagine, this blog shows this as well as anything could at this stage) and gives us a sense of how each side is likely to try to frame the options. Hence the value of this piece, to me at least, is less about question wording than what it tells us about the likely competing campaign frames.
The great unknown, unexplored but likely to be key is how/whether framing might alter perceptions given much opinion appears entrenched . This will be well understood by campaigners and no doubt much thought is going into how best to campaign/frame each campaign. It seems almost certain that if there is another referendum with another binary choice that the campaigns and framing choices will be different, at least in some respects and in terms of emphases, next time not least because the context has changed even if the wording remains as last time. It is probable that Labour will avoid the close association with the Tories that caused it so much misery afterwards (Gordon Brown’s decision to campaign separately from ‘Better Together’ was criticised during the 2014 campaign but in retrospect must be seen as wise). That creates a challenge and opportunity for supporters of the union. The duration of the campaign is likely to be different and that will affect any effort to alter framing. The pro-independence campaign will struggle to present itself as united as in 2014. Brexit presents opportunities and challenges to each side… All of this and more will affect framing. But will this be enough to move opinion significantly?
But the really interesting wild card in all of this is whether a crude binary choice is even likely – each side knows well that a binary choice and keeping the issue at or near the top of the agenda assists the Conservatives and SNP electorally but also makes a referendum a massive gamble and thereby also less likely. We have the binary paradox: a simple binary choice helps the parties most associated with these positions but creates a constitutional stalemate (thought more of a problem for the SNP than the Tories).
Breaking the deadlock will require something other than changing the wording in a binary referendum even if that could be agreed. Each side is uncertain as to the likely outcome if more options were available and polls /surveys are of limited help with the absence of a meaningful campaign – though third options poll reasonably well given no party or significant body is making the case for it. A third option, as yet unspecified in detail, but not impossible to imagine, would open up debate, make the final result much more uncertain but could potentially lead to a period of constitutional stability (the Scottish Question can never be finally resolved for reasons explored in my book on the subject – the ‘settled will’ will likely be temporary if that is ever found and we are far from finding that with either option) but that is why supporters of the binary referendum are so wary of it. Constitutional stalemate is preferred over an option that might deprive either the Tories or SNP of what they want.
James Mitchell
May I ask if the data in Figure 3 including or excluding DKs?
A point not made in this article though is that many Scots (maybe especially those who are pro independence) do not accept the premise of a leave/remain question.
Great Britain was formed by the the union of Scotland and England (which as an entity included Wales). The United Kingdom was formed by the union of great Britain and Ireland (later modified as the union of great Britain and Northern Ireland).
Many Scots will feel (irrespective of the ex-ante position of the UK government) that in an independence referendum they are not being asked whether they want to remain or leave the UK, but rather do they want to, or not, end a union such that the UK no longer exists. This is clearly different from the EU situation which obviously had a continuing existence without the UK.