Could the outcome of a second Scottish independence referendum depend on the precise wording of the ballot question? Rob Ford, Rob Johns, and John Garry discuss three likely wordings and their potential implications.
Scotland’s status in the union is on the agenda once again. Another dominant victory for the ruling Scottish National Party in May’s Scottish Parliament elections was quickly followed by First Minister Nicola Sturgeon announcing on 7 September that she aimed to seek a referendum by 2023, saying the question of Scotland’s future ‘cannot be avoided, nor postponed until the die is already cast’. But does it matter how the question of Scotland’s future is asked? Could the verdict the Scots deliver in a second independence referendum hinge on the precise wording of the ballot question – and, in turn, on how the voters think about the choice before them?
What question might be asked?
The wording of the ballot question was a matter of dispute ahead of the referendum in 2014, with the SNP government initially arguing that Scots should be asked ‘Do you agree that Scotland should be an independent country?’ (Yes/No)
The Electoral Commission criticised this as a potentially leading question, and in the end the Scottish government put forward the alternative ‘yes/no’ question – ‘Should Scotland be an independent country?’ (Yes/No) – that was eventually used.
The subsequent 2016 EU referendum provided Scottish voters, like those elsewhere in the UK, with an alternative way to frame a constitutional question. Voters were not given a yes/no question but instead asked whether they wanted to ‘Leave’ or ‘Remain’ in the EU. The same formulation can be used to ask: ‘Should Scotland remain in the United Kingdom or leave the United Kingdom?’ (Remain/Leave)
All three versions of the referendum question are at least theoretical possibilities for any future Scottish independence referendum. While the likeliest outcome is a rerun of the 2014 question, the pro-independence side may press again for the ‘do you agree’ formulation. Meanwhile the Leave/Remain wording has support from the unionist camp: not only as a supposedly more neutral question but also as it might enable them to profit from the widespread (EU) Remain sympathies in Scotland.
These disagreements suggest that each side believes the choice of question is consequential. Are they right?
Does the question matter?
Survey researchers have long known that the wording of a question can influence how it is answered. Questions which begin ‘do you agree’ encourage acquiescence bias, increasing agreement, particularly among people without strong starting views on the matter at hand. A similar dynamic can apply to yes/no questions, with at least some people drawn towards the apparently more positive, assertive ‘yes’ option.
The effects of a Remain/Leave wording are harder to predict. It could change both the way people think and the way they feel about the options in front of them. Take those – clearly the largest group in the Scottish electorate in 2021 (Figure 1) – who support both independence and EU membership. Many of them are not just pro-EU but identify strongly as Remainers. Indeed, some were only converted to the cause of independence by the Leave victory in the EU referendum. How comfortably could they vote for an independence option labelled ‘Leave’? Now take those who opposed independence in 2014 but voted Remain in 2016. Describing the anti-independence option as ‘Remain’ sit comfortably with their identity but could also remind them that, in fact, ‘Remaining’ in the UK means staying out of the EU. In either case, those who feel most strongly about the EU issue may be more influenced by a Remain/Leave wording.
Source: Authors’ analysis of data from the Scottish Election Study, 2021.
Of course, many Scots feel more strongly about the independence question than about the EU. This is an issue which has dominated Scottish politics for nearly a decade and on which a majority have very firm views, views unlikely to be shifted by nuances of the question. This likely limits the overall effect of referendum wording. On the other hand, with most recent independence polling just a margin of error from 50:50, even a small effect could be enough to tip the balance.
Testing question wordings
Does the wording make a difference? With the assistance of the British Election Study (BES), we ran an experiment to find out. In May/June 2021, Scottish respondents to the BES internet panel (2,837 in total) were randomly assigned to one of the three questions described above and repeated in Figure 2.
In turns out the choice of question does matter, at least a little, as Figure 3 illustrates. Support for independence is highest in the ‘do you agree’ formulation originally favoured by the SNP, and lowest in the ‘Leave/Remain’ formulation favoured by many unionists, with the 2014 ‘Yes/No’ question falling in between the two. It seems the instincts of both nationalists and unionists about wordings are correct – each side’s preferred question is associated with higher support for their preferred outcome, and the difference between those two is just about statistically significant (as indicated by the fact that the confidence intervals around them are not clearly overlapping).
The differences are certainly politically significant. On the Remain/Leave formulation, the independence side dips below its vote share in 2014. On the ‘do you agree’ formulation, it tops 50%. Of course, exactly whether and how wording tips the balance depends on the overall state of public opinion at the time. But the results highlight an important structural feature of referendums as binary choices with much at stake. If voters are evenly divided, as they are currently on independence, then even seemingly innocuous matters such as the question options could change the course of a nation’s history.
To finish, we should acknowledge that experiments like these are better at capturing the likely direction of wording effects than their likely scale. Our experiment simply presented respondents with a question whereas, if a second referendum campaign is anything like as long and hotly contested as the first, then voters will already have given the question extensive thought before they are confronted with the ballot question.
Two points can be made in response, though. One is that the effects seen in Figure 3 occurred despite a decade of arguing about independence. Clearly some people can still be swayed. The other is that the question does not appear for the first time on the ballot paper. Rather, it can define the whole debate. And the terms ‘Remain’ and ‘Leave’ are so potent that, if they are the frame for ‘indyref2’, things could look very different from the Yes/No battle of ‘indyref1’.