While the public have voted to leave the EU, it is less clear what that means in terms of policies around key aspects of EU membership. Sara Hobolt and Thomas Leeper (LSE) examined public opinion on various dimensions of Brexit using an innovative technique for revealing preferences that asks survey respondents to evaluate bundles of negotiation outcomes. Their results suggest that while the public is largely indifferent about many aspects of the negotiations, Leave and Remain voters are divided on several key issues.
What did we do?
Measuring public preferences is commonly approached through survey questionnaires, in which individuals express a degree of favour or disfavour toward a particular object such as a policy, product, political candidate, or – recently – possible outcomes of Brexit negotiations. When objects of evaluation have many features, such as preferences over trade, preferences over immigration, etc. entangled in current negotiations, it is common to ask about those specific aspects as separate questions. Evaluating the relative importance of the different features, however, becomes empirically challenging.
An alternative approach is “conjoint” experiments, in which respondents are asked to consider “bundles” of outcomes as a whole – that is, to consider a possible Brexit negotiation deal that includes a large number of different features as a complete package, with the specific features (e.g., the amount of immigration control) randomly varied. We recently conducted a conjoint experiment about the public’s attitudes toward Brexit with a total sample size of 3,293 respondents. It was fielded 26-27 April via YouGov’s online Omnibus panel.
In our design, respondents were shown a series of possible Brexit negotiation outcomes in pairs and were asked to choose which one of the two that they liked best (there was no “don’t know” option). This requirement that they choose – that they make trade-offs between the different aspects of the negotiations – is crucial to the conjoint approach and to our results. A screenshot of what the respondents saw is below:
Conjoint experiments differ considerably from traditional public opinion surveys in a number of ways that make powerful tools for understanding preferences. First, they allow us to make comparisons between respondents’ evaluations of different bundles in order to detect the relative importance of individual features. Rather than asking respondents directly about each separate feature, we can allow their choices in these difficult trade-off scenarios to reveal the acceptability of different features. Our results, therefore, have to be understood as the preferences respondents hold over Brexit when all of the features of the negotiation are weighed together as a package. While this makes it difficult to compare our results to those of more traditional polling, we gain a considerable amount by asking respondents to engage directly with the difficult trade-offs involved in the negotiations.
Second, because the features that are shown to respondents are fully randomized in the design, we ensure that respondents do not infer or attempt to infer how different aspects of negotiations might be tied to others. For example, if we simply asked respondents about their preferences over trade policy, they are likely to make assumptions about what that might mean for immigration policy. In the conjoint, we provide information about both aspects (as well as others), thereby making any trade-off explicit rather than implicit. As well, in our design, we never use any of the most politicized labels from ongoing debate – such as “hard” or “soft” Brexit, or “freedom of movement”, “free trade”, etc. – but instead attempt to use more precise language to describe features of a possible deal.
Third, because each respondent is shown multiple pairs of trade-offs (six in our design), a conjoint design yields a very large dataset of revealed preferences, in our case just shy of 20,000 data points about what the public wants from Brexit. This gives us considerable statistical power to detect differences in Leave and Remain voters’ taste for various components of negotiations. The fact that the packages of negotiation outcomes shown to respondents consist of fully randomized combinations of features means that we can use straightforward mathematical (albeit perhaps confusing at first glance) procedures to measure those preferences.
Finally, a conjoint experiment allows us to present our results in two distinct ways. One of these describes the pattern of preferences in terms of levels and the other describes the pattern of preferences in terms of effects. Because respondents are forced to choose one of the two outcomes shown in each pair, we can draw out of the pattern of choices the proportions of respondents that would accept or reject outcomes containing a given feature (in light of the trade-offs between features and the strengths and weaknesses of the chosen negotiation bundle as a whole).
On this measure of levels, a feature scoring 100% means that respondents would always choose outcomes that included that feature, regardless of any other aspect of Brexit. For example, if the immigration policy “Full control over EU immigration and little to no EU immigration” scored 100%, the public would trade-off everything else to have that immigration policy in a bundle consisting of any other combination of features. If it instead scored 0%, it would mean the public would never accept this policy and would similarly trade-off everything else to avoid it. If, finally, it scored 50% that would mean that the public was largely indifferent – they would accept it 50% of the time and reject it 50% in light of other possible features of the negotiation. These are not unconstrained preferences as in typical polling, but instead patterns of opinions reflecting the inherent complexity of the decisions at hand. In other words, if forced to choose (and to choose possibly between unpleasant alternatives), what percentage of the public would accept or reject whole deals based on this particular feature?
The second way of presenting the results is as effects – in the statistical language of a conjoint design, an “average marginal component effect”. In this presentation, rather than highlighting rejection versus indifference versus support, we can convey the degree to which a given feature increases or decreases support for a bundle as a whole relative to a baseline deal. The meaning of the results are identical but are presented in a way that cannot be interpreted as percentages of the public that support a feature or a bundle of features. Instead, they are weightings of the importance of different features relative to a baseline combination of features – in our case, we treat a “no deal” exit of the EU as the baseline condition, so a positive effect can be interpreted as a given feature increasing support for a negotiation outcome that contains it and a negative effect can be interpreted as a given feature decreasing support for a negotiation outcome that contains it.
The Specific Survey Procedures
Our design followed the emerging paradigm for conjoint studies, entailing a number of features of Brexit negotiation outcomes, and the request that survey participants complete multiple discrete choice tasks. At the beginning of the study, participants completed a few brief background questions, with most demographic data being drawn from YouGov’s profile variables, and then completed five sequential conjoint profile ratings. Each conjoint task presented two alternative Brexit negotiation outcome scenarios (see figure above) and asked participants: “We are interested in your opinions about possible agreements between Britain and the EU regarding Britain’s exit from the EU and future relationship. Please consider the following two possible agreements.” They were then shown two outcomes that varied along eight dimensions. After that, they were asked “Which of these two outcomes do you prefer? (Outcome A; Outcome B)” and forced to choose one of the two.
The eight dimensions (attributes) were chosen to cover the most salient aspects of the Brexit negotiations. These features were: (1) immigration controls, (2) legal sovereignty, (3) rights of EU nationals, (4) ongoing EU budget payments, (5) one-off settlement, (6) trade terms, (7) status of the Republic of Ireland/Northern Ireland border, and (8) the timeline for Brexit. The levels were designed in such a way as to range between the two most extreme negotiation outcomes: a ‘ soft Brexit’ with continued British membership of the EU’s Single Market and Customs Union and a ‘ no deal’ scenario in which negotiations break down before an agreement has been reached.
The selection of levels of each feature for each profile was fully randomized, as was the order in which those features was presented in each table. This mitigates the risk of “profile ordering effects” wherein certain features are deemed more important because they are presented first or last in the table.
LSE Blogs image, licenced under Public Domain.
Starting with the presentation of effects, we express the effects against a baseline defined by the ‘no deal’ scenario in which Britain and the EU are unable to conclude an agreement on Britain’s exit. This means that there would be no trade deal, full legal independence of Britain from EU law and the European Court of Justice, no one-off or continuing payments to the EU budget, full control over immigration with no continuing EU immigration, the loss of rights of EU citizens currently residing in the UK, and a full (customs and passport) border between Ireland and Northern Ireland. Positive effects thus indicate support for ‘softer’ Brexit outcomes and negative values indicate opposition to those scenarios. The figure below presents these results separately for Leave and Remain voters, ignoring those who did not vote, based upon a measure of vote choice which was recorded immediately after the 2016 referendum. The lines around the point estimates are 95% confidence intervals.
Relative to the baseline of a ‘no deal’ exit, Remain voters favour alternative deals in every policy area except one-off payments to settle outstanding debts to the EU. They most favour a Brexit scenario that is ‘softer’ with respect to immigration, the rights of EU nationals, trade deals, ongoing budget contributions, and the border between Ireland and Northern Ireland. Leave voters, by contrast, do not like any of these policy alternatives, except in areas related to the rights of EU nationals living in Britain and the nature of any trade deal. What people would like Brexit to mean therefore depends on how they voted in the referendum.
What people would like Brexit to mean depends on how they voted in the referendum
When we translate these results into levels, we obtain results that are mathematically identical to those above but that can be mapped out in terms of the proportions of respondents that would accept or reject outcomes that include each feature (see figure below). As should be immediately clear, most of the levels hover right around 0.5 (50%), meaning that respondents are largely indifferent about most of the aspects of the negotiations and the particularities of the deals. That said, there are some striking patterns of results (which will echo the effects analysis above).
Perhaps most striking is that few of the levels diverge particularly far from 0.5 (50%) toward either complete rejection (0%) or complete acceptance (100%). That means that for our respondents, most aspects of the negotiations are not sufficiently unacceptable to completely avoid at all costs nor are there aspects that are so important as to drive compromise of other features at all costs. The headline conclusions are therefore that the public is surprisingly willing to compromise on some aspects of the negotiations, even things that are arguably very important to them (as shown in the effects analysis above).
What we caution readers to avoid, however, is looking at these numbers as raw measures of support for particular features (as in conventional public opinion polling). This they are not. At no point did we ask respondent to evaluate individual features – they were only asked to make judgments of bundles of outcomes. The results we present are the preferences that are revealed by their choices between bundles. A value, for example, of 0.29 (29%) for Remain voters on “All must leave” does not mean – as it would in a typical polling context – that 29% of Remain voters favour all EU citizens having to leave to the UK. Instead, given the design of our study, it suggests that 71% of the time, Remain voters would reject negotiations that contained that policy feature regardless of everything else it was bundled with and accept it only 29% of the time in light of everything else it was bundled with. When Leave voters score at 0.58 (58%) on “Full control over EU immigration and little to no EU immigration” that does not mean 58% of the Leave voters want no EU immigration but rather than Leave voters will accept negotiation outcomes containing that policy feature 58% of the time and reject negotiation outcomes containing it 42% of the time.
These somewhat complex interpretations, which require looking at the whole of our results together (not just the individual levels of support for specific items) and considering precise what task was given to our survey respondents, reflects the complexity of the task of negotiating the UK’s exit from the EU. When tasked with make difficult decisions involving complex trade-offs, what is acceptable and what is not? The answers aren’t always easy and what we see consistently is that the public – both Leave and Remain voters – are willing to make trade-offs. Indeed, they appear to be almost completely indifferent over some aspects of the negotiations (such as the status of the border between Ireland and Northern Ireland and the timeline for agreeing any deal). It is not easy, as the government must now do, to make trade-offs between complex and interconnected aspects of Britain’s relationship to the EU. We designed our study to bring that complex decision-making to the public to measure their preferences in a new and different way from much other polling. Our results need to be read as a reflection of that complexity and the particular design we used.
While there appear to be few aspects of the negotiations that Leave and Remain voters demand at all cost or reject at all cost, there are aspects of the negotiations that are very important to them. Leave voters are particularly concerned about control over immigration and opposed to deals that give Britain less than “full control” over immigration. They are similarly concerned about legal sovereignty and any “divorce bill”. They also strongly prefer scenarios where EU citizens are able to apply for residence more than scenarios where all must leave. Remain voters care much more about the rights of EU citizens – indeed, no other aspect of the negotiations appears to matter more to them. They also agree with Leave voters that trade terms with fewer barriers and lower tariffs than a “no deal” scenario would bring are preferable to a hard break from the common market. Yet, ultimately, citizens are indifferent about many aspects of Brexit.
A detailed technical report is available containing all information about our study design and analysis.
This post represents the views of the author and not those of the Brexit blog, nor the LSE.
Sara Hobolt is Sutherland Chair in European Institutions at the LSE European Institute.
Thomas Leeper is Associate Professor in Political Behaviour in the Department of Government at the LSE.