Election polls play a pivotal role in shaping political action and reporting elections to the public. Jeanne Sheehan Zaino argues their underlying assumptions about the nature of society and methodologies lead to oversimplified explanations for why people vote the way they do.

---

Was 2024 was a good year for US election polls? 

Nate Silver writes, “polling did not experience much of a miss and had a reasonable year.” It’s a view that’s widely shared by everyone from John Zogby and Mike Traugott to Christopher Wlezien and G. Elliot Morris. Among the findings, polling error at the state level was the lowest in 25 years and, at <3%, the overall error rate was small by historical standards.

Their upbeat assessment, however, is not shared by the public.

Taking to the airwaves after the election, Jon Stewart spoke for many when he lashed out at pollsters saying: “You don’t know s*** about s***.” 

What explains the gulf between public perception and poll performance? If the polls weren’t far off, why do they seem so wretched to so many?

One explanation can be found in the nature of the polling post-mortems. When experts examine poll performance, they focus on issues of systemic error, such as non-response bias, weighting, sampling, likely voter models, etc. Silver’s recent autopsy is typical in this regard. He attributes the error, small as it was, to response rates, weighting techniques, and herding.

He is not wrong. But like so many engaged in post-election poll assessment, Silver fails to address a more insidious and troubling problem– the naturalist assumptions on which polling is based.

The term ‘naturalism’ in this context refers to the tendency of social scientists to apply techniques and methods used in the natural sciences to study human beings. In keeping with their colleagues in physics and chemistry who rely on the scientific method to understand atoms and molecules, social scientists use a similar approach to identify patterns, laws, and principles of society and human behaviour.

The pioneers of polling envisioned using surveys to bring scientific rigour to the study of public opinion and in doing so, to strengthen democracy.

While sometimes described as an outgrowth of ‘science envy,’ it is more fairly the result of our long term collective reverence for a particular form of the scientific method and how it has permeated almost every aspect of our lives.

The pioneers of polling envisioned using surveys to bring scientific rigour to the study of public opinion and in doing so, to strengthen democracy. What they could not have imagined was how fully society would embrace polling as the primary means by which we understand opinion or how the application of scientific methods to the study of human behaviour would impede our ability to make sense of each other and our political world.

Variables, correlation and causality

In election polling, the goal is to explain a particular human behaviour – voting. Crudely, this is the dependent variable. To do so, election pollsters turn to demographic variables such as gender, race, income, education, religion, and so on. Their goal is to find variables that correlate with the vote and use those to draw inferences.

This helps explain why demographics have become the primary lens through which we interpret and understand elections today. The problem is that while descriptive, demographic variables tend not to be explanatory.

 The problem is that while descriptive, demographic variables tend not to be explanatory.

In the run up to the 2024 U.S. election we were told that this was going to be a ‘boys versus girls election,’ due to the fact that one of the major party candidates was female and the other had taken credit for appointing the Justices who overturned Roe v. Wade.

However, this framing is not only reductive and mechanistic, it assumes gender has a level of explanatory power it lacks. If it was explanatory, how could we explain the fact that two women, such as Megyn Kelly and Rachel Maddow, supported such different candidates? Or how could we explain the fact that Donald Trump won white women from the suburbs by seven points?

Variables, like basic demographics, seldom predict belief, opinion, or action, they cannot provide the sort of deep explanations commonly attributed to them.

How do pollsters and social scientists explain these inconsistencies? They demur to probabilities and tendencies, echoing a lesson any introductory student of social science learns, ‘correlation is not causation.’ When dealing with human beings, as opposed to inanimate objects, it is not possible to speak in terms of causality, only probabilities.

Variabilizing is the equivalent of ‘judging a book by its cover,’ in both cases it is not particularly useful. Variables, like basic demographics, seldom predict belief, opinion, or action, they cannot provide the sort of deep explanations commonly attributed to them.

Probability and the neglect of context

Reducing humans to variables is a form of demographic stereotyping. In doing so, we also strip people of agency and ignore the cultural and historical context of their experience.

After the election a common question asked by pundits, was: how did a candidate who promised to deport 20 million immigrants get 45% of the Latino vote?

This question only make sense when you buy into the naturalist assumption that humans can be reduced to variables and these variables are explanatory.

If, however, we consider human behaviour not from the perspective of one ‘explanatory variable,’ but rather in the context of their history, culture, and experiences more broadly, we are likely to be far less perplexed by their actions and views.

In AfterVirtue, Alasdair MacIntyre used the example of a man gardening to illustrate the fact that even the most basic human action can be explained and understood in a variety of ways. A man tending to the soil in his back yard, MacIntyre wrote, could “be correctly described as digging, gardening, taking exercise, preparing for winter, or pleasing his wife.”

The Double H Effect 

A key difference between the natural and social sciences is that the latter is subject to what Anthony Giddens calls the “double hermeneutic” problem.

The natural sciences are ‘single hermeneutic’ disciplines because natural scientists can safely assume their theories and descriptions will not influence or change the phenomena under study.

if polls consistently show that most Americans rate the economy as the most important issue, this may lead a voter to follow suit.

In contrast, the social sciences are ‘double hermeneutic’ to the extent that their observations and theories can and often do shape the behaviours and views of the very thing they are analysing.

During election cycles, we regularly see the ‘double h’ effect in action.

In the months leading up to an election people are inundated with polling data, which they often take as authoritative. We tend to forget that this research is part of the political process and that there’s a complex relationship between how the public interprets this data and how they act.

If, for example, surveys show that a voters’ preferred candidate is leading by a large margin, she might decide there’s no reason to take time off work to vote. Likewise, if polls consistently show that most Americans rate the economy as the most important issue, this may lead a voter to follow suit.

This is why, according to a new book on the 2024 UK election, Tory campaign director Isaac Levido and Keir Starmer’s chief of staff Morgan McSweeney agree that there is a case to be made for a polling blackout in the run-up to election day.

The power and commodification of polls

The scientific revolution has imbued naturalism with unmatched technocratic power and authority; and for good reason. Being backed by a ‘science of society’ remains the best way for an expert to position herself as someone who has the knowledge, expertise, and tools to make rational decisions and to lead public discussions about key issues.

During elections, the media rely on these experts to interpret what voters are thinking and how they might act. This reliance isn’t solely due to a belief in the methodology or their expertise, it is also because this type of knowledge is easily commodified. Moreover, polling is a skill that can be marketed and one that fits neatly into the horse-race aspects of election coverage.

polling is a skill that can be marketed and one that fits neatly into the horse-race aspects of election coverage.

Detailed personal narratives may offer deeper understanding, but the costs of acquiring this type of information are high, not only in terms of time and money, but training, as well. Moreover, these types of narratives are not as easily packaged for mass consumption or the type of gamification so popular in the media today. Would the so-called ‘chart-bros’ (Steve Kornacki, Bill Hemmer, and John King) whose big-wall-map-skills have become a staple of election coverage in the U.S. be nearly as popular if they were tasked with digging into and making digestible complex personal narratives?

In the modern era, election polling has become so ubiquitous it has all but obscured other ways of understanding mass opinion. A recent study by Kathy Frankovic et al found that 85% of the countries examined (157) used polls to understand public opinion. A large percentage of the countries studied have already adopted pre-election silence periods of the kind Levido and McSweeney endorsed.

In addition to these types of policy prescriptions, we also need to balance our reliance on surveys with more narrative, interpretivist, and qualitative research; approaches that are better able to capture the rich tapestry of opinion while respecting the diversity and nuances of the lived experience of the people under examination.

---

The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.

Image Credit: Pressmaster on Shutterstock.

---