The role of ‘fake news’ in shaping political behaviour has received extensive attention in recent years, with Facebook and other websites undertaking a number of measures to try and address the problem. Drawing on an experimental study during the 2017 German federal election campaign, Bernhard Clemm von Hohenberg illustrates that people rely far more on their pre-existing political attitudes when judging the accuracy of a story on Facebook than they do on the reputation of a news source.
In the digital age, anyone can publish. This has been hailed as a democratisation of news, but it has also raised the question of what sources to trust. The abundance of information available online further invites people to process political news according to their worldview. So how do citizens determine what facts to believe, especially in the heated context of a political campaign?
In an innovative experimental study during the 2017 German election campaign, I tested the impact of two factors on people’s accuracy in judging news stories relevant to the upcoming election: firstly, people’s pre-existing political attitudes, and secondly, the professionalism of the source posting a story. My results imply that people’s pre-existing attitudes play an important role in determining their belief in a story – much more so than the source. This goes across the political spectrum, although the picture varies across different topics.
Observers have already suggested that the success of fake news is due to people having a tendency to accept ideologically comfortable information. But as my research suggests, in our current ecosystem of political communication, this dynamic cannot be easily countered by a source’s reputation – at least for now.
People rely on their attitudes when judging the accuracy of a story
For many politically relevant factual questions in society, there are no clear-cut answers. But new factual claims emerge every day from the news. Do people, given their attitudes, accept one factual answer more readily than another?
To test for this “congruence bias”, I picked a few empirical questions that were little discussed in public, and constructed two stories with contrasting facts for each. I did this for several policy areas that were at the centre of the 2017 campaign: distributive justice, domestic security, migration and European integration.
Take the migration topic: there have been efforts in Germany to integrate refugees into the labour market by giving them traineeship positions. Have these efforts been successful? Real data on this question is limited and inconclusive. I therefore randomly allocated a story to one half of the study’s participants that presented numbers showing that refugees had traineeship completion rates equal to natives. The opposite story reported that the percentage was significantly lower.
Since many people take their information from Facebook – two-thirds of Americans and about one-third in Germany – I presented the news as Facebook posts, and asked my sample whether they believed the reported story was accurate. As Figure 1 shows, it turned out that participants agreeing that “it is good that Germany has taken in many refugees recently” (and similar positions) typically judged the story of successful refugees as being more accurate. The average difference was about three points on a scale from zero to ten for the most pro-migration respondents. Similarly, those with more anti-migration positions judged the opposite report as being more accurate.
Figure 1: Link between attitudes toward refugees and belief in positive/negative stories about refugees
Note: The figure illustrates that those who agreed with the statement “it is good that Germany has taken in many refugees recently” were more inclined to believe a story about refugees being hard-working (and less likely to believe a story about refugees not being hard-working) than those who disagreed with the statement.
Congruence bias also influenced the judgement of stories on redistributive justice. Those with positions very favourable to a stronger welfare state found a pro-welfare report more accurate than an anti-welfare report. However, those opposing stronger welfare did not make a difference. Similarly, only one side fell for the congruence bias when judging the accuracy of a story about EU bureaucracy, namely those with Eurosceptic attitudes. Europhiles treated the reports equally.
It seems that the congruence bias is not always equally strong for opposing political camps. But both liberals and conservatives fall for it one time or another. Only for the topic of domestic security did I observe no impact of a respondent’s attitudes at all on story acceptance. So, attitudes guide accuracy judgements, but not for all topics. What exactly makes a topic susceptible to this bias is an issue that needs further exploration.
The reputation of a source has little impact on judging accuracy
A Facebook post comes with more than the content. Most importantly, a post is attributed to a source. Whether the source is a professional organisation or not should serve as an important credibility cue. To explore this, I made the Facebook posts look like they were published by “Tagesschau”, one of the most well-known outlets employing many professional journalists, or by “Neueste Nachrichten”, a website publishing completely made up information, but with a neutral-sounding name (literally: “Newest News”).
My expectation was that people would judge the same story as being more accurate when attributed to the professional source. But only in the case of stories on domestic security did this make any difference to participants. Even here, the “Tagesschau” source increased perceived accuracy only by less than one point on a scale from zero to ten. This topic happened to be the only one for which no congruence effect was found, which implies that the source only matters when pre-existing attitudes do not.
But perhaps the results differ for people with different general media trust levels? Media mistrust in Western societies is growing, but not all groups are similarly distrusting. Participants who trust the professional mainstream media might be more impressed by the difference between the two sources than those who are suspicious of mainstream media organisations.
I therefore asked participants to rate a few media outlets according to their trustworthiness. Indeed, those with more trust in the mainstream media were more likely to judge a story by a professional source as being more accurate. But these differences were small and mostly negligible. The implication for the whole population remains: for news on Facebook, the source does not have a major impact on people’s accuracy perceptions.
In combination with people’s tendency to believe what they like, this seems to explain a good deal of the success of fake news. Of course, it is hard to say why sources in a Facebook post have little effect. It could be because people see them, but do not know which source to trust, or because they do not properly notice them. After all, Facebook posts visually flatten out differences between news organisations, be they professional or not. However, my study produced further evidence that people’s perceptions of trustworthiness are confused. Figure 2 below shows how participants, on average, rated the media outlets’ trustworthiness (the data are weighted to be representative of the German population on a number of demographic variables).
Figure 2: Average trustworthiness score for German media outlets
Source: Compiled by the author
The five professional outlets included in this question did score highest. But they were closely followed by three scam sites with neutral sounding names, which were active in the market of misinformation at the time. More strikingly, they rated these similarly or even more trustworthy than two partisan, but professional outlets. This is also a warning sign for Facebook’s most recent effort to ask users to rate “trusted sources”.
Observers are right to urge citizens to be conscious of their biases and check sources. But my findings are also bad news for the professional media who rely heavily on social media platforms. It has never been easier for journalistic imposters to appear legitimate, and people do not seem to make a great distinction between sources on average. News organisations must therefore find ways to make their reputation known and visible during news consumption.
Bernhard Clemm von Hohenberg – European University Institute
Bernhard Clemm von Hohenberg is a PhD candidate at the European University Institute Florence, having previously worked at a media startup and as a freelance journalist in Germany. He is currently doing research in political psychology and public opinion, with a special focus on the digital. He tweets @bernhardclemm