Evaluating media literacy initiatives is both extremely important and extremely complex. Gianfranco Polizzi of the University of Liverpool, and Tom Harrison and Shane McLaughlin of the University of Birmingham, explain the findings of their research project that conducted a rigorous evaluation of a UK-based news literacy programme, NewsWise.
For a democracy to function, citizens need to be able to make informed decisions about their communities and the socio-political system. In the UK, amidst growing concerns about the spread of mis/disinformation, the opaque algorithms used by online platforms and the rise of generative AI, many have called for better regulation of the digital environment, which is the rationale behind the Online Safety Act. At the same time, and considering the likely limitations of regulation, many are also aware that it is imperative to equip citizens with media (or digital) literacy – i.e., the skills and knowledge they need to evaluate information and the impact of the internet. A form of media literacy, news literacy refers to the skills and knowledge required to engage with the news, evaluate news stories and understand the broader news ecosystem.
Young children are at the forefront of using digital technologies but also the most vulnerable online. Their engagement with online (mis/dis)information can exacerbate their vulnerability and affect their trust in evidence. This is why it is important that children develop news literacy from a young age. News literacy, furthermore, relies on engagement with news, which we need to promote among younger generations if we are to develop their socio-political awareness. Citizens’ active participation in society is based on multiple factors, including their motivation to take part in their own communities and in broader socio-political processes. Engagement with news is key to gaining an understanding of these processes and of keeping abreast of current events both within communities and in society at large.
To generate new insights for research, practice and policy aimed at promoting news literacy education, we recently conducted a project to evaluate the effectiveness of NewsWise, an award-winning school programme designed to develop 9-11-year-olds’ news literacy in the UK.
Evaluating NewsWise
Developed by the Guardian Foundation in partnership with the National Literacy Trust and the PSHE Association, NewsWise is a primary school programme designed to encourage children to engage with the news, to understand the meaning of terms such as ‘bias’ and ‘fake news’, to learn about different strategies to identify dis/misinformation, to learn about news consumption and production, and to write a news report.
From March 2022 to February 2024, researchers from the University of Birmingham and the University of Liverpool worked together to evaluate the second iteration of NewsWise. As part of this project, which was funded by Nuffield, we were interested in exploring whether NewsWise was effective in increasing 9-11-year-olds’ levels of news literacy, and whether such an increase corresponded to an increase in their civic engagement (understood as community involvement, acts of service and political participation that is appropriate to their age). Since the literature in this area is scarce and has seldom focused on this age group, we also aimed to find better ways to measure both their news literacy and civic engagement.
Key findings
We conducted a randomised control trial with 40 schools located across the four nations of the UK, administering pre- and post-intervention surveys before and after the delivery of NewsWise. In addition, we conducted focus groups with pupils and interviews with teachers in a select number of schools, to explore their views and experiences of the programme. The final report about the evaluation of NewsWise can be accessed here. Our key findings:
1. The instruments that we created and validated are suitable to measure news literacy and civic engagement in children aged 9-11.
These measures include multiple dimensions. The news literacy measure includes 1) attitudes towards news reporting and trustworthiness (comprising a motivation to engage with the news and the belief that this should be honest and balanced), 2) knowledge of news production and consumption, and 3) evaluation skills and strategies (comprising the self-reported ability to evaluate news stories and the actions taken if/when confronted with dis/misinformation). A twelve-item performance test was also created and used to assess 9-11-year-olds’ ability to spot “fake news”. The civic engagement measure includes 1) awareness of socio-political issues, 2) positive attitudes towards community/political life, and 3) participation in community/political activities.
2. 86% of 9–11-year-olds in the UK believe the news should be truthful, but less than half know how to spot fake news and only three in ten are interested in the news. Meanwhile, 90% have positive attitudes towards civic engagement. However, five in ten think they will vote when they grow up, and only 35% talk about politics and social issues with their parents.
These baseline findings from the study show that many children in the UK value the importance of trustworthy news and civic engagement. Not as many, however, are interested in the news or in the opportunity to vote in future elections. Many children, furthermore, find it hard to spot fake news and do not have conversations with their parents about socio-political issues.
3. The NewsWise programme improved 9-11-year-olds’ ability to detect fake news
Children were presented with twelve online news items and asked to select whether each item was real, fake, or whether they did not have enough information to decide. The real items were news stories taken from reputable sources. The fake items consisted of news stories that had been altered from their original version (these included, for example, misspellings, wrong URLs, pictures that bore no relationship to the headlines). The items that could not be assessed were news stories that lacked details that had been redacted (e.g., the URL, the name of the author). Not only did we find that NewsWise was effective in improving children’s ability to spot fake news, but this improvement endured over time.
4. Analysis of the focus group and interview data found many benefits of the NewsWise programme as well as areas for improvement.
Three overarching themes suggest that the programme 1) presents multiple benefits for developing pupils’ news literacy, 2) has the potential to develop pupils’ civic engagement through their engagement with news; and 3) was well received by both pupils and teachers and could be improved for future delivery. Teachers and pupils particularly enjoyed the lessons on how to spot misinformation online and the activity on how to produce a news report. Meanwhile, some teachers thought that a more condensed version of NewsWise, with more differentiation in terms of resources, could work better in the future.
5. A strong positive relationship between news literacy and civic engagement was found, for the first time, among 9-11-year-olds.
We found strong correlations between all dimensions of news literacy and civic engagement among 9-11-year-olds, including, for example, between their awareness of socio-political issues and motivation to follow the news, such awareness and their self-reported ability to evaluate news stories, and their participation in civic life (e.g., through community involvement) and interest in the news. This finding is particularly important, since it shows that children’s news literacy, which enables them to assess the reliability of news content, is foundational to their civic engagement.
Recommendations
The findings above have implications for how to better promote media and news literacy education for young children. Here is what we recommend in terms of future directions for research, practice and policy:
- The new measures created as part of the project are suitable for use by further studies and evaluations seeking to measure the news literacy and/or civic engagement of 9-11-year-old.
- Our findings about children’s engagement with the news and civic life show what dimensions of their news literacy and civic engagement may be less developed and require more attention through formal education.
- Our findings about the effectiveness of NewsWise can be utilised when designing new, and/or making revisions to current, news literacy programmes.
- We found a positive link between 9-11-year-olds’ development of news literacy and their civic engagement. Future studies and evaluations could build on the NewsWise evaluation to explore this link further.
- News literacy education should be promoted in ways that account for its potential to develop not just news literacy but also civic engagement among primary school children.
- News (and media) literacy (and programmes like NewsWise) should be more firmly embedded in the school curriculum.
It is of paramount importance that children develop the skills and knowledge required to navigate the complex news and information environment that digital technologies have contributed to shape. Equipping children with news literacy is an urgent task. Promisingly, the benefits that news literacy has for their civic engagement make this task even more valuable.
This post represents the views of the authors and not the position of the Media@LSE blog, nor of the London School of Economics and Political Science.
