The COVID-19 pandemic meant an almost overnight shift to online learning for young people across the world. In new research, Matt Lamb looks at the effectiveness of online versus in-person learning for civic education. Studying the effects of a civic education youth conference’s move online, he finds that learning outcomes for self-efficacy, community consciousness, and civic knowledge were the same online as they were in-person, but that there may have been a negative impact on students’ confidence about their knowledge.
The endurance of any multicultural democracy hinges on the quality of the civic education it provides to its youth. Since youth engagement currently does not match the power of its potential, social scientists should be concerned with the quality of civics education youth are receiving. While the increasing popularity of online learning was apparent prior to 2020, the COVID-19 pandemic forced almost all in-person functions of schools, colleges, and universities to shut down and be hastily replaced with online learning. While many educators raised alarm bells in the aftermath of the pandemic regarding decreased math and reading scores due to the conversion to fully online education, what of civic education? Unlike many other disciplines, civic education necessitates instilling feelings of community mindedness. This can be difficult to copy in an online setting when people can only communicate via video conferencing software or a messenger app. What might the increasing popularity of online learning mean for civic education?
How civic education is taught has been under-studied
The literature on the effectiveness of various forms of civic education pedagogy has been growing but is still limited in its usefulness. Previous research comparing online and in-person instruction mostly consists of impromptu assessments of student outcomes when a course ends, comparisons of different students in different settings, and often lacks pre- and post-testing of outcomes. While political scientists strive for rigor in how they study other parts of the discipline, research on pedagogy in civic education has not been well developed, and is often lacking important aspects of research design, such as control groups.
In a new work, I offer a more rigorous approach to examining the effect of online pedagogy. My study took advantage of a natural experiment by examining a civic education youth conference organized by civic mobilization organizations Mi Familia Vota (MFV) and OCA-Houston. While normally held in-person at Rice University in Houston, Texas, the conference was held online in the summer of 2020. However, the organizers still utilized a similar curriculum and the same pre and post-test assessments to measure learning outcomes of self-efficacy, community consciousness, and civic knowledge.
I find that outcomes in self-efficacy, community consciousness, and civic knowledge between online and in-person modes of instruction were comparable and statistically indistinguishable from each other. However, there is evidence that in-person students are more confident about their knowledge.
COVID-19 was an opportunity to look more closely at pedagogy.
The Youth Advocacy Summit (YAS), organized by MFV takes place over of several days and focuses on civic engagement workshops and educational programming for high school and college aged students who are recruited through government classes and after school activities. The students take part in community building exercises, hear from local community leaders, and learn important information regarding the political and civic process. In the summer of 2020, all conference activities were held online. Because MFV utilized similar programming, and used the exact same pre and post-assessment tools as they did in 2019, it provided the perfect natural experiment to measure the difference in outcomes between in-person and online civic education.
We used a Google survey in 2019 and 2020, which we sent to students before and after their participation in YAS. The survey questions were designed to measure three dimensions of civic education: community consciousness, self-efficacy, and civic knowledge. The civic knowledge questions were further categorized into questions which had dimensions that are recognized measures of the effectiveness of civic education outcomes.
Photo by Headway on Unsplash
Moving to online education had limited effects
The experience of going through the YAS conference itself had positive effects on all dimensions of civic education. The greatest increase in scores for both conferences were in the overall civic knowledge and knowledge confidence scores. In 2019, overall knowledge scores increased by 21 percent after YAS. In 2020, knowledge scores increased by 14.6 percent after the YAS. The knowledge confidence score increased by 28.2 percent in 2019 and by 17 percent in 2020. The difference in means pre- and post- was statistically significant in both years. The verifiable knowledge score increases, however, were much smaller. In 2019, the verifiable knowledge score increased by 6.9 percent. In 2020, it increased by 2.7 percent. However, it is important to note that, due to the small number of respondents, this small increase represents a change in one response. Figure 1 shows the mean scores.
Figure 1 – Improvement in Civic Knowledge
As Figure 2 shows, in 2019, community consciousness scores increased by 7 percent after YAS. In 2020, community consciousness increased by 6.2 percent after YAS. The difference in means pre- and post- was statistically significant in both years. Though there were marked improvements in community consciousness scores, the increases were not that different between 2019 and 2020.
Figure 2- Improvement in Community Consciousness
We can see in Figure 3 that t in 2019, self-efficacy scores increased by 6.6 percent after YAS, and that in 2020, they increased by 6.4 percent after YAS. The difference in means pre- and post- was statistically significant. However, once again the increases in 2019 and 2020 are not distinguishable from one another. For similar reasons as those about community consciousness, it could be that these results come from some unseen characteristics amongst those participants in the sample that make the mode of instruction irrelevant to increases in self-efficacy.
Figure 3 – Improvements in Self-Efficacy
My modelling suggested that improvements in all outcomes were smaller in the online condition, however, none of the difference in means between the online and in-person conditions are statistically significant. What is suggestive, though, is that the substantive effect of online pedagogy for civic knowledge confidence is over 10 times larger than for self-efficacy and around one and a half times larger than for community consciousness. This suggests that though the differences between the mean scores in the in-person and online conditions were not statistically different, the negative impact of online learning was much bigger in its effect on knowledge confidence.
More research so that online education can be as effective as possible
My study yielded mixed results; however, this should only encourage further study of pedagogy in civic education and political science. Civic educators must examine classroom tools and pedagogical style with the same empirical rigor that researchers approach other major questions in the social sciences. Further analysis of pedagogical techniques and modes of instruction must incorporate more pre- and post-testing, control and treatment groups, times-series designs, and larger-scale designs to determine the most effective learning tools at our disposal. As online modes of education become more popular, more must be done especially to examine how social science and civics instructors can use online tools to maximum effect.
- This article is based on, ‘Logging in to Learn: The Effects of Online Civic Education Pedagogy on a Latinx and AAPI Civic Engagement Youth Conference’, in PS: Political Science & Politics
- Please read our comments policy before commenting.
- Note: This article gives the views of the author, and not the position of USAPP – American Politics and Policy, nor the London School of Economics.
- Shortened URL for this post: https://bit.ly/3HJezmW