Lee Elliot Major and Andy Eyles write that when it comes to education reform, it is not so much what you do that counts, but how you do it. Policies need to be implemented well to make a difference and delivery needs to be consistent.
Variation, variation, variation. These should be the words resonating in the minds of Nadhim Zahawi and his new Ministerial team for Education as they scramble to get up to speed with Whitehall’s battles over the future funding for England’s schools. Forget shiny new schemes or ideas: education’s biggest challenge is ensuring consistency in what is being done across the school system, within each school, and indeed within each classroom. That’s the lesson from education research – whether we’re talking about teaching, tutoring, extending school time or deploying funds for poorer pupils.
It’s refreshing that research has featured prominently in the opening salvos of this government spending review. This saw former education catch-up tsar Sir Kevan Collins resign in the summer. Treasury officials were unconvinced by the evidence behind his proposals for an extra £15 billion for schools – including a bid to extend classroom school time by an extra 30 minutes a day.
Ministers and civil servants yearn for absolute certainties. But researchers can only offer best bets, good hunches, and lots of ifs and buts. We only know so much, and classrooms are bundles of complex human interactions. You need to cast your net wide to capture the most reliable studies. Evidence tends to be mixed and mostly correlational rather than causal. This is noted in this helpful summary, which discusses how supported learning and a lengthening of the school day might help stem learning losses caused by school closures.
Our own research shows that millions of pupils missed over half of their classroom days during the pandemic and suffered lost learning that threatens to scar a whole generation. Several other studies have confirmed stark gaps in test scores between children from poorer backgrounds and their more privileged counterparts in the wake of the pandemic. We believe these losses are important to document. But what is the best approach to help pupils recover over the coming years?
One obvious suggestion is to extend school time. This Danish study is one of the most robust large-scale, cluster-randomised trials investigating a causal link between more instruction time and higher rates of student learning. The trial involved 90 schools and 1,931 fourth-grade students who received three hours extra teaching in reading, writing, and literature a week for 16 weeks. (The cost was approximately $182 per student.) What makes these findings particularly interesting is that the extra instruction time was impactful (raising reading or comprehension scores by 0.15 standard deviations) regardless of whether teachers were free over how and what they taught, or whether they were told to follow a specific set of guidance under an evidence-based programme.
Another profitable approach might be small group instruction or mentoring programmes for children most affected by learning losses. This study compares outcomes between pupils in Germany who were randomly selected to receive one-to-one mentorship from a university student and those who were not. The findings suggest that across a wide range of outcomes that are indicative of later life success – such as math grades and social skills – mentorship improves outcomes for disadvantaged pupils. Interestingly, researchers have looked at how mentoring programmes can help stem learning losses for pupils during lockdown. The results from a recent randomised trial in Italian middle schools show that disadvantaged pupils who were randomly selected for 3-6 hours of free online tutoring, again from university students, did substantially better than their peers who were not given the same opportunity.
High quality studies such as above are particularly useful in that they control for cofounders in a way that, say, studies of cross-country correlations between instruction time and outcomes cannot. But, as this paper highlights, even high quality randomised trials like these can only tell us what has worked (or not worked) in a particular context. What works for the small sample of pupils in Denmark, Germany, and Italy may not work for the large number of pupils in the UK whose learning has been severely disrupted by COVID-19. Policymakers want to know what benefits extended time will have for pupils in schools in range of contexts across England.
Rather than rely on one study, we can also look to summaries of research. The latest synthesis in the Education Endowment Foundation’s Teaching and Learning Toolkit suggests extending school ‘may not be cost-effective for schools to implement’. Before and after school programmes need a clear structure, and well-qualified and well-trained staff to generate academic benefits. Schools also need to consider the workload and wellbeing of their staff. The synthesis is based on 74 studies over a number of years, mostly in the USA, once again calling into question the transferability of the findings.
When one of us commissioned and co-authored the original toolkit ten years ago, it came to similar conclusions. Now, as then, there are several nuances beneath the simplistic headlines. It’s not so much what you do, but how you do it that counts. Things need to be implemented well to make a difference. Delivery needs to be consistent. To understand evidence-informed education is to know this core lesson. Many argue that we should improve teaching in the current school day, before extending it. Others point out that school days already vary in time across the country; what we need is to level up so that all pupils benefit from the same school time.
For the very same reasons, we have critiqued the government’s flagship social mobility policies. Providing extra tutoring and pupil premium funds for pupils are sensible things for policymakers to prioritise; but they will only be effective if they are consistently delivered (and targeted at the poorest pupils). This is the difficult, unsexy slog of education reform, the challenge for any Education Minister if they are to make a genuine difference.
_____________________
Lee Elliot Major is Professor of Social Mobility at the University of Exeter and an Associate of the Centre for Economic Performance at the London School of Economics.
Andrew Eyles is a research economist in the Education and Skills programme at the Centre for Economic Performance, LSE. He is also a PhD (Economics) student at University College London.
Photo by Laura Rivera on Unsplash.
