LSE - Small Logo
LSE - Small Logo

Blog Admin

May 15th, 2015

Literacy as Numbers: The efficacy, merits and validity of transnational literacy assessment programmes.

2 comments | 1 shares

Estimated reading time: 5 minutes

Blog Admin

May 15th, 2015

Literacy as Numbers: The efficacy, merits and validity of transnational literacy assessment programmes.

2 comments | 1 shares

Estimated reading time: 5 minutes

Debates about the nature of literacy and how to account for the diversity of learning are far from resolved. A new book, Literacy as Numbers, looks at how literacy itself is being reframed around globalized assessment regimes. Camilla Addey delves into how these comparable numbers, now so heavily relied on in national policy, are produced, and how they are shaping our understanding of the meanings and purposes of literacy. 

The OECD’s well-marketed and mediatized PISA – the Programme for International Student Assessment – is the international education assessment programme most people have heard of. But PISA is just the tip of a much bigger phenomenon. With less catchy names, a whole plethora of international programmes measure skills and competencies across the globe. International assessments are increasingly growing in reach and scope, and have seen a steep rise in terms of countries joining in the last decade.

This has sparked the development of more international assessments catering for lower and middle income countries, or for levels of education which have not yet been assessed on the same global scale. Publications discussing international assessments are now flourishing in disciplines as diverse as political sciences, educational sciences, psychometrics and linguistics and this has been recognized as an emerging field of enquiry known as International Assessment Studies (Addey 2014). But how is the phenomenon of international educational assessments transforming education?

Measuring learning outcomes in a comparative way not only creates performance competition among countries, but also processes of governance by ranking and rating (Lehmkuhl 2005) or governance by data comparison (Martens 2007). The horse-race mentality that has grown around international assessment league tables has a naming and shaming effect, which pressures and induces countries to conform and compete through reform. By showing the world those who are leading the competition and those who are falling behind, the comparative data turn in education puts peer pressure on countries to borrow best practices and policies from the league leaders – creating educational pilgrimages (Sahlberg 2011) towards countries like Finland and South Korea and Singapore (which have more than once been the league leaders). But can this comparative data-based knowledge tell us which educational system is the best solution for all? How can international assessments validly assess the diversity of  learning through a set of standardized tests?

literacy as numbersLiteracy as numbers: Researching the politics and practices of international literarcy assessment. Cambridge University Press.

Milestone publications like PISA, Power and Policy (2013) and PISA Under Examination (2011) have examined how international assessments are reframing education globally. The latest book on international assessments published in 2015 by Cambridge University Press, Literacy as Numbers, focuses on such programmes from a lifelong learning perspective, leading the debate to reflect on how the phenomenon is not only reframing education but changing the way we conceive of literacy and numeracy.

Debates about the nature of literacy and how to account for the diversity of everyday practices are far from resolved. In fact, these debates are more fascinating and challenging than ever before with the advent of digital technologies. In Literacy as Numbers, literacy is presented as a contested territory, which is currently being subjected to new forms of codification and institutionalisation. This makes it a particularly important arena for exploring the processes whereby the diversity of everyday experiences and practices gives way to an ordered field of measurement. We can observe how the field of literacy is being re-positioned within the discourse of large-scale assessments, and trace the links that are established with themes such as employability, citizenship and opportunity.

One of the powerful aspects of contemporary quantification of literacy – literacy as numbers – is that the evidence produced through quantification seems to offer certainty and closure on what literacy is and who it is for. The technology of numbers and the comparative data-based knowledge produced through international assessments are creating a globally dominant literacy, a quantifiable commodity, a universally spoken language that has changed the way literacy is understood and enacted in policy and practice.

Although numbers carry values and ideological choices through the conceptualizations and methodologies intrinsic to their production, they are widely regarded as objective, scientific facts, considered impartial, value-free and hard to argue against. Gorur states that numbers are considered by ‘many policy makers today as a neutral and apolitical representation of reality, a weapon against prejudice. The use of scientific evidence has come to be seen as a hallmark of integrity in policy making’ (2011: 90). But Rose reminds us that ‘numbers render invisible and hence incontestable the complex array of judgments and decisions that go into a measurement, a scale, a number (1999: 208).

The project of reframing literacy around globalized assessment regimes of standardised literacy assessment is clearly still in progress. While the testing agencies – acting as ‘centres of calculation’ in the sense described by Latour and Woolgar (1979) – are motivated to present assessment programmes and their methods as unproblematic and routinised procedures, it appears that their black boxes are not yet sealed. In these processes of innovation and expansion, debates are necessarily raised about the efficacy, merits and validity of transnational assessment programmes, about their new institutional architecture and their governance and accountability. The insider approach taken by the contributors of Literacy as Numbers reveals that such debates are happening at all levels of the production and use of the survey findings, though much of the discussion remains inaccessible to the public gaze. Literacy as Numbers opens up the processes of international assessment  and uncovers the complexity and the political agendas hidden by the simplicity and usability of literacy as numbers.

Literacy as Numbers is a co-edited collection of papers, most of which were presented at the Literacy as Numbers International Symposiumwhich took place at the Institute of Education in London in June 2013. It is edited by Mary Hamilton, Bryan Maddox and Camilla Addey, and was published by Cambridge University Press in March 2015. For more information see here.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.

About the Author

Dr Camilla Addey is director of the Laboratory of International Assessment Studies and research associate at the University of East Anglia. She recently completed her PhD on the rationales for participation in international literacy assessments in Mongolia and Laos. Her current research enquires into PISA for Development from a governance perspective in lower and middle income countries. Her research has established International Assessment Studies as a field of enquiry. Dr Addey previously worked at UNESCO in Paris in the Adult Literacy and Non-Formal Education section. She is author of Readers and Non-Readers.

The Laboratory of International Assessment Studies is an inter-disciplinary network and space that brings together academic researchers, testing agencies, policy makers and the end users of international educational assessment data. It supports international debate on the potentials, practices and politics of international assessments. It is presently running an ESRC seminar series on ‘The Potentials, Politics and Practices of International Assessments’ – further information is available here.

 

Print Friendly, PDF & Email

About the author

Blog Admin

Posted In: Big data | Data science | Evidence-based policy | Government | Rankings

2 Comments