May 28 2015

Assessment practices in mathematics

Leave a comment

As academic staff at LSE continue to develop their assessment and feedback statements, today’s post offers a round-up of resources that may be of interest to those teaching and convening undergraduate mathematics courses. They cover a wide spectrum of thinking on assessment and a variety of practices that are distinguished by contrasts in:

  • timing – from one-off summative exams through to continuous assessment policies;
  • type – from written examinations through presentations to purely oral exams;
  • implementation – from manual, paper based testing to computer assisted assessment;
  • the nature of the task – from individual dissertations to group based project work;
  • the nature of the assessors – from traditional exam markers through to peer to peer.

The resources are provided not to dictate how mathematics should be assessed, but rather to promote and inform discussion on the topic.

Mapping University Mathematics Assessment Practices Project (MU-MAP)

The MU-MAP Project (MU-MAP, 2011) undertook a broad survey of assessment practices in UK higher education institutions to support the ongoing discussions on the state and future of mathematics assessment in the UK.

Unsurprisingly, closed-book examinations were observed to dominate the assessment landscape, and there seems little evidence to suggest that such assessments are falling out of favour with either academics or students. However, a growing presence of alternative forms of assessment was also noted (primarily within statistics, and the less ‘pure’ and more ‘applied’ applications such as ‘The history of mathematics’ and ‘Business and finance mathematics’). Here, several motivating factors were identified, of which two key ones were:

  1. Growing numbers of students in quantitative courses were asking institutions to consider alternatives to traditional exams, with technology increasingly being seen as a possible solution to many of the logistical challenges those alternatives raised.
  2. The changing market demands for quantitative skills (particularly in the graduate employment market) were seen as contributing to the pressure to change and re-align assessment practices.

Concerns about reliability and validity tended to restrict or constrain radical changes in contemporary practices, with innovations typically seen in individual courses (though usually with the full support of departments).

The results of the survey, and the project’s final report, can be found online at the MU-MAP Project website. While the project, and its various funding opportunities, are now closed, the site remains as a valuable resource for educators in undergraduate mathematics. In particular, while there is much literature promoting and detailing alternative assessments both within the quantitative disciplines and outside of them, the project observed that there was a paucity of empirical research available to teachers to inform and help their choice of assessments. Thus, as well as an assessment literature library, the project has also produced a book, Mapping University Mathematics Assessment Practices (Iannone and Simpson, 2012), that contains 21 case studies describing particular forms of assessment practices and including specific examples of how they have been implemented and reviewed.

Supporting good practice in assessment in mathematics, statistics and operational research

The Higher Education Authority’s Mathematics, Statistics and Operational Research (MSOR) Network has produced a variety of useful resources for both teachers and students. As part of an occasional series of briefings, it has released Supporting Good Practice in Assessment in Mathematics, Statistics and Operational Research Briefings (Challis et al., 2004) which discusses assessment and what constitutes good practice in the MSOR disciplines. While there are a handful of case studies presented, and many additional examples of assessment practices, this document may be more theoretical than the MU-MAP resources listed above. However, it is supported by a wealth of references on assessment in higher education.

Assessment practices in US undergraduate mathematics courses

The final resource is the Mathematical Association of America’s Assessment Practices in Undergraduate Mathematics (Gold et al., 1999), which contains over 60 contributions on assessment practices in a wide variety of mathematics courses in the US. Rather than theoretical discussions on assessment, these articles cover particular practices that have been implemented by their contributors (sometimes just as they have begun to implement them!). Though somewhat older than the two resources listed above, it nonetheless offers some valuable insights on topics ranging from assessment of degree programmes, through various forms of group and individual assessment both inside and outside the classroom, to the assessment of teaching itself.

A companion volume, Supporting Assessment in Undergraduate Mathematics (Steen, 2006), contains 26 case studies of assessment activities, from the development of strategy to innovative practice, in mathematics departments across the US.

References

Challis N., Houston, K. and Stirling, D. (2004), Supporting Good Practice in Assessment in Mathematics, Statistics and Operational Research, MSOR Network, University of Birmingham

Gold, B., Keith, S. and Marion, W.A. (1999), Assessment Practices in Undergraduate Mathematics, Washington DC: Mathematical Association of America

Iannone, P. and Simpson, A. (eds) (2012), Mapping University Mathematics Assessment Practices, Durham University and University of East Anglia

MU-MAP Project (2011), Mapping University Mathematics Assessment Practices, Durham University and University of East Anglia

Steen, L.A. (ed.) (2006), Supporting Assessment in Undergraduate Mathematics, Washington DC: Mathematical Association of America

With thanks to Mark Baltovic in LSE’s Teaching and Learning Centre for contributing this post.

 

Print Friendly
Share
Posted by: Posted on by Managing Editor Tagged with: ,

May 25 2015

Resource of the week

Leave a comment

As departments across LSE are engaged in reviewing and defining their assessment practices, today’s resource highlights a pair of interesting papers that consider the perspectives of students in undergraduate mathematics’ assessment. In Students’ perceptions of assessment in undergraduate mathematics (Research in Mathematics Education, vol. 15, no. 1, pp. 17-33, 2013) , authors Paola Iannone and Adrian Simpson look at how students perceive a variety of different assessment types, with a particular view on how those assessment types test for both memory and understanding. Exam types considered included both closed-book and open-book written exams, as well as multiple choice and oral exams. Additional assessment types considered were project coursework, project presentations, dissertations and weekly examples sheets. The authors’ results are in stark contrast to much of the literature in the soft disciplines and applied disciplines. When ranking assessment types as tests of memory, closed-book exams were rated significantly higher than the others, with multiple choice and oral exams a close second. When ranking assessment types as tests of understanding, the results were more surprising: multiple choice exams were seen as the least effective, while oral exams were seen as the most effective, narrowly beating closed-book exams.

In a subsequent paper, Students’ preferences in undergraduate mathematics assessment (Studies in Higher Education 2014, pp. 1-22), the same authors investigate the assessment preferences of students. Again, contrary to their peers from other disciplines, students validated the dominance of closed-book written exams (coupled with weekly example sheets) as a staple form of assessment, with the researchers unable to ascribe this to any inherent conservatism on the students’ part. Moreover, the results suggest that not only do maths students see such exams as better suited to differentiating ability levels of students, but also that they prefer such forms of assessment over ones that can award high marks to less able students (for example, multiple choice exams). The latter result might suggest that maths undergraduates typify mastery or performance goals, and prefer assessment techniques that most accurately gauge their achievements against such goals. However, while students preferred such assessment mechanisms over all others, most expressed a desire to see a more balanced portfolio of assessment techniques, with a surprising number viewing oral exams as a fair and accurate gauge of understanding (something they valued over mere memorisation).

These are just the highlights of two papers that provide an interesting glimpse into the perspectives of undergraduate mathematics students, and are recommended readings for those interested in assessment in quantitative fields.

Print Friendly
Share
Posted by: Posted on by Managing Editor Tagged with: ,

May 21 2015

Educating in the discipline

Leave a comment

Monday’s resource focused on a strand of educational research literature that explores the ontological and epistemological bases of academic disciplines and the implications these have for the way we educate our students. Academics on LSE’s Postgraduate Certificate in Higher Education course are asked to draw on this literature to reflect on the fundamental nature of their discipline(s) and to consider how this affects the way they approach teaching and the instilling of disciplinary ways of thinking, seeing and practising in their students. Here we republish excerpts from some of their writings.

Andrea Ascani from the Department of Geography and Environment reflects on economic geography

Economic Geography is a complex and multidisciplinary field of study at the crossroads between different disciplines, mainly Economics and Geography, but not only (Crang, 1994). For this reason, being a social scientist in such a field requires an eclectic attitude towards research practice and teaching, which should be aimed at encompassing and integrating different perspectives on the object of study, attempting to prize the complementary aspects of diverse approaches while bearing in mind the weaknesses and puzzles of each of them. In this respect, the field of Economic Geography is highly specific in terms of the knowledge bases that the researcher, the teacher and the student need to reconcile when carrying out their work. Such complexity should be interpreted as a strength of the discipline. It provides the opportunity to adopt different angles of study and to exploit the synergies between alternative, and sometimes competing, approaches, methodologies and implications. At a more general level, the multidisciplinary nature of Economic Geography represents its own value added, offering a unique pair of lenses through which the researcher and the student are able to interpret economic dynamics with strong spatial implications, such as international trade patterns, and to develop sound critical conceptual tools to read and assess real world phenomena.

Given the complexity of the discipline, the Economic Geographer has to wield very diverse conceptual and methodological tools, some of which are very specific to the field or to the particular area of study on which one is focussing. In terms of knowledge bases, the economic should incorporate the technical, abstract and orthodox knowledge associated with the Economics discipline with the wider, concrete and more heterodox knowledge base associated with a discipline such as Geography. This requires not only a deep understanding of the advantages and disadvantages of these approaches, but most of all an acknowledgement that the integration of the two offers a unique mix for the analysis and the study of complex social and economic processes, which allows one to overcome the puzzles related to each individual approach.

Sarah Cefai from the Department of the Media and Communications reflects on the field of media and communications

Media and communications is a contested social and political terrain and as such different theoretical approaches constitute different social and political agendas. Given the nature of the discipline, as one that engages its subjects (here, students and teachers) in the field of power it analyses, students are themselves implicated in critique. The key disciplinary attitude that is fostered by media and communications is self-reflexivity. Modern subjectivity is characteristically self-reflexive—we live in a period known as reflexive modernity (Giddens, 1991). Self-reflexivity requires the ability to think through the relationship between one’s own thinking and feeling and one’s ideas about the wider world. It also requires the ability to reflect on behaviour and to imagine how one’s behaviour or way of thinking can be interpreted from outside of one’s own position. Self-reflexivity is an important aspect of the disciplinary attitude as it is an ethical response to the fact that knowledge is perspectival, embodied and constitutive of relations of power. Much of the curriculum in media and communications propagates a normative sociological view that is ideally accompanied by a self-reflexive understanding of the types of power relations that such a view mobilises. The implications of this for teaching are that students are encouraged to reflect on the historical, cultural and geographical influences on their own styles of reasoning, the development of their intellectual interests, and the type of research they might want to pursue (in their assessments).

Jonathan Birch from the Department of Philosophy writes about threshold concepts in his discipline

In many debates, an understanding of the opposing positions in the existing literature—and of the points of disagreement between them—requires a prior understanding of crucial specialist concepts. Philosophers often construct novel concepts in order to draw attention to subtle distinctions, and these concepts are completely alien to newcomers.

Meyer and Land’s (2003) notion of a ‘threshold concept’ is relevant in this context. They characterize a threshold concept as ‘akin to a portal, opening up a new and previously inaccessible way of thinking about something’ . They propose that grasping the threshold concepts of a discipline produces a change in the student’s perspective on the subject that is transformative, irreversible, integrative, bounded and troublesome (see Meyer and Land 2003 for further discussion of each of these features).

Philosophy is a rich source of plausible instances of this phenomenon. One example, suggested by Booth (2006), is the concept of representation in the philosophy of mind. Many debates in this area take it for granted that the mind represents the world. The word ‘representation’ arises in various ordinary contexts (we might say, for example, that a painting or photograph represents its subject) but in philosophy the term is meant in a more abstract, more abstruse sense, and it is not intended to imply that the mind literally contains pictures of external phenomena. Sometimes philosophers use a jargon term—intentionality—to describe the mysterious sort of representation characteristic of mental states.

Students typically come to these debates as competent users of the ordinary concept of representation, but the concept as it is used in the philosophy of mind will be foreign to them. For this reason, courses and textbooks in philosophy of mind often begin with a discussion of mental representation, introducing the concept by way of analogy with pictorial representation (eg Crane, 2003). Once they grasp this key concept, students can comprehend a wide range of further debates about the nature and architecture of mental representation. In this sense, the concept is integrative and transformative. To the extent that it also makes room for a gap between appearance and reality, opening the door to radical scepticism, it is also ‘troublesome’, and a source of counterintuitive ‘troublesome knowledge’ (or, more accurately, troublesome scepticism).

Concepts with similar features are commonplace in philosophy teaching. Other plausible examples include computation and innateness in the philosophy of cognitive science, reference in the philosophy of language, and grounding and supervenience in metaphysics. In all these cases (with the possible exception of supervenience), students are likely to bring ordinary, pre-theoretical understandings of the concepts to the course, and part of the role of the teacher is to transform these understandings into the more technical and abstract conceptions they need in order to make sense of contemporary philosophical debates. This will often involve trying to uproot the preconceptions students have (eg that a computer must be silicon-based), but these preconceptions can sometimes facilitate useful analogies, as in the case of representation.

Federico Picinali from the Department of Law considers the interdisciplinary nature of evidence law

In the 1980s William Twining – an eminent scholar of evidence law – published an essay that was soon to become a classic in the field. The essay was titled Taking facts seriously and argued – among other theses – that the study and the teaching of evidence law had to be firmly rooted in the knowledge of other disciplines and phenomenona, namely, logic, epistemology and folk psychology (Twining, 2006). This multifaceted knowledge, Twining convincingly argued, is necessary to recognise the role that evidence, proof and fact-finding play in legal practice. Without it, scholars and students can only attain a superficial understanding of the law of evidence, as they are unable fully to appreciate the problems that this branch of law aims to solve.

Nowadays it is rare to encounter an evidence law scholar who is not familiar with the ‘Gettier problem’ or the ‘raven paradox’; who has never heard of the ‘reference class problem’ or has never seen the formula of ‘Bayes’ theorem’. Needless to say, these are not among the traditional topics of the evidence law scholarship: they have been imported or inherited from other fields (ie epistemology and probability theory). Similarly, it is now rare to encounter an evidence law scholar who is not familiar with empirical studies on the triggers of confessions and on the reliability of eyewitness evidence. Again, these works have first burgeoned outside the evidence law scholarship, in the fields of psychology and cognitive science. And so on … To be sure, the interdisciplinary nature of the evidence law scholarship and teaching is a complex phenomenon.

My first encounters with evidence law, as well as my research and teaching in the field ever since, have been characterized by a marked interdisciplinary approach. This approach is best expressed through the thesis that the most important ‘threshold concepts’ in evidence law are not legal concepts (Meyer and Land, 2003). ‘Relevance’, ‘probative value’ and, possibly, ‘fairness’ lie at the basis of evidence law. They are the key to understanding its framework and dynamics: without mastering them ‘the learner cannot progress’ (Meyer and Land, 2003) in any significant way. And yet, they are epistemological or moral concepts, prior to being legal ones. This reductive thesis has informed my scholarship and my teaching, to the extent that some of my works and lectures may strike the reader and the student as having no discernible legal implication. My greatest challenge is to convince them of the opposite.

References

Booth, J. (2006), ‘On the mastery of philosophical concepts: Socratic discourse and the unexpected “affect”‘, in Meyer J. and Land R. (eds), Overcoming Barriers to Student Understanding: Threshold Concepts and Troublesome Knowledge, London: Routledge, pp. 173-181.

Crane, T. (2003), The Mechanical Mind (2nd edn), London: Routledge.

Crang P. (1994), ‘Teaching Economic Geography – Some thought on curriculum content’, Journal of Geography in Higher Education, vol. 18, pp. 106-113.

Meyer, J. and Land, R. (2003), ‘Threshold concepts and troublesome knowledge: linkages to ways of thinking and practising within the disciplines’, ETL project, Occasional Report 4, May 2003.

Twining, W. (2006), ‘Taking facts seriously’, reprinted in Twining, W., Rethinking evidence: exploratory essays, Cambridge: CUP.

With many thanks to Dr Claire Gordon, convenor of LSE’s Postgraduate Certificate in Higher Education course for academic staff, for contributing this post, and to the course participants whose writings appear here.

Print Friendly
Share
Posted by: Posted on by Managing Editor Tagged with: , ,

May 18 2015

Resource of the week

Leave a comment

The links between disciplinary ways of thinking and seeing, and the implications these have for the way we educate our students within their academic disciplines, have been a subject of increasing interest among educational researchers in recent decades. In 2005 Lee Shulman’s Signature pedagogies in the professions (Daedalus, vol. 134, no. 3, pp. 52-59) put forward the notion of ‘signature pedagogies’ which he developed in the context of learning for the professions and defined as ‘types of teaching that organize the fundamental ways in which future practitioners are educated for their new professions’. In Shulman’s view these pedagogies combined cognitive, practical and moral dimensions. Building on this work, Dai Hounsell and Charles Anderson embarked on a study of academic disciplinary learning. Using data from course settings in history and biology their Ways of thinking and practising in biology and history: disciplinary aspects of teaching and learning environments (paper presented at the Higher Education Colloquium, Centre for Teaching, Learning and Assessment, Teaching and Learning within the Disciplines, University of Edinburgh, 2005) argued that ‘students learn ways of thinking and practising characteristic of, and particular to, each of these subject areas. These ways of thinking and practising [are] not confined to knowledge and understanding, but could also take in subject specific skills and know-how, an evolving familiarity with the values and conventions governing scholarly communication within the relevant disciplinary and professional community.’

Drawing on this body of literature participants in LSE’s PGCertHE Career Track programme are invited to ‘decode their disciplines’, to think about the specificity of their discipline in terms of particular knowledge bases, skills and attitudes that lie at the heart of their disciplines (or sub-disciplines, multi-multi-disciplines) and to reflect on the implications this may have for their approach to teaching and supporting student learning. We will be sharing some of their responses to this piece of work in our longer post later this week.

 

 

Print Friendly
Share
Posted by: Posted on by Managing Editor Tagged with: , ,

May 14 2015

Developing doctoral study in LSE’s Department of Social Psychology

Leave a comment

Building on this week’s resource about the PhD supervisor-supervisee relationship, today’s post provides an illustration of how that relationship can be strengthened at the departmental level. Among several developments to its doctoral programme in recent years, LSE’s Department of Social Psychology has had particular success with a series of research seminars redesigned by its current doctoral programme director Professor Saadi Lahlou.

In their new format, the research seminars have several objectives:

  • to develop a greater sense of community both amongst PhD students themselves and between them and academic colleagues
  • to promote the acquisition of a number of transferable skills that would prove useful whether they went for a job within or outside academia
  • to encourage PhD students to complete their thesis on time.

They are held weekly during term times, last two hours each and follow one of two formats: in some weeks, two students present their work in front of peers and academic staff, while in other weeks a member of staff gives a ‘lecture’ about a specific skill or technique related to doctoral studies, that is followed by an informal discussion during which the academic colleague presenting shares his/her personal experience.

Student presentations follow a set format. All students are expected to present their work at least once a year to their colleagues, with the help of a PowerPoint presentation, and to conform to a strict agenda in terms of time and format. Each of the presentations is chaired by a student, thus providing them with an opportunity to develop useful skills (eg, presenting the speaker, time keeping, managing question time). Each student presenting is introduced for one minute by the Chair and has 20 minutes (sharp!) to present their results to date. Students are encouraged to provide a paper copy of their presentation to facilitate note-taking by the audience and most of them do so. This is followed by a 35-minute discussion. The idea here is to prepare students for conference presentations and to enable them to receive feedback from their peers and academic staff on their work to date. (Supervisors and co-supervisors are systematically invited to attend their student(s)’ presentations during research seminars and an agenda of the sessions for the current academic year is circulated early in the year, so that colleagues can plan well in advance.) Continue reading

Print Friendly
Share
Posted by: Posted on by Managing Editor Tagged with: ,