In a previous blog post, we described how transformer AI systems can generate entire student essays at the press of a button. Type in the essay title and some opening words, press ‘submit’ and the program responds with a few hundred words of grammatically correct, plausible-looking text. Add another prompt, such as ‘example’ or ‘conclusion,’ and the transformer will continue to write. The result is an original essay or report that will pass a plagiarism check, because the text is not copied from the web, but generated by an AI program trained on the language of billions of books, articles, blog posts, and web pages.
However, transformer systems have a fatal flaw: they are models of language, not of knowledge. A transformer can write in the style of a student essay, scientific paper, or news report, but it has no deep understanding of how the world works. It, therefore, makes up research studies, invents facts to back up its arguments, and spouts references to fake papers. As the developers of GPT-3, the best-known transformer, put it: ‘large pretrained language models are not grounded in other domains of experience, such as video or real-world physical interaction, and thus lack a large amount of context about the world.’
Students now have access to cheap, easy-to-use AI programs, such as transformers, that generate plausible, yet flawed, essays, reports, and exam answers. This seems like an academic’s worst nightmare. However, this powerful new technology can also benefit education. (See video below.)
If an essay question can be answered by an AI program, albeit with flaws, then maybe it is time to rethink assessment. Transformer technology can be used creatively in the classroom to teach academic writing. It can be a starting point for teaching critical digital literacy, and for collectively exploring the ethics and limits of AI.
If an essay question can be answered by an AI program, then maybe it’s time to rethink assessment.
Teachers could set reflective and contextualised written assignments that cannot be generated by AI. For example, each student could carry out an independent research project, then produce a written report on that specific project, get feedback on the report, then write a critical reflection on the feedback and the issues raised by the project.
Universities could also embrace AI-assisted writing. Students would then be allowed to employ whatever tools they wish, to develop their skills in critical reading, accuracy, argumentation and structure. Assignments where AI is not allowed (say, in an invigilated exam) would also be assessed, as currently, for style, expression, voice and personal reflection.
An imaginative way to teach academic writing could be for the teacher to generate a set of essays with the aid of a transformer AI, then ask students to critique these and write their own, better, versions. Alternatively, students could themselves set exam questions on a topic, command the AI to answer them, then evaluate the answers. Or the teacher could first set an open-ended question, then ask each student to generate AI responses to it. Students, in small groups, could discuss these responses in relation to the marking criteria. Following this, each student could write an integrative essay drawing on the AI answers to address the original question. Students would have the option to experiment with a variety of prompts to elicit different perspectives or arguments.
Transformer AI is a gamechanger
Mike Sharples, co-author of Story Machines, discusses:
- What is a transformer AI program, and why is it a gamechanger?
- Could AI be used to disrupt testing for lower-level learning?
- How could AI help students develop critical digital literacy?
Here’s a brief example, with the prompt or question in bold italics and two alternative answers generated by GPT-3 (verbatim) below.
Question: Compare Marx and Nietzsche in their analysis of the underlying problems of their day in 19th century European society.
Answer 1: There are a few key ways in which Marx and Nietzsche differ in their analysis of 19th century European society. For one, Marx was a materialist who believed that economic forces were the primary drivers of history, while Nietzsche was a idealist who believed that ideas and culture were the primary drivers of history. Additionally, Marx believed that class conflict was the main source of social tension and conflict, while Nietzsche believed that the main source of social tension was the tension between the “herd mentality” and the “individual will.” Finally, Marx believed that the way to solve the problems of society was through revolution and the overthrow of the capitalist system, while Nietzsche believed that the way to solve the problems of society was through the development of a new, higher culture.
Answer 2: Both Marx and Nietzsche were concerned with the problems of 19th century European society, such as the growing divide between the rich and the poor, the increasing number of industrial accidents, and the spread of disease. However, they had different solutions to these problems. Marx believed that the only way to solve these problems was through a revolution that would overthrow the capitalist system. Nietzsche, on the other hand, believed that the only way to solve these problems was through a revaluation of all values, which would lead to a new way of living.
Transformer AI can also become a tool for creative writing. For example, the student writes a first paragraph, the AI continues with the second paragraph, and so on. The AI writing partner helps to maintain a flow of words and may take the story in unexpected directions, to which the student must respond.
Last, but by no means least, students could experiment with transformer systems to explore the ethics and limits of generative AI. How does it feel to interact with an expert wordsmith that has no intrinsic morals and no experience of the world? Is writing with AI tantamount to plagiarism? Is generative AI a threat or a boon to higher education, or perhaps both? Because of the strong impact that AI can have on education, we need to consider what critical technical skills students need to develop, particularly in the humanities and the social sciences.
How does it feel to interact with an expert wordsmith that has no intrinsic morals and no experience of the world?
Transformer AI systems have many more acts to perform. They can translate languages, summarise scientific papers, converse as chatbots, explain computer code, and generate computer programs from written descriptions. To consider them to be solely a threat to education is to misunderstand both their potential and their limitations.
 These suggestions are based on responses by @jennicarr8 to a Twitter discussion on rethinking assessment in an era of generative AI. https://twitter.com/sharplm/status/1534051131978047494
Disclaimer: This post is opinion-based and does not reflect the views of the London School of Economics and Political Science or any of its constituent departments and divisions.