In May, the White House’s Make America Healthy Again commission published the Make Our Children Healthy Again report, which was then found to contain references to non-existent and hallucinated sources, strong markers that some content was created by generative artificial intelligence (GenAI) tools. Peter Finn writes that the use of AI tools in reports such as this one cast doubt on the truth of the US Official Record – the material created from the operations of a government. This in turn, he suggests, could undermine the formulation and implementation of evidence-based policies and potentially ultimately undermine US democracy.
Educators like me are becoming increasingly used to encountering writing that may initially seem sound, but it quickly becomes apparent that something is amiss with the text in front of us. Reading further it becomes clear that the text, or at least significant parts of it, are the result of requests (prompts) for some kind of content (a sentence, a paragraph, a table, an image, a reference or source…) put into one of an increasing array of generative artificial intelligence (GenAI) tools.
Sometimes the content will be the product of one GenAI tool, and perhaps even a single prompt. In other cases, parts of the content may have been drafted by a student and then put into a GenAI tool to ‘improve’ it. In another common strategy, students use a combination of different GenAI tools and ultimately do not actually know where any part of an assessment submission is originally from.
Concern over the use of GenAI tools to create texts now no longer appears to be confined to educators. On May 22nd, the White House’s Make America Healthy Again (MAHA) commission – headed by Secretary of Health and Human Services Robert F. Kennedy Jr – released the Make Our Children Healthy Again report. As first detailed by NOTUS, and documented by many other media outlets, the report contained dozens of non-existent references with some likely hallucinations (composites of information related to real references and authors rearranged to create real looking references to ultimately non-existent sources).
As I have explored with others in recent work, such non-existent and hallucinated sources are the hallmarks of content generated by GenAI tools. The White House, however, attributed the presence of non-existent sources to ‘formatting issues’. In an odd coda, a new version of the report was later released to ‘remove non-existent studies’.
The GenAI bigger picture
Unfortunately, the problem of people and organisations relying on GenAI tools as a shortcut to doing the hard work of writing, research, gathering data, and understanding the world is widespread.
In journalism, a reporter for the Wyoming newspaper Cody Enterprise was caught making up quotes last year by CJ Baker, a journalist at the competing Powell Tribune. The quotes had been attributed to, among others, the Governor of Wyoming. More structurally, CNN has reported that, even when used in a transparent manner and with the best of intentions, “experts warn that relying too heavily” on GenAI tools “could wreck the credibility of news organizations and potentially supercharge the spread of misinformation if not kept in close check.” They further explain that:
“[m]edia companies integrating AI in news publishing have […] seen it backfire, resulting in public embarrassments. Tech outlet CNET’s AI-generated articles made embarrassing factual errors. The nation’s largest newspaper chain owner, Gannett, pulled back on an AI experiment reporting on high school sports games after public mockery. Sports Illustrated deleted several articles from its website after they were found to have been published under fake author names.”
In the legal sphere, meanwhile, data consultant and HEC Paris Lecturer, Damien Charlotin, is developing a database that, as of early July, contains 168 “legal decisions in cases where generative AI produced hallucinated content – typically fake citations, but also other types of arguments”. These examples stem from a broad range of jurisdictions including the UK, the US, Israel, and Canada. In the UK, the High Court has had to issue a rebuke to the legal profession ‘after dozens of fake case-law citations were put before the courts that were either completely fictitious or contained made-up passages.’ In extreme cases in academia, meanwhile, Jon Whittle and Stefan Harrer of the Australian national science agency, CSIRO, note that ‘AI may end up writing research papers, which are then [peer] reviewed by another AI.’
“P20250522MR-0456” by The White House is United States government work
In short, the MAHA report was reflective of a much broader and troubling trend. A trend which presents problems for those interested in, among other things, the Official Record, sound government, and democracy.
What GenAI means for the US’ Official Record
As I explored in a recent co-edited volume, The Official Record, the US version of which the MAHA report has now entered, can be broadly conceived of as the sum total of material created as a result of the operations of a government – everything from the statements made by government agencies, policy reports, and the Congressional Record and Federal Register. Given the size and complexity of modern governments, official records are diverse, dynamic, and important for understanding their operations, along with the thinking of those working within, and making and implementing policy for, governments. As such, it is a key (for some, the key) source for understanding events. Yet, the MAHA report, along with the examples I mentioned earlier related to the courts and the Governor of Wyoming, illustrates the precarious nature of the trust that we might put in the Official Record moving forward.
As with academia, sound government and a vibrant democracy depend on at least some agreement on what counts as data and evidence as well as the maintenance of honesty in the creation and maintenance of the Official Record. This holds even if one accepts that the Official Record is an imperfect ledger, is often used as a political tool, that it can be selectively collated and, as importantly, that there will always be significant differences in how it is interpreted and what one should do because of such interpretations. Indeed, such differences in interpretation are to be encouraged, and should hopefully lead to developments in our understanding of the world and (if we are lucky) better public services and (if the jackpot is truly struck) a more engaged citizenry.
However, the MAHA report shows at the least (and if one accepts the pretty unlikely explanation provided by the White House) sloppy referencing that would see undergraduate students docked marks. More likely it represents a willingness to outsource parts of the research and writing processes for a US government report to GenAI tools, and a failure to even bother to check if the references it provided existed, let alone whether they were accurate for the points being made.
If this approach to evidence and the Official Record are replicated in other parts of the US government, and similar examples suggest the MAHA report is not isolated in the use of GenAI, then a mistaken faith placed in such tools will undermine the formulation and implementation of evidence based policies and can be added to the list of factors that may undermine US democracy itself.
- Subscribe to LSE USAPP’s email newsletter to receive a weekly article roundup.
- Please read our comments policy before commenting.
- Note: This article gives the views of the author, and not the position of USAPP – American Politics and Policy, nor the London School of Economics.
