Whilst the COVID-19 pandemic promoted faster and more open research practices, it also revealed ongoing issues of research waste, and the widespread duplication of research efforts. Till Bruckner provides evidence for how research waste continues to impact medicine and other fields and highlights the positive steps several funders have taken to promote the responsible publication and collaborative use of negative findings.

Medical researchers gained a lot of well-deserved respect during the pandemic, but media headlines only tell part of the story. While vaccines were developed and brought to market efficiently and at record speed, the hunt for effective treatments for Covid was a shambles.

In the early months of the pandemic, academic research teams around the world raced to set up hundreds of clinical trials. Tens of thousands of hospitalised patients were randomised to receive either placebos or a drug candidate. However, few of those tiny trials yielded useful evidence, and the results of many have still not been made public. Virtually all useful insights came from a handful of large, efficient trials that rapidly recruited patients – and rapidly made their results public.

In the end, the majority of Covid drug trials ended up as medical research waste, a persistent phenomenon that costs the world an estimated $170 billion every single year, chiefly because many results are never made public.

Virtually all useful insights came from a handful of large, efficient trials that rapidly recruited patients – and rapidly made their results public.

Non-publication and incomplete publication of clinical trial results is pervasive in medicine. When a once promising-seeming treatment turns out not to help patients after all, scientists are tempted to move on and launch the next trial rather than ‘wasting’ time on publishing disappointing results in a lower-tier medical journal.

While this makes complete sense from the perspective of the team that ran the trial, the cumulative effect on medicine is disastrous. Different research teams repeatedly stumble down the same blind alleys.

Even worse, because trials with ‘positive’ outcomes are more likely to find their way into the literature, the entire medical evidence base has been severely distorted. The scientific literature systematically overstates the benefits and underplays the harms of treatments, as the example below involving a cohort of antidepressant trials shows.

Fig.1: TranspariMED graphic, based on Turner at el 2022

This problem is not confined to clinical trials. Similar patterns have been documented in psychology and more broadly throughout the social sciences including economics, undermining evidence-based policymaking.

The key driver of non-reporting are academic incentives, which reward the trumpeting of spectacular results in prestigious journals, while punishing researchers who ‘waste’ their time on publishing less exciting but scientifically equally valuable findings. Thus, year by year, the costly mountain of research waste keeps piling up.

But why should taxpayers pay for invisible research that does not benefit science, let alone benefit wider society?

Medical research funders are increasingly asking themselves this question. In 2017, some of the world’s biggest funders formally committed themselves to require all grantees to pre-register their clinical trials before they start (to discourage subsequent selective reporting of partial results only) and then make a summary of their results public on so-called trial registries within 12 months (to ensure rapid publication regardless of whether academic journals accept a scientific paper).

Meanwhile, medical journals have committed not to consider results uploaded to registries as “prior publication” – so researchers do not have to fear having their papers rejected because a summary of the findings has already been made public elsewhere. Importantly, journals have kept this promise.

Recently, a team I was part of checked whether research funders in Europe were getting serious about tackling research waste. We found that UK funders perform best by far.

Fig.2: Source, TranspariMED graphic, based on Bruckner et al 2022

Today, if you receive a grant from the  National Institute of Health Research or the Medical Research Council to run a clinical trial, they will not only verify whether you made the results public as required – they will also refuse to give you further grants if you fail to deliver.

Other funders had fewer safeguards in place, but many promised us that they would address the remaining gaps in the near future. Overall, the feedback we received from European funders was strikingly positive.

Meanwhile, across the Atlantic, tackling research waste is also moving up funders’ agenda. In early August, an audit found that the results of over a third of clinical trials funded by the National Institutes of Health had not been made public. In response, the world’s largest medical research funder promised to finally get its act together.

So, what can funders in other fields do to curb research waste in their portfolios?

First and foremost, funders should stop talking about “changing research culture.”

The culture change discourse suggests that if only we train enough early career researchers in open science principles, everything will magically get better once the old guard has shuffled off its mortal coil.

As long as funders keep punishing good people for doing the right thing, nothing will change.

In reality, as long as funders keep punishing good people for doing the right thing, nothing will change. For example, using publication metrics to inform grant decisions actively perpetuates a culture that disincentivises publication of ‘negative’ results. No amount of high-minded declarations or training sessions for PhD students will change that. Only when funders change incentives will research culture change.

Second, funders should set clear rules and clearly communicate them.

One third of medical research funders in our European cohort now explicitly requires researchers to make their clinical results public, on a trial registry, within 12 months, full stop. That’s a clear rule. (Hint: “we encourage X and Y” is not a clear rule, and “we require all research to be conducted in line with this hyperlinked 200-page document” does not qualify as clear communication.)

Third, funders should monitor whether their grantees are following the rules.

The National Institutes of Health had long clearly communicated that its grantees had to make trial results public on a registry within 12 months, but over a third of its grantees nonetheless failed to do so – and their funder never contacted them about this. If funders fail to follow up on non-compliance, they effectively penalise conscientious researchers who invest time into registry reporting (without ever been thanked for it) while their peers can use the same time to write the next grant proposal.

Monitoring results reporting is not about “policing” researchers. In most cases, a friendly reminder email is all that is needed to ensure that results are made public on time – the problem is usually sheer forgetfulness, not evil intent. Several medical research funders have demonstrated that results can be monitored centrally without making grantees fill out additional forms. Similarly, the UK’s ground-breaking national #MakeItPublic transparency strategy was deliberately designed to make trial reporting as easy as possible for researchers, and has therefore received a warm welcome from the UK’s medical research community.

Finally, funders should make Registered Reports the norm in scholarly publication.

In this format, journals decide whether or not to publish a study based on the quality of the research protocol before the results are known. That way, researchers get rewarded for doing excellent science, rather than being incentivised to cherry pick data and perform other dubious statistical acrobatics to generate the spectacular results that their careers currently depend on.

To date, only a few hundred academic journals offer this format. Funders could easily turbo-charge the rollout of Registered Reports to more journals by launching dedicated funding windows now, and loudly proclaiming that from 2030 onwards they will require all applicable research to be published in that format. While not all types of research fit into a Registered Report framework, this simple step could significantly reduce research waste for the many studies that do.

So, dear funders, please stop talking about culture change – and start to make it happen.

You can read all of Till’s LSE Impact Blogposts here.

The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.

Image Credits: Figure.1 Featured image and infographic, TranspariMED, based on Turner at el 2022; Table.1, TranspariMED, based on Bruckner et al 2022