Need a new search?

If you didn't find what you were looking for, try a new search!

  • Permalink Gallery

    The overall incidence of published replication studies in economics is minuscule – greater incentives are required

The overall incidence of published replication studies in economics is minuscule – greater incentives are required

Replicability is considered a hallmark of good scientific practice, an important post-publication quality check. But how many studies are chosen for replication? Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff, Gert G. Wagner have examined the economics literature and find that only one in one thousand publications are replication studies. The introduction of mandatory data disclosure policies may help to increase […]

  • Permalink Gallery

    Replication is both possible and desirable in the humanities, just as it is in the sciences

Replication is both possible and desirable in the humanities, just as it is in the sciences

Some scholars have claimed that replication – the independent repetition of an earlier study, answering the same study question, using the same or similar methods under the same or similar circumstances – is not possible in the humanities. The reasoning is that the humanities search for cultural meaning can yield multiple valid answers, and that research objects are people […]

  • Permalink Gallery

    Software updates: the “unknown unknown” of the replication crisis

Software updates: the “unknown unknown” of the replication crisis

The replication crisis is largely concerned with known problems, such as the lack of replication standards, non-availability of data, or p-hacking. One hitherto unknown problem is the potential for software companies’ changes to the algorithms used for calculations to cause discrepancies between two sets of reported results. Anastasia Ershova and Gerald Schneider encountered this very problem in the course […]

  • Permalink Gallery

    Looking to solve the replication crisis in psychology? Limitations of questionnaire methods must be considered.

Looking to solve the replication crisis in psychology? Limitations of questionnaire methods must be considered.

Throughout its history, psychology has been faced with fundamental crises that all revolve around its disciplinary rigour. Current debates – led in Nature, Science and high-ranking psychology journals – are geared towards the frequent lack of replicability of many psychological findings. New research led by Jana Uher highlights methodological limitations of the widely used questionnaire methods. These limitations may […]

  • Permalink Gallery

    To understand the replication crisis, imagine a world in which everything was published.

To understand the replication crisis, imagine a world in which everything was published.

Countering the claim that failed replications merely reflect the underlying truth of the scientific process and are not a matter of concern, Andrew Gelman argues that actually, the process is largely broken and we are in danger of dismissing the value of replication efforts. Current practice, centered on publication thresholds, is not filtering out poorly designed and executed studies.

John Snow points me […]

  • Permalink Gallery

    A clear distinction is needed between replication tests and the evaluation of robustness in social science literature

A clear distinction is needed between replication tests and the evaluation of robustness in social science literature

Confusion over the meaning of replication is harming social science, argues Michael Clemens. There has been a profound evolution in methods and concepts, particularly with the rise of empirical social science, but our terminology has not yet caught up. The meaning of replication must be standardized so that researchers can easily distinguish between replication efforts and the evaluation of robustness. 

In Milan Kundera’s The Unbearable […]

  • Permalink Gallery

    Replication is not about making or breaking careers: it is about providing an opportunity to move science forward.

Replication is not about making or breaking careers: it is about providing an opportunity to move science forward.

Replication and closer scrutiny of published findings are generally welcome in the scientific community, but questions have been raised over how replication attempts are being reported. Whilst there are certainly arguments for more friendly and cooperative tones to scientific debate, Dorothy Bishop welcomes this next chapter in rigorous debate. Reputation and career prospects will, at the end of the day, […]

  • Permalink Gallery

    Replication of government research uncovers shaky evidence on relationship between school and degree performance.

Replication of government research uncovers shaky evidence on relationship between school and degree performance.

Interested in the statistical analysis used to justify the Department of Education’s reforms, Ron Johnston, Kelvyn Jones, David Manley, Tony Hoare and Richard Harris requested the data related to school performance and degree results via a Freedom of Information request. One year later the dataset was finally made available and they were able to identify some substantial flaws in the government research including sample […]

  • Permalink Gallery

    The FIRST Act’s demand for relevance at the expense of replication puts the entire scientific enterprise at risk.

The FIRST Act’s demand for relevance at the expense of replication puts the entire scientific enterprise at risk.

The United States’ controversial FIRST Act would have profound  implications for how social science research is managed and its funding allocated. David Takeuchi argues that even if the act doesn’t pass, it is clear that politicians are demanding more of a say in federally funded research. While a push to ensure research remains relevant can be a good thing, scientists and […]

  • Permalink Gallery

    Grimpact – Time to acknowledge the dark side of the impact agenda

Grimpact – Time to acknowledge the dark side of the impact agenda

A critical blind spot in the impact agenda has been that impact is understood and defined solely in positive terms. In this post Gemma Derrick and Paul Benneworth introduce the concept of ‘Grimpact’, to describe instances where research negatively impacts society, and argue that the implicit optimism of research assessment has rendered researchers and science systems poorly equipped to […]

  • Permalink Gallery

    Death of the author? AI generated books and the production of scientific knowledge

Death of the author? AI generated books and the production of scientific knowledge

Artificial Intelligence (AI) has been applied to an increasing number of creative tasks from the composition of music, to painting and more recently the creation of academic texts. Reflecting on this development Harry Collins, considers how we might understand AI in the context of academic writing and warns that we should not confuse the work of algorithms with tacit […]

  • Permalink Gallery

    Self-plagiarism: When is re-purposing text ethically justifiable?

Self-plagiarism: When is re-purposing text ethically justifiable?

Self-plagiarism, or publishing substantially similar work twice, is frowned upon in academia as a way of gaining an unfair advantage in a competitive ‘publish or perish’ environment. However, the increasingly open and digital nature of academic publishing means that replication is now easier than ever before. In this post, Mark Israel explores the ethics of self-plagiarism and asks, when is it […]

  • Permalink Gallery

    2018 in review: round-up of our top posts on academic publishing

2018 in review: round-up of our top posts on academic publishing

The future for academic publishers lies in navigating research, not distributing it
The world of scholarly publishing is in upheaval. As the open science and open research movements rapidly gain momentum, the access restrictions and paywalls of many publishers put them at odds with growing parts of the research community. Mattias Björnmalm suggests there is one way for publishers to once again […]

  • Permalink Gallery

    The growing, high-stakes audit culture within the academy has brought about a different kind of publishing crisis

The growing, high-stakes audit culture within the academy has brought about a different kind of publishing crisis

The spate of high-profile cases of fraudulent publications has revealed a widening replication, or outright deception, crisis in the social sciences. To Marc Spooner, researchers “cooking up” findings and the deliberate faking of science is a result of extreme pressures to publish, brought about by an increasingly pervasive audit culture within the academy.

By now most readers will have heard […]

  • Permalink Gallery

    It is advisor attitudes that are likely to shape students’ attitudes towards questionable research practices

It is advisor attitudes that are likely to shape students’ attitudes towards questionable research practices

In debates on the validity of academic research findings, focus has been drawn to so-called questionable research practices, commonly understood to encompass a laundry list of behaviours that can increase the likelihood of statistically significant (and so more publishable) results. Anand Krishna and Sebastian M. Peter report on research examining attitudes to questionable research practices among students who have […]

  • Permalink Gallery

    Beyond #FakeScience: how to overcome shallow certainty in scholarly communication

Beyond #FakeScience: how to overcome shallow certainty in scholarly communication

Recent media reports in Germany have brought renewed focus on predatory publishing practices and seen a notably increased use of the term “fake science”. But to what extent is this a worsening problem? Lambert Heller argues that predatory publishing has never really become a big thing, and that it became a thing at all is largely attributable to the simple […]

  • Permalink Gallery

    Contrary to common belief, randomised controlled trials inevitably produce biased results

Contrary to common belief, randomised controlled trials inevitably produce biased results

Much of the social and medical sciences depend on randomised control trials. But while this may be considered the foundational experimental method, a certain degree of bias inevitably arises in any trial; whether this is sample bias, selection bias, or measurement bias. This is important as the level of validity of a trial’s causal claims can be a matter […]

  • Permalink Gallery

    Unhelpful, caustic and slow: the academic community should rethink the way publications are reviewed

Unhelpful, caustic and slow: the academic community should rethink the way publications are reviewed

The current review system for many academic articles is flawed, hindering the publication of excellent, timely research. There is a lack of education for peer reviewers, either during PhD programmes or from journal publishers, and the lack of incentives to review compounds the problem. Thomas Wagenknecht offers up some solutions to the current system, including encouraging associate editors to […]

  • Permalink Gallery

    To save the research literature, let’s make literature reviews reproducible

To save the research literature, let’s make literature reviews reproducible

Last week the Impact Blog featured a post from Richard P. Phelps, in which he proposed that journals get rid of their requirement for a literature review. Arnaud Vaganay agrees with much of what Phelps said, literature reviews are erratic and self-serving, but suggests doing away with them altogether is likely to make science less efficient and less credible. […]

  • Permalink Gallery

    To save the research literature, get rid of the literature review

To save the research literature, get rid of the literature review

The literature review is a staple of the scholarly article. It allows authors to summarise previous work in the field and highlight what makes their own contribution an original or novel one. But when those previous studies are misrepresented by an author, or even dismissed altogether amid claims of a “paucity of research”, isn’t the knowledge base in fact […]

This work by LSE Impact of Social Sciences blog is licensed under a Creative Commons Attribution 3.0 Unported.