Drawing on the research from his recent study into rising retraction rates R. Grant Steen argues retractions alone may be a poor surrogate measure of scientific misconduct. Science cultures are shifting to become more aware of certain “crimes” and publishing cultures may also be more willing to take immediate action. Nevertheless, there is reason to suspect that misconduct may really be increasing in prevalence and it is important to consider how best the community can measure and address this.
My colleagues, Drs. Arturo Casadevall and Ferric Fang, and I recently published a PLoS ONE study that asks, “Why has the number of scientific retractions increased?” We analyzed 2,047 retractions that had been indexed in PubMed prior to May, 2012, with a particular focus on how long it took to retract flawed articles. We sought insight into whether the recent increase in retractions reflects “an increase in the publication rate of flawed articles or an increase in the rate at which flawed articles are withdrawn.” We found that the rate of publication and the rate of retraction of flawed articles are both increasing. We concluded that the increase in retracted articles reflects “changes in the behavior of both authors and institutions. Lower barriers to publication of flawed articles are seen in the increase in number and proportion of retractions by authors with a single retraction. Lower barriers to retraction are apparent in an increase in retraction for “new” offenses such as plagiarism and a decrease in time-to-retraction of flawed work.”
These findings raise an interesting set of questions. Is scientific misconduct actually increasing? Or are we getting better at detecting misconduct? Or are we simply more willing to “out” what has been the dirty little secret of science? These are difficult questions to answer, for several reasons.
Retractions may be a poor surrogate measure of misconduct, in the same sense that speeding citations are a poor surrogate measure of reckless driving. Perhaps only the most flagrant examples of misconduct or reckless driving ever get cited. Speeding tickets issued by the Highway Patrol simply cannot inform us as to the average driving speed on our highways. We’ve probably all had the experience of being passed by someone driving too fast, but then not seen that person pulled over by the police as we continue on our way. Some of us may have had the experience of driving over the speed limit ourselves, yet getting no tickets. Whether or not a driver gets a speeding ticket is actually a function of many other things than simply driving speed: a late-model luxury car is less likely to be ticketed than an old dilapidated car; a Volkswagen is not catnip to cops the way a Porsche is; an older driver is less likely to be stopped than a teenager. Similarly, scientific misconduct may be less likely to be discovered if it is committed by certain protected classes of scientists (e.g., university scientists, who are often assumed to have no pecuniary motivation for fabrication or falsification).
Misconduct is hard to define rigorously. In the absence of clear consensus, the sin is in the eye of the beholder. I personally regard plagiarism as a venial rather than a mortal sin, since I think it can happen by accident or it can be done by novices who are unfamiliar with the culture of science. I am therefore reluctant to think of plagiarism as being sufficient for a verdict of misconduct. Yet I know that many of my colleagues regard plagiarism with less tolerance than I do, and some few regard theft of words as being on a par with theft of data. Different authors may be discussing different kinds of sins when they talk about misconduct.
Misconduct is very hard to measure. Scientific retractions may be a useful surrogate for misconduct, but there is almost certainly more misconduct than has been formally noted in retractions. Yet, if we do not use retractions as a surrogate for misconduct, how can we measure misconduct? I do not accept that surveys are a way to estimate the rate of misconduct, because surveys are generally not random, so results cannot be transformed into a prevalence estimate. Scientists who feel guilty about a past transgression are far more likely to respond to a survey than are the guiltless, so the number of sinners in a population can be characterized. But the total population of scientists—those who saw the survey, but didn’t feel compelled to respond—is unknown. Hence, surveys may over-estimate the prevalence of misconduct. Clever strategies designed to overcome this form of “saliency bias” (or attentional bias) may be ineffective. Furthermore, the elastic nature of the definition of misconduct may mean that certain guilt-motivated scientists confess to “crimes” that others may not see as crimes.
Nevertheless, there is reason to suspect that misconduct may really be increasing in prevalence. Federal support for research has been declining for a long time; Science reported that 71% of a sample of 1,300 scientists believed that the current grant funding situation is damaging to scientific institutions (2013;340:429). Success rate of grants has also declined precipitously in recent years; from FY 2000 to FY 2007, the RO1 success rate fell from 20.3% to 7.2% (Science 2008;322:189). Salary support for faculty by universities has declined as universities trim research services while demanding more overhead from the government. Although funding sources for scientists are drying up, mortgage payments and car loans and school costs creep upward inevitably. Universities are playing a game of musical chairs with their faculty, with money the music and endowed chairs the prize. In short, faculty have a growing need for new data in the face of a growing inability to get that data legitimately.
So, let me tap into the hive mind here: How can misconduct be measured objectively?
Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Dr. R. Grant Steen was an Assistant or Associate Professor for 18 years before leaving academia to start his own consultancy in 2006. Since then, he has helped various clients to publish clinical research or to educate the public about clinical research findings. In that position, he has come to appreciate the importance of medical ethics in the development of clinical science.
Thank you for your nice essay. Reading your article reminds me an experience I had with ‘detecting’ of two clear cases of plagiarisms of a Ph.D. graduate that conducted in two articles from his dissertation. The articles co-authored with his supervisors, as usual case. I was following other articles of him which they also were plagiarized ones and once I notified him about such scientific misconduct in one article of him. However, seemed to me that he does not care about such issues at all as published the new two articles. This time I moved further and notified the editors of such articles and the editors found ‘clear’ cases of plagiarism and decided to retract. But, since the supervisor believed he has nothing to do with these cases (as he have not) put huge amount of pressure on the editors to not to retract. Finally, my identity revealed to the supervisor and i faced tons of pressure from him because he described my actions as traitor and … Now, I am dubious whether I should have informed the editors or not?
I think the one of the reasons (at least for this case described) is the researchers do the research and publish article just for grant/tenure/etc. there is no real interest in research per se.
Thank you for reading this very long comment.
Retractions reflect a small percentage of misconduct. David Vaux’s excellent guest post on Retraction Watch provides insight into the hurdles institutions place on retraction. The only solution would appear to be something along the lines of PubPeer, with a requirement for all PhD students to submit critiques of papers they have read online prior to thesis defence. Why? To ensure that all scientists in training develop their critical abilities: we should never have to place total reliance on the output form one lab until it is separately verified.
I do not think there is a way to measure misconduct objectively. You lay out the problems very well. By looking at a specific article, it shouldn’t be to difficult to tell whether it is misconduct or not. Although even here, there is a grey area for which it is difficult to tell (not to mention attempts to judge whether it is willful or not). Beyond that, I think one can only estimate the incidence of misconduct based on known instances of misconduct. Without having seen such a model so far (I guess there are some out there), my hunch is that the margin of uncertainty will be huge.