Drawing on the research from his recent study into rising retraction rates R. Grant Steen argues retractions alone may be a poor surrogate measure of scientific misconduct. Science cultures are shifting to become more aware of certain “crimes” and publishing cultures may also be more willing to take immediate action. Nevertheless, there is reason to suspect that misconduct may really be increasing in prevalence and it is important to consider how best the community can measure and address this.
My colleagues, Drs. Arturo Casadevall and Ferric Fang, and I recently published a PLoS ONE study that asks, “Why has the number of scientific retractions increased?” We analyzed 2,047 retractions that had been indexed in PubMed prior to May, 2012, with a particular focus on how long it took to retract flawed articles. We sought insight into whether the recent increase in retractions reflects “an increase in the publication rate of flawed articles or an increase in the rate at which flawed articles are withdrawn.” We found that the rate of publication and the rate of retraction of flawed articles are both increasing. We concluded that the increase in retracted articles reflects “changes in the behavior of both authors and institutions. Lower barriers to publication of flawed articles are seen in the increase in number and proportion of retractions by authors with a single retraction. Lower barriers to retraction are apparent in an increase in retraction for “new” offenses such as plagiarism and a decrease in time-to-retraction of flawed work.”
These findings raise an interesting set of questions. Is scientific misconduct actually increasing? Or are we getting better at detecting misconduct? Or are we simply more willing to “out” what has been the dirty little secret of science? These are difficult questions to answer, for several reasons.
Retractions may be a poor surrogate measure of misconduct, in the same sense that speeding citations are a poor surrogate measure of reckless driving. Perhaps only the most flagrant examples of misconduct or reckless driving ever get cited. Speeding tickets issued by the Highway Patrol simply cannot inform us as to the average driving speed on our highways. We’ve probably all had the experience of being passed by someone driving too fast, but then not seen that person pulled over by the police as we continue on our way. Some of us may have had the experience of driving over the speed limit ourselves, yet getting no tickets. Whether or not a driver gets a speeding ticket is actually a function of many other things than simply driving speed: a late-model luxury car is less likely to be ticketed than an old dilapidated car; a Volkswagen is not catnip to cops the way a Porsche is; an older driver is less likely to be stopped than a teenager. Similarly, scientific misconduct may be less likely to be discovered if it is committed by certain protected classes of scientists (e.g., university scientists, who are often assumed to have no pecuniary motivation for fabrication or falsification).
Misconduct is hard to define rigorously. In the absence of clear consensus, the sin is in the eye of the beholder. I personally regard plagiarism as a venial rather than a mortal sin, since I think it can happen by accident or it can be done by novices who are unfamiliar with the culture of science. I am therefore reluctant to think of plagiarism as being sufficient for a verdict of misconduct. Yet I know that many of my colleagues regard plagiarism with less tolerance than I do, and some few regard theft of words as being on a par with theft of data. Different authors may be discussing different kinds of sins when they talk about misconduct.
Misconduct is very hard to measure. Scientific retractions may be a useful surrogate for misconduct, but there is almost certainly more misconduct than has been formally noted in retractions. Yet, if we do not use retractions as a surrogate for misconduct, how can we measure misconduct? I do not accept that surveys are a way to estimate the rate of misconduct, because surveys are generally not random, so results cannot be transformed into a prevalence estimate. Scientists who feel guilty about a past transgression are far more likely to respond to a survey than are the guiltless, so the number of sinners in a population can be characterized. But the total population of scientists—those who saw the survey, but didn’t feel compelled to respond—is unknown. Hence, surveys may over-estimate the prevalence of misconduct. Clever strategies designed to overcome this form of “saliency bias” (or attentional bias) may be ineffective. Furthermore, the elastic nature of the definition of misconduct may mean that certain guilt-motivated scientists confess to “crimes” that others may not see as crimes.
Nevertheless, there is reason to suspect that misconduct may really be increasing in prevalence. Federal support for research has been declining for a long time; Science reported that 71% of a sample of 1,300 scientists believed that the current grant funding situation is damaging to scientific institutions (2013;340:429). Success rate of grants has also declined precipitously in recent years; from FY 2000 to FY 2007, the RO1 success rate fell from 20.3% to 7.2% (Science 2008;322:189). Salary support for faculty by universities has declined as universities trim research services while demanding more overhead from the government. Although funding sources for scientists are drying up, mortgage payments and car loans and school costs creep upward inevitably. Universities are playing a game of musical chairs with their faculty, with money the music and endowed chairs the prize. In short, faculty have a growing need for new data in the face of a growing inability to get that data legitimately.
So, let me tap into the hive mind here: How can misconduct be measured objectively?
Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Dr. R. Grant Steen was an Assistant or Associate Professor for 18 years before leaving academia to start his own consultancy in 2006. Since then, he has helped various clients to publish clinical research or to educate the public about clinical research findings. In that position, he has come to appreciate the importance of medical ethics in the development of clinical science.