How many times have you read an article that confidently states there are few other studies in this area? And how confident are you that this is the case? Richard Phelps argues outside of the sciences there is rarely a lack of pre-existing literature, but claiming so is a rhetorical move to give priority to one’s own research.
Doesn’t all scholarship add to our storehouse of knowledge, you ask? Well, yes, in an absolute sense. Isaac Newton’s generous acknowledgement of his predecessors seeing farther by standing on the shoulders of giants comes to mind.
In a relative sense, however, a new piece of scholarship may subtract from our collective understanding with a most un-Newtonian device: the dismissive literature review. With a dismissive review, a scholar assures an audience that little or no previous research exists on a topic or, if it does, it is so poorly done it is not worth citing.
With a dismissive review, a scholar assures an audience that little or no previous research exists on a topic or, if it does, it is so poorly done it is not worth citing.
Moreover, over time dismissive reviews recursively continue subtracting. For subsequent researchers, a dismissive review discourages any further search of the research literature as, allegedly, there would be nothing to find. When a scholar declares they are the first in the history of the world to have studied a topic, they make a “firstness claim” – one type of dismissive review.
Dismissive reviews can be accurate, for example with genuinely new technologies; but most are probably not. Regardless, they are common. See for yourself with a few simple internet searches. In the search field, enter such phrases as: “this is the first study”, “paucity of research”, “few studies”, “little prior research”, and so on.
Their proliferation is a natural response to prevailing incentives. For the ambitious scholar, a dismissive review conveys several advantages over engaging the wider research literature. A scholar:
- saves much time and avoids the tedium of reading the research literature;
- adds to the citation totals of his or her in-group while not adding to rivals’;
- gives readers no help in finding rival evidence (by not even citing it);
- establishes bona fides as an “expert” on the topic (as experts are expected to know the research literature);
- attracts more attention by allegedly being “first,” “original,” “a pioneer;”
- increases the likelihood of press coverage for the same reason; and
- increases prospects for research grant funding to “fill knowledge gaps.”
By contrast, a scholar better serves the public interest by conducting honest, thorough, and very time-consuming literature reviews. But such moral rectitude won’t get one tenure.
The benefits of dismissive reviews accrue to individuals and small groups; the costs accrue to society as a whole. Dismissive reviewers not only divert our attention away from those more familiar with the research literature (genuine experts) they claim expertise they lack. With each dismissive review, we become less informed, and any resulting public policies are formed with less, and more skewed, information.
The benefits of dismissive reviews accrue to individuals and small groups; the costs accrue to society as a whole.
To be sure, the dismissed research literature remains available, somewhere. But if we do not know it exists, we are unlikely to know where to find it. These days, a mass of new information floods our purview every second, cluttering the view.
A few years ago, I began collecting dismissive reviews from some of the best-known scholars in my field of education policy. I had at first thought the behaviour was exceptional and limited to certain misunderstood topics. But my collection has now surpassed a thousand entries and reveals some patterns.
Some scholars flit quickly from topic to topic serially dismissing multiple, separate research literatures over time, their speed of movement uninhibited by time-consuming literature searches.
Some scholars persistently limit their citations not only to their own academic discipline, but to a certain group of cooperating colleagues. Collectively, they form a “citation cartel.”
dismissive reviewers number among the very few with access to policymakers, at least at the national level. Consequently, whether intentional or not, their dismissals serve a gatekeeping function
Once invested in their dismissive claims, these dismissive reviewers tend not to look back. In the process of collecting over a thousand dismissive reviews, I have yet to see a correction to or retraction of a dismissal that has already been formally published.
These dismissive reviewers number among the very few with access to policymakers, at least at the national level. Consequently, whether intentional or not, their dismissals serve a gatekeeping function, blocking the majority of available policy-relevant research from reaching those policymakers.
All of which led me to consider that these top scholars may have attained their status in part because of their dismissive reviews, not despite them. Or, as one might say in Silicon Valley, dismissive reviews may be a feature of successful careers, not a bug.
The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.
Image Credit: Stephen Plaster on Shutterstock.
Surely every PhD thesis is required to contain a “dismissive review”. How could you possibly make “an original contribution to the knowledge” in an area if that area has been adequately previously researched? Surely the first task of ANY researcher is to find a question which hasn’t been asked before?
We should, I think, question whether the notion that every paper must contain a literature review really is a sensible academic norm to have in the first place. Clearly, it is reasonable for journals to assess submissions based on whether they make a substantial contribution to the field. It is much less clear that it is appropriate to demand that authors effectively spoon-feed reviewers (who ought to be au fait enough with the the literature to make their own judgement) with the answer to the question of whether a substantial contribution is being made. This, as so much of what happens in academia, is allowing the ease-of-assessment tail to wag the substance dog. The downsides to the practice are great. Authors are effectively asked to be judges in their own cause, which is a recipe for mischief; dismissiveness of others’ work becomes almost inevitable in those circumstances. Literature reviews are often incredibly tedious for readers – again because even the best-intentioned author is unlikely to give a balanced or complete account, one’s eyes tend to glaze over reading them. You may say, “but it is important for readers to see how a piece of research is situated within the wider field”. Indeed it is. But it hardly follows that the original research paper itself is the place place to acquire that sense. If I want to get an overview of the field, I would rather go to a dedicated review article or a textbook. It’s also not really clear that all brilliant researchers actually do, or should have, encyclopedic knowledge of their own field (let alone other fields). Feynmann, for instance, notoriously hated reading the literature, as did Wittgenstein. Did even Newton, gracious as he was in acknowledging at a sweep the contributions of others, really know everything in the literature to date? In short, it seems to me that the norm that every article include a literature enforces a quite bizarre and inefficient division of academic labour.
Good post – now I feel I need to check a few reviews by a team in my field who churn out mediocre reviews by the dozen. At least one is a HiCi.
It’s dishonest. No true scholar should be doing this.