Recent findings suggest interdisciplinary research is less likely to be funded than discipline-based research proposals. Gabriele Bammer looks at how interdisciplinary research is currently peer reviewed and argues different review processes may well be required to do justice to these different kinds of interdisciplinarity. Discipline-based researchers may be ill-equipped to evaluate the integrative processes that an interdisciplinary proposal plans to use.
The first empirical support for a long-standing complaint by interdisciplinary researchers was recently published in the leading journal Nature. The Australian National University’s Lindell Bromham, Russell Dinnage and Xia Hua showed that interdisciplinary research is less likely to be funded than discipline-based research proposals (“Interdisciplinary research has consistently lower funding success”, Nature 534, 684–687. 30 June doi:10.1038/nature18315).
They cleverly applied a technique from evolutionary biology that examines relatedness between biological lineages, using a hierarchical classification of research fields rather than an evolutionary tree. The relative representation of different field of research codes and their degree of difference were used as a proxy measure for interdisciplinarity. The results, based on 5 years of data from the Australian Research Council’s Discovery program, are robust and are unaffected when number of collaborators, primary research field and type of institution are taken into account.
What does it mean? Both main potential interpretations raise concerns and require action. The results may support the grumbling of interdisciplinary researchers about an unfair review processes, where reviewers with no interdisciplinary research experience hold sway over which interdisciplinary research is published. It is also possible that interdisciplinary research proposals are not as good as those which are discipline based.
What is required for adequate peer-review of interdisciplinary grant applications? I recently laid out the key issues in a paper for Palgrave Communications (Bammer, G. 2016 ‘What constitutes appropriate peer review for interdisciplinary research?’ Palgrave Communications, 2 (16017), doi:10.1057/palcomms.2016.17). This paper argued that only involving reviewers from the disciplines was likely to significantly short-change an interdisciplinary proposal.
First, the unknowns (expressed as research questions) that are addressed in interdisciplinary proposals may include issues of concern to disciplines, but are unlikely to be confined to those. Interdisciplinary research is also likely to address unknowns that worry stakeholders and unknowns that are central to the problem, both of which may well be outside the purview of disciplines. For example, in research I conducted on the feasibility of prescribing pharmaceutical heroin to dependent heroin users as a new treatment option, we addressed concerns raised by police, one of the major stakeholders, about the possibility that heroin prescription would cause a “honeypot effect”, an issue that was not on the radar of any of the disciplines involved. Similarly we addressed concerns about fostering more permissive attitudes to illicit drug use, which are integral to changing prohibitionist policies. Such issues may seem irrelevant to discipline-based reviewers and certainly cannot be adequately assessed by them.
Further, the discipline based unknowns that are relevant to the interdisciplinary research, such as estimating the number of heroin users (demography), assessing the ethics of heroin prescription (philosophy), and estimating the likely impact on the illicit drug market (economics) may seem pedestrian to discipline-based researchers, even though they are critical to assessing an interdisciplinary problem such as the feasibility of heroin prescription. Indeed in this project only the economists undertook research that was ground-breaking from a disciplinary perspective.
Discipline-based researchers are also ill-equipped to evaluate the integrative processes that an interdisciplinary proposal plans to use, such as assessing the systems view taken, the rationale by which disciplines and stakeholders were chosen for consideration in the study and how all the different insights will be combined. Along with the different unknowns addressed, integrative methods are what set interdisciplinary research apart.
These considerations are relevant to one kind of interdisciplinary research, where several diverse disciplines and stakeholders tackle a complex real-world problem. But there are also other kinds of interdisciplinary research, ranging from one person borrowing from other disciplines to tackle a problem in a new way (as Bromham and colleagues did in their paper), to teams working at the interface of two or more disciplines (which may lead to the development of a new discipline, as when quantum physics and biology combine to form quantum biology), to closely related disciplines converging to develop a new technology. Different peer review processes may well be required to do justice to these different kinds of interdisciplinarity.
At the moment these issues are in the funders’ too hard basket. And who can blame them? Apart from complaints about the inadequacy of current processes, the interdisciplinary community is too unorganised to suggest adequate ways forward. Key tasks include providing a useable definition of interdisciplinarity (that encompasses the main kinds), highlighting central issues for peer-review of each kind, and providing an adequate college of peers to undertake effective peer-review.
And what if the results by Bromham and colleagues result from the quality of the interdisciplinary research proposals submitted for funding not being of a high enough standard? That’s also the purview of the interdisciplinary community. Just as the disciplines police their own standards, interdisciplinarians need to make clear which parameters mean that interdisciplinary research is outstanding and which mean it is poor.
If we don’t organise, our research will always be judged by outsiders, who are less well equipped to do so than we ourselves are.
Note: This article gives the views of the author, and not the position of the LSE Impact blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Gabriele Bammer is developing the new discipline of Integration and Implementation Sciences (I2S) to improve research strengths for tackling complex real-world problems through synthesis of disciplinary and stakeholder knowledge, understanding and managing diverse unknowns and providing integrated research support for policy and practice change (see i2s.anu.edu.au and I2Insights.org). She is a professor in the Research School of Population Health at The Australian National University (ANU) and an ANU Public Policy Fellow. She co-convenes (with Michael Smithson) an edX Massive Open Online Course (MOOC) on ‘Ignorance!’. Her books include Disciplining Interdisciplinarity: Integration and Implementation Sciences for Researching Complex Real-World Problems (author, 2013), Change! Combining analytic approaches with street wisdom (editor, 2015), Research Integration Using Dialogue Methods (co-author, 2009), and Uncertainty and Risk: Multidisciplinary Perspectives (co-editor, 2008).