Reflecting on the complexity of influence an individual research project can have, Adam Briggle, Robert Frodeman and Britt Holbrook try to get a handle on their own research activities and some of their impacts over the last few years. Their project led to a wide variety of results: scholarly articles, a forthcoming book, blogs and a number of ‘likes’ and ‘shares’. But what exactly is a share or a like? There is a need for further reflection on how philosophy – and the humanities more generally – can achieve broader impacts.
Today even the humanities are expected to have an impact. In the 2014 REF, for instance, philosophy formed one of 36 ‘units of assessment’. In a previous post we raised the question of impact for the field of applied philosophy. We argued that applied philosophers, a field that should be brimming with impacts, have fallen prey to ‘disciplinary capture’: even when they have sought to be practical they have played an inside game, writing for other philosophers rather than for a wider audience.
Looking back at our own earlier work in applied philosophy we see that we have committed the same mistake. We offered analyses of environmental problems, but never sent our papers to the parties concerned, much less sought their participation at the front end of the research. In recent years we’ve tried to do better. These efforts that have led us to reflect on what could be called ‘the philosophy of impact’ – what counts as impact, how is it achieved, and how can it be demonstrated.
We offer two brief accounts of projects that seem to have had societal impact, but which also have raised questions about how to understand impact. We then distill some lessons for the philosophic community, and we believe for other disciplines, for getting a grasp on the question of broader impact.
The Comparative Assessment of Peer Review
The Comparative Assessment of Peer Review (CAPR) was a four year (2008 – 2012) project funded by the US National Science Foundation’s SciSIP program. CAPR examined the peer review processes at six science agencies worldwide, focusing on how different agencies integrate broader societal impacts issues into the ex ante review of grant proposals.
CAPR’s products included 10 publications examining the development of peer review at federal agencies worldwide. (These include a post on the LSE Impact Blog. For our work on NSF’s broader impacts criterion, click here.) So far, normal academic fare; but what about the project’s societal impact? We held a series of workshops with user groups to increase societal access to decision making and conducted a survey of stakeholders concerning the relationship between science, society, and the role of peer review. And we had contacts with both the NSF and the US Congress about our work. The graph below tries to get a handle on our activities and some of their impacts.
Blue boxes = scholarly activities: grants, research workshops, and publications; Gray boxes = broader activities: e.g., meetings w/ various stakeholders; Green arrows = direct link between activities and results; Orange arrows = looser connections.
Of course, these arrows may conceal as much as they reveal. For example, our meetings with NSF representatives and policy makers didn’t simply follow from our publication. That took extra effort (the kinds of things we didn’t do in our early work) – getting on the phone and on e-mail with the audiences we wanted to influence, wheedling invitations, and the like. Nor are we committing the error of post hoc, ergo propter hoc: we acknowledge that just because an event follows upon our work doesn’t mean that it was caused by our work. In addition, events are multi-causal: if we had an impact, our work was unlikely to be the only factor involved. Finally, we note what could be called the ‘Freudian’ aspects of impact: people do not know always know when they have been impacted; and when asked, they may deny it for any one of a number of reasons.
Hydraulic Fracturing and Horizontal Drilling in the City of Denton, Texas
In 2011, a Denton City Councilor asked us for help in providing citizen input into the city’s revisions of their natural gas drilling and production ordinance. Denton has roughly 280 gas wells within its city limits, which have created ongoing conflicts related to property rights, health, safety, neighborhood integrity, and land use compatibility. One of us (Briggle) formed and chaired the Denton Drilling Awareness Group (DAG), a citizen-expert stakeholder advisory body. As part of these efforts Briggle also maintained a blog and website as well as Facebook and Twitter accounts and an e-mail list.
This project led to a wide variety of results: scholarly articles and a forthcoming book; coverage in prominent papers such as The New York Times, Wall Street Journal, and the LA Times; write-ups in the U.K., the Netherlands, France, Romania, Russia, and Denmark; and meetings, rallies, public referenda, ordinances, and lawsuits. There have also been a number of ‘likes’ and ‘shares’ of Briggle’s blog posts: on one occasion there were 8,705 people “reached,” on another blog 39,216 “reached”, with 642 likes, and 148 shares.
What are we to make of these ‘likes’, ‘shares’, and ‘reached’? Our peer reviewed work will never get the kind of exposure of the blog posts; nor will it foster the level of public discussion that these numbers suggest. But what exactly is a share or a like? For comparison, former Star Trek star George Takei received 38,000 likes and 20,000 shares in less than a day, for a post about “Dropping the F-Bomb” on distractify.com. Is this the kind of comparison we want to start drawing when it comes to philosophy – or any other kind of scholarship for that matter? Or is this the democratization of research? Which principle is stronger: ‘not all impacts are created equal’ or ‘any impact is good impact’?
Toward a Philosophy of Impact
Our attempts to understand the impacts of philosophy have driven us to questions surrounding the philosophy of impact. It has led us to the following (tentative) conclusions.
A philosophy of impact should begin by questioning the term that has become standard: ‘Impact’ is a dubious choice of metaphor. The term is too Newtonian, too visual and physicalistic in nature. It suggests a car crash when most outcomes are much gentler than that. Terms like ‘effect’ or ‘influence’ or ‘inflect’ better represent the complex processes involved.
Nor can we measure the ‘impact’ of research in the same way we can measure the impact of a car hitting a wall. There is no F = ma equation to be had. This in turn implies that every set of metrics is full of interpretive moments and axiological (i.e., value-laden) assumptions. Black-boxing these in a spurious attempt at being ‘scientific’ does no one any good. For instance, recognizing that bibliometric tools may be misused, the recent Leiden Manifesto by Hicks et al. outlines ten principles that should be followed when using bibliometrics for evaluation. Even if it is only taken as an ethical code for bibliometricians, we view the Leiden Manifesto as a move in the direction of a philosophy of bibliometrics.
A code of conduct for bibliometricians is a good start. But a philosophy of impact (by whatever name) must also consider broader and more diffuse influences across long timelines. And it must ask about the reasons we want to have impact rather than merely how it can be achieved. For instance, has ‘broader impact’ been primarily economic in nature because everyone agrees that they would like to have more money? What about those cases like the Exxon Valdez where economic growth came at a cost to the environment?
There is a need for further reflection on how philosophy – and the humanities more generally – can achieve broader impacts. We call our own thoughts about this ‘field philosophy,’ which can be put in terms of its own set of maxims:
- Be case-based: begin your thinking not with abstract concerns, but with a specific problem of concern to non-philosophers
- Involve your audience in the framing of the problem and the design of the research
- Check back in with them regularly for mid-course corrections. Better yet, work with them on a weekly basis
- Adopt a contextual definition of rigor: what counts as rigorous research should adjust to the needs, timeline, and economics of the situation
- Include non-disciplinary standards for success
These maxims don’t constitute a method that can be followed to produce a guaranteed result (like baking bread, for instance). Judgment always plays a central role. But engaging those upon whom we hope to have an impact both increases the chances of having broader impacts and improves our work.
Of course, some will ask not how to have an impact or what impact means, but whether philosophy is the sort of thing that should have an impact at all. Some will say that philosophy is its own reward. Or they might claim that its impacts unfold over very long time horizons and work deep underground in the roots of culture. Many believe that philosophers ought to leave the world as it is, and that an inward focus is necessary for protecting scholarship from interference from those outside the profession who fail to understand it.
But a philosophy of no-impact leaves us open to Marx’s criticism that philosophers have missed the point, which is to change the world, rather than merely interpret it. It also ignores the fact that most academics are employed by the state. We do not need to tell the authorities what they want to hear, but we do have an obligation to address questions that they think are important.
We believe that philosophy ought to change the world. The tricky part is figuring out how and whether one is changing it for the better.
This material is based upon work supported by the National Science Foundation under Grant No. 1445121. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF)
Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Adam Briggle is an Associate Professor in the Department of Philosophy and Religion at the University of North Texas. His research interests lie at the intersection of science, technology, ethics, and policy. He is author of A Rich Bioethics (2010, University of Notre Dame Press) and the forthcoming A Field Philosopher’s Guide to Fracking (2015, W.W. Norton).
Robert Frodeman is Professor of Philosophy at the University of North Texas. His work ranges across environmental philosophy, the philosophy of science and technology policy, and the philosophy of interdisciplinarity. Sustainable Knowledge: a theory of interdisciplinarity (Palgrave McMillan) was published in 2014.
Britt Holbrook is Visiting Assistant Professor in the School of Public Policy at Georgia Institute of Technology. Beginning in August 2015, Holbrook will become Assistant Professor in the Department of Humanities at New Jersey Institute of Technology. He has also held positions in philosophy at Emory University, Georgia State University, and the University of North Texas. Holbrook’s current research focuses on interdisciplinarity, peer review, scholarly communication, and the relationship between science, technology, and society. Follow him on twitter @jbrittholbrook and read his Impactstory.