LSE - Small Logo
LSE - Small Logo

Blog Admin

February 19th, 2018

From invisibility to impact: radically different measures are needed to capture the true impact of research

0 comments

Estimated reading time: 5 minutes

Blog Admin

February 19th, 2018

From invisibility to impact: radically different measures are needed to capture the true impact of research

0 comments

Estimated reading time: 5 minutes

Academics are increasingly expected to produce directly applicable solutions to hard-to-solve “real-world” problems such as poverty, development, and environmental degradation. However, conventional assessments of science have not yet been adequately adapted to capture the diverse effects of this type of problem-centred research. Examining a prominent recent example of multidisciplinary research on consumption, environment and sustainability in Ireland, Henrike Rau, Gary Goggins and Frances Fahy show how certain narrowly defined measures of scientific relevance can fail to capture the actual impact of research.

Sustainability researchers are experiencing increasing pressure to produce practical, as well as politically and socially acceptable, solutions for addressing complex societal challenges such as climate change and the rapid depletion of natural resources. To do so, researchers must work with a wide variety of scientific and non-scientific actors to produce research that is relevant, accessible, timely, and context-dependent. These shifting expectations are reflected in the growing number of public and private funding streams that favour “policy-relevant” sustainability research and require funding recipients to demonstrate “value for money”, defined in terms of cost-benefit and calculated using one or more quantitative measurements.

At the same time, an increasingly protracted debate is currently underway concerning how different science, arts, and humanities subjects contribute in diverse and sometimes contradictory ways to societal development. There is a growing body of evidence that conventional forms of impact assessment, which broadly insist on quantifying research outputs, cannot adequately capture this diversity. For example, the use of citation scores and journal impact factors cannot be used to measure the societal impact of research. In fact, these measurements may not even represent scientific impact very well, because they ignore less tangible aspects of academics’ success, such as being well connected to others in the field. Calls thus abound to radically reinvent the impact agenda and make current ways of assessing impact more wide-ranging, diverse and inclusive.

Alternative forms of impact assessment that capture the complexity of the short-, medium- and long-term effects of research, and that work across different areas and disciplines, clearly demand new ways of thinking and measuring. Here, the concept of “science usability” has emerged as a credible alternative to conventional research quality assessment. This innovative approach promotes an improved understanding of the opportunities and constraints concerning the application of knowledge in society. The identification of different types of users and divergent forms of use add further nuance to the science usability debate. Some attempts have also been made to develop indicators that go beyond conventional bibliometric analyses. Proposals also exist to include in research quality assessment many of the less tangible influences on the trajectory and outcomes of research, such as intuition and passion.

Image credit: Wilderness by Ravi Roshan, via Unsplash (licensed under a CC0 1.0 license).

Drawing on a large-scale interdisciplinary research project undertaken in the Republic of Ireland between 2009 and 2015, our study demonstrates the range and diversity of outputs and outcomes that fall outside of most traditional research impact assessments. Funded by Ireland’s Environmental Protection Agency, CONSENSUS (consumption, environment and sustainability) examined key challenges relating to the domestic use of water, energy, food, and mobility options. Linking qualitative and quantitative methods, it brought together a broad range of experiential and scientific expertise to produce robust scientific insights into social and environmental aspects of domestic consumption. The project engaged over 100,000 members of the public and more than 150 representatives from business, policy, and civil society organisations to identify myriad opportunities and constraints for lowering unsustainable resource use in households.

In carrying out a quasi-longitudinal qualitative impact assessment of the project, we were able to demonstrate the significant impact of CONSENSUS on the local and national sustainability research landscape in Ireland, its multi-faceted influence on international research agendas and activities, and its contribution to shaping sustainability policy and practice. For example, the production of a CONSENSUS Briefing Note on the subject of mobility biographies in Ireland presented complex content on the relationship between mobility practice and life events in an easily digestible way that was intended to appeal to time-poor political and civil society actors. Similarly, the CONSENSUS Transition Frameworks proposed concrete measures for policymakers, education and community workers, research and business innovators alike, to achieve greater sustainability in home-heating, washing and eating over the short-, medium- and long-term. Finally, key findings from CONSENSUS fed directly into the National Economic and Social Council’s 2012 report on social and behavioural aspects of climate change, which subsequently informed Ireland’s climate change policy. Many of these impacts would not, and could not, be captured by traditional quantitative assessment procedures.

Based on these findings, we argue that new and innovative forms of impact assessment not only need to be more inclusive but also time-sensitive. Scientific and societal impacts of research may only become visible after an extended period of time; usually one that exceeds the time span of most conventional impact assessment procedures. Moreover, short-term thinking often dominates the domain of research management and governance, creating tensions between those who conduct research and those who manage and assess it. Instead, continuous long-term monitoring of research endeavours such as CONSENSUS promises to provide a much more accurate picture of their actual impact. Importantly, the recent push for novel ways of conceptualising and measuring impact has provided fresh opportunities for both academics and non-academics to debate the societal relevance of research.

This blog is based on the authors’ article “From invisibility to impact: Recognising the scientific and societal relevance of interdisciplinary sustainability research” published in Research Policy (DOI: 10.1016/j.respol.2017.11.005).

Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.

About the authors

Henrike Rau is Professor of Social Geography and Sustainability Research at LMU Munich (Germany) and specialises in society-environment relations and social and cultural aspects of resource use. She has made internationally recognised contributions to the conceptual and methodological advancement of social-scientific sustainability research, including a critical appraisal of the ‘new mobilities paradigm’ and theory-led inquiries into consumption practices. She is currently leading an interdisciplinary initiative to progress cross-cultural research on domestic energy use as part of the EU Horizon 2020 funded ENERGISE project.

Gary Goggins is a postdoctoral researcher based at the National University of Ireland, Galway. His main research interests are in relation to sustainable transitions and individual and socio-material influences on consumption and production patterns.

 

Frances Fahy is a Senior Lecturer at the National University of Ireland, Galway. Her primary research interests are in the fields of sustainable consumption and environmental planning, specifically the social and cultural consequences of environmental change. She is currently the Lead Coordinator of Horizon 2020 project, ENERGISE, an innovative pan-European research initiative to achieve a greater scientific understanding of the social and cultural influences on energy consumption (http://www.energise-project.eu).

Print Friendly, PDF & Email

About the author

Blog Admin

Posted In: Higher education | Impact | Measuring Research | Research evaluation

Leave a Reply

Your email address will not be published. Required fields are marked *