To grapple with the the substantial amount of data generated by research evaluations and impact assessments, funders and institutions must look to improve their communication systems. Birge Wolf, Jürgen Heß and Anna Maria Häring are looking to combine evaluation concepts for inter- and trans-disciplinary research with funders’ increasing interests in societal impact data. Improved data sharing mechanisms will provide more support to researchers and boost efficiency of the system.

Agricultural research as problem-related system science has diverse sub-disciplines including social sciences. It is increasingly concerned about societal impact of research, because agriculture is highly related to societal needs and public goods (food! biomass! clean water and air! landscape!) and their associated conflicting interests. Agriculture is driving and being affected by grand global challenges (climate change, over-exploitation of natural resources, loss of ecosystem services, urbanisation). Accordingly, agricultural research needs to be scientifically evident as well as related to relevant problems, public welfare and often requires interdisciplinary and trans-disciplinary approaches. However, incentive effects for research to apply such approaches consequently are seen to be inadequate, especially regarding evaluation and reputation mechanisms, while financing of such approaches is already improving.

Many varieties of corn by Sam Fentress CC BY-SA

In our view changes in evaluation systems require both: (1) more support for evaluation approaches beyond scientific impact and (2) the reduction of effort associated with such evaluation approaches. For (1) it is required to build up more synergies between interest groups: There is an increasing interest of science, research funders and civil society organisations towards trans-disciplinary knowledge production and evaluation beyond scientific impact, especially in the contexts of sustainable development and global challenges. These endeavors are also supported by critiques on distorting effects within current scientific impact evaluation and attempts of the open access and open science movements to increase access, transparency, plurality and benefit in science communication. The fast development of communication technologies provides a great chance to use them for improved societal impact documentation. Synergies in the interests of these groups are subsumed in Figure 1 and detailed in our current publication.

Figure 1: Supporting movements and joint measures to facilitate evaluation beyond scientific impact

Source:http://www.librelloph.com/organicfarming/article/view/of-1.1.3.

The reduction of effort is the focus of the research team of Organic Agricultural Sciences of the University of Kassel and the Center for Evaluation (CEval) of Saarland University. We try to combine the state of the art of evaluation concepts for inter- and trans-disciplinary research and societal impact assessment with funders’ interests in societal impact data and the possibilities of Current Research Information Systems (CRIS). Research funders ask increasingly details about dissemination, exploitation and societal benefit but mostly in text documents – that are rarely useful for evaluation. CRIS are active in developing standards (CERIF, CASRAI) to serve interoperability and make once-assessed data available for multiple use, but they cover only partly the information that is needed for evaluation beyond scientific impact.

Correspondingly, we are developing a concept to integrate additional information in CRIS: towards evaluation beyond scientific impact and in alliance with documentation requirements of research funders (in this instance, carried out for German federal research). This should enable funders to replace parts of the proposals and reports by CRIS that include societal impact data and share these data with research institutions. , Thus, all can benefit from available data for evaluation beyond scientific impact (see Figure 2).

The approach intends to provide a high completeness and a certain degree of verification of the data, due to the connection with funding processes. Additionally, such approaches increase the awareness of researchers to create feasible pathways to impact of their research and might provide open access to scientific outputs that are tailored for non-scientific audiences.

Figure 2: Possibilities for using and developing Current Research Information Systems (CRIS) for interoperable data transfer between funders and institutions to assess and use societal impact data without additional effort.

Source: http://www.librelloph.com/organicfarming/article/view/of-1.1.3

Research is still in progress, but some preliminary results are reported here.

Core elements of the current version of the concept are:

• all processes and outputs can be attributed to persons, organisational units and projects, which is the core feature of current CERIF-CRIS
  • additionally, they are attributed to target groups

• publications, conferences and patents and other intellectual property rights, spin-offs and promotion of young scientists are frequently covered in institutional CRIS. They are supplemented by
  • processes and outputs that facilitate societal impact – also referred to as productive interactions
  • non-scientific use of research results, e.g. via increase in knowledge, skills, perceptions, further use of structures and networks, changed behaviour/services/management, changed regulations and policies, changed products, including failure or negative/unintended outcomes
  • impact of use of research results

• opportunities to integrate feedback and references of target groups and stakeholders
• connection of the description of aims, work packages and exploitation plans in proposals and reports with the structured data assessment of processes, outputs, outcomes and impacts regarding practice and society.

The concept is developed and tested from three perspectives: researchers, research funders and evaluators. The testing process is currently planned until the end of 2015, but we intend to apply for an enlargement of this testing period.

For more on this topic see Developing a Documentation System for Evaluating the Societal Impact of Science and Strategies towards Evaluation beyond Scientific Impact. Pathways not only for Agricultural Research.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.

About the Authors

Birge Wolf studied Organic Agriculture at the University of Kassel and works since 2009 on the issue of societal impact assessment of agricultural research. Currently, she focuses on the possibilities to extend Research Information Systems (RIS) with societal impact data and to make them useable for funding procedures in order to replace parts of proposals and reports with such RIS. The aim is, to provide societal impact data for funders, institutions and the scientists themselves – without additional burden for documentation.

Jürgen Heß is professor for Organic Farming at the University of Kassel-Witzenhausen, scientific head of the experimental farm Frankenhausen and member of several scientific boards on the national level in Germany. His work focusses on the different challenges of sustainable agriculture, such as the optimization of yield and quality of organic products through alternative cultivation methods regarding nature and water protection. A special area of research within the last five years were concepts to evaluate the impact of science on society and practice.

Anna Maria Häring works at the University for Sustainable Development Eberswalde as Professor for Policy and Markets in the Agro-Food Sector. Her research focuses on economic analyses of sustainable farming systems, ex-ante and ex-post impact analyses of agricultural and rural development policies, the development of markets and supply chains for food and farming businesses, the analyses and the development of innovation networks in the food and farming sector, and the design of multi-stakeholder processes.