“The most important thing in the Olympic Games is not winning but taking part.” So goes the famous saying by Pierre de Coubertin, the father of modern Olympic Games. But does the same apply for competitive research grants? Charles Ayoubi, Michele Pezzoni and Fabiana Visentin report on their study which finds that simply taking part in an application process has a positive effect on researchers’ publication rates and on the average impact factor of the journals in which they publish. Participating in a competitive grant also allows applicants to enhance their learning, explore new trends of research, and extend their collaboration networks.
Are researchers wasting their time writing grant proposals with a low probability of success? This question is of the utmost importance to policymakers and funding agencies looking to promote scientific research. But it is also a key question for scientists choosing to spend more and more of their time on grant proposals where the probability of being awarded funding is low (between 10% and 35% in the UK, according to the UK Research Councils). In our study, recently published in Research Policy, we evaluated the impact of the two phases of a grant competition race – applying and being awarded with funds – on the main scientific outcomes of a researcher. Relying on detailed records of a Swiss funding programme, we found that the time and effort spent preparing the proposal stimulates the researcher’s quantitative and qualitative productivity regardless of the results of the competition. For researchers, simply taking part in an application process has a positive effect on their publication rates and on the average impact factor of the journals in which they publish. Participating in a competitive grant also allows applicants to enhance their learning, explore new trends of research, and extend their collaboration networks. Interestingly, receiving the desired funds is an incentive to establish a co-authorship with co-applicants but has no additional impact on the individual productivity of researchers.
The scarcity of data on both awarded and non-awarded researchers has been a major limitation of studies evaluating what impact applying for research grants has on the scientific outcomes of researchers. We hope our study can open the way for better consideration of the incentive mechanisms triggered by funding programmes, rather than simply evaluating a return on investment based on the scientists’ financial endowments.
To precisely evaluate the effect of applying, we compared two groups of scientists with the same characteristics, differing only in their decision to participate in a grant competition. Adopting a “difference-in-differences” approach, we assessed whether or not scientists who decided to apply perform differently from others. We used a novel dataset of grant applicants to SINERGIA, a funding programme of the Swiss National Science Foundation (SNSF) sponsoring interdisciplinary team collaboration where researchers are asked to submit a common project to access funds. We then selected a control sample of scientists with observable characteristics as close as possible to the applicants in our sample, using a propensity score matching approach.
We found that applicants boosted their productivity, producing 43% more papers in the five years following the application. Applicants’ papers also appeared in higher impact factor journals (+7%) and showed an increased breadth of references (+36%), suggesting new research directions. The attempt to explore new research directions, far from the current research interests of the applicants, might explain the observed loss of citations to their work (-33%). After having evaluated the impact of applying, we focused our attention on the subsample of applicant scientists and considered the effect of being awarded. On average, awarded applicants do not perform significantly better than non-awarded ones regarding quantity and quality of their scientific production, although they have a 17% greater chance to establish a co-authorship with their co-applicants.
Image credit: _D704362, by los_bandito_anthony. This work is licensed under a CC BY 2.0 license.
Interestingly, our results suggest that scientists can sometimes find greater benefits in the time spent writing proposals than from actually receiving the funds. This being the case, scientists should be less reluctant to invest time and effort entering grant race competitions since these could represent opportunities to launch new strands of research, build working ties with fellow researchers, and acquire new knowledge. For funding agencies, our results suggest that publicising the calls more widely and encouraging scientists to apply could be as efficient as increasing the funds dedicated to finance research projects. For instance, organising dedicated days during which funding agencies meet researchers to inform them about the funding opportunities available could help the calls for projects to reach a larger pool of potential applicants. Alternatively, the introduction of a refunding schema compensating the application costs for those proposals passing a minimum quality threshold would also encourage more researchers to apply.
During the exploratory phase of our work, discussions with the SNSF and actual applicants to SINERGIA revealed that a peculiarity of the SINERGIA grant is the fact that the administrative requirements of the submitted proposals are very limited compared to other grants. In other words, most of the work scientists do when applying for the grant is directly related to the scientific project they are crafting and could therefore be useful for further research regardless of the result of the competition. This characteristic of SINERGIA could be partly responsible for our findings and could also, more importantly, suggest that other funding agencies would benefit from following a similar pattern in designing their calls for grant proposals.
Competitive grant application processes are similar in many aspects to other kinds of competition such as start-ups vying for venture capital funding or firms applying to calls for public procurement contracts. Hence, our main results could be extended to business and managerial contexts. When looking for investors to sustain their business, entrepreneurs are asked to write demanding business plans and to demonstrate a clear strategy. The efforts spent in fulfilling these requirements could be useful to improve the business performance of the start-up regardless of the result of the funding decision of investors. Similarly, firms could boost the efficiency of their projects when working on meeting the requirements of the public procurement call and benefit whether they succeed in winning the contract or not.
This blog post is based on the authors’ article, “The important thing is not to win, it is to take part: What if scientists benefit from participating in competitive grant races?”, published in Research Policy (DOI: 10.1016/j.respol.2018.07.021).
Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
About the authors
Charles Ayoubi is a final-year PhD student in the economics of science and innovation at the École Polytechnique Fédérale de Lausanne (EPFL). His thesis focuses on knowledge production and diffusion from science to innovation. His ORCID is: 0000-0002-0752-3328.
Michele Pezzoni is an associate professor at Université Côte d’Azur. Michele’s current research investigates the determinants of researchers’ productivity and careers. His ORCID is: 0000-0002-2255-8166.
Fabiana Visentin is an assistant professor at the School of Business and Economics at Maastricht University and UNU-MERIT. Her research interests focus on the microeconomics of innovation and on the economics of science. Her ORCID is: 0000-0002-2779-9655.
I’m not sure that the paper establishes that participating in a research proposal is causal of the benefits mentioned, both may be the consequence of an underlying characteristic of the researcher.
Dear Peter, Thank you for rising the causality issue that is crucial in this kind of studies. To address it we identified and compared twins, i.e. researchers as similar as possible differing only in their choice to apply. We also run several robustness checks (IV and RDD) that can be found in the paper for more details