Using survey data on the evaluations of 150 economics journals, a recent study explored the relationship between economics journals’ reputation and perceived relevance amongst economists working in the field. Justus Haucap shares some of the headline findings from the analysis based on the survey data. The findings suggest that a journal’s relevance is driven by average article quality, while reputation depends more on truly exceptional articles.
Academic careers depend on the reputation of the journals in which the respective scientists publish. Hiring and tenure decisions typically depend, at least to some degree, on publications in reputed outlets, and they also play an important role in the evaluation of research institutes and departments as well as for funding decisions by science funding organizations. At the same time, a number of prominent scientists have voiced concerns over whether the most reputed journals really publish the most relevant research. Particularly (but not only) in economics, a number of critics have argued that technical research excellence tends to drive journals’ and scientists’ reputation while the research questions’ relevance is not adequately reflected in journal reputation. In consequence, critics fear that hiring and funding decisions that are heavily based on the reputation of journal publications may undervalue research relevance. As is argued, the balance between rigor and relevance has tilted towards rigor.
This criticism has gained additional prominence in the wake of the great financial and economic crisis which lead to a vivid discussion, among both academic economists themselves as well as within the general public, about the proper role of economists and economics as a science. One important aspect of this debate has been the concern that the connection between reputation and relevance of publications may be too loose. Put differently, already in 1997, Marc Blaug criticized that “modern economics is sick; economics has increasingly become an intellectual game played for its own sake and not for its practical consequences for understanding the economic world. Economists have converted the subject into a sort of social mathematics in which analytical rigor is everything and practical relevance is nothing” (Blaug, 1997, p. 3), something also criticized recently by Piketty in the introduction to his famous book.
Image credit: M.Imran – Wikimedia Public Domain
In a recent paper, published in Scientometrics, we have tried to shed a bit more light on the relationship between reputation and relevance of economics journals. Even though our focus has been on journals’ relevance for the own profession (and not its relevance for economic policy or other purposes), our empirical analysis has revealed that there are already significant differences in a journal’s reputation and its perceived relevance among economists themselves. Hence, we would expect this discrepancy to be even wider among policy makers.
To study the determinants of journals’ relevance and reputation as perceived by their readers we have used survey data on the evaluations of 150 economics journals. The survey was conducted among German-speaking economists in April 2011, and the 150 journals included the most important economics journals listed in international journal rankings as well as most journals published in Germany, Austria, and Switzerland. For each journal, respondents were asked to indicate (1) how relevant the journal is for their own work and (2) how they assess the journal’s reputation. Both assessments were made on a six-point Likert scale, ranging from 0 (no relevance / reputation) to 5 (very high relevance / reputation). Additionally, we asked respondents to provide information about their age, affiliation, academic status, place of work, and current research focus. The survey was sent via an individualized email-link to all 2991 individual members of the German Economic Association (GEA). On average, 478 (408) respondents evaluated a journal’s relevance (reputation).
Our empirical results can be summarized as follows: Firstly, most journals are considered highly relevant or highly reputed only by a small fraction of economists. For instance, 134 (105) journals are considered highly relevant (reputed) by no more than 20% of respondents while only 8 (17) journals exist, which are regarded as highly relevant (reputed) by more than 30% (50%) of respondents. Secondly, we also found a strong interrelationship between journal reputation and relevance. However, a journal’s perceived relevance has a much stronger impact on the journal’s reputation than reputation on relevance. While a 1% increase in the fraction of economists who consider a journal’s relevance as “high” or “very high” leads to an increase in the fraction of economists who consider the journal highly or very highly reputed by 1.3%, the corresponding number for the reverse effect is only a 0.4% increase.
We also found that past journal ratings conducted by the Handelsblatt (the leading German business daily) and the German Economic Association (GEA) directly affect journals’ reputation among German economists and indirectly also their perceived relevance. However, the effect on reputation is more than twice as large as the effect on perceived relevance.
Citations appear to have a non-linear impact on perceived journal reputation and relevance. While the number of landmark articles published in a journal, reflected by the H-Index, appears to increase journals’ reputation, an increase in the so-called H-index even tends to decrease a journal’s perceived relevance as long as this is not simultaneously reflected in a higher Handelsblatt or GEA rating. This suggests that a journal’s relevance is driven by average article quality, while reputation depends more on truly exceptional articles.
Our analysis has also revealed that Elsevier and Springer have a positive impact on a journal’s relevance. However, there is a countervailing effect on reputation, given a journal’s relevance. In total, the two effects cancel out so that journals published by Elsevier and Springer are, ceteris paribus, not more reputed or more relevant than other publishers’ journals.
Finally, we also found that German-speaking economists consider German-language publications, ceteris paribus, more relevant. A more fine-grained analysis of various age groups revealed that this is largely driven by economists who are older than 45. In contrast, a journal’s Handelsblatt rating only positively influences a journal’s reputation for economists aged 55 and younger. Quite generally, we have found significant differences in the views on journal relevance and reputation between different age groups where our regression analysis shows a better fit for the younger survey respondents. One potential reason could be that younger economists might be more homogeneous in their views about their journals than their older colleagues. Another reason may be that older economists’ views are less affected by recent citation numbers and patterns. Interestingly, the models are also better in explaining journal reputation than relevance, implying that views about journal reputation may be less heterogeneous than views about journals’ relevance.
Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Justus Haucap is the director of the Düsseldorf Institute for Competition Economics (DICE) at the University of Düsseldorf, Germany. His research mostly focuses on competition and regulation in regulated markets such as telecommunications, media, electricity and transport. In recent times, he has also published on competition in Internet markets and the economic implications of digitalisation processes. As a sideline, he also has a long lasting interest in the sociology and economics of science.
1 Comments