LSE - Small Logo
LSE - Small Logo

Blog Admin

June 22nd, 2016

What do mathematicians think about their journals? Peer review quality tops list of stated issues

4 comments

Estimated reading time: 10 minutes

Blog Admin

June 22nd, 2016

What do mathematicians think about their journals? Peer review quality tops list of stated issues

4 comments

Estimated reading time: 10 minutes

Cameron NeylonCC BY-SA (Wikimedia) (Curtin University), David Michael Roberts (University of Adelaide) and Mark C Wilson (University of Auckland) have conducted a large-scale survey of what mathematicians think of their scholarly publishing options and what improvements are required. Covering topics like open access, peer review and editorial processes, the survey findings reveal some fascinating insights into the scholarly communication system as it currently stands and what changes could be made to make it better.

The last few years have seen intensive debate about the state of academic publishing, but this debate is dominated by loud voices advocating for either radical change or for the status quo. Meanwhile the silent majority seems uninterested. This divide is perhaps most obvious in Mathematics. Maths was the epicentre of the Cost of Knowledge protest of Elsevier pricing, and disciplinary home of some of the most radical experiments, yet the mainstream of mathematical publishing remains largely unchanged.

We wanted to probe whether ordinary rank and file mathematicians want change in their publishing options. What do they see as the problems? What are the opportunities? And if there is change, what form should it take? Since April we have been running an anonymous online survey to get at some of these questions. The purpose of this post is to give a snapshot of the 842 responses received so far and to encourage more people to take part. Anyone who as acted as an author, reader, editor, or reviewer for a mathematical journal in the past three years is invited to participate.

Any online survey will have a response bias. Nonetheless there is a good range of roles represented. Around a third of respondents have acted as an editor for a mathematics journal and over half have a permanent academic position. A quarter are PhD students or postdocs. There is currently a bias towards European (54%) and North American (24%) respondents, but all continents are represented.

Does this sample of mathematicians want change? On a five point scale from 1 being “the status-quo is completely acceptable” to 5 being “almost all [journals] need serious work” 78% of respondents selected 3, 4 or 5. At least amongst this sample there is a strong desire for change. Free text answers describing the major perceived problems revealed serious concerns which indicate systemic issues: over 25 publishers and 100 journals were mentioned by name as needing serious improvement, ranging from journals at large commercial publishers and university presses to small ‘diamond’ OA outfits. The following table is a classification of the stated issues into main categories, from the 401 respondents who named a journal.

Table 1: What are the major problems of scholarly publishing?

table1 maths survey

On this question those who had acted as editors did not differ substantially from those who had not. To protect anonymity, the survey did not ask which journals editors worked for, but with over 250 editors this sample must include many associated with traditionally run journals. In terms of the specifics of change, editors are less keen on Open Access, possibly related to their having a substantially stronger view that author payments for publication are unacceptable. Editors were also less favourable towards open review, editor elections and editor term limits. The latter two items were supported by 30-40% of all respondents.

A diversity of studies continue to show that journal reputation or prestige is an important factor for authors in selecting a journal. In two sets of questions we asked respondents how important they thought specific aspects were for journal reputation, and how important they thought those same aspects were for the community’s view of reputation. The most important factors for the respondents were the quality of peer review (average score 4.3) and the editorial board (4.0) and historical reputation (4.0). Selectivity of the journal (3.5) was more important than Open Access (3.0) and these were seen as more important than the Journal Impact Factor (2.8), external rankings (2.5) or the publisher (2.2).

You_vs_community_views

When we asked the respondents’ assessment of the importance of these factors for the community’s view, a striking pattern emerged. For “traditional” factors (publisher, external rankings, JIF, historical reputation, selectivity) respondents thought that they mattered far more to the community than to themselves (around 0.5 change in the average score). For Open Access respondents thought that the community saw it as far less important (2.2). Respondents were also pessimistic about the community’s focus on the quality of peer review (3.4, a shift in the average of almost a whole unit). These differences matter. Change is risky. If mathematicians are pessimistic about our colleagues’ desire for change then working for change is much less appealing. Itis one thing for the status quo to be supported by peer pressure, but it appears it may be supported by the perception of peer pressure.

If there is change, what should it look like? When asked to rate the importance of elements of journal publishing, high ethical standards (4.6) and timely and thorough peer review (4.5) were rated the most important. Community control, transparent accounting and non-profit status, use of technology, and Open Access were all rated as important (average of 3.5 or above). Perhaps more telling there is greater variance in responses for that second set of factors than for ethics or review. In terms of new practices almost a quarter supported open peer review as a default (with opt-out) and half supported post publication review with moderated comments and commenter identities revealed. Nearly half supported the publication of anonymous referee reports, suitably presented, to help readers.

Because mathematics is a discipline with relatively little funding and therefore has limited discretionary resources, it is commonly believed that there is a strong aversion to author publication charges. However opinions on APCs were split, with a quarter believing them unacceptable in principle, a quarter saying they should be paid by library consortia, and a quarter saying they were “OK if they are sufficiently low”. However, only 2% believed that they were “not a problem, and competition in the journal market will take care of them”.

Eds_vs_non-eds_journal_aspects

Overall we interpret these results as showing that respondents are strongly in favour of change in the publishing system, but pessimistic about the support the efforts for such change would get from their colleagues. There is strong support for high ethical standards and high quality peer review. This is also the subject of serious concerns raised in free-text answers. Editors and publishers should take note of these concerns, alongside the demand for greater transparency in editor selection and editorial processes. On several of these issues editors’ views diverge from that of the community and this should be a subject of some concern. However, there is substantial agreement between editors and non-editors on many issues.

When asked what should happen if efforts by editors to reform a journal are blocked by the publisher, over half of respondents favoured resigning to join a better journal (33%) or to create a new one (29%). Only a very small proportion (< 5%) favoured settling for the status quo. For this set of respondents at least, the appetite for change is there, and community support for bold moves by editors is clear. Nonetheless like any survey, the sample size and potential selection bias place limits on reaching firm conclusions. For that reason we hope that more mathematicians will take part to give us a richer view of the needs and concerns of the mathematics community. The survey can be accessed here.

Note: This article gives the views of the author, and not the position of the LSE Impact blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.

About the Authors

Cameron Neylon is Professor of Research Communications and the Centre for Culture and Technology, Curtin University, and a well known agitator for opening up the process of research. He speaks regularly on issues of Open Science including Open Access publication, Open Data, and Open Source as well as the wider technical and social issues of applying the opportunities the internet brings to the practice of science. He was named as a SPARC Innovator in July 2010 for work on the Panton Principles was a co-author of the Altmetrics manifesto and is a proud recipient of the Blue Obelisk for contributions to open data. He writes regularly at his blog, Science in the Open.

David M. Roberts is a pure mathematician working on differential geometry and category theory, and has also worked on national-scale surveys in the vocational education sector in Australia. He is a proponent of open access in all its forms and fond of placing his research in the public domain.

Mark C. Wilson works in the Computer Science Department at the University of Auckland. His PhD (University of Wisconsin-Madison) was in pure mathematics. Recently his research interests have turned to mathematical and computational social science, including electoral systems and social networks. He has been deeply interested in scientific publishing for several years and has several years painful experience as editor of a free open access journal.

Print Friendly, PDF & Email

About the author

Blog Admin

Posted In: Academic publishing | Open Access | Peer review

4 Comments