Peer review is under constant scrutiny due to its failure to adapt to a more effective model in the digital age. Steve Fuller argues that academic evaluation proceeds much too slowly for the quite simple reason that academics are valued mainly for being productive and not evaluative. It may be the job of publishers to rescue the academic brand – from academia itself — by hiring peer reviewers directly. To ensure quality control in academic knowledge production, peer review may need to be internalised as a regular publishing cost.
Suppose you’re an executive at a transnational academic publishing firm struggling with the problem of getting academics to referee articles for publication. Your starting assumption so far has been that academics should be willing to do this for free because they realize that their credibility relies on their mutual policing practices, aka ‘peer review’. So your policy has been not to pay them anything – or at least not anything substantial (e.g. the 30-day free access to journals that their universities already supply for free).
But the strategy doesn’t seem to work. Academics either refuse to referee outright or they accept the task but then turn in the report well beyond the deadline, as if they know that you have no alternative. And they’re right. Meanwhile academics are increasingly taking refuge in ‘open source’ internet-based platforms that appear to operate with minimal overhead yet profess some measure of peer review. The net result is a haemorrhaging of your core client base. What to do?
The solution is pretty simple, really. Treat peer reviewing as an in-house publishing function, on the model of the R&D division of the modern corporation. The only reason why no one has proposed this is that the ‘lean and mean’ postmodern corporation sheds its R&D division and outsources innovation to universities who, faced with their own crisis of legitimation, gladly do capital’s bidding in a way that saves firms money while making themselves look ‘relevant’.
Image credit: Frederick Taylor’s consultancy applied to practice (Wikimedia, Public Domain)
This postmodern strategy of managing the production of knowledge may well prove a ‘win-win’ strategy for the corporate and academic worlds in our neo-liberal system, but it cannot be generalised to the evaluation of knowledge. Neo-liberalism depends on clear market signals, not least in the knowledge market. Not only must these signals be reliable but they must also match the pulse of the market. From this standpoint, academic evaluation proceeds much too slowly for the quite simple reason that academics are valued mainly for being productive and not evaluative.
You cannot get a professorship simply on the basis of vast connoisseurship of the scientific literature, which enables you to spot frauds, fallacies, prospects and brilliance at a glance. People who spend more time reading than writing tend not to meet the threshold of chair-worthiness. Nevertheless, it is precisely such people who uphold the distinctiveness of the ‘academic’ brand of knowledge production. In that case, it may be the job of publishers to rescue the academic brand – from academia itself — by hiring these people on a part- or full-time basis. In strict economic terms, to ensure quality control in academic knowledge production, peer review may need to be internalised as a regular publishing cost.
It would be hard to see how universities could object to this policy, as they provide little if any incentive for academics to become involved in the evaluation of their peers’ work. From a neo-liberal perspective, the development would be easily explained. It simply marks a segmentation of the knowledge market: On the one hand, universities appear ill-equipped to support activities surrounding research evaluation; on the other, academic publishers have an interest in promoting the value of academic research. A match made in heaven, right?
If this scenario comes to pass – which is entirely possible – it will be because academia has failed to play its hand properly in the neo-liberal game. Neo-liberal policy makers already know that much truly innovative knowledge isn’t – and never has been — produced in universities. Nevertheless, they have usually expected universities to provide the testing ground or selection mechanism for these various ideas and schemes, to see which merit more widespread distribution. But if universities fail to step up to the plate, perhaps because they overestimate the value of sheer knowledge production, then it is only reasonable that publishers – whose own viability is directly affected by problems in academic quality control – take matters into their own hands and encourage universities to outsource their evaluative functions to them. In any case, a market should be created for knowledge connoisseurs, and in today’s world publishers are in the best position to do it.
Note: This article gives the views of the authors, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Steve Fuller (@profstevefuller) holds the Auguste Comte Chair in Social Epistemology at the University of Warwick. He is the author of Knowledge: The Philosophical Quest in History (Routledge). His homepage can be found here.
Seems like a good idea to me – I know that I would be more likely to do speedy reviews if pay was involved. But one thing: won’t this increase the price of academic books and journals, which are already ridiculously over-priced? And if that happens, won’t we see further ship-jumping into non-traditional, free, open-source platforms for publishing?
journalism once was overseen by inhouse processes at the magazine or the newspaper. we see the death throws due to its neglect. we see a morphing to something less, but still viable in the marketplace. I shudder to think such a morphing might “save” scientific journals.. lets hope academic publishers have some good old fashoned integrity and listen to this most modest proposal. the immodest alternative is too much to contemplate.
Thank you for the post, but I’m afraid I don’t believe this is such a good idea.
Clearly, finding reviewers willing to turn good evaluations in a timely fashion is an issue. But paying them to do so won’t work.
Note that this solution has already been tried: journals published by the American Economic Association (who else?) as well as many journals in the field of finance do so. I’m not aware of studies systematically assessing this practice, but based on what I hear from my colleagues, this hasn’t solved the problem at all. Worse, it can backfire. Suppose a reviewer is offered $100 for a timely review (as with AEA journals), chances are that this person will quickly do the math and figure that s/he cannot spend more than, say, half an hour on the review. So the flip side of the system is that is that you risk ending up with poor quality, rushed reviews and/or mostly bad reviewers.
I also would find dubious a journal published with in-house quality checks by the like of Elsevier. Their profit rates seem to indicate they are way more driven by money making than by anything else. So they would most probably recoup the additional cost, raising yet again their already outrageous prices, or cut on reviewing expenses.
I believe we have to make do with the difficulty of finding reviewers and should tackle with the issue in two ways, however limited they are: insist during PhD training that the ethics of the job require that you review two to three papers for each one you submit, and insist that hiring and promotion committees take reviewing into account.
Finally, Phil Davis has an interesting post on this topic at Scholarly Kitchen: http://scholarlykitchen.sspnet.org/2013/02/22/rewarding-reviewers-money-prestige-or-some-of-both/
Bernard Forgeus wrote:
“Suppose a reviewer is offered $100 for a timely review (as with AEA journals), chances are that this person will quickly do the math and figure that s/he cannot spend more than, say, half an hour on the review. So the flip side of the system is that is that you risk ending up with poor quality, rushed reviews and/or mostly bad reviewers.”
Lets extend this a little. If you offer $100 and get poor quality, rushed reviews, then it is unclear to me how offering no payment for peer review would improve the situation. The term “you get what you pay for” seems to ring true here.
$0 ≈ 0 mins review time
$100 ≈ 30 mins review time
$200 ≈ 60 mins review time
$300 ≈ 90 mins review time
I am regularly asked to review papers. When a postdoc, I would review everything, pleased to contribute and proud to be asked for my opinion. As I have got busier and taken on more responsibilities (grants, students, admin, other pro bono stuff) I have to be selective. The problem, as I see it, is there is little or no reward for more experienced senior academics/ scientists to peer-review. As they become more and more overburdened with responsibilities, peer-reviewing becomes a cost rather than a rewarding endeavour. This cost needs to offset in some way. Financial compensation for the hour (or so) that it can take to read a paper and construct an intelligent and constructive review seems only fair. Furthermore, it is also consistent with way we exchange services in almost every other industry or sector.
I’m not sure about Steve’s suggestion of in-house publishing function. It’s the payment that is most important! This could easily be incorporated within the current system. It’s high time these enormous publishing houses began to pay for the academic services they require.
To Bernard Forgues: I think you miss the spirit of my proposal. I’d like article evaluators to draw their principal salary from the publishers. These would be academic experts who have effectively left the rat race of publishing but are interested in maintaining the quality of their fields. This is quite different, both economically and psychologically, from simply paying publishing academics to review papers that may be effectively competing with their own.