Rapoport’s Rules for critically handling literature and reviewing articles.

Anatole (or Anatol) Rapoport was a mathematical psychologist. Image: AZ Quotes.

One of the hardest things to get right in academic contexts is how to comment fairly on other people’s work. On the one hand, scientific and academic work legitimately demands that we should criticise what seems erroneous, perhaps especially if something is both wrong and likely none the less to be assigned plausibility. The price of staying silent may be to see a poor or misleading ‘meme’ further disseminated; or a faulty equation or index taken at face value; or a flawed method adopted by other researchers less alert to its flaws or limitations. The current scale of retractions of scientific papers shows that it is relevant to worry that wrong ideas left unexposed can still be influential.

On the other hand, academics and researchers put a lot of time and effort into preparing and publishing new work. So making criticisms needs to be handled carefully and appropriately, ensuring that they both need to be made and are legitimately motivated. It is important to be sure that criticisms are not caricaturing rival viewpoints, nor unconstructively belittling the hard work and honest effort that goes into almost all academic and serious research.

A classic locale for commenting to go awry is in the early literature review chapter or chapters of many doctorates, especially those written in ‘A record of what I did in my PhD’ sequence. Here the opening over-long chapter or set of chapters really says just: ‘In my first year, I did a really long literature review, to help get my supervisor up to speed on this topic’. (This often leads on a further chapter discoursing on methodology, in the abstract, often covering only routine matters, and presented far in advance of any actual empirical finding. This kind of chapter just says: ‘In my second year, I did a lot more methods courses’).

PhDers often over-engage in the literature review, maxing out on finding holes or limitations in work previously done. Some seem unaware that every criticism of other authors builds up a set of hurdles which their own work must then be able to jump over. It is all too easy for intensively critical comments here to become disabling, setting impossibilist standards that the researcher will not be able to pass in the more original chapters to follow. Other PhDers are alert to this danger, but then seek to avoid it by being obsequiously bland about everyone in their sub-field. The problem here is that their ‘hurdle race’ becomes an exercise in jumping over matchboxes laid horizontally (the test imposed in Monty Python’s ‘Upper Class Twit of the Year’ contest) — which is unlikely to impress a professional audience.

Further on in professional life, two of the four contexts where I have most often seen unethical or erratic behavior in academia relate to commenting fairly — in examining doctoral theses and reviewing articles for journals. (The other contexts are appointment and promotion or review committees). A PhD examiner or a reviewer for a journal is in a position of power, and as Nietzsche aptly remarked: ‘Power makes stupid’. It is still too common to see PhD examiners (in the UK) mistaking their own fixed ideas or academic tics as axioms of their discipline. And all too many journal reviewers make over-strong or un-evidenced comments on papers, that seem directed to simply squashing views that deviate from their own approach, without acknowledging any scope for professional debate and disagreement.

I don’t myself claim to have ever got the balance right in commenting fairly on other people’s work. I guess most of the time I’m way too critical. So in what follows it’s a case of ‘Do what I say, not what I do’. But reading Daniel Dennet’s great book Intuition Pumps and Other Tools for Thinking I came across a short chapter (3) that seemed so well framed that I wished I had better followed this line of practice for myself.

The centrepiece of Dennet’s argument is actually his remembered reconstruction of guidelines for commenting fairly and legitimately developed by the famous game theorist Anatole Rapoport, but never written down explicitly:

Dennett concludes: ‘It is worth reminding yourself that a heroic attempt to find a defensible interpretation of an author, if it comes up empty, can be even more devastating than an angry hatchet job. I recommend it’.

To put these ideas in a wider context, you might find it helpful to read parts of my book: Patrick Dunleavy, ‘Authoring a PhD’ (Palgrave, 2003), especially the chapters on the macro-structure of a thesis and the micro-structure of a chapter. See also useful material on the LSE’s Impact blog and on Twitter: @Write4Research.