Paul J. Silvia is creeped out by the correlation between quality and quantity in academic publishing, but why do the people who publish the most also publish the work that has greatest influence?
Gregory Feist—a distinguished creativity researcher at San Jose State University—is not a haunting man, but his research on scientific eminence creeps me out. One of his early papers—“Quantity, Quality, and Depth of Research as Influences on Scientific Eminence: Is Quantity Most Important?”—strikes chills in the hearts of thwarted writers who suspect they aren’t publishing enough. As you’d suspect from the title, his research (on university biologists, chemists, and physicists) found that the mere quantity of publications was the largest predictor of eminence, assessed via citation rates, awards and distinctions, professional visibility, and peer evaluations of research contributions.
This is creepy stuff indeed, especially to those of us who are reading or writing blog posts on writing as part of a sophisticated and self-deceiving procrastination strategy. Before picking at the nits on Feist’s study, Feist isn’t the only scientist to find this effect. A huge correlation between quality and quantity is found for nearly anyone who looks. This fact forms the basis for many theories of scientific impact and eminence, such as Dean Kean Simonton’s influential writings.
Why are quality and quantity related? Why do people who publish a lot of work also publish work that has a greater influence? The quality-quantity correlation, like any other correlation, can reflect many causal directions. Here are a few speculations:
- Writing both improves and creates ideas. Most people think of writing as a kind of transcription: we gather our facts, form our ideas, and hit the mental “Print” button to output what we know. Instead of being the endpoint of a knowledge-creation cycle, however, writing is often the beginning. Quality and quantity might be linked because the process of writing improves quality, by forcing us to confront and sharpen our ideas, and quantity, by sparking more ideas. In Writing to Learn, William Zinsser argued that writing was a way to create knowledge, a way to understand what we half-know. Grappling with our ideas makes them more sophisticated and eventually sparks some new ones. As anyone who writes regularly knows, writing about one thing leads to ideas for new projects. Robert Boice, in Professors as Writers, showed that forcing professors to write daily caused a sharp increase in text output but also a many-fold increase in the number of new ideas for writing. Writing begets good ideas, which beget more writing.
- Early quality or quantity attracts resources that foster both. Here in the United States—where we are free from the REF madness but afflicted by a few peculiar American maladies—early publishing success opens access to contexts, cultures, and institutions that foster more success. Someone with a hot early career might receive a federal training grant—thus jumpstarting a research program and reducing time spent teaching—and get hired at a resource-rich department with low teaching loads, energetic doctoral students, and a warm intellectual climate. In this case, the quality-quantity correlation is a spurious result of what creativity science calls a “Matthew Effect”—the rich get richer by virtue of access to training, resources, and opportunities.
- Quantity attracts positive attention from peers. Writing is hard, so people who do a lot of it stand out. Because quantity is noticed and valued, it can spark a cycle that leads to markers of quality, such as more citations and a stronger reputation. In a small subfield, one person can generate a notable proportion of publications. High quantity then attracts positive attention from peers, which leads them to read the papers, assign them to students, and cite them in their own work. Beyond attracting attention, quantity attracts citations through mere probability: more papers, however humble, mean more potential things for a domain of scholars to cite. Unlike the prior two, this explanation implies that quantity begets an impoverished kind of quality—the work merely gets more attention and citations, not genuine improvement.
These three paths strike me as reasonable possibilities—all three might have some merit, but the quantity-quality correlation still vexes me. Have some explanations of your own? The comments section—a warm home for creators and procrastinators alike—awaits.
Note: This article gives the views of the author(s), and not the position of the Impact of Social Sciences blog, nor of the London School of Economics.
About the author:
Paul J. Silvia, an Associate Professor of Psychology at the University of North Carolina at Greensboro, is the author of How to Write A Lot: A Practical Guide to Productive Academic Writing.
Eminent scholars publish lots of papers. Therefore, if I want to be an eminent scholar, I should publish lots of papers.
Similarly:
Charlie Parker took lots of heroin. If I want to be a brilliant saxophonist like Charlie Parker, I should take lots of heroin.
Two possible explanations come to mind (in addition to yours which are all plausible):
First, famous influential people may get more invitations to co-author (or, let’s say, to “endorse” with a co-authorship credit that might not reflect a major contribution to the actual hard work of writing a paper). This is a variant of your “resources” explanation.
More banally, it may simply be the case that every paper from every author has a more or less equal chance of being any good. Therefore the more you publish, the better chance you have of hitting the lottery of a brilliant result that makes an impact.
Finally, it could be that people who have good ideas also have lots of ideas. If you’re good at one dimension, maybe you’re good at the other.
(there could also be an interaction between my second and third hypotheses – if you have loads of ideas then sooner or later some of them may turn out to be really good ones.)
This correlation has always puzzled me too. I publish quite a lot (average 10 per year, 3 as first author) but don’t rate my work as being particularly great – pretty average really. And I seem to have the same correlation – an H score of 18, only 5 years out of my PhD. Looking at my most cited article though (130 cites for a 2008 paper in Biological Conservation), the vast majority of people don’t appear to have read or engaged with it. Most of them have a similar sentence in their introduction – they’ve taken a participatory approach and need to find an article to cite to justify that this is valid and important. I think that because my name is out there a lot and usually linked to participation in some shape or form because I’m publishing a lot, people think “hmmm, I wonder who I can cite to justify this claim – oh that Reed bloke is always doing stuff on participation – he must be the key person to cite.” Then they choose the article by me with the most generic looking title that justifies their point without them having to read the paper. Is that a bit cynical??
In my discipline there is an emerging phenomenon I have termed “the cartwheel.” A group of “scholars” get together and each of them authors a couple of papers per year. Irrespective of contribution (or lack thereof) all members of the group are listed as coauthors on everyone else’s papers. This then gives each “coauthor” the appearance of much higher production than they objectively deserve. Even so, many people are oblivious to the charade and its poisonous distortion of scholarship. So pervasive is this subterfuge, that involvement is almost a prerequisite to employment. I saw a graduate student in his final year of a doctoral program lauded because he had two dozen journal publications on his CV. With all that “activity” when do we suppose he got around to his doctoral studies? He is now an up and coming professor; and generally regarded as some kind of academic star. The people involved in this game use high production numbers as support for grants… which supports yet higher production… lather, rinse repeat. As an extension, those who write books or sole authored pieces are regarded as pariahs. If you’re looking for the death knell of the academy, this is the first clap.
This is much more common than you think
Clearly, the solution is to find a way to properly assess and value the other things academics do besides publish papers. For example, there’s no reason in this modern era that a scholarly blog post should be treated differently than a published paper. It’s just that one is peer reviewed after it’s published (which is a good thing because it gets the ideas out faster).
Another potential explanation (even for the ‘cited as part of the introduction’ element noted above) is that writing many (decent) papers means that many other academics will be involved in reviewing the papers. Later, when those other academics need a citation to the topic, they may be more likely to draw on (i.e. remember) a paper they reviewed than any other paper in the field (assuming that we are referring to general papers in the field rather than groundbreaking ones).
Common sense tells me that hardworking, conscientious people spend more time writing papers, and also spend more time making sure that the papers that they write are decent…