Scientific progress can accelerate when scientists are less than fully informed about the advances their peers are achieving. The successes of particle physicists in the Soviet Union during the cold war, who worked in isolation from their Western counterparts, furnish a prominent example.

My work shows that even though isolated scientists may have to work on projects whose productivity is less promising, they are also forced to select riskier research projects. Strangely, it is the riskier scientific projects that in the long run can lead to the greatest progress.

One implication is that greater progress can sometimes be achieved when scientists are unaware of each other’s work, something that has become rare in a world where the internet allows new ideas and findings to flow without hindrance.

In the creation of knowledge, risk delivers an inherent benefit. If an innovative scientific project turns out to be a great success, follow-up projects will become available that also have great promise. If, on the other hand, the innovative project is a failure then it can be abandoned.

Since there is a potential upside gain but no downside loss, greater risk brings a net benefit. This benefit is big enough that it will sometimes be efficient for scientists to skip over the projects likely to bring the greatest gains and instead undertake riskier, more innovative alternatives. For the same reason, scientists can gain from a measure of isolation since then it is harder to follow in the footsteps of the work that others are pursuing.

Drawing on the ideas of the physicist Rafael Sorkin, I use the archetype of successful scientific development – particle physics – to illustrate the costs and benefits of scientists working in (a measure of) isolation. The end of the cold war allowed the world’s scientists to form a more unified community, seemingly an unambiguous benefit, but the change also brought a surprising drawback.

By the late 1960s, most particle physicists had rejected quantum field theory and instead followed the latest fashion, the ‘bootstrap model’. But various camps remained out of the loop, and a group of physicists in the Soviet Union – an isolated academic island – pursued a theory of gauge fields that would eventually describe the three fundamental forces in today’s standard model of particle physics. With the triumph of the standard model, the bootstrap model faded away.

The moral of this story is that it can be valuable to have several scientific schools following different lines of research in ignorance of each other’s work. When everyone knows exactly what other researchers are doing and they all judge the value of research projects in the same way, then the pursuit of the projects with the greatest value will lead individuals to herd, with all researchers pursuing similar lines of attack.

In those circumstances, the free flow of information can be harmful. In an alternative history where Soviet scientists were better tuned to Western research, the development of the standard model might have been slower.

A provocative question raised by this research is whether in our age the rapid flow of ideas via the internet might be bringing similar harmful effects. Of course, on the other side of the ledger, the internet has also delivered manifold benefits that aid the progress of science. It has become unusual for scientists to duplicate unknowingly the work that others have undertaken.

After analysing the drawbacks of a system where scientists are guided primarily by the anticipated productivity of their own research projects, I consider alternative ways of organising scientific research.

These include markets for the right to undertake follow-up projects and the pursuit of citations rather than anticipated productivity as a goal for scientists to follow. Citations have the advantage that scientists pay attention not only to what they accomplish but to the research their work might spark. But it is a crude incentive. Citation-seekers will avoid the ‘isolated gem’ projects that, while important, are unlikely to lead to follow-up work. They can also chase too many wild innovations that are bad bets for the scientific community as a whole but that when successful bring enough citations to make the gamble worthwhile for the individual scientist.

In the light of this research, some of the social idiosyncrasies of academia start to make more sense. Consider the scientists who cagily refuse to discuss their work. Even if motivated by paranoia, their secrecy can foster the initiation of new lines of inquiry, a socially productive goal. By maintaining secrecy a scientist will get to publish more of an entire line of research, including follow-up work, and thus get more of the credit that stems from an innovation. This could be beneficial, since if you don’t get the lion’s share of the credit you might not bother to do the work.

♣♣♣

Notes:


Michael Mandler is an economist whose research covers many areas of economic theory: markets, games, preferences, production, social choice, and elections. Before coming to Royal Holloway College, Mandler taught in the economics departments of the University of Pennsylvania and Harvard.