The leading journals for experimental social psychology rely disproportionally on undergraduate students for their findings. James Hartley argues necessary steps need to be taken to widen the sample size to restore credibility to the studies and to the discipline as a whole. Digital communication methods and a firmer commitment to replication studies with different populations could help to improve the situation.
When I began my academic career fifty years ago it seemed almost inconceivable that social psychology could be an experimental science. Designing experiments to simulate social situations was almost unheard of. How things have progressed! Next year the Journal of Experimental Social Psychology will reach Volume 50, and the Journal of Personal and Social Psychology Vol. 105. But I am now disconcerted by something else. Why is it that experimental social psychology is largely based on data gathered from undergraduates? Such students hardly form a normal sample of the population – differing hugely in age, ability and interests – so it seems to me unwise, if not foolish, to limit social psychology to such a base.
Such remarks are not new – they have been made steadily since the 1960s (and they have also been applied to other areas of psychology). But the real question is why has nothing changed? And why are social psychologists so defensive about it, and deny that it is a serious problem?
The following data, gathered over the years by some more concerned colleagues, substantiate my claims and indicate, yet again, that something needs to be done:
Of course many published papers with participants at different ages and abilities appear in their own specialist psychology journal, e.g.: Child Development, Journal of Adolescence, Journal of Applied Psychology, Legal and Criminological Psychology, Psychology of Ageing, Studies in Higher Education, etc. But for the journals shown above that claim to be leaders in the field of psychology, it is somewhat surprising to find that readers are treated mainly to findings derived from undergraduate samples. Furthermore, these undergraduates are often taking part in these studies in order to complete course-requirements – sometimes for credit or some other form of reward.
So I maintain that there is a problem here – and it seems that nothing has been done about it for years. Some possible solutions that might help to alleviate this problem as are follows:
Investigators today can more easily use e-mail and the mailing lists of other groups apart from students to gather participants. Experiments can and are being set up electronically to gather data from a much wider range of respondents.
Editors of social psychology journals might agree to publish a certain proportion of the papers in each issue of their journals if these papers also include the findings of replications of their studies with different populations. It is of interest to note here that as I write the Association for Psychological Science (APS) is developing proposals to launch a new psychology journal devoted to such studies.
Other technological developments might help, for example, as authors in some journals can now electronically link earlier papers to their new replications.
A rather different idea is to suggest that PhD and Masters’ students working with undergraduates include in their theses at least one replication study from a non-student population. The November 2012 edition of Perspectives in Cognitive Science (Vol. 23, No. 11) specifically draws attention to the difficulties of carrying out replication studies satisfactorily, but I submit that postgraduates in (social) psychology would learn a good deal from such practice. Further discussion of these issues can be found by Googling APS Replication Conference.
Finally, of course, we could all go mad and acknowledge the fact that most studies in psychology are WEIRD – that is based on student samples from Western, Educated, Industrialised, Rich and Democratic societies (Henrich, Heine and Norenzayan, 2010). Henrich et al. suggest ways in which we might tackle these more complex problems, but that is a more complicated story…
This article was originally published on LSE’s Impact of Social Sciences blog.
Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics.
James Hartley is Research Professor in the School of Psychology at Keele University, UK
