Quantitative measures of the effect of caring for children on research outputs (published papers and citations) have been used by some universities as a tool to address gender bias in academic grant and job applications. In this post Adrian Barnett argues that these adjustments fail to capture the real impacts of caring for children and should be replaced with contextual qualitative assessments.
Knowing how caring for children effects women’s research output would be incredibly useful, because it could potentially make job and grant applications fairer by compensating for “missing” outputs. For example, if caring reduced research output by two papers per year and 50 citations per year, then these could simply be added to the track records of women with children.
Such a simple numerical adjustment is unlikely to be possible, because the impact of caring for children is so varied that the average impact is meaningless. However, in at least one recorded instance, this has not prevented universities from implementing such adjustments in attempts to address gender bias.
To test how applicable such measures might be, in a recent study we tracked the research outputs of 95 female Australian health and medical researchers, examining over 1,000 years of their combined careers. We examined the yearly number of publications and citations, and looked for a change in output after women began caring for children.
We found that average citations counts declined after children, particularly after the second child, which might be due to reduced networking opportunities, which are particularly important to Australian researchers. Supporting this idea we also found that after caring for children women had fewer co-authors outside Australia. As one participant simply stated:
“Child comes first always; this means missed opportunities for travel to research conferences.”
Robust statistical predictions?
Whilst this evidence pointed to a general trend, we also wanted to test the robustness of our statistical models using the simple technique of leaving out each woman in turn and re-calculating the predicted impacts. If the impacts change because of just one woman, then we know our model is unstable. This is exactly what happened, a number of high-achieving women were highly influential, which led us to conclude that trying to find an average impact was a fool’s errand.
This finding corresponds with a previous qualitative study, that examined a formula used by Monash University, which was similarly sceptical about the value of a simple average adjustment, and concluded that it was controversial to ask women how many papers a baby was “worth”.
An average is a poor indicator because the actual impacts will be complex. They will likely depend on (amongst other things) previous research experience, the field they work in, the support from their university and department, the support from their family, the time away from work, and the age of their children.
Previous research that tried to find an average effect has been strikingly heterogeneous, with studies finding both positive and negative effects on research outputs. Again this heterogeneity indicates the futility of searching for an average impact.
The statistical failure to find an average impact does not mean that we should not try to assess the impact of caring for children. The historical ignorance of this issue is likely part of the reason why female researchers are still under-represented in many funding schemes and senior research positions. Adopting a relatively simplistic approach, such as a numerical adjustment, may also deflect attention from addressing wider systemic issues surrounding inequality.
Instead of a numerical adjustment, impact should be assessed qualitatively, such as in funding applications for the Australian Research Council, where researchers can provide a description of career interruptions and how they impacted their research productivity. Researchers might mention the reduced opportunities to travel or the reduced opportunities to join collaborations. This might not just be because of caring for children, but could also include caring for other family members.
A metric for everything and a value for nothing
More generally, these practices reflect a wider obsession with finding numerical formulae to represent complex phenomena that is prevalent in higher education. Two metrics that have been widely criticised are the impact factor for journals and h-index for researchers. The h-index might get used when there are hundreds of candidates for just a few positions. It helps to thin the field down to a manageable workload.
However, research careers are complex and cannot be reduced to simple numbers. No simple formula will ever capture the myriad benefits of research. What formula could capture research that led to changing individual behaviour, improving practice at a local hospital, and reducing health inequality?
By using numeric comparisons such as these, a decision maker may feel they have been scientific, but instead they are being lazy and avoiding the difficult task of comparing our richly diverse research world.
This post is based on the author’s co-authored paper, The impact of caring for children on women’s research output: A retrospective cohort study, published in PloS ONE
Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below
About the author
Adrian Barnett is a statistician and president of the Statistical Society of Australia. He works at the Australian Centre for Health Services Innovation at Queensland University of Technology. He has an NHMRC Senior Research Fellowship in meta-research which uses research to improve research. Adrian’s ORCID iD is 0000-0001-6339-0374, and he can be found on Twitter @aidybarnett.