The Policy Institute at King’s College London, along with colleagues in the digital humanities department, teamed up with technology company Digital Science to build a searchable database and produce a rich analysis of the impact case studies for the 2014 Research Excellence Framework. Saba Hinrichs and Jonathan Grant introduce the key findings of the analysis and explain how the resource […]
The weight placed on p-values and significance testing has come under increasing criticism, with one social psychology journal banning their use entirely. Nicole Radziwill argues that many of the issues come down to sampling errors. Inferential statistics is good because it lets us make decisions about a whole population based on one sample. But inferential statistics is bad if your sample size is […]
Choosing the Right Criteria: Universities need to identify what behaviours should be valued and reward accordingly.
If universities are to thrive, evaluation criteria must capture and recognise exemplary contributions. To discontinue the legacy of poor indicators, Athene Donald calls for the focus on quality to be in more fundamental ways. For example, the mobility of an individual to accept overseas posts and attend international conferences has been frequently used as a proxy for excellence. […]
Interrogating ‘excellence’: Implicit bias in academic promotion decisions perpetuates gender inequality.
The ostensibly objective criteria outlined in many Key Performance Indicators of excellence can become highly subjective and gendered when applied in practice. Pat O’Connor and Clare O’Hagan share the findings of a cross-national project concerned with women’s underrepresentation at senior levels in STEM disciplines. Structural aspects, including the bureaucratisation of the promotion process and the composition of the boards, were seen […]
City University of New York (CUNY) is a public university system throughout New York City and was established to improve access to quality education for a rapidly growing and diverse population. With campuses spread across a vast metropolitan area, tracking and recording the impact of its research and teaching activities is a big task. The Futures Initiative project is […]
Given the enormous amount of new knowledge produced every day, keeping up-to-date on all the literature is increasingly difficult. Peter Kraker argues that visualizations could serve as universal guides to knowledge domains. He and colleagues have come up with an interactive way of automating the visualisations of entire fields along with relevant articles. Through similarity measures identified in a Mendeley-powered data-set, a researcher can see the […]
Volha Piotukh and Simon Lightfoot look at the links between research and teaching in Politics and International Relations and find they are not as visible as one would expect, in part due to them being insufficiently developed institutionally. While departments/schools of the Russell Group universities explicitly discussed the research-teaching nexus, very few of the individual departments demonstrated a variety […]
The failure of a web address to link to the appropriate online source is a significant problem facing scholarly material. Martin Klein and Herbert Van de Sompel together with their collaborators have investigated the extent of reference rot on scholarly domains and their results show an alarming link rot ratio. The authors also explore ways to mitigate it through more systematic […]
What’s the point of open academic data? Redefining valuable research based on re-use and reproducibility.
The benefits of open research to both those who fund it and to wider society mean that academia is on a course which cannot be altered, or returned to a previous state. Mark Hahnel discusses the momentum behind making all research outputs openly available online and what changes may be in store for universities in this transition. Funder mandates could shift […]
Expectations for All: Universities and supervisors have a responsibility to manage PhD career prospects.
Athene Donald highlights an important document published by the Royal Society which addresses the plight of PhD students and their job prospects. The overall shape of the pipeline feeding through from research student to professor is not in doubt. Few make it. Supervisors and institutions can do more to manage expectations and improve job prospects. It is vital that students […]
Season’s Greetings from the Impact Blog! We wish our readers a restful holiday ahead of next year’s research-filled excitement. In 2014 the blog featured a range of evidence-based analysis and fresh perspectives on academic impact, from low citation rates in the humanities to reports of economists accepting sex in exchange for co-authorship. In case you missed them the first time […]
Predicting the results of the REF using departmental h-index: A look at biology, chemistry, physics, and sociology.
Can metrics be used instead of peer review for REF-type assessments? With the stakes so high, any replacement would have to be extremely accurate. Olesya Mryglod, Ralph Kenna, Yurij Holovatch and Bertrand Berche looked at two metric candidates, including the departmental h-index, and four subject areas: biology, chemistry, physics and sociology. The correlations are significant, but comparisons with RAE indicate that […]
Qualitative and quantitative research are fundamentally distinct and differences are paramount to the social sciences
Matt Vidal calls for clear distinctions to be made between qualitative and quantitative research. Using as an example the impartial data generated by surveys, Vidal argues that such quantitative data are fundamentally important, but incomplete. Data based on methods of prolonged engagement with respondents are qualitative, also important, but incomplete. Both are united in their goal of advancing knowledge and […]
Striving for gender equity in science: Conference participation behaviour contributes to gender disparity in academia
The issue of gender equity in science (and other areas of academia) is not new; however, it is remarkably persistent. In a recent paper, Kerry Fanson, Therésa Jones, Matthew Symonds, and Megan Higgie found evidence that women may inadvertently contribute to observed gender disparities in conference presentations through their decision to request lower profile roles. In conjunction with efforts to end gender […]
Stand Up and Be Counted: Why social science should stop using the qualitative/quantitative dichotomy
Qualitative and quantitative research methods have long been asserted as distinctly separate, but to what end? Howard Aldrich argues the simple dichotomy fails to account for the breadth of collection and analysis techniques currently in use. But institutional norms and practices keep alive the implicit message that non-statistical approaches are somehow less rigorous than statistical ones.
Over the past year, I’ve met with many doctoral […]
Evaluation systems need not be perfect: University research assessment and the ongoing quest for simplicity.
In order to get a perfect assessment method, are we at risk of developing systems that are ever more complex and time-consuming? Dorothy Bishop looks at the differences between readily available measures to award research funding and the highly complicated RAE formula. An evaluation system need not be perfect – it just needs to be ‘good enough’ to provide a […]
What is the difference between an impact and an outcome? Impact is the longer term effect of an outcome.
Andrew Harding looks closely at the terminology and theoretical differences between an impact and an outcome in welfare research. Outcomes tend to be pre-defined and can be measured objectively, but the personal experiences and nature of impact is intuitively subjective. A mixed methods approach that focuses on delineating outcomes and exploring impact might be appropriate.
In recent years ‘impact’ and […]