David Carless walks us through his recipe for academic research impact

How might one develop an impactful paper that transcends a modest sub-field and enters the mainstream? What are some of the characteristics of highly cited papers that capture attention in unexpected ways? What is the respective importance of the publication venue, the authors, the topic, and the quality of the research? These are some of the issues I address in this post in attempting to unpack the process of developing and publishing an impactful article.

My research specialism lies in feedback for student learning in higher education, a modest niche within a Cinderella sub-field: higher education pedagogic research. Up until 2017-18, the term ‘feedback literacy’ generated some buzz at higher education conferences, but was infrequently found in the literature. I resolved to fill this space by developing a conceptual article setting out the scope of feedback literacy, and how it could be useful in tackling some of the challenges of feedback practice. The essence of feedback literacy is that learners of all ages need a set of competencies to make the most of feedback opportunities of different kinds.

Key ingredients

I prepared a draft of the paper, presented it at a relevant international conference, and then discussed it with a highly esteemed collaborator, David Boud from Deakin University, Australia, inviting him to join as second author. The involvement of international co-authors  tends to enhance research impact and citations because of the wider geographical spread of dissemination.

The article went through a number of iterations through which we used our broad experience of feedback research to define feedback literacy, and propose a framework for student feedback literacy. This involved synthesizing and re-formulating relevant strands of literature in striving to produce a foundational article: a pioneering attempt to open up and establish principles and practices for a pertinent theme. Adding weight to our endeavors, feedback has been identified as a topic that engenders dissatisfaction among both students and teachers across the higher education sector, so represents a key quality enhancement focal point across numerous disciplines.

Rejections are ubiquitous in academia, and represent an opportunity to reflect, revise, and move forward

The article was rejected by the first target journal with the negative reviewer finding it reliant on creative synthesis by the authors as opposed to using more clearly defined processes of systematic review methodology. Rejections are ubiquitous in academia, and represent an opportunity to reflect, revise, and move forward. Scholars benefit from academic feedback literacy if they are to respond constructively to peer review. Strategies include working with emotions in dealing with rejection, addressing comments from peer reviewers skillfully, and taking informed enhancement actions.

Accordingly, we re-worked the paper via further iterations. We did not change our methodological orientation but added some caveats justifying our approach as well as acknowledging its limitations. We centered on the core arguments, and eliminated interesting side-tracks more suitable for another paper. These processes of rejection, revision, and re-submission offer opportunities to enhance overall argumentation.

What counts

A useful principle is steady enhancement of an article until it finds a suitable home. There is evidence that the more revisions an article undergoes, the more that work contributes to the enhanced impact of the final product. We worked on the paper for eighteen months. After review, further revisions, and final acceptance, the article was published open access in Assessment and Evaluation in Higher Education, the premier journal for assessment and feedback research in HE. If funding is available, publishing open access is an obvious way of increasing readership and potential citations. The paper was published online on 3 May 2018 and came out in the print version six months later.

The metrics for the paper are striking: currently more than 180,000 reads/downloads, making it the most read article in the journal by a wide margin. The citation profile for the article is impressive by the standards of higher education research. More than 500 citations in 2023, and an increase to almost 600 citations in 2024. The overall citation ranking is 6 out of 237,838 articles (top 0.0025%) in the Education category from 2018 to 2023 in the Web of Science database. While citations are not necessarily a proxy for quality, they are an indicator of reach and a commonly used measure of research impact.

Offering a definition of a relatively new or ill-defined term seems to attract citations

A lot of the citations related to the definition of student feedback literacy as “the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies”. Offering a definition of a relatively new or ill-defined term seems to attract citations. Also striking is the multi-disciplinary reach of feedback literacy with sustained pockets of research interest in disciplines as diverse as applied linguistics and medicine.

In terms of academic impact, the graph below represents in blue the number of papers in which ‘feedback literacy’ occurs in the article using Google Scholar data. Superimposed on the graph in orange is the citation trajectory for Carless and Boud (2018). These illustrate steady and significant increases in coverage of feedback literacy.

Building on the above, a heuristic for academic research impact might be:

Plentiful peer review + pertinent topic + international co-authors + impactful journal + open access = citations potential

This heuristic offers a starting point but a paper needs to do more. In the this case study, offering multi-disciplinary appeal beyond the branch of knowledge in which it is conceived seemed to increase impact. The positive reception to the paper and the spread of feedback literacy as a new research sub-area exceeded my most optimistic dreams. Feedback literacy seems to be widely appreciated because it offers a concept that acknowledges the challenges and potentials of making use of feedback inputs of different kinds. And as we look to the future, feedback literacy for productive GenAI use represents a fascinating avenue for further research.

Image: Florian Schmetz/Unsplash

_____________________________________________________________________________________________ This post is opinion-based and does not reflect the views of the London School of Economics and Political Science or any of its constituent departments and divisions.    ______________________________________________________________________________________________