Last week there was an eruption of discussion, debate and acerbic wit surrounding a published article by biologist Neil Hall proposing the use of a new metric for academics to evaluate their performance. Named after celebrity Kim Kardashian, the K-index analyses the popularity of scientists on Twitter against the citation impact of their publications in peer-reviewed journals, suggesting academics should be wary of too high a social media presence and should be more concerned with traditional research outputs. Needless to say, academic Twitter was ablaze and contributed a range of other celebrity-inspired indices compiled at #AlternateScienceMetrics.
Hall’s piece was certainly intended to be a satire (of sorts), but there is a grain of truth to every joke. His paper is filled with assumptions about public engagement, social media and the impact of academic work – all topics we at the Impact blog feel are worthy of more sustained reflection. Here we have compiled three informed responses to the Kardashian Index by Kathryn Clancy, Jean-François Gariépy and Micah Allen which engage directly with power dynamics within the academy, metrics used to assess academic success, and the democratizing nature of social media and which all take the discussion in more productive directions.
Jokes That Don’t Work
Kathryn Clancy, Assistant Professor of Anthropology at the University of Illinois.
Genome Biology published a satirical piece by Neil Hall, and since I’m American and he’s British I don’t find it funny. No wait, it’s that I’m female and he’s male. Or maybe that I’m junior and he’s senior. I’ve got it, it’s because he has a ton of publications (many times the number I have), and I have a ton of Twitter followers (many times the number he has). Meaning, my K-index knocks his out of the park.
Let me back up. You see, Hall created a joke metric he calls the Kardashian Index, which is one’s Twitter followers divided by one’s scientific citations. He writes:
Hence a high K-index is a warning to the community that researcher X may have built their public profile on shaky foundations, while a very low K-index suggests that a scientist is being undervalued.
Hall selected a non-random sample of 40 scientists, only 14 of whom were women. He never explains the methods of how he chose these 40. Hall himself says he “had intended to collect more data but it took a long time and I therefore decided 40 would be enough to make a point. Please don’t take this as representative of my normal research rigor.” Why be rigorous and recruit equally when it’s just social science research, amirite?
In the discussion, Hall observes that the 14 female scientists he sampled tended to have a low K-index, meaning they were underappreciated for their work. He rather nonchalantly throws this bone out there to show he’s on the side of the ladies. Strangely, many ladies have not taken up the bone and its rather meager gristle, a sign, Janet Stemwedel once pointed out, that the joke-teller isn’t as aware of his in-group as he might think.
I feel the need to draw the reader’s attention, again, to the question of how to make jokes work. As Emily Finke pointed out in a conversation with me recently, this joke punches down. How is this punching down? Consider the community. The people with power tend to be the ones who:
- Are dubious of any time spent doing outreach, science communication, education, or social media.
- Are scientists in fields considered more “hard,” like engineering and the physical and (some of the) life sciences.
- Are older, white, and male.
- Have a lot of publications.
- Would have a K-index near or at zero.
That means that a joke intended to problematize how we quantify metrics in academic science should probably punch in that direction. Instead, Hall punches at people with less power, who tend to be:
- Committed to social media outreach.
- Are scientists in fields considered less “hard,” like (some of the) life and the social sciences.
- Younger, less white, and less male.
- Have fewer publications.
- Have a higher K-index than you.
So yes, he’s punching down, and that makes it not funny. There is no dark corner of academic metrics to expose when the people you’re mocking are the ones least well positioned to respond. I would never have gotten that paper published – in a journal with an impact factor of 10.5, no less – because I am one of ones whose profile is built on “shaky foundations.”
All I can do… is blog about it.
This piece originally appeared on the author’s personal blog and is reposted under CC BY-NC.
Scientists on Twitter
Jean-François Gariépy, Postdoctoral Researcher at the Duke Institute for Brain Sciences, Duke University.
Neil Hall from the University of Liverpool has published a very interesting mini-study on scientists and Twitter. He developed a metric that compares the popularity of scientists on Twitter to the impact of their publications within peer-reviewed journals. The metric is called the Kardashian Index, a reference to the fact that Kim Kardashian became wildly popular for no apparent reason, and a wink at those scientists who get Twitter popularity without having accomplished as much as others in their scientific career. Neil Hall is not necessarily critiquing the individuals who use Twitter to their advantage – he simply creates a metric that finds discrepancies between Twitter popularity and scientific popularity. The idea is brilliant, but in my view the short article is based on an incorrect premise. The premise is that science and social media contributions are two fundamentally separate things that can be compared to each other. He writes:
I am concerned that phenomena similar to that of Kim Kardashian may also exist in the scientific community. I think it is possible that there are individuals who are famous for being famous (or, to put it in science jargon, renowned for being renowned). We are all aware that certain people are seemingly invited as keynote speakers, not because of their contributions to the published literature but because of who they are. In the age of social media there are people who have high-profile scientific blogs or twitter feeds but have not actually published many peer-reviewed papers of significance; in essence, scientists who are seen as leaders in their field simply because of their notoriety.
The view that emerges from the article is that what scientists do on Twitter is not science or education – it is self-promotion, shouting, or mindless small talk. I hope that sharing part of my experience on social media will convince you that this is wrong.
Graph from Hall, 2014 with my own point manually added. See the discussion on Twitter that followed.
As I read about the new index developed by Neil Hall, I quickly wanted to know where I would lie on his graph. I was not surprised to find myself within the top of the Kardashian Index. I am a postdoc who started Twitter very early, systematically releasing tweets about recent publications in neuroscience and cognitive science for almost 4 years. I can easily gather statistics about my contribution to Twitter since most of my links are tracked for the number of clicks they generate. Over the last 4 years, I drove tens of thousands of people to peer-reviewed research and PubMed abstracts, responded to hundreds of questions from the general public about the scientific research of other labs, and engaged in direct conversation with experts about current issues in neuroscience research. I also founded NEURO.tv, publishing 1-hour long conversations between experts in neuroscience on YouTube. We do not talk about the weather on NEURO.tv, we talk about what matters to current experts in the many branches of neuroscience.
I propose that we stop seeing social media as a side to science and academia – science is advancing and being disseminated on social media, and therefore Twitter is not different from the conference room of a University where future directions of research are determined, or knowledge is being exchanged. If the attendance at a scientific symposium might be somewhat indicative of its relevance to current scientific research, then we might want to consider that the following of a Twitter account may also be somewhat indicative of the quality and relevance of the information it provides. If we are to recognize that science is not just a list of results from experiments, but that it includes the process by which we think of hypotheses and exchange ideas, then we are bound to accept that social media is an institution where science happens.
I once asked Richard Dawkins how he would summarize his philosophy of science education. He said something along those lines: “Just put the science out there and people will find it interesting.” The more we follow Dawkins’ mantra, the more the line is blurred between science education and science itself. If I discuss with Micah Allen (piece below) about the fascinating question of why thoughts and feelings have evolved on NEURO.tv, we engage in the same thought process that would happen in a laboratory or scientific meeting. As a result of this discussion, if people found it fascinating, I will get more Twitter followers. This is not the result of a defect of the system that needs fixing. It is science, happening right there, in a public, transparent way that was not possible before.
When I discuss with Katherine Bryant on Twitter about the relation between hard sciences and the humanities, the reflection we engage in is similar to that happening in scientific symposiums. When I convince 170 backers, many of whom have PhDs and are professors, to fund a YouTube show about neuroscience on Kickstarter, the process is not so different from that of grant applications for scientific and educational projects, which are reviewed by three experts to determine if they get funded by the government. Scientific experiments can also be funded on crowdfunding platforms, as many have shown. Finally, when I link to a research article from my Twitter account, I am “quoting” it in the same way as I would in a peer-reviewed article. I am acknowledging that I have read it and that it may influence my current thoughts or future experiments.
The popular scientists on Twitter typically have a recipe that led to their success: consistency, sharing of good peer-reviewed research, honest interventions and high-quality information. Those are exactly the characteristics we would be looking for when hiring a lecturer or professor.
I am reminded of a discussion I had with a young scientist for whom I have the greatest respect. I mentioned an idea I had been thinking about for a long time which consisted in creating an hybrid molecular device using a polymerase and a voltage-sensitive channel to record neural activity on DNA strands. This is pure genius, he said. He then asked if I had published the idea in a peer-reviewed journal, and recommended I do so. I then told him that I had already published the idea in a blog post. Oh, was his reaction. It seemed as if the idea was much less exciting to him because it was published in a blog post. This story shows that the snobbish attitude toward everything that is open and transparent is not just affecting old scientists, we are currently training scientists younger than me to think that science is not about the ideas and knowledge it produces, it is about the building of CVs.
Neil Hall shows a discrepancy between science citations and Twitter popularity, but perhaps the resolution to this problem is not in the rejection of Twitter following as a measure of scientific success. Rather, we may want to recognize that, yes, science is now happening on Twitter too. The inclusion of a metric of scientific success based on the number of Twitter followers may sound ludicrous, but other metrics of scientific success are just as ridiculous, so we may simply be bound to acknowledge that, in general, numbers aren’t good at quantifying scientific success. The Kardashian index may be one more case illustrating that any two measures of scientific success will always find some outliers that get classified well in one system and not so well in the other.
Two things seem to be implied by the article. One, that we have a good basis to measure scientific success, or at least that such a measure theoretically exists. Two, that this measure can then be compared to social media profiles to look for discrepancies. I would suggest an alternative way to look at the data presented by Neil Hall. The Kardashian index not only shows the failure of Twitter follower counts to accurately represent scientific success (no one expected that), it shows that science and education are multi-dimensional endeavors, and some individuals find ways to accomplish progress and influence their peers using a different path than that which is expected by academia. These intellectuals simply have found a way to disseminate their knowledge that does not always results in increasing the number of times their research gets quoted in the research articles of the next generation. In other words, perhaps we might want to consider that the number of times a researcher is quoted in future research is not a perfect representation of the value of his contributions to science.
This view reverses the problem identified by Neil Hall: we currently have intellectuals who teach, research and engage in a thought process in the public sphere, to the benefit of their followers, and academia has not yet found a way to evaluate and reward this part of their contribution to society.
This piece originally appeared on BrainFacts and is reposted with the author’s permission.
We the Kardashians are Democratizing Science
Micah Allen, Postdoctoral Fellow at University College London.
I had a good laugh this weekend at a paper published to Genome Biology. Neil Hall, the author of the paper and well-established Liverpool biologist, writes that in the brave new era of social media, there “is a danger that this form of communication is gaining too high a value and that we are losing sight of key metrics of scientific value, such as citation indices.” Wow, what a punchline! According to Neil, we’re in danger of forgetting that tweets and blogposts are, according to him, the worthless gossip of academia. After all, who reads Nature and Science these days?? I know so many colleagues getting big grants and tenure track jobs just over their tweets! Never mind that Neil himself has about 11 papers published in Nature journals – or perhaps we are left to sympathize with the poor, untweeted author? Outside of bitter sarcasm, the article is a fun bit of satire, and I’d like to think charitably that it was aimed not only at ‘altmetrics’, but at the metric enterprise in general. Still, I agree totally with Kathryn Clancy that the joke fails insofar as it seems to be ‘punching down’ at those of us with less established CVs than Neil, who take to social media in order to network and advance our own fledgling research profiles. I think it also belies a critical misapprehension of how social media fits into the research ecosystem common among established scholars. This sentiment is expressed rather precisely by Neil when discussing his Kardashian index:
In an age dominated by the cult of celebrity we, as scientists, need to protect ourselves from mindlessly lauding shallow popularity and take an informed and critical view of the value we place on the opinion of our peers. Social media makes it very easy for people to build a seemingly impressive persona by essentially ‘shouting louder’ than others. Having an opinion on something does not make one an expert.
So there you have it. Twitter equals shallow popularity. Never mind the endless possibilities of having seamless networked interactions with peers from around the world. Never mind sharing the latest results, discussing them, and branching these interactions into blog posts that themselves evolve into papers. Forget entirely that without this infosphere of interaction, we’d be left totally at the whims of Impact Factor to find interesting papers among the thousands published daily. What it’s really all about is building a “seemingly impressive persona” by “shouting louder than others”. What then does constitute effective scientific output, Neil? The answer it seems – more high impact papers:
I propose that all scientists calculate their own K-index on an annual basis and include it in their Twitter profile. Not only does this help others decide how much weight they should give to someone’s 140 character wisdom, it can also be an incentive – if your K-index gets above 5, then it’s time to get off Twitter and write those papers.
Well then, I’m glad we covered that. I’m sure there were many scientists or scholars out there who amid the endless cycle of insane job pressure, publish or perish horse-racing, and blood feuding for grants thought, ‘gee I’d better just stop this publishing thing entirely and tweet instead’. And likewise, I’m sure every young scientist looks at ‘Kardashians’ and thinks, ‘hey I’d better suspend all critical thinking, forget all my training, and believe everything this person says’. I hope you can feel me rolling my eyes. Seriously though – this represents a fundamental and common misunderstanding of the point of all this faffing about on the internet. Followers, impact, and notoriety are all poorly understood side-effects of this process; they are neither the means nor goal. And never mind those less concrete (and misleading) contributions like freely shared code, data, or thoughts – the point here is to blather and gossip!
While a (sorta) funny joke, it is this point that is done the most disservice by Neil’s article. We (the Kardashians) are democratizing science. We are filtering the literally unending deluge of papers to try and find the most outrageous, the most interesting, and the most forgotten, so that they can see the light of day beyond wherever they were published and forgotten. We seek these papers to generate discussion and to garner attention where it is needed most. We are the academy’s newest, first line of defense, contextualizing results when the media runs wild with them. We tweet often because there is a lot to tweet, and we gain followers because the things we tweet are interesting. And we do all of this without the comfort of a lofty CV or high impact track record, with little concrete assurance that it will even benefit us, all while still trying to produce the standard signs of success. And it may not seem like it now – but in time it will be clear that what we do is just as much a part of the scientific process as those lofty Nature papers. Are we perfect? No. Do we sometimes fall victim to sensationalism or crowd mentality? Of course – we are only fallible human beings, trying to find and create utility within a new frontier. We may not be the filter science deserves – but we are the one it needs. Wear your Kardshian index with pride.
This piece originally appeared on NEUROCONSCIENCE and is reposted with the author’s permission.
Note: This article gives the views of the authors, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
Featured and top right image credit: Jason Howie (Flickr, CC BY)
Micah Allen is a Post Doctoral Researcher at University College London. His research primarily investigates the alteration and training of consciousness from neural, philosophical, and psychological perspectives.
Kathryn Clancy is Assistant Professor of Anthropology at the University of Illinois.
Jean-François Gariépy is a postdoctoral researcher at the Duke Institute for Brain Sciences, Duke University. His research interests include the neural control of social behaviors in groups of macaques and humans. Dr. Gariépy is also an active public science educator, teaching about brain research through the TV show that he founded, NEURO.tv as well as through his blog. He is also preparing a book called The Moral Signal, which will describe the biological basis and evolution of human social and moral behaviors.