Improved research sharing practices will undoubtedly help to boost the visibility of research. Richard Price, CEO of Academia.edu, explains how their social media platform is looking to incentivise data sharing by providing an outlet for researchers to share their data and code in a way that also enhances their reputations. 

Last week, Academia.edu announced that users can embed data-sets and code onto their Academia.edu profile pages. Data-sets and code can be attached to papers, or can be uploaded in a stand-alone way. Historically researchers have only shared their ideas in the form of academic papers. The DNA of academic journals came from the era of print, and it never made sense to share data and code in print form. Currently 75% of the world’s scientific data is not shared. It hasn’t been there because the distribution platforms haven’t been there, and there haven’t been the right reputation metrics to incentivize researchers to share their data.

Academia.edu’s announcement is providing an outlet for researchers to share their data and code in a way that enhances their reputations. Data-sets and code are attached to Academia.edu’s analytics engine. You can see how many views you get for your data-sets and code, and share these analytics with your tenure and grant committees.

Below is a screenshot of an embedded data-set:

Below is a screenshot of an embedded Github repo:

The importance of the sharing of data was highlighted in the media a couple of weeks ago. A couple of Harvard professors wrote an influential economics paper on national debt and growth ratios. The paper was circulated in 2009 and it had a significant impact on the policy decisions of governments around the world.

Earlier this year a graduate student asked the authors of the paper for the data-set that backed up the paper. After looking at the data-set he found an error that undermined the conclusions of the paper. Had the data been shared with this paper’s publication, the error would have been caught immediately, before it had a chance to impact the various countries’ economic policies.

History of the Science Ecosystem

400 years ago, journals had not been invented yet, and research was largely a private pursuit. Wealthy people would have private labs in their country houses, and they would keep the results of their experiments private. There was not a strong cultural norm around sharing your scientific ideas or results.

Journals were invented for the sharing of ideas towards the end of the 1600s. This sharing infrastructure helped spur the Scientific Revolution, a rapid acceleration of scientific progress.

As much as 50% of the world’s research output may not be being shared right now, because the incentives haven’t encouraged the form that the output comes in. These forms can include data, code, comments on papers, images and videos.

Academia.edu’s Mission

Part of Academia.edu’s mission is to build the incentive engine for researchers to get credit for sharing the full range of their research output: closing the feedback loop, so if a contribution they make to research has an impact, there are metrics that reflect that impact. The researcher can take those metrics and use them to improve their chances with grant and tenure committees. This announcement is part of building the new infrastructure in research, where researchers can collect credit for sharing more and more of their output.

Users’ thoughts

Some users were in the beta for this feature on Academia.edu, and they added their thoughts.

Shivendra Tewari, a Biology post-doc at the Medical College of Wisconsin, writes “I see sharing data as an advancement of science. If I’ve already done something, why should someone re-do all of the work again. They should just use whatever I’ve done and then move forward from that point.

I think it’s really good that you can provide things like code and datasets on Academia.edu because sometimes publishers don’t even ask for code. So if there is one single place where you can put papers and code, then people can get a lot of information from a single site.”

Murray Rudd, a Lecturer in the Environment department at the University of York, writes “Looking at it from the environment and economics realm, there’s generally not enough sharing of datasets. I have datasets that go back 10 or 12 years, and I always have these good intentions to get students working on them at some point. But unfortunately due to time constraints, a lot of these datasets just die out— they are never really plumbed to the extent that they could be.

Most of the people that I work with are in the same situation. So, in the case of one of my datasets, I thought why not just put it up online and if someone can use it sometime then that’s great. Also, even if I know that it’s going to take me a while to analyze the data, it still doesn’t hurt to post it online. If someone else picks up my data and publishes a paper, I’ll still get cited.”

Daniel Curtis, a History post-doc at Utrecht University, writes “I think the more material you have on Academia.edu, the more ‘visible’ you are. You are more likely to be found through search engines that way. Perhaps by arriving at my page by accident through a bibliographic reference, someone might see one of my papers and become interested.

“I do also believe in the sharing of data though. I think the future of the historical discipline is not through individual research but research in teams with international collaboration. This is just a small way of contributing to that. Plus it doesn’t really take much effort to just put a file on academia.edu, so there’s no reason not too!”

This was originally posted on the Academia.edu blog and is reposted with permission.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics.  

About the Author

Richard Price is the CEO of Academia.edu. He did a Ph.D at Oxford in philosophy, where he was a Fellow of All Souls College. His thesis was in the philosophy of perception. Prior to that, he did a BA at Oxford in philosophy, politics, and economics.

Having first documented the large-scale demise of the impact factor as a predictor of quality research, George Lozano and team examined whether this pattern also applies to the handful of elite journals. His recent study finds the proportion of top papers published by elite journals has in fact been in steady decline since the late-eighties. Journal hierarchies are breaking down and researchers will benefit from the many publishing venues now available to reach wider audiences.

The digital age has brought forth many changes to scholarly publishing. For instance, we now read papers, not journals. We used to read papers physically bound with other papers in an issue within a journal, but now we just read papers, downloaded individually, and independently of the journal. In addition, journals have become easier to produce. A physical medium is no longer necessary, so the production, transportation, dissemination and availability of papers have drastically increased. The former weakened the connection between papers and their respective journals; papers now are more likely to stand on their own. The latter allowed the creation of a vast number of new journals that, in principle, could easily compete at par with long-established journals.

In a previous blog, and paper, we documented that the most widely used index of journal quality, the impact factor, is becoming a poorer predictor of the quality of the papers therein. The IF already had many well documented and openly acknowledged problems, so that analysis just added another problem to its continued. The data set used for that analysis was as comprehensive as possible, and included thousand of journals. During subsequent discussions, the issue came up of whether the patterns we documented at a large scale also applied to the handful of elite journals that have traditionally deemed to be the best.

Hence, in a follow-up paper we examined Nature, Science Cell, Lancet, NEJM, JAMA and PNAS (just in case, the last 3 are New Engl. J. Med., J. Am. Med. Ass., and Proc. Natl. Acad. Sci.). We identified the 1% and 5% most cited papers in every year in the past 40 years, and determined the percentage of these papers being published by each of these elite journals. In all cases, except for JAMA and the Lancet, the proportion of top papers published by elite journals has been declining since the late-eighties.

To account for the fact that more papers are now being published than 40 years ago, a normalized index was used. An index of 1 indicates that the number of highly cited papers published by a journal is what would be expected by chance. So, for example, 1% of that journal’s papers that year would be among the among the 1% most cited papers. An index of 7 indicates that a journal published 7 times as many top papers as would be expected by mere chance.

Using a normalized 1% index a more complex pattern emerged.  Some journals, like Cell and PNAS had lost ground in that past 20 years. Nature and Science had remained relatively at the same level, and NEJM, Lancet and JAMA had increased their share up to about 2005, at which time Lancet continued to rise, but the other two noticeably dropped.

This normalized 1% index was also applied to several emerging journals. Most had increased their relative share of highly cited papers in the past 20 years. Chemical Reviews had a high but highly variable index. Finally PLoS One, despite having a high number of highly cited papers, had a normalized 1% index of barely above one. This means that considering the number of papers they publish, their proportion of top papers is roughly what would be expected by chance.

The digital age has brought new challenges and opportunities for journals, researchers and evaluators. Papers are now independent of the journal in which they appear, so researchers will benefit from knowing that many publishing venues are now available, and they do not differ much in their ability to reach a wide audience. Journals have been affected in different and complex ways, which suggests that their relative position in the journal hierarchy is highly dependent on the effectiveness of their past and future editorial, advertising and marketing policies. Finally, advancement, recruitment and grant-evaluation committees, administrators and other evaluators should be aware even if journal hierarchy and reputation were a valid measure of the papers therein, the hierarchy is highly variable, so identifying quality work is no longer as simple as checking the journal names where the research is published.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics.  

About the Author

George Lozano received a B.Sc. from the University of Guelph, earned an M.Sc. from the University of Western Ontario, and holds a Ph.D. from McGill University. He was subsequently awarded FCAR postdoctoral Fellowship which he took to UC Riverside, and an NSERC postdoctoral fellowship, which he took to Simon Fraser University. Since then he has taken several teaching and research positions in three continents, in a concerted effort to add to his multi-cultural experiences. George’s main research deals with the evolutionary, behavioural, and physiological ecology of animals, mostly birds. Along with the empirical research, some of his recent work deals with the evolution and maintenance of multiple sexual signals, the adaptive explanation behind anorexia, evolutionary medicine, research policy and bibliometrics

Leslie Madsen-Brooks is skeptical about the kind of disruption start-ups and tech folks promise. She highlights ways university faculty and staff are already driving thoughtful technological innovation through engaging in open source, open learning projects. Projects which focus on the individual and collective empowerment of students and communities, rather than commercialization will ensure lasting, productive disruption.

I’ve heard that higher ed needs to be “disrupted” because it’s not cost efficient, it treats students as learners rather than customers, it’s risk-averse and unproductive, it values research over teaching, it doesn’t offer enough flexibility to adult learners, it’s too focused on prestige and credit hours instead of broad-based student competencies, it’s done a lousy job of using technology to expand affordable access to degrees, faculty spend too much classroom time lecturing and faculty act as if we should be exempt from the sweeping technological change that has upended the newspaper and music industries.

I’m not opposed to disruption; rather, I’m skeptical about the kind of disruption start-ups and tech folks promise: “paradigm-shifting” technology that improves university teaching and learning. The truth is, many of these start-ups clearly have no idea what actually works in higher ed and know little about the direction university teaching and learning have moved in the last 10 years, because they’re trying to take us backward, not forward. Start-up and commercial tech are certainly proving disruptive—just in all the wrong ways.

But this is a not (merely) a post complaining about bad technology. Instead I want to highlight the ways university faculty and staff are driving thoughtful technological innovation. These are people who intimately understand students’ needs and the faculty’s interests, tech skills and psychology. And although there are some acclaimed universities launching projects with the aim of spawning start-ups or transferring commercial technology to industry, I want to showcase a few projects that take the opposite track: they’re innovative, but they tend to rely on open source technologies, and their focus is on individual and collective empowerment of students and communities, rather than commercialization.

Those who have been paying attention only to partnerships among Silicon Valley companies and the Ivies may be surprised that the beating heart of a tremendous amount of academic technology innovation is a small state university in Fredericksburg, Virginia. At the University of Mary Washington, the Division of Teaching and Learning Technology has launched at least four amazing initiatives that should be replicated widely because it’s clear to even casual observers that they advance teaching and learning in myriad ways. For one, evidence of student learning appears on the open web, and I encourage you to check out the current blogs developed for courses. Faculty, too—and I know this from first-hand experience—benefit from knowing what students are thinking (as expressed in blog posts and comments) before they convene for class.

Several years ago, UMW’s DTLT premiered UMW Blogs, termed “the Bluehost experiment” by the DTLT staff because in its first iteration, it was little more than a WordPress Multi-User installation on an inexpensive ($6.95 per month) shared server at Bluehost. Today, any UMW student, faculty, or staff can set up a blog for class or personal use on UMW Blogs—and 500 courses have been brought onto the platform since fall 2008.  Anyone can browse the courses using UMW Blogs or discover all kinds of non-course blogs by exploring the latest posts featured on the home page. The UMW archives, for example, recently put online a series of lectures by the late civil rights leader James Farmer, and Jess Rigelhaupt’s Oral History class has created Rosie the Riveter, an excellent resource that includes “firsthand accounts of what people experienced on the American home front during World War II.”

Next to emerge from this innovation engine was DS 106, an open course on digital storytelling, originally taught by Jim Groom, but since taught by several different instructors, including noted ed tech thought leaders and innovators Martha Burtis and Alan Levine, and recently by instructors at other universities as well. Because of the strong networks of the instructors and students, DS 106 took on a life of its own, with students—both those enrolled at UMW and those following the course from elsewhere—providing daily fun assignments (“the Daily Create”) that stimulate participants’ creativity and stretch their technological savvy. DS 106 spawned ds106 radio, a free-form, streaming broadcast for which anyone could volunteer to provide content.  How popular is DS 106 and its apparently endless stream of creative multimedia content? In spring 2012, Groom launched a Kickstarter campaign to fund a better web server for DS 106, and the campaign raised 600% of its goal in just a few days, providing funding for all kinds of course improvements and expansions.  While Kickstarter provided private funds for this project, I’m excited about this kind of crowdsourced funding—although I’d be even more enthusiastic about greater public funding—because it allows project creators greater future freedom than would, say, funding from investors whose motive is more likely to be profit than pedagogical revolution.

Springing next from the mind of the DTLT geniuses was Domain of One’s Own, in which each first-year student at UMW receives a domain name and space on a web server. The project encourages each student  to “reclaim the web” by “taking control of your digital identity,” gathering its artifacts “in a central place that you own and control.” And it’s offered in collaboration with the university’s Office of Information Technology Services:

The pilot gave 400 students and faculty their own domain name and web space to install a portfolio of work or map onto existing systems. In Fall of 2013 every incoming student at UMW will have the opportunity to choose their own domain and receive a web hosting account with the freedom to create subdomains, install any LAMP-compatible software, setup databases, email addresses and carve out their own space on the web that they own and control.

Then, as if granting students this creative freedom and technical autonomy wasn’t enough, this spring UMW launched Thinklab, a makerspace. According to its About page:

ThinkLab is the exciting new makerspace located in the Simpson Library at the University of Mary Washington. As a collaboration between the Division of Teaching and Learning Technologies, the College of Education, and the Library, ThinkLab hosts a variety of emerging technologies and tools for students and faculty across all disciplines. 3D printing, robotics, and electronics work using Arduinos and simple breadboard kits are just some of the many exciting things happening at ThinkLab.

The innovations and—yes, I’ll say it—disruptions, emerging from UMW exemplify some of the best practices in developing communities of learners, fostering collaboration, encouraging writing and reflection and developing curiosity about the world. Channeling George Kuh, Randall Bass emphasizes that such “high-impact practices” lead to “meaningful learning gains” as well as “high retention and persistence rates” because they encourage these specific behaviors:

• Investing time and effort
• Interacting with faculty and peers about substantive matters
• Experiencing diversity
• Responding to more frequent feedback
• Reflecting and integrating learning
• Discovering relevance of learning through real-world application

In an age when universities are pushing faculty ever harder to develop monetizable intellectual property, it’s refreshing to see faculty doubling down on using relatively inexpensive technologies to improve student learning. UMW is a case in point: it’s a modestly funded, small state university that, thanks to all the active minds (and periodic strategic hires) at DTLT and on the faculty, has become a major hub of innovation in higher education. It joins other cutting-edge departments and programs launched by other Virginia institutions, including the University of Virginia’s Scholars’ Lab and the Virginia Center for Digital History, as well as George Mason University’s Roy Rosenzweig Center for History and New Media, whose staff and fellows have created not only a lot of terrific curricular resources, but also Zotero, Omeka, ScholarPress,PressForward, and the globally popular THATCamp. It’s amazing how much scholars, programmers and others have accomplished in such a short time—and all without spinning off start-ups as seems to be so fashionable in higher education today.

This is the kind of disruption I’d like to see at more universities, especially out here in the Intermountain West, Pacific Northwest and Great Basin. That’s going to be difficult because, in Idaho at least, we aren’t developing or attracting people with the programming training to do this kind of work. Still, we can go a long way using inexpensive but high-quality, open-source tools.  And in fact, in my teaching, I have relied on a number of open-source tools, including WordPress and LocalWiki, and (to a lesser extent) Sakai, an increasingly robust alternative to the unwieldy course management system Blackboard. I require my students to create digital products and imagine new digital services they might provide, and I teach them about Creative Commons and the public domain.

I admit I feel a good deal of pride that this movement toward open source, open access learning founded on creative uses of inexpensive technologies is driven by digital humanists, faculty, librarians and academic technologists—including some people who manage to occupy all of those fields simultaneously. If you follow any of these innovators on Twitter or read their blogs, you can see their conversations and collaborations unfold, illustrating, as Scott Leslie points out, that disruption emerges from networks that enable open learning. Their collaborations and projects are excellent case studies of why Jon Boeckenstedt’s term “punctuated equilibrium” makes more sense than “disruption” when discussing changes in the digital landscape of higher ed.

I’m no detractor of entrepreneurship; I encourage my public history graduate students to make their own way in the world, and if I wasn’t so busy with my faculty responsibilities, I’d dabble in it myself. But what if, instead of investing so much time, effort and money in start-ups, MOOCs, lecture capture, unwieldy learning management systems, overzealous intellectual property protections and the like, we redoubled our efforts in open access, open learning and open source? These are the efforts that would prove truly disruptive of business-as-usual at the university.

Of course, I’m not the only one thinking along these lines. Aaron Bady muses on what makes a good MOOC (hint: it’s open and free), and then points out that what most folks are talking about when they invoke “disruption” is a further corporatization of the university:

So I want to shift the debate a bit. [Clay] Shirky thinks in terms of “disruption” and what can come of it, in theory. I think in terms of what the “disruption” of the University of California system looks like in practice, as a complex of politicians, financiers, and career administrators move in lock-step to transform it into a self-sufficient corporate entity, and to enrich private industry in the bargain. I see a group of decision-makers who quite manifestly do not know what they are talking about and who barely try to disguise it, for whom “online” is code word for privatization. If I am against MOOC’s, I am against the way “MOOC” is being experienced in California, in practice: as an excuse to cheapen education and free the state budget from its responsibility to educate its citizenry.

There’s little need to hire Udacity or Coursera or any other ed tech company to disrupt higher education because faculty and staff representing key nodes in the network are already evolving the theory and practice of teaching, learning, research and outreach in ways that are incredibly productive, if not always recognized. Take a moment to explore some of the projects and networks I’ve discussed here and then ask yourself: who exactly is so invested in interrupting this productive disruption, and why? Why are universities considering spending $2 million to affiliate with a MOOC provider, when tremendous faculty creativity and the $6.95 Bluehost experiment are at hand?

This was first posted at The Blue Review, Boise State University’s journal of popular scholarship, and is republished with permission.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics.  

About the Author

Dr. Leslie Madsen-Brooks joined the Department of History at Boise State University in 2010 as an Assistant Professor. She came to Boise State with a broad backgroundincluding work in American, museum, and technocultural studies, culminating with a Ph.D. in Cultural Studies from the University of California, Davis.

Peer review may be favoured as the best measure of scientific assessment ahead of the REF, but can it be properly implemented? Peter Coles does the maths on what the Physics panel face and finds there simply won’t be enough time to do what the REF administrators claim. Rather, closed-access bibliometrics will have to be substituted at the expense of legitimate assessment of outputs. 

What I want to do first of all is to draw attention to a very nice blog post by a certain Professor Moriarty who, in case you did not realise it, dragged himself away from his hiding place beneath the Reichenbach Falls and started a new life as Professor of Physics at Nottingham University.  Phil Moriarty’s piece basically argues that the only way to really judge the quality of a scientific publication is not by looking at where it is published, but by peer review (i.e. by getting knowledgeable people to read it). This isn’t a controversial point of view, but it does run counter to the current mania for dubious bibliometric indicators, such as journal impact factors and citation counts.

The forthcoming Research Excellence Framework involves an assessment of the research that has been carried out in UK universities over the past five years or so, and a major part of the REF will be the assessment of up to four “outputs” submitted by research-active members of staff over the relevant period (from 2008 to 2013). Reading Phil’s piece might persuade you to be happy that the assessment of the research outputs involved in the REF will be primarily based on peer review. If you are then I suggest you read on because, as I have blogged about before, although peer review is fine in principle, the way that it will be implemented as part of the REF has me deeply worried.

The first problem arises from the scale of the task facing members of the panel undertaking this assessment. Each research active member of staff is requested to submit four research publications (“outputs”) to the panel, and we are told that each of these will be read by at least two panel members. The panel comprises 20 members.

As a rough guess let’s assume that the UK has about 40 Physics departments, and the average number of research-active staff in each is probably about 40. That gives about 1600 individuals for the REF. Actually the number of category A staff submitted to the 2008 RAE was 1,685.57 FTE (Full-Time Equivalent), pretty  close to this figure. At 4 outputs per person that gives 6400 papers to be read. We’re told that each will be read by at least two members of the panel, so that gives an overall job size of 12800 paper-readings. There is some uncertainty in these figures because (a) there is plenty of evidence that departments are going to be more selective in who is entered than was the case in 2008 and (b) some departments have increased their staff numbers significantly since 2008. These two factors work in opposite directions so not knowing the size of either it seems sensible to go with the numbers from the previous round for the purposes of my argument.

There are 20 members of the panel so 6400 papers submitted means that, between 29th November 2013 (the deadline for submissions) and the announcement of the results in December 2014 each member of the panel will have to have read 640 research papers. That’s an average of about two a day…

It is therefore blindingly obvious that whatever the panel does do will not be a thorough peer review of each paper, equivalent to refereeing it for publication in a journal. The panel members simply won’t have the time to do what the REF administrators claim they will do. We will be lucky if they manage a quick skim of each paper before moving on. In other words, it’s a sham.

Now we are also told the panel will use their expert judgment to decide which outputs belong to the following categories:

• 4*  World Leading
• 3* Internationally Excellent
• 2* Internationally Recognized
• 1* Nationally Recognized
• U   Unclassified

There is an expectation that the so-called QR  funding allocated as a result of the 2013 REF will be heavily weighted towards 4*, with perhaps a small allocation to 3* and probably nothing at all for lower grades. The word on the street is that the weighting for 4* will be 9 and that for 3* only 1. “Internationally recognized”  will be regarded as worthless in the view of HEFCE. Will the papers belonging to the category “Not really understood by the panel member” suffer the same fate?

The panel members will apparently know enough about every single one of the papers they are going to read in order to place them  into one of the above categories, especially the crucial ones “world-leading” or “internationally excellent”, both of which are obviously defined in a completely transparent and objective manner. Not. The steep increase in weighting between 3* and 4* means that this judgment could mean a drop of funding that could spell closure for a department.

We are told that after forming this judgement based on their expertise the panel members will “check” the citation information for the papers. This will be done using the SCOPUS service provided (no doubt at considerable cost) by Elsevier, which by sheer coincidence also happens to be a purveyor of ridiculously overpriced academic journals.  No doubt Elsevier are on a nice little earner peddling meaningless data for the HECFE bean-counters, but I have no confidence that they will add any value to the assessment process.

There have been high-profile statements to the effect that the REF will take no account of where the relevant “outputs”  are published, including a pronouncement by David Willetts. On the face of it, that would suggest that a paper published in the spirit of Open Access in a free archive would not be disadvantaged. However, I very much doubt that will be the case.

I think if you look at the volume of work facing the REF panel members it’s pretty clear that citation statistics will be much more important for the Physics panel than we’ve been led to believe. The panel simply won’t have the time or the breadth of understanding to do an in-depth assessment of every paper, so will inevitably in many cases be led by bibliometric information. The fact that SCOPUS doesn’t cover the arXiv means that citation information will be entirely missing from papers just published there.

The involvement of  a company like Elsevier in this system just demonstrates the extent to which the machinery of research assessment is driven by the academic publishing industry. The REF is now pretty much the only reason why we have to use traditional journals. It would be better for research, better for public accountability and better economically if we all published our research free of charge in open archives. It wouldn’t be good for academic publishing houses, however, so they’re naturally very keen to keep things just the way they are. The saddest thing is that we’re all so cowed by the system that we see no alternative but to participate in this scam.

Incidentally we were told before the 2008 Research Assessment Exercise that citation data would emphatically not be used;  we were also told afterwards that citation data had been used by the Physics panel. That’s just one of the reasons why I’m very sceptical about the veracity of some of the pronouncements coming out from the REF establishment. Who knows what they actually do behind closed doors?  All the documentation is shredded after the results are published. Who can trust such a system?

To put it bluntly, the apparatus of research assessment has done what most bureaucracies eventually do; it has become  entirely self-serving. It is imposing increasingly ridiculous administrative burdens on researchers, inventing increasingly arbitrary assessment criteria and wasting increasing amounts of money on red tape which should actually be going to fund research.

And that’s all just about “outputs”. I haven’t even started on “impact”….

This was originally posted on Peter Coles’ personal blog and is reposted with permission.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics.  

About the Author

Peter Coles is Professor of Theoretical Astrophysics and Head of the School of Mathematical and Physical Sciences at the University of Sussex. His research is in the area of cosmology and the large-scale structure of the Universe.

Evidence-based social policy depends on access to rich supplies of high-quality data. But how can we create, curate, enrich and reuse data already collected by government departments and researchers? James Nazroo and Matthew Woollard of the UK Data Service explore the network of trust and expertise that ensures a cost-effective pipeline of productive, policy-relevant data.

James Nazroo, a Deputy Director of the UK Data Service writes from a researcher’s point of view:

The launch of the UK Data Service signals a step-change in the way we use and reuse the products of our research. It is about making high-quality data (of all types) easy to get hold of, as easy as possible to use, and providing support for the use of such data. And, by providing an exemplar, it is also about encouraging and supporting others to set up ‘data stores’ that provide easy access to data either directly or through the UK Data Service. Doing this is not straightforward, taking the efforts of a large number of people and involving significant funds. So it is worth thinking about why it is important.

I’m an active researcher, involved in running data collections (most notably the English Longitudinal Study of Ageing, or ELSA) and in using data produced by others (for example, this will be a major element of the Centre on the Dynamics of Ethnicity, or CoDE). The UK Data Service is crucial to such research activities, as was its predecessor. The millions of pounds invested in data collections like ELSA are best used to generate data that a wide range of academic, policy and lay analysts can get their hands on. The ELSA data are being used to provide evidence on a wide range of issues — locally, nationally, internationally, short- and long-term — because they are accessible (as well as high quality). A recent IFS working paper exploring how the 2008-09 financial downturn affected older households in England is just one rich example.

Of course academics typically design data collections to address our particular research concerns. If we plan for wider use we must understand and design for others, and to do this we need to involve a broad constituency. For ELSA this has meant consulting with a wide range of disciplines and with policy analysts in a number of sectors, but also consulting internationally so a cross-national research agenda could be supported. A by-product of designing for a wide constituency and making data easily accessible is that our data outputs may be used by academic rivals, but to do otherwise would be wasteful. The longstanding social sciences culture of sharing data – making data accessible and usable – is one to value and one that is at the heart of the UK Data Service.

Race and ethnicity in Philadelphia, USA. Photo by Eric Fischer. CC-BY-SA

This is clearly demonstrated in the work we plan for CoDE. A central theme in our work is the proposition that the changing ways in which ethnicity is categorised reflect changing meanings of ethnicity – which identities become relevant in particular periods and contexts, why they are relevant, and how they are lived and racialised – and changing patterns of inequality. Following from this, to understand ethnicity now we need to also understand how it has ‘evolved’. So in CoDE we are planning to use data generated from the 1950s to the present day – local surveys, national studies, census data, etc. Almost none of these data have been generated by the CoDE team. So our exciting (at least from my perspective) agenda would not be possible without the past and current efforts of researchers to make their data available, and the current efforts of the UK Data Service to make data accessible and to support their use.

Support is important, because secondary use of data is not without problems. Some of these problems are obvious. For example, survey questionnaires may not adequately cover the concepts needed for the research, or sample designs might not be quite right, both of which might result in research agendas being modified. But there are also problems in getting to grips with and understanding the details of the methods used to generate the data and the implications of these for planned analysis. This is another area where the UK Data Service excels, not only making data accessible, but also providing support for their use.

Matthew Woollard, Director of the UK Data Service, adds:

James makes a convincing case for sharing research data and the support necessary to ensure that data can be used by researchers not involved with its collection. But what most researchers aren’t aware of is the infrastructure necessary to make digital data available and reusable over time. This is where archivists and digital preservation specialists play an important role.

A key part of the UK Data Service is to ensure the data received from researchers, survey subjects and government departments are formatted, contextualised and enriched in a way which ensures they are as usable as possible to researchers, while any curatorial decisions retain the purity of the original deposited data. These curation activities must also remain transparent to the data producers and users to ensure the archival phase of the data lifecycle is valued and trusted. Essentially, there must be a trust relationship between all players in these activities: researchers must trust that archivists are giving them the ‘right’ data, and data owners and producers must trust that the archivists are not damaging the integrity of their data.

Broadly, this is what the UK Data Service does every day. While many of our behind-the-scenes processes may be invisible to researchers, they are foundations necessary to support the needs of data producers, data users, funders and policy makers. In contrast with some of the giant datasets (such as earth observation satellites), social science data require more than simply looking after the bits and ensuring that file formats remain readable over time. An organisation hoping to support continued access to resources while keeping them understandable must be prepared to provide sufficient context not only in terms of documentation but also by defining how the data are related to each other.

The UK Data Service is based around a functional model, which in turn is based on the Open Archival Information System (an ISO standard). This means that we can work with proper standards for archiving digital materials building trust relationships. Over and above the need for a trusted archival storage system is to guarantee data integrity. The archival terms ‘fixity’ (assurance that any content alterations are accurately documented), ‘context’ and ‘provenance’ are critical for data to be used appropriately, whether for secondary analysis or the validation and replication of results. In addition, archives play a key role in tagging digital objects with unique and persistent identifiers (such as DOIs), which supports unambiguous reference and citation, simplifies the assignment of credit where it’s due (at both the data level and the publication level), and allows us to further enrich the environment of the data through appropriate linkage.

It is through the development of these supporting structures around the data — and crucially, making the data available as widely as safely possible through the application of clear rights and access — that trust and transparency are achieved. Many of these critical concepts are created and managed based on recognised metadata standards;  at the UK Data Service, that includes the Data Documentation Initiative (DDI). It is generally acknowledged that the best time to generate rich, accurate metadata is as close as possible to data creation. Through the development of data lifecycle models, we can provide guidance and develop tools to create metadata which are packaged and maintained with the data throughout its journey.

In summary, our work to ensure easily referenced, authentic, linked data with clear provenance underpins the layers of supporting services, the assignment of credit and the assurance of accountability. These concepts are not only necessary to support further work, evaluation or decisions based on these valuable data assets; they are the building blocks for demonstrating true impact.

The UK Data Service is established and funded by the ESRC to provide seamless and flexible access to a wide range of data resources in order to facilitate high quality social and economic research and education. 

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics.  

About the Authors

James Nazroo is a Professor of Sociology and Director of the Cathie Marsh Centre for Census and Survey Research (CCSR) at the University of Manchester.

Matthew Woollard is Director of both the UK Data Service and the UK Data Archive, based at the University of Essex.

Social thinkers in all fields are faced with one unavoidable question: what does it mean to be ‘human’ in the 21st century? As definitions between what is ‘animal’ and what is ‘human’ break down, and as emerging technologies such as artificial intelligence and nano- and bio- technologies develop, accepted notions of humanity are rapidly evolving. Francis Remedios finds that although Humanity 2.0 offers challenging ideas, readers who work through those ideas will be rewarded.

This was originally published on LSE Review of Books.

Humanity 2.0: What it Means to be Human Past, Present and Future. Steve Fuller. Palgrave Macmillan. 

Find this book 

As biotechnology, genetic engineering and synthetic biology are changing humanity, what does it mean to be human?  What is the distinctiveness of humanity? Given humanity is the locus of the social sciences, this book focusses on the changing boundary conditions of biology (race) and ideology (religion) for humanity. With the welfare state set as the location of the battle between biology and ideology on humanity, Fuller defends the distinctiveness of humanity.

The author first diagnoses the problem of humanity as a bipolar disorder between our animal nature (biology) and our search for transcendence of nature (ideology). Are we closer to animals as indicated by Darwinism or are we closer to God as indicated by Christianity? In today’s terms, the positions can be portrayed to be between the poles of Peter Singer’s animal liberation or Ray Kurzweil’s spiritual machines.

For Fuller, humanity, which is moral, is the central project of the social sciences. Humanity consists of socially organised resistance to the natural selection and natural forces through collective projects such as Christianity, the University and the State. Participation in large-scale projects allows humans to control or even reverse the effects of natural selection. For Fuller, the classical sociologists Durkheim, Marx, and Weber all concur with his characterisation of the project of humanity. Essential to Fuller’s concept is the redistribution of wealth through the state. Fuller recognises Foucault’s notion that the human sciences as a body of knowledge was created in the 19th century and by the 20th century, man has died – human sciences as a body of knowledge are in question. Fuller connects humanity to transhumanism, which is the view that humanity can be enhanced or redesigned through technology. With converging technologies such as biotechnology, nanotechnology and computer technology, humanity can be transformed to an enhanced version of humanity – humanity 2.0. For Fuller, humanity 2.0 is an emerging object of social science and social policy. Fuller has indicated the core principle of social science is humanity, and he has extended it to the possible future of transhumanism. In my view, whether transhumanism occurs is difficult to say because even with converging technologies, it is not a linear progression from humanity to transhumanism.

How did the project of humanity start? Fuller avers that John Duns Scotus started the project of humanity with a univocal theory which predicates God’s attributes to man, while Thomas Aquinas has an equivocal or analogical theory of predication of God’s attributes to man. Fuller’s view is that for Scotus, man’s difference to God’s attributes is by degree, while for Aquinas, man’s difference to God is by kind. Humanity is created in the image and likeness of God. For scientists, Bacon, Newton and Mendel, who are Christians, doing science is participating in the mind of God. With the advance of the nanosciences, biotechnology and genetic engineering with which the future of life can be engineered, there have been many voices which claim science is playing God. From Fuller’s perspective, ‘doing’ science, particularly the nanosciences, biotechnology and genetic engineering, is to participate in God’s mind. However, this reviewer is sceptical. It would be very difficult to convince the public of this because many of the public take Aquinas’ view that humanity and God are different in kind and it is God who created life. The public fears that engineering of life may have the potential to do more harm than good.

Fuller takes on Darwninism with intelligent design theory (ID). For Fuller, ID is the view of the role of divine design in western science. In 2005, Fuller was an expert witness to defend ID to be taught in schools at the Kitzmiller vs. Dover trial. The judge disagreed that ID is science since it is based in theology. In a controversial move, Fuller recommends the promotion of an Abrahamic theological perspective to motivate students to become scientists in the United States because of Abrahamic theology’s view that humans are privileged to understand and control nature as they are in created in the image and likeness of God. Many critics will disagree that Fuller needs this controversial move since many scientists in the West are not motivated by Abrahamic theology and scientists in countries such as China or India are not brought up as Christians.

As humanity 2.0 will push against boundaries of morality, Fuller links theodicy to humanity 2.0. Theodicy is the problem of evil in a world created by God. Fuller’s answer to alleviating suffering, which occurs with natural disasters or human deeds, is to suffer smart. He recommends moral entrepreneurship, which is to recycle evil into good through an agent who did evil deeds but has decided to do good. Fuller’s examples of moral entrepreneurs are Jeffrey Sachs, George Soros and Robert McNamara. McNamara, who was US Secretary of Defense during the Vietnam war, later became the President of the World Bank. He lent money to the Third World to reduce poverty. His lending policies to the Third World had a negative impact because many poor countries were unable to repay their loans and there was corruption in some governments who were recipients of loan money.

Humanity 2.0 can be considered a milestone in Fuller’s work since it forms the locus of his discussions in his other works on the foundations of the social sciences. It is a complex book brimming with ideas on what it means to be human. As Fuller’s social epistemology is concerned with social transformation of knowledge, the exploration of the changing boundary conditions of the knower is critical. With the enhancement of humanity through biotechnology, genetic engineering and synthetic biology, the knower’s identity and social epistemic role can change. With advancement of computer technology and digital technology, avatars can be created and the identity of knower and social epistemic role is extended through avatars. The interface between the knower and the world has changed because the knower can be changed either through human enhancement or avatars.  Those who are interested in the foundation of the social sciences and its intersection with biology, theology and transhumanism would benefit from reading this book. I recommend this book since it goes beyond traditional issues of social science to include discussions of biology, theology, transhumanism and the history of sociology in the UK such as the founding of the first chair of sociology at the London School of Economics in 1907. Though the book offers challenging ideas, readers who work through those ideas will be rewarded.

—————————————————————————

Francis Remedios is a Canadian independent scholar with his PhD from Institute of Philosophy, University of Leuven. His research areas are social epistemology, philosophy of science and philosophy of the social sciences.  His 2003 book, Legitimizing Scientific Knowledge, was on Steve Fuller’s social epistemology. He has published several articles and book reviews on social epistemology and he is a member of the editorial board of the journal Social Epistemology and the Social Epistemology Review and Reply Collective. Read more reviews by Francis.

The public response to Elsevier’s takeover of Mendeley prompted Thomson Reuters to release an enhanced, free version of their referencing management tool, EndNote Basic. Paul Horsler examines the new features. While certainly not a tool for sharing PDFs, the web-based platform can be accessed remotely and allows for wider group collaboration and reference sharing. 

Endnote has two versions, Endnote and Endnote Basic.  The latter, and the topic of this post, was formerly branded as Endnote Web, the free web-based referencing software from Thomson Reuters.  Prior to this rebranding, there were three distinct levels of Endnote Web account: (one can even say that there still is)

• A free one via Thomson Reuters which had very limited features
• One via Web of Knowledge, which provided those at subscribing institutions with some enhanced features
• An Endnote Web account that was linked to a version of the Endnote Desktop software.  This provided even more features

Endnote Basic now provides users with the ability to store 50,000 references, 2GB of file storage, 21 bibliographic styles, 5 online search connectors, and 9 import filters.  With Endnote Basic it is possible to create groups which you can share with others.  Groups can be created on the basis of either read or read/write.  Whilst references can be shared within a group, PDF attachments cannot due to licensing.  This has long been Thomson Reuters’ position on the matter. Simply put, Endnote Basic has very few of the community sharing aspects of Mendeley.

There is however, a slightly enhanced level of Endnote Basic account which is available to subscribers to Web of Knowledge.  When Endnote Basic recognises a user as having access to Web of Knowledge, it automatically upgrades the account.  This enables the user to access the institution’s chosen styles, connectors and import filters.  It is at this point that the community aspect of the Thomson Reuters’ product suite begins to come into play with the links between Researcher ID and Web of Knowledge.  Further information on Researcher ID can be found here.

Pick the right tool for the right job is certainly something to consider when it comes to choosing which software to use for managing your references.  For me this means Endnote as it offers all of the functionality that I require such as the ability to add references types and amend reference styles on my own without having to contact the supplier.  This has been especially useful for my doctoral research, which I’m doing part-time whilst working full-time.  My full-time job includes amongst other things, training people to use Endnote and solving problems relating to it  Despite being an avid fan of Endnote, I am not against other reference management software packages (even if I do have a loathing for Zotero).  Each to their own, I say, as people need to choose the software that does what they need it to do. Endnote Basic is essentially a reference management tool rather than a community, unlike Mendeley, and this is where the power of the Endnote product will continue to be for the foreseeable future.

I could not finish without saying a quick word about the desktop version and how it works with the web version.  It is possible to link the desktop version to the web version.  This allows users to synchronise their libraries and increase their online storage space to 5GB for free.  This facility is valid for 2 years from the date of linking the account.  In addition there is the recently released Endnote app for the iPad.  This synchronises with the web version and allows you to take your library with you.  It is a fairly easy to use app but there is still a bit of work to be done with the PDF viewing facility.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics.  

About the Author

Paul Horsler is an academic support librarian for LSE Library Services and a part-time research student with the Department for International History.  His research focuses on British public opinion at a local level in the lead up to the Second World War.  He is a heavy user of the desktop version of Endnote, as well as being one of the LSE’s 3 Endnote trainers, and could not imagine undertaking his research without it.

In response to last week’s piece on how open access will enhance academic freedom, Kyle Grayson responds by outlining three key reasons why open access will directly–and indirectly–erode academic freedom in the arts, humanities, and social sciences. He argues that gold access will catalyse more intensive forms of managerialism based on crude metrics and that the scope and size of research projects are equally at risk.

In light of the formalisation of core aspects of the open access regime by the Higher Education Funding Council (HEFCE) on 1 April 2013, there was an interesting piece by Curt Rice on the LSE Impact blog last week. He argues that open access will enhance academic freedom. While I would agree that his argument is plausible in theory–and I have presented similar arguments in favour of open access–his position completely ignores the institutional context that is shaping how open access is being implemented in the UK.

In fairness, Rice is based in Norway at the University of Tromsø. Therefore, it is uncharitable to expect him to understand and account for the pathologies of the British higher education system. Nevertheless, my argument is that open access requirements will become a means of supplementing the quotidian forms of monitoring and managerialism that plague UK universities. As discussions thus far have concentrated on ‘greedy publishers’, the value of peer review, and the future viability of disciplinary societies, I fear that the negative impacts on academic freedom catalysed by open access are going to blindside many colleagues in the arts, humanities, and social sciences.

Academic Freedom

I take the definition of academic freedom offered by the UCU as my starting reference point. But it is not wholly sufficient. Beyond the standard ‘freedoms to conduct research, teach, speak, and publish without interference or penalty’, academic freedom should also encompass the freedoms to:

• conduct research that adheres to your methodological and topic-area preferences based on your own evaluations of your expertise;
• and pursue opportunities to place your research where you believe it will have the biggest impact on the audience that you are trying to reach.

I would also argue that academic freedom includes the freedom from being professionally evaluated by managers on the basis of crude metrics that do not engage qualitatively with the substance of one’s academic work.

Open Access and Three Threats to Academic Freedom

Unfortunately, there are three ways that open access in the UK will threaten academic freedom in the arts, humanities, and social sciences:

• First, although HEFCE mercifully backed down from demanding that all post-REF 2014 eligible research be gold open access–i.e., authors pay journal publishers to have the accepted copy-set version of an article immediately available to the general public on a CC-BY license– gold access will catalyse more intensive forms of managerialism. Senior research managers–who invariably look for quick measures of research quality that do not require reading articles–will use gold open access publications as a proxy for quality. Their assumption will be that gold pieces must have received UK Research Council funding as there are not sufficient funds to pay for gold open access from internal sources. Given that strike rates on awards in the arts, humanities, and social sciences range between 5-20% depending on the scheme, the use of this short-hand does not bode well for most colleagues.

• Second, it is often American-based journals that are considered to be the ‘best’ in individual fields based on reputation and ranking metrics. As the United States has the largest national research sector in the Anglo-European world, these measures of quality are often reflections of the size of the epistemic communities advocating for the merits of specific outlets and citation patterns within communities that reproduce ranking hierarchies. However, UK research managers have a tendency to go with ranking metrics because these make monitoring ‘academic performance’ easier for them. The position of a journal in a ranking system–generally the Thomson-Reuters ISI–is said to capture the quality of any individual research article published in the journal itself. Thus, research managers routinely pressurise academics to publish in these journals even when their areas of expertise and methodological commitments mean that the chances of acceptance are next to nil on the basis of fit–regardless of the quality of the work itself.

• But the threat to academic freedom will be more pernicious with open access. I have already heard rumors from well-placed sources that US-based disciplinary journals are not going to offer open access options that meet HEFCE guidelines and/or will operate an unofficial policy of rejecting all articles from the UK that come with open access requirements. In particular, many American editors and disciplinary associations are worried that colleagues based outside of the UK will perceive gold open access as ‘paying to publish’. They are not prepared to wage the information campaign that will be required to convince people otherwise. When it comes to green open access, editors and publishers will be loathe to have to navigate the production of different licensing agreements in order to cater to UK academics. The easiest solution will be to eliminate the problem through desk rejections.

• And here is where a perfect-catch 22 will arise. For those in the arts, humanities, and social sciences who have funding that will allow them to pursue gold open access, placing their work in ‘top outlets’ based in North America may well become impossible. And when they seek access to their research funds to obtain open access licenses for other outlets, they are going to have to convince sceptical research managers–who control how research monies are spent–that they should be given access to these resources. And one can only imagine the bureaucratic regime–and accompanying paperwork– that will be instituted to evaluate these requests across UK universities. UK academics–whether funded or unfunded–will lose.

• Third, open access will also lead to a significant impact on the amount of research funding available as the size of grants–particularly smaller grants like ESRC and AHRC doctoral studentships–are adjusted to provide resources for securing gold open access licenses. Given the current allocations of funding and the broader austerity drive, these adjustments will reduce the total number of awards that can be given. They will also narrow the scope of research that is funded away from non-responsive and inquiry-based initiatives towards specific topics and themes that are determined by funders–and influenced by government ideology.

For these reasons–and others that will likely become manifest in the coming months–open access is going to directly–and indirectly–erode academic freedom in the arts, humanities, and social sciences in the UK. It is on the principles of academic freedom that colleagues and disciplinary associations should be demanding that open access requirements be reformulated to avoid these negative externalities.

[full disclosure: I am an editor-in-chief of the UK Political Association's journal Politics, associate editor of the journal Critical Studies on Security, and co-editor of the Popular Culture and World Politics book series].

This was originally posted on Kyle Grayson’s personal blog and is reposted with permission.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics.  

About the Author

Kyle Grayson is a Senior Lecturer in International Politics at Newcastle University, UK. He is a lead editor of the journal Politics, an associate editor of Critical Studies on Security and a co-editor of the Popular Culture and World Politics book series. He is a contributor to the CSI-Newcastle blog and can be found on twitter @chasing_dragons.

Academic blogging has become an increasingly popular form, but key questions still remain over whether blog posts should feature more prominently in formal academic discourse. Jenny Davis clarifies the pros and cons of blog citation and sees the remaining ambiguity as indicative of a changing professional landscape. The wider scholarly community must learn how to grapple with these ethical and professional questions of rigor in standards of academic sourcing.

In this post I attempt to tackle a complex but increasingly important question: Should writers cite blog posts in formal academic writing (i.e. journal articles and books)? To begin with full disclosure: I cite blog posts in my own formal academic writing. But not just any blog posts. I am highly discriminate in what I cite, but my discriminations are not of the cleanly methodical type which can be written, shared, and handed out as even a suggested guide.

Mostly, I cite Cyborgology and a select few blogs that I know really, really well. I have done so in my last three formally published works (two of which are Encyclopedia entries), and successfully suggested blog posts to others via peer-review. When pressed for a rationale (as I have been in conversations with colleagues), I less-than-confidently ramble something like:

Well I mean, I know these bloggers to be good theorists, and I find their work useful for my own. Some of their work is published only in blog form, and I need those ideas to build my argument. I also don’t want to ignore something good that I know is out there. But I mean, I know there are other good things out there that I don’t know about, or don’t know enough to trust. And I know I’ve written bad ideas on Cyborgology, or ideas that I further developed later, so I guess quality is not a sure thing, but reviewers and editors have accepted it so…[insert sheepish grin].

With this poorly articulated rationale in mind, I present first, some pros and cons to citing blogs within formal academic writing. Next, I put forth three main sub-questions that I think will help us—and by “us” I mean myself and the readers who grapple with the ethical and professional questions of rigor in standards of academic sourcing—organize our thoughts. 

Pros and Cons of Blog Citation

The following pros and cons of blog citation are far from exhaustive. Rather, I highlight some key tensions. Please feel free to address other benefits, complications, or tensions in the comments section.

Pros

There are several benefits to citing blogs. Importantly, as I list these benefits—largely through juxtaposition to traditional publication venues— I by no means eschew the benefits of traditional venues. Rather, I explore the possibility of a broader citation base.

Without university affiliation, it would cost me $25 to access my own article for 24hrs.

• First, peer-reviewed journals are slow, jargon ridden, and often financially pay-walled (amiright David Banks, PJ Rey!?). Blogs are fast, self-published, and usually free. That is, the content of a blog becomes available far faster than that of a journal article, and is accessible to a wider audience. Including blogs within formal academic writing allows authors to utilize ideas that may not yet be available through traditional channels, and provides source materials for those without access to content hidden behind publishers’ blockades.
• Second, blogs can be written by anyone. Peer-reviewed journal articles and books are almost always authored by academics. This academic bias, like pay-walls and jargon, limits discursive participants, whereas blogs can potentially open discursive boundaries.
• Third, traditional journals rely on existing experts to decide what can/should be published. If an idea or methodology does not fit within an existing framework, its chances of acceptance diminish. Blogs are less susceptible this type of censorship, providing a wider breadth of theoretical building blocks and facilitating new theoretical directions.

Cons

• The most obvious problem with citing blogs is that they are not peer-reviewed. They sit outside the agreed upon standard of academia, taking away the insurance policy (however flimsy that policy may be) of the peer-review process. Anyone with minimal computer literacy and access to a computer can publish a blog. Although this opens the discursive boundaries, it also means the discourse is far more crowded, and an academic writer must navigate the crowds with no clear rubric to discern rigor.
• Second, bloggers tend to write in piecemeal fashion. On Cyborgology, for instance, theories of Digital Dualism and Augmented Reality continue to develop over time. Early posts do not necessarily reflect current thinking, and current posts may have significant problems yet revealed. To cite a blog is therefore to run the risk of citing an incomplete idea, or an idea out of context.
• Third, blog posts are impermanent. Individual posts can be deleted, entire sites can be deleted, and texts can be edited with or without notification to the reader. This transience means that what one cites may not always exist, or may exist in a form that completely alters the meaning. On Cyborgology, we make an effort to notify readers of textual changes, but this is our editorial policy, not an across-the-board standard.

With these benefits and deterrents in mind, we can further explore what blog citation might look like, and how we can make decisions about its usefulness. In this spirit, I present some key orienting questions.

Three Orienting Questions

1.) When is it okay to cite blogs in a formal academic paper?    

Or, in other words, how can academic writers use blogs effectively in their writing? Thankfully, the use of blog posts in academic writing is not always ambiguous. Few would debate the use of blog posts as data sources within a discourse analysis. For example, if I wanted to analyze discourses on body size, food, and health, it would make sense to cite content from Health At Every Size,  HungryGirl, and food scholar Marion Nestle’s blog Food Politics.

More ambiguous, of course, is the question of using content from these blogs, in their own right, as building blocks or even a foundation for, theoretical arguments. The journal publishing process tends to be frustratingly slow, and the contents of journals are often hidden behind financially prohibitive pay-walls. Blogs are a fast and usually free way to disseminate information. If the ideas are there, available to all, it seems like we can and should use them. I may need some idea to build a theoretical argument, and it may not be available to me for months, years, or ever if I rely solely on peer-reviewed publications. Moreover, as Whitney Erin Boesel (@phenatypical)  points out, it is important to give credit where credit is due, and if I find a blog post useful for building a theoretical argument, I should cite that blog post, giving credit to the author(s).

At the same time, blogs are not peer-reviewed—if they were, they would be open access online journals and decisions about citation would be far less contentious. In the “information age,” one of the biggest challenges is sorting through an abundance of content. The peer-review process is an important tool here. It becomes a boundary within which the reader can feel relatively comfortable with the methodological soundness and theoretical rigor of a piece (although certainly not completely comfortable). To give the “ok” to cite blog posts opens the proverbial information floodgates, and unbounds the corpus of citable literature. What would such an anything-goes literature review look like?

To be clear, I do not think anyone (or at least very few people) would argue for the anything-goes model. Even those who strongly believe that blog posts are legitimately citable material would likely agree that some blogs are better (read: more sophisticated, more rigorous) than others. This leads to the next question:

2.) Which blogs are okay to cite, and how do we know?  

Not all blogs are created equal. As mentioned in the opening paragraph, I cite some blogs (e.g. Cyborgology, Racism Review) but not all blogs. Going back to the food/health/body example from above, I would be pretty comfortable citing Marion Nestle’s blog but would balk at the idea of citing HungryGirl’s Lisa Lillien.  Basically, I cite the blogs that have writers I know and like. These writers are usually academics. I do not cite blogs with writers who I do not know, and who are not academics.

The requirement that citable blogs be written by academics makes intuitive sense, but is highly problematic. First, it assumes that academics are always rigorous in their writings.  As we know from the high rejection rates of peer-reviewed journals, this is certainly not the case. In turn, it discounts those who are not affiliated with the academy. This is not only elitist, but reinforces institutional power hierarchies within knowledge production.

However, one can counter the former point by noting the poor quality of some published articles, problematizing the false-security that comes along with a legitimizing label of “peer-reviewed.” One can then counter the latter point by noting that sole reliance on peer-reviewed materials necessarily creates an exclusionary power-knowledge relationship, one even more firmly cemented than author-credential based decisions.

My imperfect solution is therefore to make subjective decisions on a case-by-case basis. Reviewers and readers are free to check my sources and make judgments of their own.

3.) Who can cite blogs?

Okay, now here comes the real hypocrisy. Although I cite blogs within academic writing, I explicitly forbid my undergraduate students from doing so. Their papers must include only peer-reviewed work unless I specifically approve of a non-peer-reviewed source.

Oh, hi Privilege, nice to see you again. The key difference between my students and me (besides, of course, our taste in music and repertoire of Seinfeld quotes), is that I have a Ph.D. and they are working on Bachelor’s degrees. That is, we are differentiated by levels of education, and having a higher level of education gives me the privilege and power to determine the value of piece of writing, and denies this power and privilege to those with less formal education. To say it out loud feels like the academic equivalent of “Because I Said So.”

At the same time, I have been trained in a particular field for several years. I have read the jargon-ridden journal articles, trudged through the 5-chapters-too-long books, and even contributed a few pieces of my own. Moreover, I have been a peer-reviewer, charged with making formal decisions about what is, and is not, a publishable piece of research. And so I take this training and I use it, again imperfectly, as a privilege, allowing myself to discern quality while urging others to wait until they have enough knowledge and practice to make such discernments. What “enough” is, however, remains quite nebulous. Perhaps as a brand new Ph.D. I grant myself too much license. Again, I leave it to my peer-reviewers to determine.

And the Final Answer Is:

Hey, I said at the beginning I was not going to provide a definitive answer. I think the ambiguity is indicative of a changing professional landscape. Decisions we make about citation—and ultimately, the legitimacy of different forms of work—will shape how intellectualism develops. The real question then, is how do we want the intellectual landscape to look?

This was originally posted  at Cyborgology and is reposted with the author’s permission.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics.  

About the Author

Jenny Davis is a postdoctoral researcher in the Department of Sociology at Texas A&M University. She will begin an Assistant Professor appointment at James Madison University in Fall 2013. Jenny is also a weekly contributor for Cyborgology.org. You can follow Jenny on Twitter @Jup83.

Amidst the time-consuming intensity of compiling submissions for the Research Excellence Framework (REF), Athene Donald reflects on how the necessity of demonstrating research impact has been understood over the years. While there are even identifiable shifts from 2008′s RAE assessment, more substantially divergent is the 18th century view where practical applications of research could actually do great damage to the status and funding of a researcher.

Up and down the land, academics from Vice Chancellors down are sweating over 3 letters: REF. This dread acronym, standing for the Research Excellence Framework, must be absorbing a fantastic number of hours of time for many people and it is not something to be taken lightly. Many millions of pounds are at stake, as well as departmental honour. Some aspects merely require simple metrics – such as on PhD student numbers and grant income from different sources – but others are much more challenging to construct. Long narratives are required about the environment in which we work and how we ensure it is an appropriate and supportive place for everyone. HEFCE guidelines make it very clear that they want to hear about diversity issues this time around and have acted sensibly (in the end) about reducing outputs for women who’ve had time out on maternity leave. We need to describe our strategic plans and how well we’ve accomplished what we said we’d do in 2008 during REF’s predecessor the RAE (Research Excellence Framework). Staff have to choose their ‘best’ 4 outputs, without using journal impact factors as the crude measure of what makes ‘best’ – although not everyone is convinced that institutions are abiding by the rules on this one (see here and here).

Most of this is comparable to what happened in the RAE, although with some shifts in the emphasis and detail required. What is different is the new section on ‘impact’. Impact in the sense of something rather tangible coming out of research carried out in the previous 15 years. It isn’t sufficient to say that a patent was forthcoming unless that patent spawned a spin-out or a product on the market. Simply having done a lot of outreach isn’t going to count unless you can show somehow the outreach led to a measurable change: a fiendishly difficult challenge to link attendance at a splendid school’s talk or an evening at an observatory to an upturn in astronomy degrees awarded, for instance! So, impact in the REF sense is very different from impact in the RCUK sense of ‘pathways to impact’. REF wants impact signed and delivered, with a stress on both ‘reach’ and ‘significance’; RCUK grant proposals only require the alleged promise of impact tomorrow.

I chair my local Unit of Assessment’s REF panel and my summer will be consumed, I fear, by sorting out the details of our submission. I guess it makes a change from the summer I spent reading everyone else’s outputs because I was on the RAE panel itself, but I’m not sure it’s actually an improvement. I had thought that summer of 2008 was difficult (it also involved some extraordinarily wet stays in the Lake District where our panel meetings were all held. I couldn’t savour the location because the location was constantly shrouded in low cloud and horizontal sheets of rain) but this time the weight of responsibility feels very different, indeed even more substantial.

Recently, what has struck me forcibly is what the passage of years can do to transform acceptable behaviour.  I am currently reading a rather old (1992) book by Jan Golinski called Science as Public Culture: Chemistry and Enlightenment in Britain 1760-1820. The first main chapter discusses William Cullen  and Joseph Black, two early professors of Chemistry in Scotland during the 18^th century. Both were determined to be free of any taint of ‘projecting’, a word in its day associated with the sordid practice of making money, doing nefarious deals and taking out patents.

Projecting was seen by many as something associated with the mercantile class rather than as an appropriate activity for men of intellect; the kind of dodgy deal people associated with this sort of action might be exemplified by the South Sea Bubble, clearly a distinctly dubious operation. Nevertheless it is striking that these early professors of chemistry felt their reputations would suffer if they did anything that could smack of projection. Despite the importance of chemistry for fields such as agriculture, dyeing and bleaching, all very important to 18th century commercial activities, it was not regarded as comme il faut to get too close to these practical matters. Projectors were not gentlemen and could not expect to receive aristocratic patronage, important to impecunious academics, so the practical side of Cullen and Black’s research had to be played down, to be made to look as if it was less useful than it was, so that their standing was not damaged.

Golinski says the following of Cullen

‘Practical achievements were highly problematic, and a maladroit insistence on the relevance of chemical theory to practice could be taken for arrogant projecting or self-serving puffery.’

As I read the book, it must have been a very fine line to tread between being thought to be not-quite-a-gentleman who was at risk of ostracism, and actually doing something useful with the subject. For chemistry, a discipline that was barely taught in English universities (i.e. Oxford and Cambridge) at the time, the usual emphasis was on chemistry as relevant to medicine – something that was of course acceptable although extremely empirical.

Times have changed radically. Now we all are scraping around trying to prove, not only that our research has potential for the future, but that we already have done something significant with it. Not everyone will view it as a badge of honour to have produced a tangible outcome from their research, as comments on this blog make very clear. It is, however, something likely to find favour with a head of department in the current climate. Even more so if it is an outcome worth quite a bit of cash – whether that cash is for the originator/inventor or someone else. Funders also are putting a high premium on such matters, because they see the demonstration of impact of funded research as a powerful argument with Treasury, rightly or wrongly. It is of course the case that I doubt any academic worries over much as to whether they are considered to be gentleman (or ladies of course), so that particular pitfall has ceased to be relevant. But the attitudes of the UK powers-that-be to doing something useful has been totally transformed into a world view where utility may sometimes seem to override any other consideration.

This was originally posted on Athene Donald’s personal blog and is reposted with permission.

Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of the London School of Economics.  

About the Author

Athene Donald is Professor of Experimental Physics at Cavendish Laboratory at the University of Cambridge. Athene has been at Cavendish since 1983, and became a professor in 1998.