LSE - Small Logo
LSE - Small Logo

Sierra Williams

June 23rd, 2014

The FIRST Act’s demand for relevance at the expense of replication puts the entire scientific enterprise at risk.

4 comments

Estimated reading time: 5 minutes

Sierra Williams

June 23rd, 2014

The FIRST Act’s demand for relevance at the expense of replication puts the entire scientific enterprise at risk.

4 comments

Estimated reading time: 5 minutes

david-takeuchi-513x336The United States’ controversial FIRST Act would have profound  implications for how social science research is managed and its funding allocated. David Takeuchi argues that even if the act doesn’t pass, it is clear that politicians are demanding more of a say in federally funded research. While a push to ensure research remains relevant can be a good thing, scientists and politicians must not forget that initial outcomes do not constitute substantive evidence. Scientific integrity and replication shouldn’t have to be sacrificed in order to meet political timeframes.

A few weeks ago, I penned an Op Ed for the Boston Globe alerting colleagues and the general public about impending legislation that could have severe negative implications for those engaged in social science inquiry. Called the Frontiers in Innovation, Research, Science, and Technology Act, or FIRST Act, the new law, if enacted, would limit funding directed to American social science projects, while empowering politicians, at the expense of scientists, to define our nation’s priorities in research.

It’s probable that FIRST will come out of committee and before our Congress in the coming weeks, and that, as is often the case, a potential vote would be determined according to party lines. By and large, Republicans would support the legislation, using fiscal austerity as a catch phrase to gather more control on the kinds of science they’d like to fund (“areas that promote economic growth,” according to one prominent Republican congressman), and away from the kind of science they’d prefer not to fund (such as climate change). Most Democrats, on the other hand, would be likely to vote against the bill, supporting the idea that funding decisions should be left to those who have devoted their life’s work to research.

1280px-United_States_Capitol_model_at_Disneyland_(side_view)Image credit: United States Capitol model at Disneyland, Josh Hallett (Wikimedia, CC BY-SA 2.0)

Currently, the U.S. Congress earmarks about $7.2 billion to the National Science Foundation, or NSF, an organization that funds nearly a quarter of all “federally supported basic research conducted by America’s colleges and universities.” Within the NSF, review committees made up of accomplished researchers in appropriate fields evaluate the merits of proposed studies. FIRST, however, allows politicians more say as to what they’d like to see funded. The bill’s authors have already said that the social and behavioral sciences are not among those preferred fields of research, and therefore, monies directed to these areas would be slashed (early estimates were by as much as 40 percent).

One tracker of congressional bills suggests that FIRST has only a 29 percent chance of being enacted. So, why be concerned about this bill? Even if FIRST were to fail, politicians will almost certainly continue to pursue legislation giving them more say in choosing the types of science that’s funded by federal sources. And there’s more going on here — at risk is the very fabric of scientific enterprise. Once we’re forced to cater our research to the whims and fancies of politicians, we also become more likely to forget to maintain our integrity. When this happens, science suffers, and on a disconcertingly broad level. Certainly, the reach of politics into science goes far beyond U.S. borders.

More and more in the current academic landscape, we feel the pressure to make our research more relevant to the general public. While this can be a good thing, there can be a rush to report findings and we forget that initial outcomes aren’t evidence, and, particularly where people are affected, that interventions should only be introduced following substantive, verifiable inquiry.

I’m reminded of an important article that ran in the Economist last fall about the “culture of novel findings” which exposed the fact that “science’s self-correcting mechanism,” namely replication, is no longer in “working order.”

“Replication is hard and thankless,” wrote the magazine.  “Journals, thirsty for novelty, show little interest in it; though minimum-threshold journals could change this, they have yet to do so in a big way. Most academic researchers would rather spend time on work that is more likely to enhance their careers.”

This is, of course, all too true. Researchers, particularly those on the tenure track, are evaluated based on their published research, the reputation of the journals that accept their manuscripts, and how often their peers cite their work. There’s nothing wrong with that, but we shouldn’t have to sacrifice our scientific integrity to get ahead. Replication of novel findings matters deeply — without it, budding science can’t effect real change. But popular (i.e. political) oversight of research (such as is being proposed through FIRST) would only further ensure that the kind of scientific scrutiny that leads to real progress becomes less and less valued.

Take, for example, all of the excitement surrounding Google Flu Trends. Back in 2008, the Internet search giant successfully predicted flu outbreak prevalence based on search results on its site. The American Centers for Disease Control (CDC) hailed the technology and heaps of positive press followed suit. But since then, Google Flu Trends has failed miserably; in 2013 Nature found it was predicting more than 50 percent as many physician visits than were reported by the CDC.

What happened? In short: Google’s algorithm incorporating data from the Web and social media needed some serious tweaking before it should have been accepted as a viable means for tracking the flu. But, as is increasingly the case in science, the cart came before the proverbial horse, and the technology was hailed as a next, great innovation. For all the data our tech giants can aggregate, we’re reminded that crowdsourcing should not be confused with scientific inquiry.

The mistakes made with Google Flu Trends should serve as a cautionary tale, and above all, as a reminder to all of us in the field of research that replication matters.  One set of findings does not make a trend. Politicians build careers upon “what have you done for me latelys,” and their stances are often manipulated by the tenor of a specific moment in history. Science, on the other hand, doesn’t work this way. We have a process, and if this process is threatened, we risk losing more than just our jobs. Without establishing foundational science, effective interventions to solving problems won’t follow. This will not only affect science, and the careers of scientists, but it could cost lives.

Note: This article gives the views of the authors, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.

About the Author

David T. Takeuchi is Professor and the inaugural Dorothy Book Scholar at the Graduate School of Social Work at Boston College. He is also the School’s Associate Dean for Research. Dr. Takeuchi is a sociologist with postdoctoral training in epidemiology and health services research.  His research focuses on the social, structural, and cultural contexts that are associated with different health outcomes, especially among racial and ethnic minorities. He also examines the use of health services in different communities.

Print Friendly, PDF & Email

About the author

Sierra Williams

Posted In: Government | Higher education | Research funding

4 Comments