Ideas often have a way of spreading. This is certainly true in the case of American states; innovative policies are often adopted by others in a process known as policy diffusion. But in trying to determine which policies spread, could we be giving too much of a focus on those that are successful? In new research which examines interstate compacts, Andrew Karch, Sean C. Nicholson-Crotty, Neal D. Woods, and Ann O’M. Bowman find that an emphasis on successful policies may be leading scholars to overestimate the importance of some factors, such as neighboring state activity, and underestimate the importance of others, like the number of previous adopters.
How political phenomena spread from one jurisdiction to another, typically known as diffusion, is a central concern in political science. Nearly 800 articles on the subject have been published in political science journals, over half in the last decade. Interest in diffusion cuts across virtually every subfield of the discipline. Scholars of comparative politics, international relations, and American politics have examined the diffusion of democracy, international norms about human rights, pension reform, political institutions, and many other political phenomena. Few topics have simultaneously engaged so many scholars in so many disparate areas of the discipline.
Existing studies, however, often examine only episodes during which an innovation was adopted by a large number of political jurisdictions. For instance a recent review looks at studies that investigated the diffusion of 23 distinct policies in the American states. The policies that researchers chose to study had diffused fairly widely; the average number of adopters across these studies was just over 29.
This could be problematic because “successful” (i.e. widely-diffused) policies may not be representative of all policies. Critics argue that constructing a compelling account of why diffusion occurs, or fails to occur, requires that scholars investigate both “successes” and episodes of limited adoption or non-adoption. Failing to do so may produce a cumulative knowledge base that is built on a systematically unrepresentative sample of policies, thus leading scholars to make incorrect or questionable inferences about the general nature of diffusion dynamics. This “pro-innovation bias” could take several forms. Researchers could identify certain factors as influential even though they do not have a significant effect on program adoption when the full range of policies is considered. The reverse could also occur. Finally, researchers could identify the appropriate factors as influential but inflate or deflate their substantive impact. The appearance of any of these patterns would expose the limitations of thinking about diffusion from only the perspective of successful cases.
The main purpose of our recent research is to assess whether these concerns are warranted. To our knowledge ours is the first study to quantitatively assess the potential effects of a pro-innovation bias in diffusion research. We estimate this bias by turning to the American states to examine state participation in interstate compacts, constitutionally-sanctioned agreements between two or more states that are used to address joint problems in many policy areas. Interstate compacts are not perfect analogues for the policies on which diffusion research tends to focus, yet in both cases state officials have the opportunity to adopt a preexisting template. Moreover, previous research suggests that compact membership and policy innovation are driven by similar forces. A major advantage of our approach is that we can observe the entire universe of interstate compacts rather than examining a small purposive sample of innovations.
Figure 1 gives an indication of the range of adoption for various interstate compacts. Of the 43 national compacts in existence, 20 have only been adopted by ten or fewer states. Eight, however, have been adopted by 41 or more. The wide variation in state participation helps address the pro-innovation bias issue. While all 50 states entered into the original Interstate Compact for Juveniles and 47 states joined the Emergency Management Assistance Compact, the only states to join the Interstate Insurance Receivership Compact are Illinois, Michigan, and Nebraska. Five national compacts technically have not gone into effect because they claim only a single member. Thus national compacts include examples of widespread adoption, limited adoption, and non-adoption, a rarity in diffusion research. They are also substantively meaningful policy initiatives that address the most salient issues in state policymaking—criminal justice, education, social welfare, and business practices.
Figure 1 – Rates of American state adoption of national interstate compacts
Our empirical strategy relies on pooled event history analysis, which can be used to examine multiple policy innovations and ensure that any results are not due to the idiosyncrasies of an individual policy or state. Specifically, we estimate a model that allows us to assess whether the effects of many well-known predictors of adoption vary between a traditional sample of widely adopted compacts (the 19 national compacts joined by 20 states or more) and the full universe of 43 national compacts.
Our analysis shows that an emphasis on diffusion “successes” may lead scholars to overestimate the impact of some factors but understate the effect of others.
This can be seen by looking at the differences in the marginal effects of these variables across the traditional and full samples, which are plotted in Figure 2 along with their 95% confidence intervals. Each point in the plot represents the effect of a 0 to 1 shift for indicators which are one of two possibilities and a one-standard-deviation shift for continuous measures.
Figure 2 – Differences between models looking only at “successful” compacts and at all compacts (with 95 percent confidence interval)
In some cases, our results highlight how restricting the sample to widely diffused policies produces an overestimation of key effects. The impact of neighboring state activity, the most common measure of interdependence, is over twice as large in the traditional sample as it is in the full model. The effect of policy complexity is also much larger in the traditional sample than in the full model. Indeed the observed impact is more than four times as large.
In other cases looking solely at widely diffused policies appears to underestimate important effects. A notable difference in this regard concerns the impact of the number of previous adopters, a proxy for policy learning. The impact of this variable on the probability of adoption is positive and significant in both samples, but its effect is almost twice as large in the full sample relative to the traditional sample. The general impact of professional associations and interest groups also appears to be underestimated in diffusion research. In the traditional sample, the involvement of organizations like the Council of State Governments in compact development had no observable effect on the likelihood of adoption. In the full model, however, their involvement has a significant positive effect.
Our analysis suggests that focusing exclusively on episodes of widespread adoption may cause scholars to draw unwarranted inferences about the factors that influence policy decisions. While these results must be interpreted carefully given the substantive differences between interstate compacts and “conventional” policy innovations, they imply that scholars must be cognizant of the ways in which case selection affects our understanding of the diffusion process. Our findings suggest that scholars must engage in more careful theorizing about the conditions under which external forces and policy attributes are likely to influence the adoption decision. Rather than presuming that developments in neighboring states will have an impact, for example, they must articulate specific mechanisms—such as communications networks, program externalities, and demographic similarities—that promote interdependence. Additional theoretical development along these lines is necessary in order to more fully understand what factors differentiate diffusion successes from instances of limited adoption.
This article is based on the paper, ”Policy Diffusion and the Pro-Innovation Bias” which appeared in Political Research Quarterly.
Featured image credit: Aaron Brown (Flickr, CC-BY-NC-2.0)
Please read our comments policy before commenting.
Note: This article gives the views of the author, and not the position of USApp– American Politics and Policy, nor of the London School of Economics.
Shortened URL for this post: http://bit.ly/1ZDyFAV
______________________
Andrew Karch – University of Minnesota
Andrew Karch is Arleen C. Carlson Associate Professor of Political Science at the University of Minnesota. He is the author of Democratic Laboratories: Policy Diffusion among the American States and Early Start: Preschool Politics in the United States.
Sean C. Nicholson-Crotty – Indiana University
Sean C. Nicholson-Crotty is an Associate Professor in the School of Public and Environmental Affairs at Indiana University. His research interests include public management, federalism and intergovernmental relations, and comparative state policy. He is the author of Governors, Grants, and Elections: Fiscal Federalism in the American States.
Neal D. Woods – University of South Carolina
Neal D. Woods is an Associate Professor of Political Science at the University of South Carolina. His research focuses on public policy and public administration.
Ann O’M. Bowman – Texas A&M University
Ann O’M. Bowman is Professor and Hazel Davis and Robert Kennedy Endowed Chair in the Department of Public Service and Administration at the Bush School at Texas A&M University. Her research interests center on state and local government institutions and policy.
