cumulation

Banerjee, Duflo, and Kremer have had an enormous impact on scholarship on the political economy of development. But as RCTs have become more central in this field, political scientists have struggled to draw implications from proliferating micro-level studies for longstanding macro­ level problems. We describe these challenges and point to recent innovations to help address them.

Critics of field experiments lament a turn away from theory and criticize findings for weak external validity. In this chapter, we outline strategies to address these challenges. Highlighting the connection between these twin critiques, we discuss how structural approaches can both help design experiments that maximize the researcher’s ability to learn about theories and enable researchers to judge to what extent the results of one experiment can travel to other settings. We illustrate with a simulated analysis of a bargaining problem to show how theory can help make external claims with respect to both populations and treatments and how combining random assignment and theory can both sharpen learning and alert researchers to over-dependence on theory.

Throughout the world, voters lack access to information about politicians, government performance, and public services. Efforts to remedy these informational deficits are numerous. Yet do informational campaigns influence voter behavior and increase democratic accountability? Through the first project of the Metaketa Initiative, sponsored by the Evidence in Governance and Politics (EGAP) research network, this book aims to address this substantive question and at the same time introduce a new model for cumulative learning that increases coordination among otherwise independent researcher teams. It presents the overall results (using meta-analysis) from six independently conducted but coordinated field experimental studies, the results from each individual study, and the findings from a related evaluation of whether practitioners utilize this information as expected. It also discusses lessons learned from EGAP’s efforts to coordinate field experiments, increase replication of theoretically important studies across contexts, and increase the external validity of field experimental research.

Voters may be unable to hold politicians to account if they lack basic information about their representatives’ performance. Civil society groups and international donors therefore advocate using voter information campaigns to improve democratic accountability. Yet, are these campaigns effective? Limited replication, measurement heterogeneity, and publication biases may undermine the reliability of published research. We implemented a new approach to cumulative learning, coordinating the design of seven randomized controlled trials to be fielded in six countries by independent research teams. Uncommon for multisite trials in the social sciences, we jointly preregistered a meta-analysis of results in advance of seeing the data. We find no evidence overall that typical, nonpartisan voter information campaigns shape voter behavior, although exploratory and subgroup analyses suggest conditions under which informational campaigns could be more effective. Such null estimated effects are too seldom published, yet they can be critical for scientific progress and cumulative, policy-relevant learning.

We develop an approach to multimethod research that generates joint learning from quantitative and qualitative evidence. The framework—Bayesian integration of quantitative and qualitative data (BIQQ)—allows researchers to draw causal inferences from combinations of correlational (cross-case) and process-level (within-case) observations, given prior beliefs about causal effects, assignment propensities, and the informativeness of different kinds of causal-process evidence. In addition to posterior estimates of causal effects, the framework yields updating on the analytical assumptions underlying correlational analysis and process tracing. We illustrate the BIQQ approach with two applications to substantive issues that have received significant quantitative and qualitative treatment in political science: the origins of electoral systems and the causes of civil war. Finally, we demonstrate how the framework can yield guidance on multimethod research design, presenting results on the optimal combinations of qualitative and quantitative data collection under different research conditions.