At the IPI group at WZB we have five projects innovating on aspects of what we call the “aggregation challenge.” The aggregation challenge is the challenge of combining findings from multiple studies to foster cumulative learning. We think of it as one of the biggest challenges facing social scientists and that addressing it will be key for strengthening the relevance and reliability of social science findings.
- Coordinated trials 1. IPI contributes to the "metaketa" initiative housed at EGAP. Metaketas are coordinated randomized trials across multiple sites with harmonized measures and analysis strategies. See this summary paper in Science Advances summarizing the results from Metaketa 1 and this "shiny app" we developed that lets readers explore analyses and robustness of findings to the inclusion or exclusion of different studies.
- Coordinated trials 2. We are working on the structure for a "rolling" metaketa on the contact hypotheses, building on work by Alexandra Scacco and Shana Warren and others. The framework connects existing contact experiments using a common causal logic and sets up a structure to allow future studies to enter a live meta-analysis.
- Coordinated analysis of strategies to measures hidden populations. We are leading the meta-analysis for a multicountry collection of studies coordinated by APRIES to assess the prevalence of human trafficking prevalence estimation.
- Integrated inferences. We are developing the CausalQueries R package that lets users define and combine causal models. See our guide for examples with applications to combine inferences from qualitative and quantitative analyses, inferences from observational and experimental studies, and inferences from multiple trials examining different parts of a common causal model.
- Meta-models: Our Correlates of Corona project examines socioeconomic predictors of Covid mortality. Experimental stages now in the field (with Miriam Golden, Alexandra Scacco and colleagues) focus on aggregating disciplinary beliefs about logics driving Corona and strategies to connect observational patterns with causal logics. See here for our challenge.
- Meta-analysis can be used not just to guess about effects out-of-sample but also to re-evaluate effects in sample (Declare Design Team)
- The aggregation challengeWorld Development 2020
- ChapterField experiments, theory, and external validitySAGE Handbook of Research Methods in Political Science and International Relations 2020
- CUPInformation, accountability, and cumulative learning: Lessons from Metaketa I2019
- SciAdvVoter information campaigns and political accountability: Cumulative findings from a preregistered meta-analysis of coordinated trialsScience advances 2019
- Mixing methods: A Bayesian approachAmerican Political Science Review 2015