At the IPI group at WZB we have five projects innovating on aspects of what we call the “aggregation challenge.” The aggregation challenge is the challenge of combining findings from multiple studies to foster cumulative learning. We think of it as one of the biggest challenges facing social scientists and that addressing it will be key for strengthening the relevance and reliability of social science findings.

  • Coordinated trials 1. IPI contributes to the "metaketa" initiative housed at EGAP.  Metaketas are coordinated randomized trials across multiple sites with harmonized measures and analysis strategies. See this summary paper in Science Advances summarizing the results from Metaketa 1 and this "shiny app" we developed that lets readers explore analyses and robustness of findings to the inclusion or exclusion of  different studies.
  • Coordinated trials 2. We are working on the structure for a "rolling" metaketa on the contact hypotheses, building on work by Alexandra Scacco and Shana Warren and others. The framework connects existing contact experiments using a common causal logic and sets up a structure to allow future studies to enter a live meta-analysis.
  • Coordinated analysis of strategies to measures hidden populations. We are leading the meta-analysis for a multicountry collection of studies coordinated by  APRIES  to assess the prevalence of human trafficking prevalence estimation.
  • Integrated inferences. We are developing the CausalQueries R package that lets users define and combine causal models. See our guide for examples with applications to combine inferences from qualitative and quantitative analyses, inferences from observational and experimental studies, and inferences from multiple trials examining different parts of a common causal model.
  • Meta-models: Our Correlates of Corona project examines socioeconomic predictors of Covid mortality. Experimental stages now in the field (with Miriam Golden, Alexandra Scacco and colleagues) focus on aggregating disciplinary beliefs about logics driving Corona and strategies to connect observational patterns with causal logics. See here for our challenge.

Related writing:

  1. The aggregation challenge
    Macartan Humphreys, and Alexandra Scacco
    World Development 2020
  2. Chapter
    Field experiments, theory, and external validity
    Anna Wilke, and Macartan Humphreys
    SAGE Handbook of Research Methods in Political Science and International Relations 2020
  3. CUP
    Information, accountability, and cumulative learning: Lessons from Metaketa I
    Thad DunningGuy GrossmanMacartan HumphreysSusan D Hyde, and 2 more authors
  4. SciAdv
    Voter information campaigns and political accountability: Cumulative findings from a preregistered meta-analysis of coordinated trials
    Thad DunningGuy GrossmanMacartan HumphreysSusan D Hyde, and 7 more authors
    Science advances 2019
  5. Mixing methods: A Bayesian approach
    Macartan Humphreys, and Alan M Jacobs
    American Political Science Review 2015