1.1.1 Aims

• Deep understanding of key ideas in causal inference
• Transportable tools for understanding how to evaluate and improve design
• Applied skills for design and analysis
• Exposure to open science practices
• Deeper dive into some specific topics (see survey)

1.1.2 Syllabus and resources

• https://macartan.github.io/ci for all resources
• https://macartan.github.io/ci/syllabus.pdf
• Git repo https://github.com/macartan/ci
• Student survey: Please fill this out today.

1.1.7 Good coding rules

• https://bookdown.org/content/d1e53ac9-28ce-472f-bc2c-f499f18264a3/code.html
• https://www.r-bloggers.com/2018/09/r-code-best-practices/

1.1.9 Aim

• First best: If someone has access to your .Rmd/.qmd file they can hit render or compile and the whole thing reproduces first time. So: Nothing local, everything relative: so please do not include hardcoded paths to your computer

• But: often you need ancillary files for data and code. That’s OK but aims should still be that with a self contained folder someone can open a main.Rmd file, hit compile and get everything. I usually have an input and an output subfolder.

2.2.1 Four elements of any research design

• Model: set of models of what causes what and how
• Inquiry: a question stated in terms of the model
• Data strategy: the set of procedures we use to gather information from the world (sampling, assignment, measurement)
• Answer strategy: how we summarize the data produced by the data strategy

2.2.2 Four elements of any research design

2.2.3 Declaration

Design declaration is telling the computer (and readers) what M, I, D, and A are.

2.2.4 Diagnosis

• Design diagnosis is figuring out how the design will perform under imagined conditions.

• Estimating “diagnosands” like power, bias, rmse, error rates, ethical harm, “amount learned”.

• Diagnosis takes account of model uncertainty: it aims to identify models for which the design works well and models for which it does not

2.2.5 Redesign

Redesign is the fine-tuning of features of the data- and answer strategies to understand how changing them affects the diagnosands

• Different sample sizes
• Different randomization procedures
• Different estimation strategies
• Implementation: effort into compliance versus more effort into sample size

2.2.6 Very often you have to simulate!

• Doing all this is often too hard to work out from rules of thumb or power calculators
• Specialized formulas exist for some diagnosands, but not all

2.3.1 Key commands for making a design

• declare_model()

• declare_inquiry()

• declare_sampling()

• declare_assignment()

• declare_measurement()

• declare_estimator()

and there are more declare_ functions!

2.3.2 Key commands for using a design

• draw_data(design)
• draw_estimands(design)
• draw_estimates(design)
• get_estimates(design, data)
• run_design(design), simulate_design(design)
• diagnose_design(design)
• redesign(design, N = 200)
• compare_designs(), compare_diagnoses()

2.3.3 Pipeable commands

design |> 
  redesign(N = c(200, 400)) |>
  diagnose_designs() |> 
  tidy() |> 
  ggplot(...) 

2.3.4 Cheat sheet

https://raw.githubusercontent.com/rstudio/cheatsheets/master/declaredesign.pdf

2.3.5 Other resources

• The website: https://declaredesign.org/
• The book: https://book.declaredesign.org
• The console: ?DeclareDesign

2.4.1 The simplest possible (diagnosable) design?

mean <- 0
simplest_design <- 
  declare_model(N = 100, Y = rnorm(N, mean)) +
  declare_inquiry(Q = mean) +
  declare_estimator(Y ~ 1)

• we draw 100 units from a standard normal distribution
• we define our inquiry as the population expectation
• we estimate the average using a regression with a constant term

2.4.2 The simplest possible design?

simplest_design <- 
  declare_model(N = 100, Y = rnorm(N, mean)) +
  declare_inquiry(Q = 0) +
  declare_estimator(Y ~ 1)

• This design has three steps, with steps connected by a +
• The design itself is just a list of steps and has class design

str(simplest_design)

List of 3
 $ model    :design_step:    declare_model(N = 100, Y = rnorm(N, mean)) 
 $ Q        :design_step:    declare_inquiry(Q = 0) 
 $ estimator:design_step:    declare_estimator(Y ~ 1) 
 - attr(*, "call")= language construct_design(steps = steps)
 - attr(*, "class")= chr [1:2] "design" "dd"

2.4.3 The simplest possible design? It’s a pipe

Each step is a function (or rather: a function that generates functions) and each function presupposes what is created by previous functions.

• The ordering of steps is quite important
• Most steps take the main data frame in and push the main dataframe out; this data frame normally builds up as you move along the pipe.

2.4.4 The simplest possible design? It’s a pipe

Each step is a function (or rather: a function that generates functions) and each function presupposes what is created by previous functions.

• The ordering of steps is quite important
• declare_estimator steps take the main data frame in and send out an estimator_df dataframe
• declare_inquiry steps take the main data frame in and send out an estimand_df dataframe.

2.4.5 The simplest possible design? It’s a pipe

• You can run these functions one at a time if you like.
• For instance the third step presupposes the data from the first step:

df <- simplest_design[[1]]()
A  <- simplest_design[[3]](df)

A |> kable(digits = 2) |> kable_styling(font_size = 20)

Estimand  <- simplest_design[[2]](df)

Estimand |> kable(digits = 2) |> kable_styling(font_size = 20)

2.4.6 The simplest possible design? Run it once

You can also just run through the whole design once by typing the name of the design:

simplest_design

Research design declaration summary

Step 1 (model): declare_model(N = 100, Y = rnorm(N, mean)) ---------------------

Step 2 (inquiry): declare_inquiry(Q = 0) ---------------------------------------

Step 3 (estimator): declare_estimator(Y ~ 1) -----------------------------------

Run of the design:

 inquiry estimand estimator        term estimate std.error statistic p.value
       Q        0 estimator (Intercept)  0.00249    0.0916    0.0272   0.978
 conf.low conf.high df outcome
   -0.179     0.184 99       Y

2.4.7 The simplest possible design? Run it again

Or by asking for a run of the design

one_run <- simplest_design |> run_design()
one_run |> kable(digits = 2) |> kable_styling(font_size = 18)

A single run creates data, calculates estimands (the answer to inquiries) and calculates estimates plus ancillary statistics.

2.4.8 The simplest possible design?: Simulation

Or by asking for a run of the design

some_runs <- simplest_design |> simulate_design(sims = 1000)
some_runs |> kable(digits = 2) |> kable_styling(font_size = 16)