Topics

1. The MIDA framework and the declaration-diagnosis-redesign cycle
2. Brief intro to DeclareDesign
3. \(p\)-values review
4. Power review
5. Power via design diagnosis: the how to and advantages thereof
6. Lessons & strategies
7. Some applications

Timing

• Part I: Topics 1-4 and exercises 1
• Break: 10:13 - 10:33
• Part II: Topics 5-6, 10:33 - 11:30
• Break: 11:30 - 11:45
• Part III: Topic 7, 11:45 - 12:30

Declaration

Q: Is my research design good?

A: Well let’s simulate it to see how it performs.

Q: What should I put in the simulation?

A: All elements of a research design.

Q: What are the elements of a research design?

A: M! I! D! A!

Four elements of any research design

• Model: set of models of what causes what and how
• Inquiry: a question stated in terms of the model
• Data strategy: the set of procedures we use to gather information from the world (sampling, assignment, measurement)
• Answer strategy: how we summarize the data produced by the data strategy

Four elements of any research design

Declaration

Telling the computer what M, I, D, and A are.

Diagnosis

Estimating “diagnosands” like power, bias, rmse, error rates, ethical harm, amount learned.

• want to diagnose over model uncertainty

Redesign

Fine-tuning features of the data and answer strategies to understand how they change the diagnosands

• Different sample sizes
• Different randomization procedures
• Different estimation strategies
• Implementation: effort into compliance versus more effort into sample size

Very often you have to simulate!

• This is too hard to work out from rules of thumb or power calculators
• Specialized formulas exist for some diagnosands, but not all.

Key commands for making a design

• declare_model()
• declare_inquiry()
• declare_assignment()
• declare_measurement()
• declare_inquiry
• declare_estimator()

and there are more declare_ functions!

Key commands for using a design

• draw_data(design)
• draw_estimands(design)
• draw_estimates(design)
• get_estimates(design, data)
• run_design(design), simulate_design(design)
• diagnose_design(design)
• redesign(design, N = 200)
• design |> redesign(N = c(200, 400)) |> diagnose_designs()
• compare_designs(), compare_diagnoses()

Cheat sheet

https://raw.githubusercontent.com/rstudio/cheatsheets/master/declaredesign.pdf