1.1.1 Core idea

• Generate a Frankenstein control from weighting a set of control units so that they collectively track the unit of interest and can plausibly serve as a counterfactual.

• Identify weights \(w\) that minimize \(\left(Y_i^s(0) - \sum_{j\neq i} w_j Y_j^s(0)\right)^2\) in time period(s) \(s\), prior to treatment; use these to predict counterfactual outcomes \(Y^t_i(0)\) in later periods, \(t\)

• Note, in Abadie and Gardeazabal (2003) weights are chosen to minimize \((X_1 - X_0W)'V(X_1 - X_0W)\) where \(X_1\), \(X_0\) are covariates in the unit of interest and the “donor” units respectively. \(V\) is chosen so that the pre treatment outcomes are as close as possible.

1.2.2 Basque example

Packages and data:

library(SCtools)
library(Synth)
data("basque")

1.2.3 Basque example

Some fairly nasty data manipulation:

f_dataprep  <- function(data, time_plot = min(data$year):max(data$year))
  dataprep(
  foo = data,
  predictors = c("school.illit", "school.prim", "school.med",
     "school.high", "school.post.high", 
    "invest"),
  predictors.op = "mean",
  time.predictors.prior = 1964:1969,
  special.predictors = list(
    list("gdpcap", 1960:1969 ,"mean"),
    list("sec.agriculture", seq(1961, 1969, 2), "mean"),
    list("sec.energy", seq(1961, 1969, 2), "mean"),
    list("sec.industry", seq(1961, 1969, 2), "mean"),
    list("sec.construction", seq(1961, 1969, 2), "mean"),
    list("sec.services.venta", seq(1961, 1969, 2), "mean"),
    list("sec.services.nonventa", seq(1961, 1969, 2), "mean"),
    list("popdens",               1969,               "mean")),
  dependent = "gdpcap",
  unit.variable = "regionno",
  unit.names.variable = "regionname",
  time.variable = "year",
  treatment.identifier = 17,
  controls.identifier = c(2:16, 18),
  time.optimize.ssr = 1960:1969,
  time.plot = time_plot)

dataprep.out  <- f_dataprep(basque)

1.2.4 Implementation (with messages)

synth.out <- synth(data.prep.obj = dataprep.out, method = "BFGS")

X1, X0, Z1, Z0 all come directly from dataprep object.


**************** 
 searching for synthetic control unit  
 

**************** 
**************** 
**************** 

MSPE (LOSS V): 0.008864606 

solution.v:
 0.02773094 1.194e-07 1.60609e-05 0.0007163836 1.486e-07 0.002423908 0.0587055 0.2651997 0.02851006 0.291276 0.007994382 0.004053188 0.009398579 0.303975 

solution.w:
 2.53e-08 4.63e-08 6.44e-08 2.81e-08 3.37e-08 4.844e-07 4.2e-08 4.69e-08 0.8508145 9.75e-08 3.2e-08 5.54e-08 0.1491843 4.86e-08 9.89e-08 1.162e-07 

1.2.5 Results

This produces the following weights, selected to render the pre-treatment trends of the synthetic and actual controls as similar as possible.

1.2.9 tidysynth implementation

tidysynth a wrapper for synth and tidier for sure

Vignette on: Impact of Proposition 99 on cigarette consumption in California. https://github.com/edunford/tidysynth

data("smoking")
smoking |> head() |> kable()

1.2.11 add predictor information

smoking_out <- smoking_out %>%
  
  # Generate the aggregate predictors used to fit the weights
  
  # average log income, retail price of cigarettes, and proportion of the
  # population between 15 and 24 years of age from 1980 - 1988
  generate_predictor(time_window = 1980:1988,
                     ln_income = mean(lnincome, na.rm = T),
                     ret_price = mean(retprice, na.rm = T),
                     youth = mean(age15to24, na.rm = T)) %>%
  
  # average beer consumption in the donor pool from 1984 - 1988
  generate_predictor(time_window = 1984:1988, 
                     beer_sales = mean(beer, na.rm = T)) %>%
  
  # Lagged cigarette sales 
  generate_predictor(time_window = 1975, cigsale_1975 = cigsale) %>%
  generate_predictor(time_window = 1980, cigsale_1980 = cigsale) %>%
  generate_predictor(time_window = 1988, cigsale_1988 = cigsale) 

1.2.12 implement

smoking_out <- smoking_out %>%
  
  
  # Generate the fitted weights for the synthetic control
  generate_weights(optimization_window = 1970:1988, # time for optimization
                   margin_ipop = .02,sigf_ipop = 7,bound_ipop = 6 # options
  ) %>%
  
  # Generate the synthetic control
  generate_control()

1.2.16 Design declaration

sc_helper <- function(data, treatment_time = 2010)

  data |>
  
  # initial the synthetic control object
  synthetic_control(
    outcome = Y, # outcome
    unit = state, # unit index in the panel data
    time = year, # time index in the panel data
    i_unit = "01", # unit where the intervention occurred
    i_time = treatment_time, # time period when the intervention occurred
    generate_placebos = F  # generate placebo synthetic controls
    )  |>
  
  generate_predictor(time_window = 2000:2020,
                     lnincome = mean(ln_income, na.rm = T),
                     youth = mean(age15to24, na.rm = T)) %>%
  generate_weights(optimization_window = 2000:2010, # time for optimization
                   margin_ipop = .02,sigf_ipop = 7,bound_ipop = 6 # options
  ) |>
  generate_control()

1.3.1 Local Average Treatment Effects

Sometimes you give a medicine but only a nonrandom sample of people actually try to use it. Can you still estimate the medicine’s effect?

Say that people are one of 3 types:

1. \(n_a\) “always takers” have \(X=1\) no matter what and have average outcome \(\overline{y}_a\)
2. \(n_n\) “never takers” have \(X=0\) no matter what with outcome \(\overline{y}_n\)
3. \(n_c\) “compliers have” \(X=Z\) and average outcomes \(\overline{y}^1_c\) if treated and \(\overline{y}^0_c\) if not.

1.3.2 Local Average Treatment Effects

Sometimes you give a medicine but only a non random sample of people actually try to use it. Can you still estimate the medicine’s effect?

We can figure something about types:

1.3.3 Local Average Treatment Effects

You give a medicine to 50% but only a non random sample of people actually try to use it. Can you still estimate the medicine’s effect?

Key insight: the contributions of the \(a\)s and \(n\)s are the same in the \(Z=0\) and \(Z=1\) groups so if you difference you are left with the changes in the contributions of the \(c\)s.

1.3.4 Local Average Treatment Effects

Average in \(Z=0\) group: \(\frac{{n_c} \overline{y}^0_{c}+ \left(n_{n}\overline{y}_{n} +{n_a} \overline{y}_a\right)}{n_a+n_c+n_n}\)

Average in \(Z=1\) group: \(\frac{{n_c} \overline{y}^1_{c} + \left(n_{n}\overline{y}_{n} +{n_a} \overline{y}_a \right)}{n_a+n_c+n_n}\)

So, the difference is the ITT: \(({\overline{y}^1_c-\overline{y}^0_c})\frac{n_c}{n}\)

Last step:

\[ITT = ({\overline{y}^1_c-\overline{y}^0_c})\frac{n_c}{n}\]

\[\leftrightarrow\]

\[LATE = \frac{ITT}{\frac{n_c}{n}}= \frac{\text{Intent to treat effect}}{\text{First stage effect}}\]

1.3.5 The good and the bad of LATE

• You get a well-defined estimate even when there is non-random take-up
• May sometimes be used to assess mediation or knock-on effects
• But:
  • You need assumptions (monotonicity and the exclusion restriction – where were these used above?)
  • Your estimate is only for a subpopulation
  • The subpopulation is not chosen by you and is unknown
  • Different encouragements may yield different estimates since they may encourage different subgroups

1.4.1 Constraint: Is it ethical to manipulate subjects for research purposes?

• There is no foundationless answer to this question.

• Belmont principles commonly used for guidance:

  1. Respect for persons
  2. Beneficence
  3. Justice

• Unfortunately, operationalizing these requires further ethical theories. (1) is often operationalized by informed consent (a very liberal idea). (2) and (3) sometimes by more utiliarian principles

• The major focus on (1) by IRBs might follow from the view that if subjects consent, then they endorse the ethical calculations made for 2 and 3 — they think that it is good and fair.

• Trickiness: can a study be good or fair because of implications for non-subjects?

1.4.2 Is it ethical to manipulate subjects for research purposes?

• Many (many) field experiments have nothing like informed consent.

• For example, whether the government builds a school in your village, whether an ad appears on your favorite radio show, and so on.

• Consider three cases:

  1. You work with a nonprofit to post (true?) posters about the crimes of politicians on billboards to see effects on voters
  2. You hire confederates to offer bribes to police officers to see if they are more likely to bend the law for coethnics
  3. The British government asks you to work on figuring out how the use of water cannons helps stop rioters rioting

1.4.3 Is it ethical to manipulate subjects for research purposes?

• Consider three cases:

  • You work with a nonprofit to post (true?) posters about the crimes of politicians on billboards to see effects on voters
  • You hire confederates to offer bribes to police officers to see if they are more likely to bend the law for coethnics
  • The British government asks you to work on figuring out how the use of water cannons helps stop rioters rioting

• In all cases, there is no consent given by subjects.

• In 2 and 3, the treatment is possibly harmful for subjects, and the results might also be harmful. But even in case 1, there could be major unintended harmful consequences.

• In cases 1 and 3, however, the “intervention” is within the sphere of normal activities for the implementer.

1.4.4 Constraint: Is it ethical to manipulate subjects for research purposes?

• Sometimes it is possible to use this point of difference to make a “spheres of ethics” argument for “embedded experimentation.”

• Spheres of Ethics Argument: Experimental research that involves manipulations that are not normally appropriate for researchers may nevertheless be ethical if:

  • Researchers and implementers agree on a division of responsibility where implementers take on responsibility for actions
  • Implementers have legitimacy to make these decisions within the sphere of the intervention
  • Implementers are indeed materially independent of researchers (no swapping hats)

1.4.5 Constraint: Is it ethical to manipulate subjects for research purposes?

• Difficulty with this argument:
  • Question begging: How to determine the legitimacy of the implementer? (Can we rule out Nazi doctors?)

Otherwise keep focus on consent and desist if this is not possible

1.4.7 APSA Ethics: General [Abbr]

1. Political science researchers should respect autonomy, consider the wellbeing of participants and other people affected by their research, and be open about the ethical issues they face.

2. Political science researchers have an individual responsibility to consider the ethics of their research related activities and cannot outsource ethical reflection to review boards, other institutional bodies, or regulatory agencies.

3. These principles describe the standards of conduct and reflexive openness that are expected of political science researchers. … [In cases of reasonable deviations], researchers should acknowledge and justify deviations in scholarly publications and presentations of their work.

1.4.8 APSA Ethics: Power

4. When designing and conducting research, political scientists should be aware of power differentials between researcher and researched, and the ways in which such power differentials can affect the voluntariness of consent and the evaluation of risk and benefit.

1. especially with low-power or vulnerable participants
2. covert or deceptive research with more than minimal harm may sometimes be ethically permissible in research with powerful parties

1.4.9 APSA Ethics: Consent

5. Political science researchers should generally seek informed consent from individuals who are directly engaged by the research process, especially if research involves more than minimal risk of harm or if it is plausible to expect that engaged individuals would withhold consent if consent were sought.

1. informed and voluntary
2. continuing consent
3. observation of public behavior does not usually invoke this principle of consent.
4. forgoing perhaps if minimal risk, or seeking consent increases the risks for participants
5. discuss consent in publications and presentations

1.4.11 APSA Ethics: Harm and Trauma

7. Political science researchers should consider the harms associated with their research.

• Researchers should generally avoid harm when possible, minimize harm when avoidance is not possible, and not conduct research when harm is excessive.

• do not limit concern to physical and psychological risks to the participant.

8. Political science researchers should anticipate and protect individual participants from trauma stemming from participation in research.

1.4.12 APSA Ethics: Confidentiality

9. Political science researchers should generally keep the identities of research participants confidential; when circumstances require, researchers should adopt the higher standard of ensuring anonymity.

1. Researchers should clearly communicate assurances of confidentiality / anonymity
2. If confidentiality bit provided, communicate this and justify c./d. consider risks at all stages
3. Researchers who determine that it would be unethical to share materials derived from human subjects should be prepared to justify their decision to journal editors, to reviewers, etc

1.4.13 APSA Ethics: Impact

10. Political science researchers conducting studies on political processes should consider the broader social impacts of the research process as well as the impact on the experience of individuals directly engaged by the research. In general, political science researchers should not compromise the integrity of political processes for research purposes without the consent of individuals that are directly engaged by the research process.

1. cases in which research that produces impacts on political processes without consent of individuals directly engaged by the research might be appropriate. [examples]

2. Studies of interventions by third parties do not usually invoke this principle on impact. [details]

3. This principle is not intended to discourage any form of political engagement by political scientists in their non-research activities or private lives.

4. researchers should report likely impacts

1.4.15 APSA Ethics: Shared Responsibility

12. The responsibility to promote ethical research goes beyond the individual researcher or research team.

1. Mentors, advisors, dissertation committee members, and instructors

2. Graduate programs in political science should include ethics instruction in their formal and informal graduate curricula;

3. Editors and reviewers should encourage researchers to be open about the ethical decisions …

4. Journals, departments, and associations should incorporate ethical commitments into their mission, bylaws, instruction, practices, and procedures.

1.5.4 The list experiment: Strategy

• Respondents are given a short list and a long list.

• The long list differs from the short list in having one extra item—the sensitive item

• We ask how many items in each list does a respondent agree with:

  • \(Y_i(0)\) is the number of elements on a short list that a respondent agrees with
  • \(Y_i(1)\) is the number of elements on a long list that a respondent agrees with
  • \(Y_i(1) - Y_i(0)\) is an indicator for whether an individual agrees with the sensitive item
  • \(\mathbb{E}[Y_i(1) - Y_i(0)]\) is the share of people agreeing with sensitive item

1.5.5 The list experiment: Simplified example

How many of these do you agree with:

[Note: this is obviously not a good list. Why not?]

1.5.14 Individual or group effects?

• This is typically used to estimate average levels

• However you can use it in the obvious way to get average levels for groups: this is equivalent to calculating group level heterogeneous effects

• Extending the idea you can even get individual level estimates: for instance you might use causal forests

• You can also use this to estimate the effect of an experimental treatment on an item that’s measured using a list, without requiring individual level estimates:

\[Y_i = \beta_0 + \beta_1Z_i + \beta_2Long_i + \beta_3Z_iLong_i\]

1.6.2 Intuition

• Kids born on 31 August start school a year younger than kids born on 1 September: does starting younger help or hurt?

• Kids born on 12 September 1983 are more likely to register Republican than kids born on 10 September 1983: can this identify the effects of registration on long term voting?

• A district in which Republicans got 50.1% of the vote get a Republican representative while districts in which Republicans got 49.9% of the vote do not: does having a Republican representative make a difference for these districts?

1.6.3 Argument for identification

Setting:

• Typically the decision is based on a value on a “running variable”, \(X\). e.g. Treatment if \(X > 0\)
• The estimand is \(\mathbb{E}[Y(1) - Y(0)|X=0]\)

Two arguments:

1. Continuity: \(\mathbb{E}[Y(1)|X=x]\) and \(\mathbb{E}[Y(0)|X=x]\) are continuous (at \(x=0\)) in \(x\): so \(\lim_{\hat x \rightarrow 0}\mathbb{E}[Y(0)|X=\hat x] = \mathbb{E}[Y(0)|X=\hat 0]\)

2. Local randomization: tiny things that determine exact values of \(x\) are as if random and so we can think of a local experiment around \(X=0\).

1.6.4 Argument for identification

Note:

• continuity argument requires continuous \(x\): granularity
• also builds off a conditional expectation function defined at \(X=0\)

Exclusion restriction is implicit in continuity: If something else happens at the threshold then the conditional expectation functions jump at the thresholds

Implicit: \(X\) is exogenous in the sense that units cannot adjust \(X\) in order to be on one or the other side of the threshold

1.6.11 Bandwidth tradeoff

rdd_design |> 
  redesign(bandwidth = seq(from = 0.05, to = 0.5, by = 0.05)) |> 
  diagnose_designs()

• As we increase the bandwidth, the lm bias gets worse, but slowly, while the error falls.
• The best bandwidth is relatively wide.
• This is more true for the optimal estimator.