1.1.3 Potential Outcomes

Idea: A causal claim is (in part) a claim about something that did not happen. This makes it metaphysical.

1.1.4 Potential Outcomes

Now that we have a concept of causal effects available, let’s answer two questions:

• TRANSITIVITY: If for a given unit \(A\) causes \(B\) and \(B\) causes \(C\), does that mean that \(A\) causes \(C\)?

1.1.5 Potential Outcomes

Now that we have a concept of causal effects available, let’s answer two questions:

• TRANSITIVITY: If for a given unit \(A\) causes \(B\) and \(B\) causes \(C\), does that mean that \(A\) causes \(C\)?

• A boulder is flying down a mountain. You duck. This saves your life.

• So the boulder caused the ducking and the ducking caused you to survive.

• So: did the boulder cause you to survive?

1.1.6 Potential Outcomes

CONNECTEDNESS Say \(A\) causes \(B\) — does that mean that there is a spatiotemporally continuous sequence of causal intermediates?

1.1.7 Potential Outcomes

CONNECTEDNESS Say \(A\) causes \(B\) — does that mean that there is a spatiotemporally continuous sequence of causal intermediates?

• Person A is planning some action \(Y\); Person B sets out to stop them; person X intervenes and prevents person B from stopping person A. In this case Person A may complete their action, producing Y, without any knowledge that B and X even exist; in particular B and X need not be anywhere close to the action. So: did X cause Y?

1.1.8 Causal claims: Contribution or attribution?

The counterfactual model is about contribution and attribution in a very specific sense.

• Focus is on non-rival contributions
• Focus is on conditional attribution. Not: “what caused \(Y\)?” or “What is the cause of \(Y\)?”, but “did \(X\) cause \(Y\) given all other factors were what they were?”

1.1.9 Causal claims: Contribution or attribution?

Consider an outcome \(Y\) that might depend on two causes \(X_1\) and \(X_2\):

\[Y(0,0) = 0\] \[Y(1,0) = 0\] \[Y(0,1) = 0\] \[Y(1,1) = 1\]

What caused \(Y\)? Which cause was most important?

1.1.11 Causal claims: Contribution or attribution?

Erdogan’s increasing authoritarianism was the most important reason for the attempted coup

• More important than Turkey’s history of coups?
• What does that mean?

1.1.12 Causal claims: No causation without manipulation

• Some seemingly causal claims not admissible.
• To get the definition off the ground, manipulation must be imaginable (whether practical or not)
• This renders thinking about effects of race and gender difficult
• What does it mean to say that Aunt Pat voted for Brexit because she is old?

1.1.13 Causal claims: No causation without manipulation

• Some seemingly causal claims not admissible.
• To get the definition off the ground, manipulation must be imaginable (whether practical or not)
• This renders thinking about effects of race and gender difficult
• Compare: What does it mean to say that Southern counties voted for Brexit because they have many old people?

1.1.14 Causal claims: No causation without manipulation

More uncomfortably:

What does it mean to say that the tides are caused by the moon? What exactly do we have to imagine…

1.1.15 Causal claims: Causal claims are everywhere

• Jack exploited Jill

• It’s Jill’s fault that bucket fell

• Jack is the most obstructionist member of Congress

• Melania Trump stole from Michelle Obama’s speech

• Activists need causal claims

1.1.16 Causal claims: What is actually seen?

• We have talked about what’s potential, now what do we observe?
• Say \(Z_i\) indicates whether the unit \(i\) is assigned to treatment \((Z_i=1)\) or not \((Z_i=0)\). It describes the treatment process. Then what we observe is: \[ Y_i = Z_iY_i(1) + (1-Z_i)Y_i(0) \]

This is sometimes called a “switching equation”

In DeclareDesign \(Y\) is realised from potential outcomes and assignment in this way using reveal_outcomes

1.1.17 Causal claims: What is actually seen?

• Say \(Z\) is a random variable, then this is a sort of data generating process. BUT the key thing to note is

  • \(Y_i\) is random but the randomness comes from \(Z_i\) — the potential outcomes, \(Y_i(1)\), \(Y_i(0)\) are fixed
  • Compare this to a regression approach in which \(Y\) is random but the \(X\)’s are fixed. eg: \[ Y \sim N(\beta X, \sigma^2) \text{ or } Y=\alpha+\beta X+\epsilon, \epsilon\sim N(0, \sigma^2) \]

1.1.18 Causal claims: The estimand and the rub

• The causal effect of Treatment (relative to Control) is: \[\tau_i = Y_i(1) - Y_i(0)\]
• This is what we want to estimate.
• BUT: We never can observe both \(Y_i(1)\) and \(Y_i(0)\)!
• This is the fundamental problem (Holland (1986))

1.1.22 Causal claims: The rub and the solution

This provides a positive argument for causal inference from randomization, rather than simply saying with randomization “everything else is controlled for”

Let’s discuss:

• Does the fact that an estimate is unbiased mean that it is right?
• Can a randomization “fail”?
• Where are the covariates?

1.1.23 Causal claims: The rub and the solution

Idea: random assignment is random sampling from potential worlds: to understand anything you find, you need to know the sampling weights

1.1.24 Reflection

Idea: We now have a positive argument for claiming unbiased estimation of the average treatment effect following random assignment

But is the average treatment effect a quantity of social scientific interest?

1.1.25 Potential outcomes: why randomization works

The average of the differences \(\approx\) difference of averages

1.1.26 Potential outcomes: heterogeneous effects

The average of the differences \(\approx\) difference of averages

1.1.27 Potential outcomes: heterogeneous effects

Question: \(\approx\) or \(=\)?

1.3.1 Endogeneous Subgroups

Experiments often give rise to endogenous subgroups. The potential outcomes framework can make it clear why this can cause problems.

We are going to look at three examples in which you might be tempted to condition on an endogeneous subgroup:

1. focusing on “reporting” subgroups
2. handling attrition
3. analyzing multistage behavior

[We will split up and try to make sense of each of these in groups]

1.3.2 Endogeneous Subgroups

General problem:

• is usually of the form: \(Y[X1, X2]\) — outcomes depend on multiple features, but you condition on the observed or revealed value of one

• usually requires heterogeneity

• involves conditioning on some post treatment feature that varies across types

1.3.3 Heterogeneous Effects with Endogeneous Categories

• Problems arise in analyses of subgroups when the categories themselves are affected by treatment

• Example from our work:

  • You want to know if an intervention affects reporting on violence against women
  • You measure the share of all subjects that experienced violence that file reports
  • The problem is that which subjects experienced violence is itself a function of treatment

1.3.5 Heterogeneous Effects with Endogeneous Categories

It is possible that in truth no one’s reporting behavior has changed, what has changed is the propensity of people with different propensities to report to experience violence:

1.3.6 Heterogeneous Effects with Endogeneous Categories

This problem can arise as easily in seemingly simple field experiments. Example:

• In one study we provided constituents with information about performance of politicians
• We told politicians in advance so that they could take action
• We wanted to see whether voters punished poorly performing politicians

What’s the problem?

1.3.7 Endogeneous Categories: Test yourself

Question for us:

• Quotas for women are randomly placed in a set of constituencies in year 1. All winners in these areas are women; in other areas only some are.
• In year 2 these quotas are then lifted.

Which problems face an endogenous subgroup issue?:

1.3.8 Endogeneous Categories: Test yourself

Which problems face an endogenous subgroup issue?:

1. You want to estimate the likelihood that a woman will stand for reelection in treatment versus control areas in year 2.
2. You want to estimate whether incumbents are more likely to be reelected in treatment versus control areas in year 2
3. You want to estimate how much treatment areas have more re-elected incumbents in elections in year 2 compared to control

1.3.9 Endogeneous Categories: Responses

In such cases you can:

• Examine the joint distribution of multiple outcomes
• Condition on pretreatment features only
• Engage in mediation analysis

1.3.13 Missing data

Note: The right way to think about this is that bias is a property of the strategy over possible realizations of data and not normally a property of the estimator conditional on the data.

1.3.14 Multistage games

Multistage games can also present an endogenous group problem since collections of late stage players facing a given choice have been created by early stage players.

1.3.16 Multistage games

Imagine you could see all the potential outcomes, and they looked like this:

Conclusion: Both the offer and the information condition are completely irrelevant for all subjects.

1.3.17 Multistage games

Unfortunately you only see a sample of the potential outcomes, and that looks like this:

False Conclusion: When not protected, responders condition behavior strongly on offers (because offerers can select on type accurately)

In fact: The nice types invest more because they are nice. The responders return more to the nice types because they are nice.

1.3.18 Multistage games

Unfortunately you only see a (noisier!) sample of the potential outcomes, and that looks like this:

False Conclusion: When protected, responders condition behavior less strongly on offers (because offerers can select on type less accurately)

2.1.3 Estimands: ATE, ATT, ATC, S-, P-, C-

In addition, each of these can be targets of interest:

• for the population, in which case we refer to PATE, PATT, PATC and \(\widehat{PATE}, \widehat{PATT}, \widehat{PATC}\)
• for a sample, in which case we refer to SATE, SATT, SATC, and \(\widehat{SATE}, \widehat{SATT}, \widehat{SATC}\)

And for different subgroups,

• given some value on a covariate, in which case we refer to CATE (conditional average treatment effect)

2.1.5 Quantile estimands

Other ways to condition on potential outcomes:

• A quantile treatment effect: You might be interested in the difference between the median \(Y(1)\) and the median \(Y(0)\) (Imbens and Rubin (2015) 20.3.1)
• or even be interested in the median \(Y(1) - Y(0)\). Similarly for other quantiles.

2.1.6 Model estimands

Many inquiries are averages of individual effects, even if the groups are not known, but they do not have to be:

• The RDD estimand is a statement about what effects would be at a threshold; it can be defined under a model even if no actual individuals are at the threshold. We imagine average potential outcomes as a function of treatment \(Z\) and running variable \(X\), \(f(z, x)\) and define: \[\tau_{RDD} := f(1, x^*) - f(0, x^*)\]

2.1.9 Estimands for a continuous treatment

Say our treatment \(X\) varies continuous over \([0,1]\), and is randomly assigned.

What estimand captures “the average effect of \(X\) on \(Y\)?

2.1.10 Estimands for a continuous treatment

In class exercise:

• Write down potential outcomes for two units that can take on up to 5 values of X.

• Assume a non linear but (largely?) homogeneous relationship (up to a constant, for example)

• Define your estimand in terms of the potential outcomes

• Imagine the multiple plot results you might get from observing two units.

2.4.2 Mediation estimands and complex counterfactuals

Again we might imagine that these are defined with respect to some group:

• \(A = \{i|Y_i(1, M(X=1)) > Y_i(0, M(X=0))\}\)
• \(\frac{1}{|A|} \sum_{i\in A}(Y(1, 1) > Y(0, 1))\)

here, among those for whom \(X\) has a positive effect on \(Y\), for what share would there be a positive effect if \(M\) were fixed at 1?

2.4.3 Causes of effects and effects of causes

In qualitative research a particularly common inquiry is “did \(X=1\) cause \(Y=1\)?

This is often given as a probability, the “probability of causation” (though at the case level we might better think of this probability as an estimate rather than an estimand):

\[\Pr(Y_i(0) = 0 | Y_i(1) = 1, X = 1)\]

2.4.4 Causes of effects and effects of causes

Intuition: What’s the probability \(X=1\) caused \(Y=1\) in an \(X=1, Y=1\) case drawn from a large population with the following experimental distribution:

2.4.5 Causes of effects and effects of causes

Intuition: What’s the probability \(X=1\) caused \(Y=1\) in an \(X=1, Y=1\) case drawn from a large population with the following experimental distribution:

2.4.6 Actual causation

Other inquiries focus on distinguishing between causes.

For the Billy Suzy problem (Hall 2004), Halpern (2016) focuses on “actual causation” as a way to distinguish between Suzy and Billy:

Imagine Suzy and Billy, simultaneously throwing stones at a bottle. Both are excellent shots and hit whatever they aim at. Suzy’s stone hits first, knocks over the bottle, and the bottle breaks. However, Billy’s stone would have hit had Suzy’s not hit, and again the bottle would have broken. Did Suzy’s throw cause the bottle to break? Did Billy’s?

2.4.7 Actual causation

Actual Causation:

1. \(X=x\) and \(Y=y\) both happened;
2. there is some set of variables, \(\mathcal W\), such that if they were fixed at the levels that they actually took on in the case, and if \(X\) were to be changed, then \(Y\) would change (where \(\mathcal W\) can also be an empty set);
3. no strict subset of \(X\) satisfies 1 and 2 (there is no redundant part of the condition, \(X=x\)).

2.4.8 Actual causation

• Suzy: Condition 2 is met if Suzy’s throw made a difference, counterfactually speaking—with the important caveat that, in determining this, we are permitted to condition on Billy’ stone not hitting the bottle.
• Billy: Condition 2 is not met.

An inquiry: for what share in a population is a possible cause an actual cause?

2.4.9 Pearl’s ladder

Pearl (e.g. Pearl and Mackenzie (2018)) describes three types of inquiry:

2.4.10 Pearl’s ladder

We can understand these as asking different types of questions about a causal model

The third is qualitatively different because it requires information about two mutually incompatible conditions for units. This is not (generally ) recoverable directly from knowledge of \(\Pr(Y(1)=1)\) and \(\Pr(Y(0)=0)\).