Problem Set 6

1. Under pressure: Interference (from G & G Ch. 8): In their study of spillover effects, Sinclair, McConnell, and Green¹ sent mailings to randomly selected households in Chicago encouraging them to vote in an upcoming special election on April 7th 2009 (Figure 1). The mailings used a form of social pressure, disclosing whether the targeted individual had voted in previous elections. Because this type of mail had proven to increase turnout by approximately 4-5 percentage points in previous experiments, the authors used it to study whether the treatment effects are transmitted across households.

   Employing a multilevel design, they randomly assigned all, half, or none of the members of each nine-digit zip code to receive mail. For purposes of this example, we focus only on households with one registered voter. The outcome variable is voter turnout as measured by the registrar of voters. The results are as follows. Among registered voters in untreated zip codes, 1,201 of 6,217 cast ballots. Among untreated voters in zip codes where half of the households received mail, 526 of 3,316 registered voters cast ballots. Among treated voters in zip codes where half of the households received mail, 620 of 2,949 voted. Finally, among treated voters in zip codes where every household received mail, turnout was 1,316 of 6,377.

   1. Identify the response variable, the units (measurement and experimental), and the experiental factor(s) and their levels.

   2. Using potential outcomes, define the direct treatment effects (parameters) of receiving mail addressed to subject \(i\).

   3. Define the spillover treatment effects (also parameters) of being in a zip code where varying fractions of households are treated.

   4. Propose an estimator for estimating the firsthand (direct) and secondhand (spillover) treatment effects. Show that the estimator is unbiased, explaining the assumptions required to reach this conclusion.

   5. Based on these data, what are your estimates of the magnitude of the mailing’s direct and spillover effects? What did we learn here scientifically?

   

   Figure 1: Example of postcard mailed to registered voters.

2. Bring Out Your Bins: Non-compliance Cotterill et al. report the results of an experiment conducted in an area of the United Kingdom where only half of the local residents recycle their trash². Canvassers visited homes and encouraged residents to recycle. Outcomes were measured by whether the home put out a recycling bin on at least one occasion during the following three weeks. We restrict our attention here to homes that did not recycle trash during a pre-experimental period of observation. When implementing the intervention, researchers encountered one-side noncompliance: 1,105 of the 1,849 homes assigned to the treatment group were successfully canvassed; none of the 1,430 homes assigned to the control group were canvassed. These researchers found that 591 homes in the treatment group recycled, as opposed to 377 in the control group. The researchers also observed that 429 of 1,105 homes that were successfully canvassed recycled, as opposed to 539 of the 2,264 homes that were not canvassed.

1. Estimate the \(ITT_Y\) and interpret what that statistic captures in the context of this study.

2. Estimate the \(ITT_D\) and interpret what that statistic captures.

3. Estimate the \(CACE\) and interpret what that statistic captures.

4. Explain why comparing the recycling rates of the treated and untreated subjects tends to produce misleading estimates of the \(CACE\) and the \(ATE\).

More questions coming soon!

Footnotes

1. Sinclair, McConnell, and Green (2012). “Detecting Spillover Effects: Design and Analysis of Multilevel Experiments” in American Journal of Political Science.

2. Cotteril el al (2009). “Mobilizing Citizen Effort to Enhance Environmental Outcomes: A Randomized Controlled Trail of a Door-to-Door Recycling Campaign.” in Journal of Environmental Management.