Blocking

Examples

Study 1

In many parts of the US, college access programs partner with high schools to provide advising and information to students and encourage them to enroll in college. Consider a study of the effectiveness of college access programs. 64 Texas high schools participated in the study, 2 from each of 32 geographic regions within the state. In each region, 1 of the high schools was randomly selected to receive support from a college access program and the other did received no support. The rates of college enrollment from each school were recorded at the end of the school year.

What are the four components of this experiment?

Study 1

In many parts of the US, college access programs partner with high schools to provide advising and information to students and encourage them to enroll in college. Consider a study of the effectiveness of college access programs. 64 Texas high schools participated in the study, 2 from each of 32 geographic regions within the state. In each region, 1 of the high schools was randomly selected to receive support from a college access program and the other did received no support. The rates of college enrollment from each school were recorded at the end of the school year.

1. Factors:
   • Experimental: college access program vs no support
   • Blocking: geographic region
2. Experimental units: high schools
3. Method of assignment: with each pair of schools in a region, one is randomly assigned to receive the college access program and the other is assigned to receive no support.
4. Response variable: rate of college enrollment from each school at the end of the school year.

Study 2

The goal of this study is to compare the effects of caffeine, theobromine, and a placebo, on people as measured by the rate at which they tap their fingers two hours after receiving the drug. Four subjects were recruited, and each subject received each of the three drugs. Their rate of finger tapping was measured two hours after each treatment.

Study 2

The goal of this study is to compare the effects of caffeine, theobromine, and a placebo, on people as measured by the rate at which they tap their fingers two hours after receiving the drug. Four subjects were recruited, and each subject received each of the three drugs. Their rate of finger tapping was measured two hours after each treatment.

1. Factors:
   • Experimental: drug (caffeine, theobromine, placebo)
   • Blocking: subject
2. Experimental units: subjects
3. Method of assignment: each subject received each of the three drugs, and the order in which they received the drugs was randomized.
4. Response variable: rate of finger tapping two hours after receiving the drug.

Study 3

In this experiment, public works projects occurring in two regions of Indonesia are selected at random for government auditing. There are six projects occurring in each region; due to resource constraints only two projects in each region can be audited. After each project concludes the amount of money unaccounted for (which is presumed stolen) is measured, in order to understand the impact of government oversight on corruption.

Study 3

In this experiment, public works projects occurring in two regions of Indonesia are selected at random for government auditing. There are six projects occurring in each region; due to resource constraints only two projects in each region can be audited. After each project concludes the amount of money unaccounted for (which is presumed stolen) is measured, in order to understand the impact of government oversight on corruption.

1. Factors:
   • Experimental: government auditing vs no auditing
   • Blocking: region
2. Experimental units: public works projects
3. Method of assignment: with each set of six projects in a region, two are randomly selected to receive government auditing and the other four are assigned to receive no auditing.
4. Response variable: amount of money unaccounted for (which is presumed stolen) at the conclusion of each project.

Block Designs

Block Designs

Complete Block Design \(CB[]\)

Units are partitioned into blocks based on a blocking factor. Treatments are assigned to units such that every treatment is applied exactly once per block. Examples: Study 1 and Study 2.

Generalized Complete Block Design \(GCB[]\)

Similar to \(CB[]\), but treatments can appear more than once per block and in unequal numbers. Example: Study 3.

In both, units are randomly assigned to treatments within each block.

Why Block?

Remove unwanted variability.

Draw data frame with columns “id”, “run#”, d, and y where id is the unit id, run# is the order in which the units were run, d is the treatment assignment, and y is the response. Show a time trend in the data by making y increase with run#.

Ask: How could I control for the “run” trend?

Answer: form blocks of size two and randomly assign the two units in each block to the two treatments. This way, the time trend is controlled for within each block, and we can more accurately estimate the treatment effect.

Forming Blocks

Blocks should be selected to maximize the variability in the response between blocks and to minimize the variability within each block.

1. Sort units into blocks based on a shared characteristic

2. Subdivide chunks into smaller pieces

3. Reuse material / subjects

Study 2: Drugs and Tapping

The Data

   taps person drug
1    11      1    P
2    20      1    T
3    26      1    C
4    83      2    C
5    56      2    P
6    71      2    T
7    41      3    T
8    15      3    P
9    34      3    C
10   32      4    T
11   13      4    C
12    6      4    P

EDA

EDA

Forming Blocks

Blocks should be selected to maximize the variability in the response between blocks and to minimize the variability within each block.

Inference

A Model for Data from \(CB[1]\)

\[ Y_i = \mu + \alpha_{j(i)} + \beta_{k(i)} + \epsilon_i \]

Estimates:

• \(\hat{\mu} = \hat{\bar{Y}}\)
• \(\hat{\alpha}_{j(i)} = \hat{\bar{Y}}_{j(i)} - \hat{\mu}\)
• \(\hat{\beta}_{k(i)} = \hat{\bar{Y}}_{k(i)} - \hat{\mu}\)

Note: \(CB[]\) doesn’t have enough data to estimate interactions (would need replicates: \(GCB[]\))

Design - Analysis Mismatch

Taps Model-based Inference

Additive model (no interaction)

anova_results <- aov(taps ~ person + drug, taps_df)
summary(anova_results)

            Df Sum Sq Mean Sq F value   Pr(>F)    
person       3   5478  1826.0   33.00 0.000399 ***
drug         2    872   436.0    7.88 0.020967 *  
Residuals    6    332    55.3                     
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Taps Model-based Inference

Additive model with interactions

anova_results <- aov(taps ~ person * drug, taps_df)
summary(anova_results)

            Df Sum Sq Mean Sq
person       3   5478  1826.0
drug         2    872   436.0
person:drug  6    332    55.3

Taps Model-based Inference

Analyzing like a \(CR[1]\)

anova_results <- aov(taps ~ drug, taps_df)
summary(anova_results)

            Df Sum Sq Mean Sq F value Pr(>F)
drug         2    872   436.0   0.675  0.533
Residuals    9   5810   645.6               

Taps Randomization-based Inference

Calculating the observed test statistic.

anova_results <- aov(taps ~ person + drug, taps_df)
obs_f_stat <- summary(anova_results)[[1]]$`F value`[2]
obs_f_stat

[1] 7.879518

Resampling within blocks.

taps_df |>
  group_by(person) |>
  mutate(Y = sample(taps))

# A tibble: 12 × 4
# Groups:   person [4]
    taps person drug      Y
   <dbl> <fct>  <fct> <dbl>
 1    11 1      P        20
 2    20 1      T        11
 3    26 1      C        26
 4    83 2      C        56
 5    56 2      P        83
 6    71 2      T        71
 7    41 3      T        41
 8    15 3      P        34
 9    34 3      C        15
10    32 4      T         6
11    13 4      C        13
12     6 4      P        32

taps_df |>
  group_by(person) |>
  mutate(Y = sample(taps))

# A tibble: 12 × 4
# Groups:   person [4]
    taps person drug      Y
   <dbl> <fct>  <fct> <dbl>
 1    11 1      P        11
 2    20 1      T        26
 3    26 1      C        20
 4    83 2      C        71
 5    56 2      P        83
 6    71 2      T        56
 7    41 3      T        41
 8    15 3      P        15
 9    34 3      C        34
10    32 4      T         6
11    13 4      C        32
12     6 4      P        13

block_shuffle <- function() { # specific to taps_df
  sim_df <- taps_df |>
    group_by(person) |>
    mutate(Y = sample(taps))
  
  anova_results <- aov(Y ~ person + drug, sim_df)
  f_stat <- summary(anova_results)[[1]]$`F value`[2]
  f_stat
}

replicate(10, block_shuffle())

 [1]  0.9714129  6.7621622  0.1908127  0.2007089  2.9653179  1.8777853
 [7]  1.8127915  1.1256425 12.8769231  5.8965517

null_stats <- replicate(500, block_shuffle())

mean(null_stats > obs_f_stat)

[1] 0.036

Randomization-based vs Model-based

Randomization-based:

mean(null_stats > obs_f_stat)

[1] 0.036

Model-based

anova_results <- aov(taps ~ person + drug, taps_df)
summary(anova_results)

            Df Sum Sq Mean Sq F value   Pr(>F)    
person       3   5478  1826.0   33.00 0.000399 ***
drug         2    872   436.0    7.88 0.020967 *  
Residuals    6    332    55.3                     
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1