To understand the behavior of estimators under a random process, we start by assuming the impossible: that we know the full schedule of potential outcomes. Here it is, for six students who went through the anchoring experiment.
# A tibble: 6 × 2
Y_0 Y_1
<dbl> <dbl>
1 15 31
2 15 22
3 19 45
4 2 20
5 10 20
6 8 15Let’s compute the individual treatment effect (ITE) for each of the size students.
# A tibble: 6 × 3
Y_0 Y_1 tau
<dbl> <dbl> <dbl>
1 15 31 16
2 15 22 7
3 19 45 26
4 2 20 18
5 10 20 10
6 8 15 7Now we can calculate summary statistics on the schedule, which are better thought of as parameters.
# A tibble: 1 × 3
Ybar_1 Ybar_0 ATE
<dbl> <dbl> <dbl>
1 25.5 11.5 14Note that the Average Treatment Effect (ATE), 14, our estimand, can either be expressed as the difference between the means of the potential outcomes, \(\bar{Y}_1 - \bar{Y}_0\), or as the average of the \(\tau_i\).
Let’s simulate a single randomized experiment conducted on these six students by randomly assigning them to get the treatment (1) or the control (0).
# A tibble: 6 × 5
Y_0 Y_1 tau d_i y_i
<dbl> <dbl> <dbl> <dbl> <dbl>
1 15 31 16 1 31
2 15 22 7 1 22
3 19 45 26 0 19
4 2 20 18 1 20
5 10 20 10 0 10
6 8 15 7 0 8Our estimator for the ATE will be the difference in the average responses of the units in the treated group and the control group.
[1] 12In a normal data analysis, all we get to see is our estimate, 12. Since we know the full schedule, here we’re effectively playing this game in “God Mode” and can know that the value of the parameter was in fact 14. So this particular estimate was a bit off, but not by much.
To learn about how much variability we can expect in our estimator, we need to know it’s sampling distribution.
To learn the sampling distribution of \(\widehat{ATE}\), we need to
To aid in step 1, we can use the permutations() function to show every permutation of the a vector containing size elements, three of which indicate the treatment (1) and three of which indicate the control (0). We can imagine that the indices of this vector correspond to the indices of the students.
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0 0 0 1 1 1
[2,] 0 0 0 1 1 1
[3,] 0 0 0 1 1 1
[4,] 0 0 0 1 1 1
[5,] 0 0 0 1 1 1
[6,] 0 0 0 1 1 1
[7,] 0 0 1 0 1 1
[8,] 0 0 1 0 1 1
[9,] 0 0 1 1 0 1
[10,] 0 0 1 1 1 0
[11,] 0 0 1 1 0 1
[12,] 0 0 1 1 1 0
[13,] 0 0 1 0 1 1
[14,] 0 0 1 0 1 1
[15,] 0 0 1 1 0 1
[16,] 0 0 1 1 1 0
[17,] 0 0 1 1 0 1
[18,] 0 0 1 1 1 0
[19,] 0 0 1 0 1 1
[20,] 0 0 1 0 1 1
[21,] 0 0 1 1 0 1
[22,] 0 0 1 1 1 0
[23,] 0 0 1 1 0 1
[24,] 0 0 1 1 1 0
[25,] 0 0 0 1 1 1
[26,] 0 0 0 1 1 1
[27,] 0 0 0 1 1 1
[28,] 0 0 0 1 1 1
[29,] 0 0 0 1 1 1
[30,] 0 0 0 1 1 1
[31,] 0 0 1 0 1 1
[32,] 0 0 1 0 1 1
[33,] 0 0 1 1 0 1
[34,] 0 0 1 1 1 0
[35,] 0 0 1 1 0 1
[36,] 0 0 1 1 1 0
[37,] 0 0 1 0 1 1
[38,] 0 0 1 0 1 1
[39,] 0 0 1 1 0 1
[40,] 0 0 1 1 1 0
[41,] 0 0 1 1 0 1
[42,] 0 0 1 1 1 0
[43,] 0 0 1 0 1 1
[44,] 0 0 1 0 1 1
[45,] 0 0 1 1 0 1
[46,] 0 0 1 1 1 0
[47,] 0 0 1 1 0 1
[48,] 0 0 1 1 1 0
[49,] 0 1 0 0 1 1
[50,] 0 1 0 0 1 1
[51,] 0 1 0 1 0 1
[52,] 0 1 0 1 1 0
[53,] 0 1 0 1 0 1
[54,] 0 1 0 1 1 0
[55,] 0 1 0 0 1 1
[56,] 0 1 0 0 1 1
[57,] 0 1 0 1 0 1
[58,] 0 1 0 1 1 0
[59,] 0 1 0 1 0 1
[60,] 0 1 0 1 1 0
[61,] 0 1 1 0 0 1
[62,] 0 1 1 0 1 0
[63,] 0 1 1 0 0 1
[64,] 0 1 1 0 1 0
[65,] 0 1 1 1 0 0
[66,] 0 1 1 1 0 0
[67,] 0 1 1 0 0 1
[68,] 0 1 1 0 1 0
[69,] 0 1 1 0 0 1
[70,] 0 1 1 0 1 0
[71,] 0 1 1 1 0 0
[72,] 0 1 1 1 0 0
[73,] 0 1 0 0 1 1
[74,] 0 1 0 0 1 1
[75,] 0 1 0 1 0 1
[76,] 0 1 0 1 1 0
[77,] 0 1 0 1 0 1
[78,] 0 1 0 1 1 0
[79,] 0 1 0 0 1 1
[80,] 0 1 0 0 1 1
[81,] 0 1 0 1 0 1
[82,] 0 1 0 1 1 0
[83,] 0 1 0 1 0 1
[84,] 0 1 0 1 1 0
[85,] 0 1 1 0 0 1
[86,] 0 1 1 0 1 0
[87,] 0 1 1 0 0 1
[88,] 0 1 1 0 1 0
[89,] 0 1 1 1 0 0
[90,] 0 1 1 1 0 0
[91,] 0 1 1 0 0 1
[92,] 0 1 1 0 1 0
[93,] 0 1 1 0 0 1
[94,] 0 1 1 0 1 0
[95,] 0 1 1 1 0 0
[96,] 0 1 1 1 0 0
[97,] 0 1 0 0 1 1
[98,] 0 1 0 0 1 1
[99,] 0 1 0 1 0 1
[100,] 0 1 0 1 1 0
[101,] 0 1 0 1 0 1
[102,] 0 1 0 1 1 0
[103,] 0 1 0 0 1 1
[104,] 0 1 0 0 1 1
[105,] 0 1 0 1 0 1
[106,] 0 1 0 1 1 0
[107,] 0 1 0 1 0 1
[108,] 0 1 0 1 1 0
[109,] 0 1 1 0 0 1
[110,] 0 1 1 0 1 0
[111,] 0 1 1 0 0 1
[112,] 0 1 1 0 1 0
[113,] 0 1 1 1 0 0
[114,] 0 1 1 1 0 0
[115,] 0 1 1 0 0 1
[116,] 0 1 1 0 1 0
[117,] 0 1 1 0 0 1
[118,] 0 1 1 0 1 0
[119,] 0 1 1 1 0 0
[120,] 0 1 1 1 0 0
[121,] 0 0 0 1 1 1
[122,] 0 0 0 1 1 1
[123,] 0 0 0 1 1 1
[124,] 0 0 0 1 1 1
[125,] 0 0 0 1 1 1
[126,] 0 0 0 1 1 1
[127,] 0 0 1 0 1 1
[128,] 0 0 1 0 1 1
[129,] 0 0 1 1 0 1
[130,] 0 0 1 1 1 0
[131,] 0 0 1 1 0 1
[132,] 0 0 1 1 1 0
[133,] 0 0 1 0 1 1
[134,] 0 0 1 0 1 1
[135,] 0 0 1 1 0 1
[136,] 0 0 1 1 1 0
[137,] 0 0 1 1 0 1
[138,] 0 0 1 1 1 0
[139,] 0 0 1 0 1 1
[140,] 0 0 1 0 1 1
[141,] 0 0 1 1 0 1
[142,] 0 0 1 1 1 0
[143,] 0 0 1 1 0 1
[144,] 0 0 1 1 1 0
[145,] 0 0 0 1 1 1
[146,] 0 0 0 1 1 1
[147,] 0 0 0 1 1 1
[148,] 0 0 0 1 1 1
[149,] 0 0 0 1 1 1
[150,] 0 0 0 1 1 1
[151,] 0 0 1 0 1 1
[152,] 0 0 1 0 1 1
[153,] 0 0 1 1 0 1
[154,] 0 0 1 1 1 0
[155,] 0 0 1 1 0 1
[156,] 0 0 1 1 1 0
[157,] 0 0 1 0 1 1
[158,] 0 0 1 0 1 1
[159,] 0 0 1 1 0 1
[160,] 0 0 1 1 1 0
[161,] 0 0 1 1 0 1
[162,] 0 0 1 1 1 0
[163,] 0 0 1 0 1 1
[164,] 0 0 1 0 1 1
[165,] 0 0 1 1 0 1
[166,] 0 0 1 1 1 0
[167,] 0 0 1 1 0 1
[168,] 0 0 1 1 1 0
[169,] 0 1 0 0 1 1
[170,] 0 1 0 0 1 1
[171,] 0 1 0 1 0 1
[172,] 0 1 0 1 1 0
[173,] 0 1 0 1 0 1
[174,] 0 1 0 1 1 0
[175,] 0 1 0 0 1 1
[176,] 0 1 0 0 1 1
[177,] 0 1 0 1 0 1
[178,] 0 1 0 1 1 0
[179,] 0 1 0 1 0 1
[180,] 0 1 0 1 1 0
[181,] 0 1 1 0 0 1
[182,] 0 1 1 0 1 0
[183,] 0 1 1 0 0 1
[184,] 0 1 1 0 1 0
[185,] 0 1 1 1 0 0
[186,] 0 1 1 1 0 0
[187,] 0 1 1 0 0 1
[188,] 0 1 1 0 1 0
[189,] 0 1 1 0 0 1
[190,] 0 1 1 0 1 0
[191,] 0 1 1 1 0 0
[192,] 0 1 1 1 0 0
[193,] 0 1 0 0 1 1
[194,] 0 1 0 0 1 1
[195,] 0 1 0 1 0 1
[196,] 0 1 0 1 1 0
[197,] 0 1 0 1 0 1
[198,] 0 1 0 1 1 0
[199,] 0 1 0 0 1 1
[200,] 0 1 0 0 1 1
[201,] 0 1 0 1 0 1
[202,] 0 1 0 1 1 0
[203,] 0 1 0 1 0 1
[204,] 0 1 0 1 1 0
[205,] 0 1 1 0 0 1
[206,] 0 1 1 0 1 0
[207,] 0 1 1 0 0 1
[208,] 0 1 1 0 1 0
[209,] 0 1 1 1 0 0
[210,] 0 1 1 1 0 0
[211,] 0 1 1 0 0 1
[212,] 0 1 1 0 1 0
[213,] 0 1 1 0 0 1
[214,] 0 1 1 0 1 0
[215,] 0 1 1 1 0 0
[216,] 0 1 1 1 0 0
[217,] 0 1 0 0 1 1
[218,] 0 1 0 0 1 1
[219,] 0 1 0 1 0 1
[220,] 0 1 0 1 1 0
[221,] 0 1 0 1 0 1
[222,] 0 1 0 1 1 0
[223,] 0 1 0 0 1 1
[224,] 0 1 0 0 1 1
[225,] 0 1 0 1 0 1
[226,] 0 1 0 1 1 0
[227,] 0 1 0 1 0 1
[228,] 0 1 0 1 1 0
[229,] 0 1 1 0 0 1
[230,] 0 1 1 0 1 0
[231,] 0 1 1 0 0 1
[232,] 0 1 1 0 1 0
[233,] 0 1 1 1 0 0
[234,] 0 1 1 1 0 0
[235,] 0 1 1 0 0 1
[236,] 0 1 1 0 1 0
[237,] 0 1 1 0 0 1
[238,] 0 1 1 0 1 0
[239,] 0 1 1 1 0 0
[240,] 0 1 1 1 0 0
[241,] 0 0 0 1 1 1
[242,] 0 0 0 1 1 1
[243,] 0 0 0 1 1 1
[244,] 0 0 0 1 1 1
[245,] 0 0 0 1 1 1
[246,] 0 0 0 1 1 1
[247,] 0 0 1 0 1 1
[248,] 0 0 1 0 1 1
[249,] 0 0 1 1 0 1
[250,] 0 0 1 1 1 0
[251,] 0 0 1 1 0 1
[252,] 0 0 1 1 1 0
[253,] 0 0 1 0 1 1
[254,] 0 0 1 0 1 1
[255,] 0 0 1 1 0 1
[256,] 0 0 1 1 1 0
[257,] 0 0 1 1 0 1
[258,] 0 0 1 1 1 0
[259,] 0 0 1 0 1 1
[260,] 0 0 1 0 1 1
[261,] 0 0 1 1 0 1
[262,] 0 0 1 1 1 0
[263,] 0 0 1 1 0 1
[264,] 0 0 1 1 1 0
[265,] 0 0 0 1 1 1
[266,] 0 0 0 1 1 1
[267,] 0 0 0 1 1 1
[268,] 0 0 0 1 1 1
[269,] 0 0 0 1 1 1
[270,] 0 0 0 1 1 1
[271,] 0 0 1 0 1 1
[272,] 0 0 1 0 1 1
[273,] 0 0 1 1 0 1
[274,] 0 0 1 1 1 0
[275,] 0 0 1 1 0 1
[276,] 0 0 1 1 1 0
[277,] 0 0 1 0 1 1
[278,] 0 0 1 0 1 1
[279,] 0 0 1 1 0 1
[280,] 0 0 1 1 1 0
[281,] 0 0 1 1 0 1
[282,] 0 0 1 1 1 0
[283,] 0 0 1 0 1 1
[284,] 0 0 1 0 1 1
[285,] 0 0 1 1 0 1
[286,] 0 0 1 1 1 0
[287,] 0 0 1 1 0 1
[288,] 0 0 1 1 1 0
[289,] 0 1 0 0 1 1
[290,] 0 1 0 0 1 1
[291,] 0 1 0 1 0 1
[292,] 0 1 0 1 1 0
[293,] 0 1 0 1 0 1
[294,] 0 1 0 1 1 0
[295,] 0 1 0 0 1 1
[296,] 0 1 0 0 1 1
[297,] 0 1 0 1 0 1
[298,] 0 1 0 1 1 0
[299,] 0 1 0 1 0 1
[300,] 0 1 0 1 1 0
[301,] 0 1 1 0 0 1
[302,] 0 1 1 0 1 0
[303,] 0 1 1 0 0 1
[304,] 0 1 1 0 1 0
[305,] 0 1 1 1 0 0
[306,] 0 1 1 1 0 0
[307,] 0 1 1 0 0 1
[308,] 0 1 1 0 1 0
[309,] 0 1 1 0 0 1
[310,] 0 1 1 0 1 0
[311,] 0 1 1 1 0 0
[312,] 0 1 1 1 0 0
[313,] 0 1 0 0 1 1
[314,] 0 1 0 0 1 1
[315,] 0 1 0 1 0 1
[316,] 0 1 0 1 1 0
[317,] 0 1 0 1 0 1
[318,] 0 1 0 1 1 0
[319,] 0 1 0 0 1 1
[320,] 0 1 0 0 1 1
[321,] 0 1 0 1 0 1
[322,] 0 1 0 1 1 0
[323,] 0 1 0 1 0 1
[324,] 0 1 0 1 1 0
[325,] 0 1 1 0 0 1
[326,] 0 1 1 0 1 0
[327,] 0 1 1 0 0 1
[328,] 0 1 1 0 1 0
[329,] 0 1 1 1 0 0
[330,] 0 1 1 1 0 0
[331,] 0 1 1 0 0 1
[332,] 0 1 1 0 1 0
[333,] 0 1 1 0 0 1
[334,] 0 1 1 0 1 0
[335,] 0 1 1 1 0 0
[336,] 0 1 1 1 0 0
[337,] 0 1 0 0 1 1
[338,] 0 1 0 0 1 1
[339,] 0 1 0 1 0 1
[340,] 0 1 0 1 1 0
[341,] 0 1 0 1 0 1
[342,] 0 1 0 1 1 0
[343,] 0 1 0 0 1 1
[344,] 0 1 0 0 1 1
[345,] 0 1 0 1 0 1
[346,] 0 1 0 1 1 0
[347,] 0 1 0 1 0 1
[348,] 0 1 0 1 1 0
[349,] 0 1 1 0 0 1
[350,] 0 1 1 0 1 0
[351,] 0 1 1 0 0 1
[352,] 0 1 1 0 1 0
[353,] 0 1 1 1 0 0
[354,] 0 1 1 1 0 0
[355,] 0 1 1 0 0 1
[356,] 0 1 1 0 1 0
[357,] 0 1 1 0 0 1
[358,] 0 1 1 0 1 0
[359,] 0 1 1 1 0 0
[360,] 0 1 1 1 0 0
[361,] 1 0 0 0 1 1
[362,] 1 0 0 0 1 1
[363,] 1 0 0 1 0 1
[364,] 1 0 0 1 1 0
[365,] 1 0 0 1 0 1
[366,] 1 0 0 1 1 0
[367,] 1 0 0 0 1 1
[368,] 1 0 0 0 1 1
[369,] 1 0 0 1 0 1
[370,] 1 0 0 1 1 0
[371,] 1 0 0 1 0 1
[372,] 1 0 0 1 1 0
[373,] 1 0 1 0 0 1
[374,] 1 0 1 0 1 0
[375,] 1 0 1 0 0 1
[376,] 1 0 1 0 1 0
[377,] 1 0 1 1 0 0
[378,] 1 0 1 1 0 0
[379,] 1 0 1 0 0 1
[380,] 1 0 1 0 1 0
[381,] 1 0 1 0 0 1
[382,] 1 0 1 0 1 0
[383,] 1 0 1 1 0 0
[384,] 1 0 1 1 0 0
[385,] 1 0 0 0 1 1
[386,] 1 0 0 0 1 1
[387,] 1 0 0 1 0 1
[388,] 1 0 0 1 1 0
[389,] 1 0 0 1 0 1
[390,] 1 0 0 1 1 0
[391,] 1 0 0 0 1 1
[392,] 1 0 0 0 1 1
[393,] 1 0 0 1 0 1
[394,] 1 0 0 1 1 0
[395,] 1 0 0 1 0 1
[396,] 1 0 0 1 1 0
[397,] 1 0 1 0 0 1
[398,] 1 0 1 0 1 0
[399,] 1 0 1 0 0 1
[400,] 1 0 1 0 1 0
[401,] 1 0 1 1 0 0
[402,] 1 0 1 1 0 0
[403,] 1 0 1 0 0 1
[404,] 1 0 1 0 1 0
[405,] 1 0 1 0 0 1
[406,] 1 0 1 0 1 0
[407,] 1 0 1 1 0 0
[408,] 1 0 1 1 0 0
[409,] 1 0 0 0 1 1
[410,] 1 0 0 0 1 1
[411,] 1 0 0 1 0 1
[412,] 1 0 0 1 1 0
[413,] 1 0 0 1 0 1
[414,] 1 0 0 1 1 0
[415,] 1 0 0 0 1 1
[416,] 1 0 0 0 1 1
[417,] 1 0 0 1 0 1
[418,] 1 0 0 1 1 0
[419,] 1 0 0 1 0 1
[420,] 1 0 0 1 1 0
[421,] 1 0 1 0 0 1
[422,] 1 0 1 0 1 0
[423,] 1 0 1 0 0 1
[424,] 1 0 1 0 1 0
[425,] 1 0 1 1 0 0
[426,] 1 0 1 1 0 0
[427,] 1 0 1 0 0 1
[428,] 1 0 1 0 1 0
[429,] 1 0 1 0 0 1
[430,] 1 0 1 0 1 0
[431,] 1 0 1 1 0 0
[432,] 1 0 1 1 0 0
[433,] 1 1 0 0 0 1
[434,] 1 1 0 0 1 0
[435,] 1 1 0 0 0 1
[436,] 1 1 0 0 1 0
[437,] 1 1 0 1 0 0
[438,] 1 1 0 1 0 0
[439,] 1 1 0 0 0 1
[440,] 1 1 0 0 1 0
[441,] 1 1 0 0 0 1
[442,] 1 1 0 0 1 0
[443,] 1 1 0 1 0 0
[444,] 1 1 0 1 0 0
[445,] 1 1 0 0 0 1
[446,] 1 1 0 0 1 0
[447,] 1 1 0 0 0 1
[448,] 1 1 0 0 1 0
[449,] 1 1 0 1 0 0
[450,] 1 1 0 1 0 0
[451,] 1 1 1 0 0 0
[452,] 1 1 1 0 0 0
[453,] 1 1 1 0 0 0
[454,] 1 1 1 0 0 0
[455,] 1 1 1 0 0 0
[456,] 1 1 1 0 0 0
[457,] 1 1 0 0 0 1
[458,] 1 1 0 0 1 0
[459,] 1 1 0 0 0 1
[460,] 1 1 0 0 1 0
[461,] 1 1 0 1 0 0
[462,] 1 1 0 1 0 0
[463,] 1 1 0 0 0 1
[464,] 1 1 0 0 1 0
[465,] 1 1 0 0 0 1
[466,] 1 1 0 0 1 0
[467,] 1 1 0 1 0 0
[468,] 1 1 0 1 0 0
[469,] 1 1 0 0 0 1
[470,] 1 1 0 0 1 0
[471,] 1 1 0 0 0 1
[472,] 1 1 0 0 1 0
[473,] 1 1 0 1 0 0
[474,] 1 1 0 1 0 0
[475,] 1 1 1 0 0 0
[476,] 1 1 1 0 0 0
[477,] 1 1 1 0 0 0
[478,] 1 1 1 0 0 0
[479,] 1 1 1 0 0 0
[480,] 1 1 1 0 0 0
[481,] 1 0 0 0 1 1
[482,] 1 0 0 0 1 1
[483,] 1 0 0 1 0 1
[484,] 1 0 0 1 1 0
[485,] 1 0 0 1 0 1
[486,] 1 0 0 1 1 0
[487,] 1 0 0 0 1 1
[488,] 1 0 0 0 1 1
[489,] 1 0 0 1 0 1
[490,] 1 0 0 1 1 0
[491,] 1 0 0 1 0 1
[492,] 1 0 0 1 1 0
[493,] 1 0 1 0 0 1
[494,] 1 0 1 0 1 0
[495,] 1 0 1 0 0 1
[496,] 1 0 1 0 1 0
[497,] 1 0 1 1 0 0
[498,] 1 0 1 1 0 0
[499,] 1 0 1 0 0 1
[500,] 1 0 1 0 1 0
[501,] 1 0 1 0 0 1
[502,] 1 0 1 0 1 0
[503,] 1 0 1 1 0 0
[504,] 1 0 1 1 0 0
[505,] 1 0 0 0 1 1
[506,] 1 0 0 0 1 1
[507,] 1 0 0 1 0 1
[508,] 1 0 0 1 1 0
[509,] 1 0 0 1 0 1
[510,] 1 0 0 1 1 0
[511,] 1 0 0 0 1 1
[512,] 1 0 0 0 1 1
[513,] 1 0 0 1 0 1
[514,] 1 0 0 1 1 0
[515,] 1 0 0 1 0 1
[516,] 1 0 0 1 1 0
[517,] 1 0 1 0 0 1
[518,] 1 0 1 0 1 0
[519,] 1 0 1 0 0 1
[520,] 1 0 1 0 1 0
[521,] 1 0 1 1 0 0
[522,] 1 0 1 1 0 0
[523,] 1 0 1 0 0 1
[524,] 1 0 1 0 1 0
[525,] 1 0 1 0 0 1
[526,] 1 0 1 0 1 0
[527,] 1 0 1 1 0 0
[528,] 1 0 1 1 0 0
[529,] 1 0 0 0 1 1
[530,] 1 0 0 0 1 1
[531,] 1 0 0 1 0 1
[532,] 1 0 0 1 1 0
[533,] 1 0 0 1 0 1
[534,] 1 0 0 1 1 0
[535,] 1 0 0 0 1 1
[536,] 1 0 0 0 1 1
[537,] 1 0 0 1 0 1
[538,] 1 0 0 1 1 0
[539,] 1 0 0 1 0 1
[540,] 1 0 0 1 1 0
[541,] 1 0 1 0 0 1
[542,] 1 0 1 0 1 0
[543,] 1 0 1 0 0 1
[544,] 1 0 1 0 1 0
[545,] 1 0 1 1 0 0
[546,] 1 0 1 1 0 0
[547,] 1 0 1 0 0 1
[548,] 1 0 1 0 1 0
[549,] 1 0 1 0 0 1
[550,] 1 0 1 0 1 0
[551,] 1 0 1 1 0 0
[552,] 1 0 1 1 0 0
[553,] 1 1 0 0 0 1
[554,] 1 1 0 0 1 0
[555,] 1 1 0 0 0 1
[556,] 1 1 0 0 1 0
[557,] 1 1 0 1 0 0
[558,] 1 1 0 1 0 0
[559,] 1 1 0 0 0 1
[560,] 1 1 0 0 1 0
[561,] 1 1 0 0 0 1
[562,] 1 1 0 0 1 0
[563,] 1 1 0 1 0 0
[564,] 1 1 0 1 0 0
[565,] 1 1 0 0 0 1
[566,] 1 1 0 0 1 0
[567,] 1 1 0 0 0 1
[568,] 1 1 0 0 1 0
[569,] 1 1 0 1 0 0
[570,] 1 1 0 1 0 0
[571,] 1 1 1 0 0 0
[572,] 1 1 1 0 0 0
[573,] 1 1 1 0 0 0
[574,] 1 1 1 0 0 0
[575,] 1 1 1 0 0 0
[576,] 1 1 1 0 0 0
[577,] 1 1 0 0 0 1
[578,] 1 1 0 0 1 0
[579,] 1 1 0 0 0 1
[580,] 1 1 0 0 1 0
[581,] 1 1 0 1 0 0
[582,] 1 1 0 1 0 0
[583,] 1 1 0 0 0 1
[584,] 1 1 0 0 1 0
[585,] 1 1 0 0 0 1
[586,] 1 1 0 0 1 0
[587,] 1 1 0 1 0 0
[588,] 1 1 0 1 0 0
[589,] 1 1 0 0 0 1
[590,] 1 1 0 0 1 0
[591,] 1 1 0 0 0 1
[592,] 1 1 0 0 1 0
[593,] 1 1 0 1 0 0
[594,] 1 1 0 1 0 0
[595,] 1 1 1 0 0 0
[596,] 1 1 1 0 0 0
[597,] 1 1 1 0 0 0
[598,] 1 1 1 0 0 0
[599,] 1 1 1 0 0 0
[600,] 1 1 1 0 0 0
[601,] 1 0 0 0 1 1
[602,] 1 0 0 0 1 1
[603,] 1 0 0 1 0 1
[604,] 1 0 0 1 1 0
[605,] 1 0 0 1 0 1
[606,] 1 0 0 1 1 0
[607,] 1 0 0 0 1 1
[608,] 1 0 0 0 1 1
[609,] 1 0 0 1 0 1
[610,] 1 0 0 1 1 0
[611,] 1 0 0 1 0 1
[612,] 1 0 0 1 1 0
[613,] 1 0 1 0 0 1
[614,] 1 0 1 0 1 0
[615,] 1 0 1 0 0 1
[616,] 1 0 1 0 1 0
[617,] 1 0 1 1 0 0
[618,] 1 0 1 1 0 0
[619,] 1 0 1 0 0 1
[620,] 1 0 1 0 1 0
[621,] 1 0 1 0 0 1
[622,] 1 0 1 0 1 0
[623,] 1 0 1 1 0 0
[624,] 1 0 1 1 0 0
[625,] 1 0 0 0 1 1
[626,] 1 0 0 0 1 1
[627,] 1 0 0 1 0 1
[628,] 1 0 0 1 1 0
[629,] 1 0 0 1 0 1
[630,] 1 0 0 1 1 0
[631,] 1 0 0 0 1 1
[632,] 1 0 0 0 1 1
[633,] 1 0 0 1 0 1
[634,] 1 0 0 1 1 0
[635,] 1 0 0 1 0 1
[636,] 1 0 0 1 1 0
[637,] 1 0 1 0 0 1
[638,] 1 0 1 0 1 0
[639,] 1 0 1 0 0 1
[640,] 1 0 1 0 1 0
[641,] 1 0 1 1 0 0
[642,] 1 0 1 1 0 0
[643,] 1 0 1 0 0 1
[644,] 1 0 1 0 1 0
[645,] 1 0 1 0 0 1
[646,] 1 0 1 0 1 0
[647,] 1 0 1 1 0 0
[648,] 1 0 1 1 0 0
[649,] 1 0 0 0 1 1
[650,] 1 0 0 0 1 1
[651,] 1 0 0 1 0 1
[652,] 1 0 0 1 1 0
[653,] 1 0 0 1 0 1
[654,] 1 0 0 1 1 0
[655,] 1 0 0 0 1 1
[656,] 1 0 0 0 1 1
[657,] 1 0 0 1 0 1
[658,] 1 0 0 1 1 0
[659,] 1 0 0 1 0 1
[660,] 1 0 0 1 1 0
[661,] 1 0 1 0 0 1
[662,] 1 0 1 0 1 0
[663,] 1 0 1 0 0 1
[664,] 1 0 1 0 1 0
[665,] 1 0 1 1 0 0
[666,] 1 0 1 1 0 0
[667,] 1 0 1 0 0 1
[668,] 1 0 1 0 1 0
[669,] 1 0 1 0 0 1
[670,] 1 0 1 0 1 0
[671,] 1 0 1 1 0 0
[672,] 1 0 1 1 0 0
[673,] 1 1 0 0 0 1
[674,] 1 1 0 0 1 0
[675,] 1 1 0 0 0 1
[676,] 1 1 0 0 1 0
[677,] 1 1 0 1 0 0
[678,] 1 1 0 1 0 0
[679,] 1 1 0 0 0 1
[680,] 1 1 0 0 1 0
[681,] 1 1 0 0 0 1
[682,] 1 1 0 0 1 0
[683,] 1 1 0 1 0 0
[684,] 1 1 0 1 0 0
[685,] 1 1 0 0 0 1
[686,] 1 1 0 0 1 0
[687,] 1 1 0 0 0 1
[688,] 1 1 0 0 1 0
[689,] 1 1 0 1 0 0
[690,] 1 1 0 1 0 0
[691,] 1 1 1 0 0 0
[692,] 1 1 1 0 0 0
[693,] 1 1 1 0 0 0
[694,] 1 1 1 0 0 0
[695,] 1 1 1 0 0 0
[696,] 1 1 1 0 0 0
[697,] 1 1 0 0 0 1
[698,] 1 1 0 0 1 0
[699,] 1 1 0 0 0 1
[700,] 1 1 0 0 1 0
[701,] 1 1 0 1 0 0
[702,] 1 1 0 1 0 0
[703,] 1 1 0 0 0 1
[704,] 1 1 0 0 1 0
[705,] 1 1 0 0 0 1
[706,] 1 1 0 0 1 0
[707,] 1 1 0 1 0 0
[708,] 1 1 0 1 0 0
[709,] 1 1 0 0 0 1
[710,] 1 1 0 0 1 0
[711,] 1 1 0 0 0 1
[712,] 1 1 0 0 1 0
[713,] 1 1 0 1 0 0
[714,] 1 1 0 1 0 0
[715,] 1 1 1 0 0 0
[716,] 1 1 1 0 0 0
[717,] 1 1 1 0 0 0
[718,] 1 1 1 0 0 0
[719,] 1 1 1 0 0 0
[720,] 1 1 1 0 0 0---
title: "Sampling Distributions"
format:
html:
code-tools:
source: true
---
To understand the behavior of estimators under a random process, we start by assuming the impossible: that we know the full schedule of potential outcomes. Here it is, for six students who went through the anchoring experiment.
```{r}
#| message: false
library(tidyverse)
anchor_mini_sched <- tibble(Y_0 = c(15, 15, 19, 2, 10, 8),
Y_1 = c(31, 22, 45, 20, 20, 15))
anchor_mini_sched
```
Let's compute the individual treatment effect (ITE) for each of the size students.
```{r}
anchor_mini_sched <- anchor_mini_sched |>
mutate(tau = Y_1 - Y_0)
anchor_mini_sched
```
Now we can calculate summary statistics on the schedule, which are better thought of as parameters.
```{r}
parameters <- anchor_mini_sched |>
summarize(Ybar_1 = mean(Y_1),
Ybar_0 = mean(Y_0),
ATE = mean(tau))
parameters
```
Note that the Average Treatment Effect (ATE), `{r} parameters$ATE`, our estimand, can either be expressed as the difference between the means of the potential outcomes, $\bar{Y}_1 - \bar{Y}_0$, or as the average of the $\tau_i$.
### A single experiment
Let's simulate a single randomized experiment conducted on these six students by randomly assigning them to get the treatment (`1`) or the control (`0`).
```{r}
set.seed(4032) # fix the randomness so we draw the same random numbers
anchor_mini_sched <- anchor_mini_sched |>
mutate(d_i = sample(c(0, 0, 0, 1, 1, 1)),
y_i = Y_1 * d_i + Y_0 * (1 - d_i))
anchor_mini_sched
```
Our estimator for the ATE will be the difference in the average responses of the units in the treated group and the control group.
```{r}
ATE_hat <- anchor_mini_sched |>
group_by(d_i) |>
summarize(Ybar_hat = mean(y_i)) |>
summarize(ATE_hat = diff(Ybar_hat)) |>
pull(ATE_hat)
ATE_hat
```
In a normal data analysis, all we get to see is our estimate, `{r} ATE_hat`. Since we know the full schedule, here we're effectively playing this game in "God Mode" and can know that the value of the parameter was in fact `{r} parameters$ATE`. So this particular estimate was a bit off, but not by much.
To learn about how much variability we can expect in our estimator, we need to know it's sampling distribution.
### Calculating the sampling distribution
To learn the sampling distribution of $\widehat{ATE}$, we need to
1. Articulate every possible way to split the six students into two groups of size three.
2. For each unique partition, calculate $\widehat{ATE}$
3. Calculate the probability of observing each of those values.
To aid in step 1, we can use the `permutations()` function to show every permutation of the a vector containing size elements, three of which indicate the treatment (`1`) and three of which indicate the control (`0`). We can imagine that the indices of this vector correspond to the indices of the students.
```{r}
library(arrangements)
d_x <- c(0, 0, 0, 1, 1, 1)
permmat <- permutations(d_x)
permmat
```