#{r setup, include=FALSE} #knitr::opts_chunk$set(echo = TRUE, warning = FALSE, message = FALSE) #
The ideal experiment would be for answering “What was the effect of their MPP degree on the future earnings of the average applicant?” is the RCT(Randomized Control Trial). To carry out this experiment I would need a data set that has a column indicating the
The first step is to compare applicants who enroll into the MPP degree and the applicants who do not. The two groups of people should be comparable. On observable characteristics including good grades in statistics they should be similar. They should also be comparable on non-observable characteristics including motivation to succeed in public policy careers. To ensure characteristics are balanced across the two groups, I will randomly assign students into the two separate groups.
I would test if the assignment into treatment and control groups is random. If it is random,
Considering the outcome variable: \(Y_{i}(1)\) measures the future earnings of average applicant i in the state of the world where i enrolls into and graduates from Harris’ MPP degree program, and \(Y_{i}(0)\) as the future earnings of average applicant i where they are not enrolled into the MPP program.
Applicants’ observable characteristics (\(X_{i}\)) should look similar (their grades in college statistics classes won’t determine their treatment status):
\(E[StatsGrades_{i}|D_{i}=Treated] = E[StatsGrades_{i}|D_{i}=Control]\)
Their un-observable characteristics (\(\varepsilon_{i}\)) should look similar (their motivation to succeed in public policy career won’t determine their treatment status):
\(E[Motiation|D_{i}=Treated] = E[Motivation|D_{i}=Control]\)
As a result, the average potential outcome is the same for either group:
\(E[Y_{i}(1)] = E[Y_{i}(1)|D_{i}=1] = E[Y_{i}(1)|D_{i}=0]\)
\(E[Y_{i}(0)] = E[Y_{i}(0)|D_{i}=0] = E[Y_{i}(0)|D_{i}=1]\)
In other words, I if the data set is randomized, the selection bias is eliminated, ensuring that the only future earnings difference between the groups is that one enrolls in the MPP program.
The impact of treatment for average applicant i would be the difference between their specific potential outcomes:
\(\tau_{i} = Y_{i}(1) - Y_{i}(0)\)
This is the difference between the future earnings for average applicant i who enrolled in the MPP degree and the average future earnings for average applicant i not who was not enrolled into the MPP program.
What was the effect of their MPP degree on future earnings for students who enrolled in the program?
Describe in math and words, using the potential outcomes framework, what they’d like to estimate. Explain how this differs from what you described in (1), and describe what component of this estimand you will be fundamentally unable to observe.
What Harris would like to estimate is technically known as the ATT (Average Treatment Effect on the Treated). It is mathematically expressed as
\(\tau^{ATT} = E[\tau_{i}(1)|D_{i}=1]\)
\(= E[Y_{i}(1) - Y_{i}(0)|D_{i}=1]\)
\(= E[Y_{i}(1)|D_{i}=1] - E[Y_{i}(0)|D_{i}=1]\)
This is interpreted as the difference between the average future earnings of student i who enrolled in the program and the average future earnings of the same student who enrolled in the program hadn’t they enrolled in the program.
This differs from what I described in (1) because of selection bias it contains which is not present in the ATE because we randomize to avoid it. It could be that a student who enrolled in the program potentially end up having higher average future earnings than a student who does not enroll in the program because those who enrolled in the program are students who for instance, excelled in statistics at college, can write convincing essays, and are highly motivated to succeed in public policy careers. These characteristics cause someone apply, be admitted and go on to enroll while those lacking these characteristics do get admitted and hence do not enroll. To control for this selection problem in ATT, in addition to observing the average future earnings of a student who enrolled in the program, we also need to observe what their average future earnings would have been hadn’t they enrolled in the program. Unfortunately, we can not observe the average future earnings of a student who enrolled in the program hadn’t they enrolled in the program.
Explain what you would recover if you simply compared admitted students to non-admitted students. Describe three concrete examples in this context of why this might be problematic
If I simply compared admitted students to non-admitted students, I would recover the biased estimate of the effect of being admitted to the program.
Three concrete examples of why this approach might be problematic are:
Another reason for being admitted to Harris could be related to being raised in a wealthy and supportive family. This could encourage a student to take statistics class in college because their parents buy them tools and pay for them tutoring to help them understand and excel in statistics. Originating from a wealthy and supportive family also increases your chances of having a higher future income because a student’s family is well-connected and can provide financial support and inheritance to a student. Students who did not study statistics in college may have lacked a similar background and resources. This estimate we recovered does not account for confounding observable characteristics like family wealth between admitted and non-admitted students and this could lead to erroneous conclusions about the effect of admission to HARRIS on future earnings of an admitted student.
Harris admits students based on whether a student took a statistics class in college. This is selection bias. It means that the treatment effect we recovered is not solely due to admission to the program but also due to history of a student having taken a statistics class in college. This is problematic because, for instance, if students have higher average scores in statistics because they took a statistics class, attributing the average future earnings to admission rather than prior training in statistics could be misleading. Those who have better preparation also tend to qualify for better jobs after graduation which increases their chances of earning more.
When HARRIS almost never admits students who did not take a statistics class in college, there are non-observable characteristics they can not directly measurable which could influence both the likelihood of of a student being admitted and their future earnings. For example, the non-observable characteristic of “motivation.” It’s possible that students who are highly motivated are more likely to take a statistics class in college, as they recognize its importance for admission to HARRIS or for their future career goals. This motivation could also drive them to pursue jobs that pay higher salaries. Without accounting for non-observable characteristics like motivation, comparing admitted to non-admitted students gives a biased estimates of the treatment effect of admission to HARRIS, as the differences in outcomes could be partially attributable to these unmeasured characteristics rather than the admission itself.
# read in data
HARRIS <- read.csv("ps2_data_24.csv")#changing the values of college_stastics from character to integer
# identifying unique values in the data set
unique(HARRIS$college_statistics)## [1] "1" "0" "One"# identifying rows with value "one" in college_statistics variable/column.
rows_with_one <- which(HARRIS$college_statistics == "One")
rows_with_one## [1] 275# replace "One" with "1" in row 275 in "college_statistics" column
HARRIS$college_statistics[275] <- "1"Use the variables contained in the data set to describe, using math and words, two (related) potential approaches to estimating the effect of HARRIS’ MPP program on earnings. Make sure to be clear about your unit of analysis and be explicit about how these designs apply to the HARRIS MPP program (ie, describe things in terms of “earnings,” not just “outcome”).
The two selection-on-observables designs we would use to estimate the effect of HARRIS’ MPP program on earnings are: 1. regression adjustment 2. matching
\(X_{i}\) includes a student’s status on whether they took a statistics class in college, whether they studied calculus in college, their GRE score and undergrad GPA, and a their age.
Conducting the regression adjustment approach,
I would run a regression of earnings on the treatment (Admission to HARRIS MPP program) and on the covariates, \(X_{i}\), to estimate the treatment effect of HARRIS MPP program on the future earnings of a student.
\(Y_{i} = \alpha + \tau D_{i} + \gamma X_{i} + \nu_{i}\)
This means that the future earnings of student, \(i\), who is admitted to the HARRIS MPP program is independent of the admission to HARRIS conditional on whether they took a statistics class in college, whether they studied calculus in college, their GRE score and undergrad GPA, and a their age. Having this conditional independence means that if we leave out any of the covariates included in \(X_{i}\), this very \(X_{i}\) will be partially or wholly included in the error term \(\nu_{i}\) meaning that our estimate of the treatment effect of HARRIS MPP program on future earnings of a student will be biased because due to Omitted Variable Bias.
It is equally important to note the following about \(X_{i}\): These characteristics should be similar or closely similar for students who were admitted to HARRIS MPP program and those that were not. If the difference between \(X_{i}\) between the two groups is large, the estimate for the treatment effect will also be biased.
Conducting the matching approach,
Steps: 1. I would divide the data into cells uniquely defined by the covariates in \(X_{i}\) that can help me satisfy the common support assumption. We will then compare students who were admitted to Harris and those that were not as long as they have identical covariate(s).
\(\hat{\tau}^{ATE} = \sum_{j=1}^{N} \frac{N_{i}}{N} \hat{(\overline{Y}_{T}-\overline{Y}_{U})}\)
N is the number of cells that is determined by the covariates and the values considered for each covariate. The weight is determined by the ratio \(\frac{N_{i}}{N}\). \(N_{i}\) counts the observations in each cell for each of the covariates. \(N\) is the total number of observations in the data set.
Produce a balance table which displays the differences between HARRIS enrollees and applicants that were not admitted to the program on observable characteristics. Interpret this table. Does this table make you feel better or worse about your concerns in (3)?
HARRIS <- HARRIS |>
mutate(did_gre = ifelse(gre_score==0,0,1),
gre_level = case_when(
gre_score <= 100 ~ "Low",
gre_score >= 101 & gre_score < 200 ~ "Medium",
gre_score >= 200 ~ "High"
))
# We will first convert the non-binary categorical variable, gender, into three dummy variable columns
HARRIS <- HARRIS %>%
fastDummies::dummy_cols(select_columns = c("gender"))
print(names(HARRIS)[grep("gender", names(HARRIS))])## [1] "gender" "gender_FEMALE" "gender_MALE"
## [4] "gender_NON-BINARY"# We will also convert the non-binary categorical variable, undergrad_GPA, into 5 dummy variable columns
HARRIS <- HARRIS %>%
fastDummies::dummy_cols(select_columns = c("undergrad_GPA"))
print(names(HARRIS)[grep("undergrad_GPA", names(HARRIS))])## [1] "undergrad_GPA" "undergrad_GPA_A" "undergrad_GPA_B" "undergrad_GPA_C"
## [5] "undergrad_GPA_D" "undergrad_GPA_F"# We will also convert the non-binary categorical variable, gre_level, into 3 dummy variable columns
HARRIS <- HARRIS %>%
fastDummies::dummy_cols(select_columns = c("gre_level"))
print(names(HARRIS)[grep("gre_level_", names(HARRIS))])## [1] "gre_level_High" "gre_level_Low" "gre_level_Medium"regressions <- HARRIS |>
dplyr::select(college_statistics, college_calculus, gre_score, age,
starts_with("undergrad_GPA_"), starts_with("gender_")) |>
lapply(function(x) lm(x ~ harris_admit, data = HARRIS))
balance_table <- regressions |>
sapply(function(x) coef(summary(x))[c(2,8)]) |>
t()
# format the table output
balance_table |>
kable(col.names = c("Differences in means", "$p$-value")) |>
kable_styling(position = "center", font_size = 13, latex_options = "hold_position")| Differences in means | \(p\)-value | |
|---|---|---|
| college_statistics | 0.6007981 | 0.0000000 |
| college_calculus | 0.1007403 | 0.0000001 |
| gre_score | 52.1262645 | 0.0000000 |
| age | 0.1456464 | 0.4238352 |
| undergrad_GPA_A | 0.1281136 | 0.0000000 |
| undergrad_GPA_B | 0.0412384 | 0.0080824 |
| undergrad_GPA_C | -0.0149358 | 0.3314871 |
| undergrad_GPA_D | -0.0330057 | 0.0333105 |
| undergrad_GPA_F | -0.1214105 | 0.0000000 |
| gender_FEMALE | 0.0043254 | 0.8136242 |
| gender_MALE | 0.0055253 | 0.7014503 |
| gender_NON-BINARY | -0.0098507 | 0.5043984 |
Produce a balance table which displays the differences between HARRIS enrollees and applicants that were not admitted to the program on observable characteristics. Interpret this table. Does this table make you feel better or worse about your concerns in (3)?
From the table above, the baseline (pre-treatment) characteristics of age, undergrad_GPA_C, gender_FEMALE, gender_MALE, gender_NON-BINARY have no statistically significant difference between enrollees and applicants that were not admitted to the program at the 1% level. On the other hand, baseline characteristics including college_statistics, college_calculus, gre_score, undergrad_GPA_A, undergrad_GPA_B, and undergrad_GPA_F have statistical significance. In other words, the difference between enrollees and applicants that were not admitted to the program is did not happen randomly.
This table makes me confirm the concerns I had in (3). Since, for instance, differences based on college_statistics, college_calculus, undergrad_GPA_A, and undergrad_GPA_B are not random, this leads to incorrect conclusions about the causal effects of admission to Harris. If a student scored higher GPA in their undergrads, earned higher GRE score, and excelled at college calculus and statistics, attributing any differences in future earnings to admission to Harris rather than their prior academic success could be misleading.
HARRIS are interested in your approach in (4), but would like to know a bit more about how much they should believe your proposal. Describe the assumptions required for these designs to be valid in math and in words. To the extent possible, assess the validity of these assumptions using the provided data.
Discuss whether you think you will be able to obtain a credible estimate of the answer to the questions described in (1) and (2) based on the data, and use concrete examples to explain why or why not.
Under these two designs, I would consider the following assumptions. 1. Conditional Indepedence Assumption
\((Y_{i}(1), Y_{i}(0)) ⊥ D_{i}|X_{i}\)
The potential earnings are independent of HARRIS’ MPP program, conditional on whether a student took a statistics class in college, whether a student studied calculus in college, a student’s GRE score and undergrad GPA.
In order to start any SOO analysis, we should first look at how the variables differ between treatment status groups:
# to view the means of the variables in each treatment status group (harris_admit)
HARRIS |>
group_by(harris_admit) |>
summarize(`College Statistics` = mean(as.integer(college_statistics)),
`College Calculus` = mean(college_calculus),
`GRE Score` = mean(gre_score),
`Age` = mean(age),
`Females` = mean(gender_FEMALE),
`Males` = mean(gender_MALE),
`Non-binary` = mean(gender_FEMALE),
`GPA A` = mean(undergrad_GPA_A),
`GPA B` = mean(undergrad_GPA_B),
`GPA C` = mean(undergrad_GPA_C),
`GPA D` = mean(undergrad_GPA_D),
`GPA F` = mean(undergrad_GPA_F))This assumption can be tested using provided data. We have to strongly assume that as long as we control for the \(X_{i}\), our comparison between the future earnings of enrollees and the future earnings of applicants who were not admitted is as good as random.
\(= E[Y_{i}(1)|X_{i}=X] - E[Y_{i}(0)|X_{i}=X]\)
\(X_{i}\) includes a student’s status on whether they took a statistics class in college, whether they studied calculus in college, their GRE_score, undergrad_GPA_A or undergrad_GPA_B, and undergrad GPA_F, and a their age.
This assumption can be tested be validated using the provided data.
# make control variable
HARRIS_1 <- HARRIS |>
mutate(non_enrollee = ifelse(harris_admit==0,1,0))
head(HARRIS_1)# are there treated and control people at each level of the college_statistics variable?
HARRIS_1 |>
group_by(college_statistics) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")We have enrollees and applicants that were not admitted for both levels of college_statistics. So, we have a common support within college_statistics.
# are there treated and control people at each level of the college_calculus variable?
HARRIS_1 |>
group_by(college_calculus) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")We have enrollees and applicants that were not admitted for both levels of college_calculus. So, we have a common support within college_calculus.
# are there treated and control people at each level of the gre_score variable?
HARRIS_1 |>
group_by(gre_score) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")We do not have enrollees and applicants that were not admitted for both levels of gre_score. So, we lack common support within gre_score.
# are there treated and control people at each level of the did_gre variable?
HARRIS_1 |>
group_by(did_gre) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")We do not have enrollees and applicants that were not admitted for both levels of did_gre. So, we lack common support within did_gre.
# are there treated and control people at each level of the did_gre variable?
HARRIS_1 |>
group_by(gre_level) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")We do not have enrollees and applicants that were not admitted for both levels of gre_level. So, we lack common support within gre_level.
# are there treated and control people at each level of the undergrad_GPA_A variable?
HARRIS_1 |>
group_by(undergrad_GPA_A) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")We have enrollees and applicants that were not admitted for both levels of college_calculus. So, we have a common support within undergrad_GPA_A.
# are there treated and control people at each level of the undergrad_GPA_B variable?
HARRIS_1 |>
group_by(undergrad_GPA_B) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")We have enrollees and applicants that were not admitted for both levels of college_calculus. So, we have a common support within undergrad_GPA_B.
# are there treated and control people at each level of the undergrad_GPA_D variable?
HARRIS_1 |>
group_by(undergrad_GPA_D) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")# are there treated and control people at each level of the undergrad_GPA_D variable?
HARRIS_1 |>
group_by(age) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")We have enrollees and applicants that were not admitted for both levels of college_calculus. So, we have a common support within undergrad_GPA_D.
# are there treated and control people at each level of the undergrad_GPA_F variable?
HARRIS_1 |>
group_by(undergrad_GPA_F) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")We have enrollees and applicants that were not admitted for both levels of college_calculus. So, we have a common support within undergrad_GPA_F.
The other assumption is SUTVA assumption. We imagine that there are not spillover effects. For example students of high performance in statistics helping lowe quality students to perform well on GRE.
Estimating the relationship between future_earnings and harris_admit by running a regression analysis with some controls:
lm1 <- lm(future_earnings ~ harris_admit + college_statistics + college_calculus + gre_score + undergrad_GPA + age + gender, data = HARRIS_1)
lm2 <- lm(future_earnings ~ harris_admit, data = HARRIS_1)
stargazer(lm1, lm2,
type="text",
header=F)##
## ============================================================================
## Dependent variable:
## --------------------------------------------------------
## future_earnings
## (1) (2)
## ----------------------------------------------------------------------------
## harris_admit -11,533.450*** 123,542.800***
## (2,316.774) (3,715.347)
##
## college_statistics1 173,152.400***
## (1,850.063)
##
## college_calculus 1,585.432
## (1,476.419)
##
## gre_score 291.607***
## (17.145)
##
## undergrad_GPAB -18,018.200***
## (2,627.753)
##
## undergrad_GPAC -56,168.320***
## (2,642.108)
##
## undergrad_GPAD -75,165.260***
## (2,633.338)
##
## undergrad_GPAF -106,376.800***
## (2,565.391)
##
## age -16.554
## (152.403)
##
## genderMALE 45,323.350***
## (1,973.391)
##
## genderNON-BINARY 5,431.486***
## (1,922.382)
##
## Constant 155,208.400*** 192,471.100***
## (4,955.932) (1,892.380)
##
## ----------------------------------------------------------------------------
## Observations 3,500 3,500
## R2 0.852 0.240
## Adjusted R2 0.852 0.240
## Residual Std. Error 42,513.440 (df = 3488) 96,344.250 (df = 3498)
## F Statistic 1,832.289*** (df = 11; 3488) 1,105.697*** (df = 1; 3498)
## ============================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Harris_admit is -11,533.450. Holding all else constant, the future earnings of students at Harris diminishes by $ 11, 553 compared to those who were not admitted to harris. Advantages: - As we add more covariates to the regression or reducing more terms in the error term we increase chances of validating the conditional independence assumption which should hold for the selection observables design to be valid. - Adding covariates to the regression reduces the standard error of the treatment effect and hence increases the precision of our estimate on the association of admission to Harris program to the the future earnings.
Challenges: - We need the covariates, \(X_{i}\), to be very similar for enrollees and the applicants that applied but were not admitted. And, this is not always guaranteed. - We must write down the right regression and this is hard because we don’t necessarily know the underlying data generating process.
The naive estimator is nearly 14 times larger than the result I get when I run a regression analysis with controls. This means that the naive estimator is positively biased and attributes a huge chunk of the future earnings of an enrollee to the HARRIS MPP program which is truly a result of an enrollee’s previous academic performance and training.
Variables included in the matching procedure: college_statistics, college_calculus, undergrad_GPA_A, undergrad_GPA_B, undergrad_GPA_D, and undergrad_GPA_F.
# make control variable
HARRIS_1 <- HARRIS_1 |>
mutate(non_enrollee = ifelse(harris_admit==0,1,0))
# are there treated and control people at each level of the X variable?
HARRIS_1 |>
group_by(college_statistics) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep")Checking a common support for multi-variable “cells” - checking if we have treated and control observations across multiple observable variables.
HARRIS_1 |>
group_by(college_statistics, college_calculus) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep") Checking a common support for multi-variable “cells” - checking if we have treated and control observations across multiple observable variables. college_statistics, college_calculus, gender, undergrad_GPA: Specificically college_statistics, college_calculus, undergrad_GPA_A, undergrad_GPA_B, undergrad_GPA_D, undergrad_GPA_F
HARRIS_1 |>
group_by(college_statistics, college_calculus, gender, undergrad_GPA) |>
summarize(n_enrollee = sum(harris_admit),
n_non_enrollee = sum(non_enrollee),
.groups = "keep") With these possible groups, let’s try to calculate a treatment effect:
# calculate the means by cell * treatment status
HARRIS_2 <- HARRIS_1 |>
group_by(college_statistics, college_calculus, undergrad_GPA_A, undergrad_GPA_B, undergrad_GPA_D, undergrad_GPA_F) |>
summarize(mean_future_earnings_harris_admit = sum(future_earnings * harris_admit) / sum(harris_admit),
mean_future_earnings_non_harris_admit = sum(future_earnings * non_enrollee) / sum(non_enrollee),
count_harris_admit = sum(harris_admit, na.rm = TRUE),
count_non_harris_admit = sum(non_enrollee, na.rm = TRUE),
.groups = "keep")
# Convert the 0-count cells to NA
HARRIS_2 <- HARRIS_2 |> mutate(mean_future_earnings_harris_admit = ifelse(count_harris_admit==0, NA, mean_future_earnings_harris_admit),
mean_future_earnings_non_harris_admit = ifelse(count_non_harris_admit==0, NA, mean_future_earnings_non_harris_admit))
# Take difference for each cell
HARRIS_2$diff <- HARRIS_2$mean_future_earnings_harris_admit - HARRIS_2$mean_future_earnings_non_harris_admit
HARRIS_2_fin <- HARRIS_2 |> filter(!is.na(diff))\(\hat{\tau}^{ATE} = \sum_{k=1}^{N} \frac{N_{k}}{N} \hat{\tau}_{k}\)
Then, estimating the answer to the question in (2): the ATT, the weight should be the counts of treated in a cell over the total counts of treated individuals.
\(\hat{\tau}^{ATT} = \sum_{k=1}^{N_{T}} \frac{N_{k, T}}{N_{T}} \hat{\tau}_{k}\)
# Estimate tau_ate as weighted average
ate <- weighted.mean(HARRIS_2_fin$diff,
w = (HARRIS_2_fin$count_harris_admit + HARRIS_2_fin$count_non_harris_admit))
ate## [1] -12543.83# Estimate the tau_att as as weighted average
att <- weighted.mean(HARRIS_2_fin$diff,
w = HARRIS_2_fin$count_harris_admit)
att## [1] 11201.43ATE and ATE are meaningfully different because ATE shows that being admitted to Harris decreases the future earnings compared to those that were not admitted. ATT shows that those who attend Harris earn 11,000 more than if they hadn’t attended Harris. In principle they should be the same because exact matching helped us find people who have the same characteristics for every covariate for both admitted and non admitted applicants group. This exact matching method is strong in a way that it creates observably identical treated and untreated comparisons so we do not worry about the values of a covariate to be different between the treated and the control groups. It also makes sure we do not worry about worry about how to control for covariates or technically called functional form. It is weak on the other hand in a way that it doesn’t work for continuous covariates. It works for only binary categorical variables. To consider the continuous variables we have to do the bandwidth and nearest neighbor matching which adds a layer of complexity.
The naive estimator is larger than the ATE and ATT in matching by 10 times in magnintude. We also run into a problem of the Curse of Dimensionality which reduced our chances of finding comparable matches between admitted and non admitted applicants because the more covariates we added the less chances we had of finding a match. Adding more covariates reduces our sample size and gave us a very small sample (20 observations) compared to the total 3,500 total observations to match from. So, it gives us an unreliable estimate. describe how it affected your approach.