#Packages list
library(stargazer)
library(AER)Discussion_3
I. Assumptions
1 & 2.
Linearity: the relationship between \(x\) and \(y\) are linear in parameters, but not necessarily in variables.
Non-collinearity: there is no exact linear relationship among any of the predictors or \(x\) variables.
Exogeneity of independent variable: Independent variables are determined from outside the model and are not affected by \(y\) or other variables in the model.
Homoscedasticity: variation of error terms is constant over time.
Random: data should be randomly generated.
Normality of errors: errors are normally distributed.
3.
Linearity: the relationship between \(x\) and \(y\) are linear in parameters, but not necessarily in variables.
Non-collinearity: there is no exact linear relationship among any of the predictors or \(x\) variables. The columns of matrix \(X\) are linearly independent, meaning they are not produced by or multiples of each other.
Exogeneity of independent variable: the disturbance average to 0 for any value of \(X\)
Homoscedasticity: the variance of \(\epsilon_i\) is the same for all \(i\).
Random: data should be randomly generated by a mechanism that is unrelated to \(\epsilon\)
Normality of errors: errors are normally distributed.
II. Linear regression
1. Running regression
#data(package = "AER")
data("CASchools")
dt <- CASchools
dt$score <- ((dt$math + dt$read)/2)\[ Score_i = \beta_0 + \beta_1computer_i + \beta_2income_i + \beta_3english_i \]
I kept the model as level-level. I am trying to analyze the effects of computers in schools, district average income and percentage of non-English speakers in schools on the overall standardized tests scores. I created a new variable called \(scores\) which takes the average of the \(math\) and \(reading\) scores.
The variable \(score\) is in Stanford 9 standardized test score
Variable \(computer\) is the number of computers
Variable \(income\) is district average income in (USD 1,000)
Variable \(english\) is the percentage of English learners or people whose English is not their first language
options(scipen = 999)
reg1 <- lm(data = dt,
formula = score ~ computer + income + english)
stargazer(reg1, type = "text")
===============================================
Dependent variable:
---------------------------
score
-----------------------------------------------
computer 0.0005
(0.001)
income 1.492***
(0.075)
english -0.493***
(0.031)
Constant 638.936***
(1.432)
-----------------------------------------------
Observations 420
R2 0.707
Adjusted R2 0.705
Residual Std. Error 10.346 (df = 416)
F Statistic 334.984*** (df = 3; 416)
===============================================
Note: *p<0.1; **p<0.05; ***p<0.01When all values of \(income\) , \(computer\) , and \(english\) are equal to 0, the average score is 638.94
For every extra computer the school gets, the average scores increase by 0.0005. This coefficient is not statistically significant.
For every extra dollar that increases in the average district income, test scores increase by 1.49. This coefficient is statistically significant at the 99% confidence interval.
For every percentage point increase in the percent of people whose first language is not English, test scores decrease by 0.493. This coefficient is statistically significant at the 99% confidence interval.
The only coefficients that display somehow a significant magnitude are \(income\) and \(english\)
2. Storing regression
my_reg <- reg1III. Residual Analysis
1. Plots
par(mfrow = c(2, 2))
plot(my_reg)
Residual vs fitted measures the difference between the residuals and the fitted values of our model. The points are are residuals, (actual - predicted values) and the line represents the fitted line through the residuals. It helps us asses two GM assumptions, linearity and homoscedasticity. The rule of thumb for this graph is that we need the line to be as straight as possible and around 0, and we also want the residuals to be randomly scattered around the line, without any recognizable pattern.
Q-Q residuals measure the distribution of the residuals. It compares the distribution of residuals with a normal distribution. The quantiles of the data are plotted against the quantiles of a theoretical normal distribution. It helps us asses linearity and normality of the distribution of errors. The rule of thumb is that the residual points form a straigth line around the 45 degree line and all points should be between 3 and -3 on the y axis (empirical rule).
Scale location measures the spread of the residuals among the predictions. It helps us asses the homoscedasticity of the errors. This plot shows the fitted values and the square root of standardized residuals. The rule of thumb is that points should be randomly distributed among the axis without an observable patter and the line should be horizontal.
Residuals vs leverage measures the influence of individual points on the fit of OLS. Leverage is how much influence does the data point have. The rule of thumb is that we want all points to be inside of the cook’s distance, if it is beyond the dashed lines, it is considered an outlier.
2. Checking the model
plot(my_reg,
which = 1
)
It looks like linearity is violated because the residuals display some kind of non-linear pattern.
plot(my_reg,
which = 2
)
It looks like the errors are normally distributed but we have a point that falls over -4 standard deviations away from the mean.
plot(my_reg,
which = 3
)
The points look normally distributed but the line is not entirely horizontal.
plot(my_reg,
which = 4
)
Overall, the model looks like a good model but the assumption of linearity might be violated. There is room for improving the model by removing some of the outliers and performing some transformations to the variables.
3. Fixing model
hist(dt$computer)
hist(dt$income)
hist(dt$english)
dtf <- dt[-c(404, 49, 405, 262, 39), ]dtf$english <- log(dtf$english + 1e-10)
dtf$income <- log(dtf$income)
dtf$computer <- log(dtf$computer + 1e-10)regf <- lm(data = dtf,
formula = score ~ computer + income + english)
stargazer(regf,
type = "text")
===============================================
Dependent variable:
---------------------------
score
-----------------------------------------------
computer 0.019
(0.142)
income 36.736***
(1.509)
english -0.515***
(0.070)
Constant 556.628***
(4.017)
-----------------------------------------------
Observations 415
R2 0.619
Adjusted R2 0.616
Residual Std. Error 11.700 (df = 411)
F Statistic 222.824*** (df = 3; 411)
===============================================
Note: *p<0.1; **p<0.05; ***p<0.01par(mfrow = c(2, 2))
plot(regf)
The adjusted R square and F statistics for this new model are worse but the residual analysis looks way better, it seems that none of the assumptions that we can check from the graphs are violated.
plot(regf,
which = 1
)
plot(regf,
which = 2
)
plot(regf,
which = 3
)
plot(regf,
which = 4
)