Homework77

##testing linearity
plot(model2,which=1)

#Durbin Watson
durbinWatsonTest(model2)

##  lag Autocorrelation D-W Statistic p-value
##    1      0.03048677      1.938161   0.344
##  Alternative hypothesis: rho != 0

#testing Homoscedasticity 
plot(model1,which=3)

#testing heteroscedasticity
bptest(model1)

## 
##  studentized Breusch-Pagan test
## 
## data:  model1
## BP = 17.482, df = 3, p-value = 0.0005623

#normality of residuals
plot(model1,which=2)

## shapiro test
shapiro.test(model1$residuals)

## 
##  Shapiro-Wilk normality test
## 
## data:  model1$residuals
## W = 0.46166, p-value < 2.2e-16

## testing multicolinearity
vif(model1)

## DPFVTOTK DPETECOP DPSTTOSA 
## 1.032720 1.028553 1.013072

• The model shows it is slightly linear. the red line is slightly tilted downward.
• The errors are independent on the Durbin Watson test
• I thought it was homoescadistic because the line seemed straight but the p value for the bptest is significant meaning less than .05 - meaning it’s heteroscedastic.
• For the Q-Q Test: there is a steep drop off between -4 to -2 of the x axis - meaning there is no normality. The P Value being significantly below .05 for the Shapiro Test confirms this.
• VIF met the assumption becuase they were close to 1

5. Which assumptions are violated: Homoscedasticity and normality were violated. Most districts have graduation rates close to 100%, and a rate can’t go higher than that. So the residuals bunch up near the top, and a few districts with low rates make a long tail on the bottom.

6. How would I fix it? I already tried to fix it by cleaning the data at the start but it did not work. Below I have tried to do log transformations but it didn’t work.

district_clean$grad_log <- log(district_clean$DAGC4X21R + 1)
model3 <- lm(grad_log ~ DPFVTOTK + DPETECOP + DPSTTOSA, data = district_clean)
summary(model3)

## 
## Call:
## lm(formula = grad_log ~ DPFVTOTK + DPETECOP + DPSTTOSA, data = district_clean)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.42699 -0.01873  0.00993  0.03055  0.10101 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  4.738e+00  1.875e-02 252.657  < 2e-16 ***
## DPFVTOTK     1.299e-09  1.032e-09   1.259    0.208    
## DPETECOP    -8.726e-04  8.129e-05 -10.734  < 2e-16 ***
## DPSTTOSA    -2.138e-06  3.294e-07  -6.488 1.39e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.04814 on 956 degrees of freedom
## Multiple R-squared:  0.1394, Adjusted R-squared:  0.1367 
## F-statistic: 51.62 on 3 and 956 DF,  p-value: < 2.2e-16

I log-transformed graduation rate to try to fix the heteroscedasticity and non-normal residuals. It did not help: the Breusch-Pagan test was still significant, and the Shapiro-Wilk W went from 0.88 to 0.84. It will need more tinkering.