DATA 605 Homework 12
Peter Fernandes
4/24/2021
Problem
The attached who.csv dataset contains real-world data from 2008. The variables included follow.
Country: name of the country
LifeExp: average life expectancy for the country in years
InfantSurvival: proportion of those surviving to one year or more
Under5Survival: proportion of those surviving to five years or more
TBFree: proportion of the population without TB.
PropMD: proportion of the population who are MDs
PropRN: proportion of the population who are RNs
PersExp: mean personal expenditures on healthcare in US dollars at average exchange rate
GovtExp: mean government expenditures per capita on healthcare, US dollars at average exchange rate
TotExp: sum of personal and government expenditures.
The file is upload on git and data is fetched from there
who <- read.csv("https://raw.githubusercontent.com/petferns/605/main/who.csv")
#View the head
head(who)## Country LifeExp InfantSurvival Under5Survival TBFree PropMD
## 1 Afghanistan 42 0.835 0.743 0.99769 0.000228841
## 2 Albania 71 0.985 0.983 0.99974 0.001143127
## 3 Algeria 71 0.967 0.962 0.99944 0.001060478
## 4 Andorra 82 0.997 0.996 0.99983 0.003297297
## 5 Angola 41 0.846 0.740 0.99656 0.000070400
## 6 Antigua and Barbuda 73 0.990 0.989 0.99991 0.000142857
## PropRN PersExp GovtExp TotExp
## 1 0.000572294 20 92 112
## 2 0.004614439 169 3128 3297
## 3 0.002091362 108 5184 5292
## 4 0.003500000 2589 169725 172314
## 5 0.001146162 36 1620 1656
## 6 0.002773810 503 12543 13046#View summary
summary(who)## Country LifeExp InfantSurvival Under5Survival
## Length:190 Min. :40.00 Min. :0.8350 Min. :0.7310
## Class :character 1st Qu.:61.25 1st Qu.:0.9433 1st Qu.:0.9253
## Mode :character Median :70.00 Median :0.9785 Median :0.9745
## Mean :67.38 Mean :0.9624 Mean :0.9459
## 3rd Qu.:75.00 3rd Qu.:0.9910 3rd Qu.:0.9900
## Max. :83.00 Max. :0.9980 Max. :0.9970
## TBFree PropMD PropRN PersExp
## Min. :0.9870 Min. :0.0000196 Min. :0.0000883 Min. : 3.00
## 1st Qu.:0.9969 1st Qu.:0.0002444 1st Qu.:0.0008455 1st Qu.: 36.25
## Median :0.9992 Median :0.0010474 Median :0.0027584 Median : 199.50
## Mean :0.9980 Mean :0.0017954 Mean :0.0041336 Mean : 742.00
## 3rd Qu.:0.9998 3rd Qu.:0.0024584 3rd Qu.:0.0057164 3rd Qu.: 515.25
## Max. :1.0000 Max. :0.0351290 Max. :0.0708387 Max. :6350.00
## GovtExp TotExp
## Min. : 10.0 Min. : 13
## 1st Qu.: 559.5 1st Qu.: 584
## Median : 5385.0 Median : 5541
## Mean : 40953.5 Mean : 41696
## 3rd Qu.: 25680.2 3rd Qu.: 26331
## Max. :476420.0 Max. :4827501. Provide a scatterplot of LifeExp~TotExp, and run simple linear regression. Do not transform the variables. Provide and interpret the F statistics, R^2, standard error,and p-values only. Discuss whether the assumptions of simple linear regression met.
#Scatterplot of LifeExp~TotExp with intercept
plot(who$TotExp, who$LifeExp, xlab = "Total Expenditures", ylab = "Life Expectancy", col=2)
abline(lm(who$LifeExp~who$TotExp), col=1)
#Applying Linear regression
linear_reg <- lm(LifeExp ~ TotExp, data = who)
summary(linear_reg)##
## Call:
## lm(formula = LifeExp ~ TotExp, data = who)
##
## Residuals:
## Min 1Q Median 3Q Max
## -24.764 -4.778 3.154 7.116 13.292
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.475e+01 7.535e-01 85.933 < 2e-16 ***
## TotExp 6.297e-05 7.795e-06 8.079 7.71e-14 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 9.371 on 188 degrees of freedom
## Multiple R-squared: 0.2577, Adjusted R-squared: 0.2537
## F-statistic: 65.26 on 1 and 188 DF, p-value: 7.714e-14F-statistic is 65.26 with p-value nearly 0, which means there is sufficient evidence that current model is better and we can reject the null hypothesis.
Multiple \(R^2\) is 0.2577 and Adjusted \(R^2\) is 0.2537, which means only around 26% of the variability of the data can be explained by the current model.
Standard error 9.371 is the typical distance of the data points from the regression line of the current model.
In the below plots, Fitted vs Residual - residuals are not randomly distributed but have a pattern, curve shape.
In QQ plot the residuals doesn’t form a straight line.
In general, the assumptions of the linear regression are not met for the current model.
plot(linear_reg$fitted.values, linear_reg$residuals, xlab="Fitted Values", ylab="Residuals",
main="Fitted Values vs.Residuals")
qqnorm(linear_reg$residuals, col=2)
qqline(linear_reg$residuals, col=1)
2 Raise life expectancy to the 4.6 power (i.e., LifeExp^4.6). Raise total expenditures to the 0.06 power (nearly a log transform, TotExp^.06). Plot LifeExp^4.6 as a function of TotExp^.06, and r re-run the simple regression model using the transformed variables. Provide and interpret the F statistics, R^2, standard error, and p-values. Which model is “better?”
#Add 2 new columns
raisedLifeExp = who$LifeExp^4.6
raisedTotExp = who$TotExp^0.06
linear_reg_new =lm(raisedLifeExp~ raisedTotExp, data = who)
summary(linear_reg_new)##
## Call:
## lm(formula = raisedLifeExp ~ raisedTotExp, data = who)
##
## Residuals:
## Min 1Q Median 3Q Max
## -308616089 -53978977 13697187 59139231 211951764
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -736527910 46817945 -15.73 <2e-16 ***
## raisedTotExp 620060216 27518940 22.53 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 90490000 on 188 degrees of freedom
## Multiple R-squared: 0.7298, Adjusted R-squared: 0.7283
## F-statistic: 507.7 on 1 and 188 DF, p-value: < 2.2e-16F Statistics & P-value - the value is 507.7 with p-value very closed to zero, which means there is significant evidence that the current model is better than the null model. Furthermore, the large F statistics shows that the current model has a strong performance compared to the null model.
\(R^2\) - the value 0.7298 means that around 73% of the variability of the data can be explained by the current model.
From the below plots :
From the plot Residuals vs Fitted, we see that the residuals appear to be randomly distributed on the plot.
According to the plot Normal Q-Q Plot, most of the residuals fall on the theoretical normal line.
In general, the assumptions of the linear regression are met for the current model.
The new model has larger F-statistic and R-squared when compared to original data model. So the new model is better.
plot(linear_reg_new$fitted.values, linear_reg_new$residuals, xlab="Fitted Values", ylab="Residuals",
main="Fitted Values vs.Residuals")
qqnorm(linear_reg_new$residuals, col=2)
qqline(linear_reg_new$residuals, col=1)
3 . Using the results from 2, forecast life expectancy when TotExp^.06 =1.5. Then forecast life expectancy when TotExp^.06=2.5.
Answer
From the new model \(LifeExp=−736527910+620060216∗TotExp\)
#TotExp^.06 =1.5
expectancy <- (-736527910 + (620060216 * 1.5))^(1/4.6)
expectancy## [1] 63.31153#TotExp^.06 = 2.5
expectancy <- (-736527910 + (620060216 * 2.5))^(1/4.6)
expectancy## [1] 86.50645Build the following multiple regression model and interpret the F Statistics, R^2, standard error, and p-values. How good is the model?
\(LifeExp = b0+b1 x PropMd + b2 x TotExp +b3 x PropMD x TotExp\)
Answer
mmodel = lm(LifeExp ~ PropMD + TotExp + PropMD*TotExp, data = who)
summary(mmodel)##
## Call:
## lm(formula = LifeExp ~ PropMD + TotExp + PropMD * TotExp, data = who)
##
## Residuals:
## Min 1Q Median 3Q Max
## -27.320 -4.132 2.098 6.540 13.074
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.277e+01 7.956e-01 78.899 < 2e-16 ***
## PropMD 1.497e+03 2.788e+02 5.371 2.32e-07 ***
## TotExp 7.233e-05 8.982e-06 8.053 9.39e-14 ***
## PropMD:TotExp -6.026e-03 1.472e-03 -4.093 6.35e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.765 on 186 degrees of freedom
## Multiple R-squared: 0.3574, Adjusted R-squared: 0.3471
## F-statistic: 34.49 on 3 and 186 DF, p-value: < 2.2e-16F Statistics & P-value: the value is 34.49 with p-value very closed to zero, which means there is significant evidence that the current model is better than the null model.
R-squared: 0.3574 which means that only around 36% of the variability of the data can be explained by the current model.
From the below plots:
Residuals vs Fitted, we see that the residuals are not randomly distributed on the plot but a significant curve shape.
According to the plot Normal Q-Q Plot, the residuals do not fall on the theoretical normal line.
In general, the assumptions of the linear regression are not met for the current model.
plot(mmodel$fitted.values, mmodel$residuals, xlab="Fitted Values", ylab="Residuals",
main="Fitted Values vs.Residuals")
qqnorm(mmodel$residuals, col=2)
qqline(mmodel$residuals, col=1)
5. Forecast LifeExp when PropMD=.03 and TotExp = 14. Does this forecast seem realistic? Why or why not?
PropMD <- 0.03
TotExp <- 14
LifeExp <- (6.277*10) + (1.497*10^3)*PropMD + (7.233*10^-5)*TotExp - (6.026*10^-3) *PropMD*TotExp
LifeExp## [1] 107.6785Life expectancy with given PropMD and TotExp is 107. This forecast does not look realistic as this is more than the max life expectancy which is 83.