Data605-Week12-HomeWork12-kamath
Vinayak Kamath
11/08/2020
Home Work 12
The attached who.csv dataset contains real-world data from 2008. The variables included follow.
- Country: name of the country
- LifeExp: average life expectancy for the country in years
- InfantSurvival: proportion of those surviving to one year or more
- Under5Survival: proportion of those surviving to five years or more
- TBFree: proportion of the population without TB.
- PropMD: proportion of the population who are MDs
- PropRN: proportion of the population who are RNs
- PersExp: mean personal expenditures on healthcare in US dollars at average exchange rate
- GovtExp: mean government expenditures per capita on healthcare, US dollars at average exchange rate
- TotExp: sum of personal and government expenditures.
#Reading the file
df_who = read.csv('who.csv', header = TRUE)
#Displaying the car dataset contents:
str(df_who)## 'data.frame': 190 obs. of 10 variables:
## $ Country : Factor w/ 190 levels "Afghanistan",..: 1 2 3 4 5 6 7 8 9 10 ...
## $ LifeExp : int 42 71 71 82 41 73 75 69 82 80 ...
## $ InfantSurvival: num 0.835 0.985 0.967 0.997 0.846 0.99 0.986 0.979 0.995 0.996 ...
## $ Under5Survival: num 0.743 0.983 0.962 0.996 0.74 0.989 0.983 0.976 0.994 0.996 ...
## $ TBFree : num 0.998 1 0.999 1 0.997 ...
## $ PropMD : num 2.29e-04 1.14e-03 1.06e-03 3.30e-03 7.04e-05 ...
## $ PropRN : num 0.000572 0.004614 0.002091 0.0035 0.001146 ...
## $ PersExp : int 20 169 108 2589 36 503 484 88 3181 3788 ...
## $ GovtExp : int 92 3128 5184 169725 1620 12543 19170 1856 187616 189354 ...
## $ TotExp : int 112 3297 5292 172314 1656 13046 19654 1944 190797 193142 ...#Showing stats with Summary function:
summary(df_who)## Country LifeExp InfantSurvival Under5Survival
## Afghanistan : 1 Min. :40.00 Min. :0.8350 Min. :0.7310
## Albania : 1 1st Qu.:61.25 1st Qu.:0.9433 1st Qu.:0.9253
## Algeria : 1 Median :70.00 Median :0.9785 Median :0.9745
## Andorra : 1 Mean :67.38 Mean :0.9624 Mean :0.9459
## Angola : 1 3rd Qu.:75.00 3rd Qu.:0.9910 3rd Qu.:0.9900
## Antigua and Barbuda: 1 Max. :83.00 Max. :0.9980 Max. :0.9970
## (Other) :184
## TBFree PropMD PropRN PersExp
## Min. :0.9870 Min. :0.0000196 Min. :0.0000883 Min. : 3.00
## 1st Qu.:0.9969 1st Qu.:0.0002444 1st Qu.:0.0008455 1st Qu.: 36.25
## Median :0.9992 Median :0.0010474 Median :0.0027584 Median : 199.50
## Mean :0.9980 Mean :0.0017954 Mean :0.0041336 Mean : 742.00
## 3rd Qu.:0.9998 3rd Qu.:0.0024584 3rd Qu.:0.0057164 3rd Qu.: 515.25
## Max. :1.0000 Max. :0.0351290 Max. :0.0708387 Max. :6350.00
##
## GovtExp TotExp
## Min. : 10.0 Min. : 13
## 1st Qu.: 559.5 1st Qu.: 584
## Median : 5385.0 Median : 5541
## Mean : 40953.5 Mean : 41696
## 3rd Qu.: 25680.2 3rd Qu.: 26331
## Max. :476420.0 Max. :482750
## Provide a scatterplot of LifeExp~TotExp, and run simple linear regression. Do not transform the variables. Provide and interpret the F statistics, R^2, standard error,and p-values only. Discuss whether the assumptions of simple linear regression met.
# Linear regression model build who_lm <- lm(LifeExp ~ TotExp, data=df_who) who_lm## ## Call: ## lm(formula = LifeExp ~ TotExp, data = df_who) ## ## Coefficients: ## (Intercept) TotExp ## 6.475e+01 6.297e-05# Scatterplot of dependent and independent variables plot(LifeExp~TotExp, data=df_who, xlab="Total Expenditures", ylab="Life Expectancy", main="Life Expectancy vs Total Expenditures") abline(who_lm, col='blue')
who_lm## ## Call: ## lm(formula = LifeExp ~ TotExp, data = df_who) ## ## Coefficients: ## (Intercept) TotExp ## 6.475e+01 6.297e-05summary(who_lm)## ## Call: ## lm(formula = LifeExp ~ TotExp, data = df_who) ## ## Residuals: ## Min 1Q Median 3Q Max ## -24.764 -4.778 3.154 7.116 13.292 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 6.475e+01 7.535e-01 85.933 < 2e-16 *** ## TotExp 6.297e-05 7.795e-06 8.079 7.71e-14 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 9.371 on 188 degrees of freedom ## Multiple R-squared: 0.2577, Adjusted R-squared: 0.2537 ## F-statistic: 65.26 on 1 and 188 DF, p-value: 7.714e-14Linear Regression Model : Life Expentancy = 64.75 + .000063 Total Expenditure The negative y intercept is an indication that any expenditure below 65 would lead to a negative life expectancy. The model is thus unrealistic.*
Multiple R-squared: 0.2577, Adjusted R-squared: 0.2537: The low R-squared value tells us that our model only explains for around 25% of the data variation.
Residual standard error: 9.371 on 188 degrees of freedom : It tells us the standard error is somewhat high (10 years). This means that some the sample data points are significantly off the fitted line and that countries that contribute significantly less in healthcare expenditure than what the model would predict, have sustain a life expectancy that is significantly higher than expected.
F-statistic: 65.26 on 1 and 188 DF, p-value: 7.714e-14: The p-value of the model is very low which means we can confindetly reject the null hypothesis that “Expenditure DOES NOT contribute to Country’s Life Expentancy”
Raise life expectancy to the 4.6 power (i.e., LifeExp^4.6). Raise total expenditures to the 0.06 power (nearly a log transform, TotExp^.06). Plot LifeExp^4.6 as a function of TotExp^.06, and r re-run the simple regression model using the transformed variables. Provide and interpret the F statistics, R^2, standard error, and p-values. Which model is “better?”
# Transformation LifeExp_1 <- df_who$LifeExp^4.6 TotExp_1 <- df_who$TotExp^0.06 # Linear regression model build who_lm_1 <- lm(LifeExp_1 ~ TotExp_1) # Scatterplot of dependent and independent variables plot(LifeExp_1~TotExp_1, xlab="Total Expenditures", ylab="Life Expectancy", main="Life Expectancy vs Total Expenditures (Modified)") abline(who_lm_1, col='red')
who_lm_1## ## Call: ## lm(formula = LifeExp_1 ~ TotExp_1) ## ## Coefficients: ## (Intercept) TotExp_1 ## -736527909 620060216summary(who_lm_1)## ## Call: ## lm(formula = LifeExp_1 ~ TotExp_1) ## ## Residuals: ## Min 1Q Median 3Q Max ## -308616089 -53978977 13697187 59139231 211951764 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -736527910 46817945 -15.73 <2e-16 *** ## TotExp_1 620060216 27518940 22.53 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 90490000 on 188 degrees of freedom ## Multiple R-squared: 0.7298, Adjusted R-squared: 0.7283 ## F-statistic: 507.7 on 1 and 188 DF, p-value: < 2.2e-16Multiple R-squared: 0.7298, Adjusted R-squared: 0.7283: The high R-squared value of close to 73% means that the response variable explains the model’s variability around the mean 75% of the time.
Residual standard error: 90,490,000 on 188 degrees of freedom: It tells is that due to the exponentioal increase in these values, the SE should also be expected to increase exponentially.
F-statistic: 507.7 on 1 and 188 DF, p-value: < 2.2e-16: The p-value of the model is very low which means we can reject the null hypothesis “Expenditure^0.06 does not contribute to a country’s Life Expentancy^4.6”.
It is evident that the variable does contribute in a greater way when compared to the initial model.
Using the results from 3, forecast life expectancy when TotExp^.06 =1.5. Then forecast life expectancy when TotExp^.06=2.5.
who_predictdata <- data.frame(TotExp_1=c(1.5,2.5)) predict(who_lm_1, who_predictdata,interval="predict")^(1/4.6)## fit lwr upr ## 1 63.31153 35.93545 73.00793 ## 2 86.50645 81.80643 90.43414Predicting the values at 1.5 adn 2.5 provides the following results:
Prediction at 1.5 is 63 years with a CI(35.93545, 73.00793).
Prediction at 2.5 is 87 year with a CI(81.80643, 90.43414).
Build the following multiple regression model and interpret the F Statistics, R^2, standard error, and p-values. How good is the model?
\[ LifeExp = b0+b1 * PropMd + b2 * TotExp +b3 * PropMD * TotExp \]# Multiple linear regression model build who_lm4 <- lm(LifeExp ~ PropMD + TotExp + TotExp:PropMD, data=df_who) who_lm4## ## Call: ## lm(formula = LifeExp ~ PropMD + TotExp + TotExp:PropMD, data = df_who) ## ## Coefficients: ## (Intercept) PropMD TotExp PropMD:TotExp ## 6.277e+01 1.497e+03 7.233e-05 -6.026e-03# Linear regression model summary summary(who_lm4)## ## Call: ## lm(formula = LifeExp ~ PropMD + TotExp + TotExp:PropMD, data = df_who) ## ## Residuals: ## Min 1Q Median 3Q Max ## -27.320 -4.132 2.098 6.540 13.074 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 6.277e+01 7.956e-01 78.899 < 2e-16 *** ## PropMD 1.497e+03 2.788e+02 5.371 2.32e-07 *** ## TotExp 7.233e-05 8.982e-06 8.053 9.39e-14 *** ## PropMD:TotExp -6.026e-03 1.472e-03 -4.093 6.35e-05 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 8.765 on 186 degrees of freedom ## Multiple R-squared: 0.3574, Adjusted R-squared: 0.3471 ## F-statistic: 34.49 on 3 and 186 DF, p-value: < 2.2e-16Multiple R-squared: 0.3574, Adjusted R-squared: 0.3471 : The adjusted R-Sqrd is too low at only 35%. The response variables in this model account for only ~35% of the variability.
Residual standard error: 8.765 on 186 degrees of freedom: The residual standard error is significant at 8.765 since the data points are on average off by 8.765 and thus the model is not a good fit to its corresponding data points.
F-statistic: 34.49 on 3 and 186 DF, p-value: < 2.2e-16: The p-value is very low meaning that we can reject the null hypothesis and confidently state that the response variables contribute to the true value of the dependent variable.
Forecast LifeExp when PropMD=.03 and TotExp = 14. Does this forecast seem realistic? Why or why not?
who_newdata <- data.frame(PropMD=0.03, TotExp=14) predict(who_lm4, who_newdata,interval="predict")## fit lwr upr ## 1 107.696 84.24791 131.1441Predicting the values at PropMD=0.03, TotExp=14 provides the following results.
Prediction is 108 years with a CI(84.24791, 131.1441).
The data maxes out about the 90-100 range. Seeing a prediction of 108 becomes unrealistic when the CI also shows 132 years.