DATA_605_Assign_12_DThonn
** DATA_605_Discussion_12_Regression_2 **
http://rpubs.com/danthonn/382369
** Real World Data **
if (!require(RCurl)) install.packages("RCurl")## Loading required package: RCurl## Loading required package: bitopsif (!require(stringr)) install.packages("stringr")## Loading required package: stringrlibrary(RCurl)
library(stringr)
library(knitr)
if (!require(tidyr)) install.packages("tidyr")## Loading required package: tidyr##
## Attaching package: 'tidyr'## The following object is masked from 'package:RCurl':
##
## completeif (!require(dplyr)) install.packages("dplyr")## Loading required package: dplyr##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(tidyr)
library(dplyr)
if (!require(gvlma)) install.packages("gvlma")## Loading required package: gvlmalibrary(gvlma)** Assign_12_Regression_Real_World Data **
The attached who.csv dataset contains real-world data from 2008. The variables included follow. Country: name of the country LifeExp: average life expectancy for the country in years InfantSurvival: proportion of those surviving to one year or more Under5Survival: proportion of those surviving to five years or more TBFree: proportion of the population without TB. PropMD: proportion of the population who are MDs PropRN: proportion of the population who are RNs PersExp: mean personal expenditures on healthcare in US dollars at average exchange rate GovtExp: mean government expenditures per capita on healthcare, US dollars at average exchange rate TotExp: sum of personal and government expenditures. 1. Provide a scatterplot of LifeExp~TotExp, and run simple linear regression. Do not transform the variables. Provide and interpret the F statistics, R^2, standard error,and p-values only. Discuss whether the assumptions of simple linear regression met. 2. Raise life expectancy to the 4.6 power (i.e., LifeExp^4.6). Raise total expenditures to the 0.06 power (nearly a log transform, TotExp^.06). Plot LifeExp^4.6 as a function of TotExp^.06, and r re-run the simple regression model using the transformed variables. Provide and interpret the F statistics, R^2, standard error, and p-values. Which model is “better?” 3. Using the results from 3, forecast life expectancy when TotExp^.06 =1.5. Then forecast life expectancy when TotExp^.06=2.5. 4. Build the following multiple regression model and interpret the F Statistics, R^2, standard error, and p-values. How good is the model? LifeExp = b0+b1 x PropMd + b2 x TotExp +b3 x PropMD x TotExp 5. Forecast LifeExp when PropMD=.03 and TotExp = 14. Does this forecast seem realistic? Why or why not?
url <- 'https://raw.githubusercontent.com/danthonn/DATA_605_Assign_12_DThonn/master/who.csv'
who1 <- read.csv(url)
str(who1)## 'data.frame': 190 obs. of 10 variables:
## $ Country : Factor w/ 190 levels "Afghanistan",..: 1 2 3 4 5 6 7 8 9 10 ...
## $ LifeExp : int 42 71 71 82 41 73 75 69 82 80 ...
## $ InfantSurvival: num 0.835 0.985 0.967 0.997 0.846 0.99 0.986 0.979 0.995 0.996 ...
## $ Under5Survival: num 0.743 0.983 0.962 0.996 0.74 0.989 0.983 0.976 0.994 0.996 ...
## $ TBFree : num 0.998 1 0.999 1 0.997 ...
## $ PropMD : num 2.29e-04 1.14e-03 1.06e-03 3.30e-03 7.04e-05 ...
## $ PropRN : num 0.000572 0.004614 0.002091 0.0035 0.001146 ...
## $ PersExp : int 20 169 108 2589 36 503 484 88 3181 3788 ...
## $ GovtExp : int 92 3128 5184 169725 1620 12543 19170 1856 187616 189354 ...
## $ TotExp : int 112 3297 5292 172314 1656 13046 19654 1944 190797 193142 ...glimpse(who1)## Observations: 190
## Variables: 10
## $ Country <fctr> Afghanistan, Albania, Algeria, Andorra, Angola...
## $ LifeExp <int> 42, 71, 71, 82, 41, 73, 75, 69, 82, 80, 64, 74,...
## $ InfantSurvival <dbl> 0.835, 0.985, 0.967, 0.997, 0.846, 0.990, 0.986...
## $ Under5Survival <dbl> 0.743, 0.983, 0.962, 0.996, 0.740, 0.989, 0.983...
## $ TBFree <dbl> 0.99769, 0.99974, 0.99944, 0.99983, 0.99656, 0....
## $ PropMD <dbl> 0.000228841, 0.001143127, 0.001060478, 0.003297...
## $ PropRN <dbl> 0.000572294, 0.004614439, 0.002091362, 0.003500...
## $ PersExp <int> 20, 169, 108, 2589, 36, 503, 484, 88, 3181, 378...
## $ GovtExp <int> 92, 3128, 5184, 169725, 1620, 12543, 19170, 185...
## $ TotExp <int> 112, 3297, 5292, 172314, 1656, 13046, 19654, 19...head(who1)## Country LifeExp InfantSurvival Under5Survival TBFree
## 1 Afghanistan 42 0.835 0.743 0.99769
## 2 Albania 71 0.985 0.983 0.99974
## 3 Algeria 71 0.967 0.962 0.99944
## 4 Andorra 82 0.997 0.996 0.99983
## 5 Angola 41 0.846 0.740 0.99656
## 6 Antigua and Barbuda 73 0.990 0.989 0.99991
## PropMD PropRN PersExp GovtExp TotExp
## 1 0.000228841 0.000572294 20 92 112
## 2 0.001143127 0.004614439 169 3128 3297
## 3 0.001060478 0.002091362 108 5184 5292
## 4 0.003297297 0.003500000 2589 169725 172314
## 5 0.000070400 0.001146162 36 1620 1656
## 6 0.000142857 0.002773810 503 12543 13046kable(head(who1))| Country | LifeExp | InfantSurvival | Under5Survival | TBFree | PropMD | PropRN | PersExp | GovtExp | TotExp |
|---|---|---|---|---|---|---|---|---|---|
| Afghanistan | 42 | 0.835 | 0.743 | 0.99769 | 0.0002288 | 0.0005723 | 20 | 92 | 112 |
| Albania | 71 | 0.985 | 0.983 | 0.99974 | 0.0011431 | 0.0046144 | 169 | 3128 | 3297 |
| Algeria | 71 | 0.967 | 0.962 | 0.99944 | 0.0010605 | 0.0020914 | 108 | 5184 | 5292 |
| Andorra | 82 | 0.997 | 0.996 | 0.99983 | 0.0032973 | 0.0035000 | 2589 | 169725 | 172314 |
| Angola | 41 | 0.846 | 0.740 | 0.99656 | 0.0000704 | 0.0011462 | 36 | 1620 | 1656 |
| Antigua and Barbuda | 73 | 0.990 | 0.989 | 0.99991 | 0.0001429 | 0.0027738 | 503 | 12543 | 13046 |
** Question-1 **
- Provide a scatterplot of LifeExp~TotExp, and run simple linear regression. Do not transform the variables. Provide and interpret the F statistics, R^2, standard error,and p-values only. Discuss whether the assumptions of simple linear regression met.
# Question-1 Answer:
# a simple linear regression - who1
lm_who1 <- lm(who1$LifeExp ~ who1$TotExp)
# a scatter plot - who1
plot(who1$LifeExp, who1$TotExp, main="Scatterplot",
xlab="LifeExp", ylab="TotExp", pch=19)
abline(lm_who1, col = "blue")
# the residuals - who1
hist(resid(lm_who1), main = "Histogram - Residuals", xlab = "residuals")
# summary - who1
summary(lm_who1)##
## Call:
## lm(formula = who1$LifeExp ~ who1$TotExp)
##
## Residuals:
## Min 1Q Median 3Q Max
## -24.764 -4.778 3.154 7.116 13.292
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.475e+01 7.535e-01 85.933 < 2e-16 ***
## who1$TotExp 6.297e-05 7.795e-06 8.079 7.71e-14 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 9.371 on 188 degrees of freedom
## Multiple R-squared: 0.2577, Adjusted R-squared: 0.2537
## F-statistic: 65.26 on 1 and 188 DF, p-value: 7.714e-14Interpretations-1: Plot: line does not appear linear and is sharply curved upward after LifeExp = 75. F statistics: large at 65.26. reject null hypothesis and no relationship between variables. R^2: R^2 is small at .2577. Only 25% of response variable explained by the independant variable. standard error: at 9.3 is large, indicating how much the avg. response variable deviates from the regression line. p-values: near zero. probability of observing this relationship is small.
** Question-2 **
- Raise life expectancy to the 4.6 power (i.e., LifeExp^4.6). Raise total expenditures to the 0.06 power (nearly a log transform, TotExp^.06). Plot LifeExp^4.6 as a function of TotExp^.06, and r re-run the simple regression model using the transformed variables. Provide and interpret the F statistics, R^2, standard error, and p-values. Which model is “better?”
# Question-2 Answer:
# a simple linear regression - who2
LifeExp4.6 <- who1$LifeExp^4.6
TotExp0.06 <- who1$TotExp^0.06
lm_who2 <- lm(LifeExp4.6 ~ TotExp0.06)
# scatter plot - who2
plot(TotExp0.06, LifeExp4.6)
abline(lm_who2, col="blue")
# residuals - who2
hist(resid(lm_who2), main = "Histogram - Residuals", xlab = "residuals")
# summary - who2
summary(lm_who2)##
## Call:
## lm(formula = LifeExp4.6 ~ TotExp0.06)
##
## Residuals:
## Min 1Q Median 3Q Max
## -308616089 -53978977 13697187 59139231 211951764
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -736527910 46817945 -15.73 <2e-16 ***
## TotExp0.06 620060216 27518940 22.53 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 90490000 on 188 degrees of freedom
## Multiple R-squared: 0.7298, Adjusted R-squared: 0.7283
## F-statistic: 507.7 on 1 and 188 DF, p-value: < 2.2e-16Interpretations-2: Plot: regression line does appear linear F statistics: larger at 507.7 and higher than who1.
R^2: R^2 is larger at .7283. 73% of response variable explained by the independant variable. standard error: at 90490000 is large, indicating how much the avg. response variable deviates from the regression line. p-values: p-value: < 2.2e-16 is statistically significant. less than who1 model. histogram: near normal distribution
Overall: The who2 model is much improved over the who1 model and a good linear regression fit.
** Question-3 **
- Using the results from 3, forecast life expectancy when TotExp^.06 =1.5. Then forecast life expectancy when TotExp^.06=2.5.
Then forecast life expectancy when TotExp^.06=2.5. y=-736527910+620060216x lifeexpectancy=y(1/4.6)
life_ex <- function(x)
{ z <- -736527910 + 620060216 * (x)
z <- z^(1/4.6)
print(z)
}
# Life expectancy with TotExp^.06 =1.5
life_ex(1.5)## [1] 63.31153## [1] 63.31153
# Life expectancy with TotExp^.06 =2.5
life_ex(2.5)## [1] 86.50645## [1] 86.50645** Question-4 **
- Build the following multiple regression model and interpret the F Statistics, R^2, standard error, and p-values. How good is the model? LifeExp = b0+b1 x PropMd + b2 x TotExp +b3 x PropMD x TotExp
lm_4 <- lm(who1$LifeExp ~ who1$PropMD + who1$TotExp + who1$PropMD*who1$TotExp)
summary(lm_4)##
## Call:
## lm(formula = who1$LifeExp ~ who1$PropMD + who1$TotExp + who1$PropMD *
## who1$TotExp)
##
## Residuals:
## Min 1Q Median 3Q Max
## -27.320 -4.132 2.098 6.540 13.074
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.277e+01 7.956e-01 78.899 < 2e-16 ***
## who1$PropMD 1.497e+03 2.788e+02 5.371 2.32e-07 ***
## who1$TotExp 7.233e-05 8.982e-06 8.053 9.39e-14 ***
## who1$PropMD:who1$TotExp -6.026e-03 1.472e-03 -4.093 6.35e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.765 on 186 degrees of freedom
## Multiple R-squared: 0.3574, Adjusted R-squared: 0.3471
## F-statistic: 34.49 on 3 and 186 DF, p-value: < 2.2e-16hist(resid(lm_4), main = "Histogram - Residuals", xlab = "residuals")
plot(fitted(lm_4), resid(lm_4))
# Call:
# lm(formula = who1$LifeExp ~ who1$PropMD + who1$TotExp + who1$PropMD *
# who1$TotExp)
#
# Residuals:
# Min 1Q Median 3Q Max
# -27.320 -4.132 2.098 6.540 13.074
#
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 6.277e+01 7.956e-01 78.899 < 2e-16 ***
# who1$PropMD 1.497e+03 2.788e+02 5.371 2.32e-07 ***
# who1$TotExp 7.233e-05 8.982e-06 8.053 9.39e-14 ***
# who1$PropMD:who1$TotExp -6.026e-03 1.472e-03 -4.093 6.35e-05 ***
# ---
# Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#
# Residual standard error: 8.765 on 186 degrees of freedom
# Multiple R-squared: 0.3574, Adjusted R-squared: 0.3471
# F-statistic: 34.49 on 3 and 186 DF, p-value: < 2.2e-16Interpretations-4:
1). p-value is less than .05, therefore model is statistically significant. 2). F-statistic is 34.49 on 3 3). Based on Rsquare only 34.7% of the variability can be explained by the 3 variables. 4). Residuals are right skewed. There, the linear model is not a good fit.
** Question-5 **
- Forecast LifeExp when PropMD=.03 and TotExp = 14. Does this forecast seem realistic? Why or why not?
LifeExp=(6.277 * 10^1) +(1.497 * 10^3) * PropMD + (7.233 * 10^(-5) TotExp - ((6.026 10^(-3) * PropMD * TotExp
LifeExp5 <- ( (6.277*10^1) + (1.497*10^3)*.03 + (7.233*10^(-5))* 14 - ((6.026*10^(-3))*0.03*14) )
LifeExp5## [1] 107.6785## [1] 107.6785Interpretations-5: 1). The forecast age of 107.6 is an outlier and is unlikely. 2). The expenditure is also low.
** END **