Czhu_Assignment12 Regression
Chunhui Zhu
April 25, 2018
library(data.table)
library(ggplot2)
library(scales)
library(RCurl)The attached who.csv dataset contains real-world data from 2008. The variables included follow.
Country: name of the country LifeExp: average life expectancy for the country in years InfantSurvival: proportion of those surviving to one year or more Under5Survival: proportion of those surviving to five years or more TBFree: proportion of the population without TB. PropMD: proportion of the population who are MDs PropRN: proportion of the population who are RNs PersExp: mean personal expenditures on healthcare in US dollars at average exchange rate GovtExp: mean government expenditures per capita on healthcare, US dollars at average exchange rate TotExp: sum of personal and government expenditures.
Firstly import data into r, there is a table with 190 observations and 10 filds. There is no missing value in the table. And except Country, all else are numberial data type.
url <- getURL("https://raw.githubusercontent.com/czhu505/Data605_W12_Regression/master/who.csv")
who <- fread(url)
print(head(who))## Country LifeExp InfantSurvival Under5Survival TBFree
## 1: Afghanistan 42 0.835 0.743 0.99769
## 2: Albania 71 0.985 0.983 0.99974
## 3: Algeria 71 0.967 0.962 0.99944
## 4: Andorra 82 0.997 0.996 0.99983
## 5: Angola 41 0.846 0.740 0.99656
## 6: Antigua and Barbuda 73 0.990 0.989 0.99991
## PropMD PropRN PersExp GovtExp TotExp
## 1: 0.000228841 0.000572294 20 92 112
## 2: 0.001143127 0.004614439 169 3128 3297
## 3: 0.001060478 0.002091362 108 5184 5292
## 4: 0.003297297 0.003500000 2589 169725 172314
## 5: 0.000070400 0.001146162 36 1620 1656
## 6: 0.000142857 0.002773810 503 12543 13046summary(who)## Country LifeExp InfantSurvival Under5Survival
## Length:190 Min. :40.00 Min. :0.8350 Min. :0.7310
## Class :character 1st Qu.:61.25 1st Qu.:0.9433 1st Qu.:0.9253
## Mode :character Median :70.00 Median :0.9785 Median :0.9745
## Mean :67.38 Mean :0.9624 Mean :0.9459
## 3rd Qu.:75.00 3rd Qu.:0.9910 3rd Qu.:0.9900
## Max. :83.00 Max. :0.9980 Max. :0.9970
## TBFree PropMD PropRN
## Min. :0.9870 Min. :0.0000196 Min. :0.0000883
## 1st Qu.:0.9969 1st Qu.:0.0002444 1st Qu.:0.0008455
## Median :0.9992 Median :0.0010474 Median :0.0027584
## Mean :0.9980 Mean :0.0017954 Mean :0.0041336
## 3rd Qu.:0.9998 3rd Qu.:0.0024584 3rd Qu.:0.0057164
## Max. :1.0000 Max. :0.0351290 Max. :0.0708387
## PersExp GovtExp TotExp
## Min. : 3.00 Min. : 10.0 Min. : 13
## 1st Qu.: 36.25 1st Qu.: 559.5 1st Qu.: 584
## Median : 199.50 Median : 5385.0 Median : 5541
## Mean : 742.00 Mean : 40953.5 Mean : 41696
## 3rd Qu.: 515.25 3rd Qu.: 25680.2 3rd Qu.: 26331
## Max. :6350.00 Max. :476420.0 Max. :4827501.
Provide a scatterplot of LifeExp~TotExp, and run simple linear regression. Do not transform the variables. Provide and interpret the F statistics, R^2, standard error,and p-values only. Discuss whether the assumptions of simple linear regression met.
ggplot(data=who, aes(x=TotExp, y=LifeExp)) +
geom_point() +
geom_point( size=0.5, shape=10, fill="white")+
ggtitle("LifeExp~TotExp") +
geom_smooth(method = "lm", se = TRUE) +
scale_x_continuous(labels = comma)
From the satter plot, it shows TotExp and LifeExp don’t have linar relationship, most point are at vertical 0 TotExp, which I can see majority people have very low TotExp versuse to 0-75 LifeExp.
lm.exp<-lm(who$LifeExp~who$TotExp)
summary(lm.exp)##
## Call:
## lm(formula = who$LifeExp ~ who$TotExp)
##
## Residuals:
## Min 1Q Median 3Q Max
## -24.764 -4.778 3.154 7.116 13.292
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.475e+01 7.535e-01 85.933 < 2e-16 ***
## who$TotExp 6.297e-05 7.795e-06 8.079 7.71e-14 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 9.371 on 188 degrees of freedom
## Multiple R-squared: 0.2577, Adjusted R-squared: 0.2537
## F-statistic: 65.26 on 1 and 188 DF, p-value: 7.714e-14In the linear model test outcome, the intercept and coefficient of the TotExp are incredible small (close to 0). However, the p-value for intercept and coefficient not relevant to the model are lesser than 0.05, which means both are significant to the model. It is because of very tiny Std. In this case, this result is against what it shows in the plot. From the multiple R-squared of interval (0,1), it shows the model only can explain 25.77% of the variance, and 74.23% data variance is not able to explain. Therefore, the model doesn’t not have trustful outcome.
2.
Raise life expectancy to the 4.6 power (i.e., LifeExp^4.6). Raise total expenditures to the 0.06 power (nearly a log transform, TotExp^.06). Plot LifeExp^4.6 as a function of TotExp^.06, and r re-run the simple regression model using the transformed variables. Provide and interpret the F statistics, R^2, standard error, and p-values. Which model is “better?”
logLifeExp<-who$LifeExp^4.6
logTotExp<-who$TotExp^.06
ggplot(data=who, aes(x=logTotExp, y=logLifeExp)) +
geom_point() +
geom_point( size=0.5, shape=10, fill="white")+
ggtitle("LifeExp^4.6~TotExp^0.06") +
geom_smooth(method = "lm", se = TRUE)+
scale_y_continuous(labels = comma)
After adjusted TotExp and LifeExp, scatter plot shows the linear relatioship with LifeExp^4.6 ~ TotExp^0.06.
lm.logexp<-lm(logLifeExp~logTotExp)
summary(lm.logexp)##
## Call:
## lm(formula = logLifeExp ~ logTotExp)
##
## Residuals:
## Min 1Q Median 3Q Max
## -308616089 -53978977 13697187 59139231 211951764
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -736527910 46817945 -15.73 <2e-16 ***
## logTotExp 620060216 27518940 22.53 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 90490000 on 188 degrees of freedom
## Multiple R-squared: 0.7298, Adjusted R-squared: 0.7283
## F-statistic: 507.7 on 1 and 188 DF, p-value: < 2.2e-16The second model logLifeExp4.6~logTotExp0.06 is obviously better than previos model.
In their linear test outcome, the intercept and coefficient of the TotExp are very large. In their linear test outcome, the p-value for intercept and coefficient are not relevant to the model are lesser than 0.05, which means both are significant to the model. From the multiple R-squared of interval (0,1), it shows the model can explain 72.98% of the variance, and 27.02% data ariance is not able to explain. The model has statistical significant result.
3.
Using the results from 2, forecast life expectancy when TotExp^.06 =1.5. Then forecast life expectancy when TotExp^.06=2.5.
predict_logLifExp<-lm.logexp$coefficients[[1]]+ lm.logexp$coefficients[[2]] * 1.5
exp(log(predict_logLifExp)/4.6)## [1] 63.31153predict_logLifExp<-lm.logexp$coefficients[[1]]+ lm.logexp$coefficients[[2]] * 2.5
exp(log(predict_logLifExp)/4.6)## [1] 86.506454.
Build the following multiple regression model and interpret the F Statistics, R^2, standard error, and p-values. How good is the model?
LifeExp = b0+b1 x PropMD+ b2 x TotExp +b3 x PropMD x TotExp
a<-who$PropMD*who$TotExp
multi_LifeExp<-lm(who$LifeExp ~who$PropRN+who$TotExp+a)
print(multi_LifeExp)##
## Call:
## lm(formula = who$LifeExp ~ who$PropRN + who$TotExp + a)
##
## Coefficients:
## (Intercept) who$PropRN who$TotExp a
## 6.182e+01 1.099e+03 5.586e-05 -7.219e-03From the given result in mutilinear regression model, the coefficients for TotExp and PropMD x TotExp are very samll. The model doesn’t show these two indepent variables have relatiship with LifeExp.
summary(multi_LifeExp)##
## Call:
## lm(formula = who$LifeExp ~ who$PropRN + who$TotExp + a)
##
## Residuals:
## Min 1Q Median 3Q Max
## -26.572 -5.007 2.301 6.599 21.310
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.182e+01 9.571e-01 64.592 < 2e-16 ***
## who$PropRN 1.099e+03 2.377e+02 4.625 6.99e-06 ***
## who$TotExp 5.586e-05 9.158e-06 6.100 6.03e-09 ***
## a -7.219e-03 1.828e-03 -3.950 0.000111 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.922 on 186 degrees of freedom
## Multiple R-squared: 0.3343, Adjusted R-squared: 0.3236
## F-statistic: 31.14 on 3 and 186 DF, p-value: 2.329e-16The model LifeExp ~ PropMD+ TotExp + PropMD x TotExp is not a good model.
In their linear test outcome, the intercept and coefficient of the TotExp are very tiny. Even the p-value for intercept and coefficient are not relevant to the model are lesser than 0.05, the model predicted outcome is not trustful. From multiple R-squared of interval (0,1), it shows the model can explain 72.98% of the variance, and 27.02% data ariance is not able to explain.
5.
Forecast LifeExp when PropMD=.03 and TotExp = 14. Does this forecast seem realistic? Why or why not?
LifeExp = multi_LifeExp$coefficients[[1]]+multi_LifeExp$coefficients[[2]] * .03 + multi_LifeExp$coefficients[[3]] *14 +multi_LifeExp$coefficients[[4]]* 0.03*14
print(LifeExp)## [1] 94.79981t<-who[order(who$TotExp)]
print(head(t,2))## Country LifeExp InfantSurvival Under5Survival TBFree PropMD
## 1: Burundi 49 0.891 0.819 0.99286 2.45e-05
## 2: Ethiopia 56 0.923 0.877 0.99359 2.39e-05
## PropRN PersExp GovtExp TotExp
## 1: 0.000164933 3 10 13
## 2: 0.000191851 6 64 70Comparing real data from TotExp =13, LifeExp is 49, the model has bad outcome in predition while given predicted LifeExp = 95 by given TotExp = 14. The reason could because of outliner, since TotExp is smallest number in TotExp column. And the other reason, the statistical test for multilinear regression is fail the test. The model doesn’t have statistical significant outcome.