library(dplyr)
library(ggplot2)

Questions:

The attached who.csv dataset contains real-world data from 2008. The variables included follow.

  • Country: name of the country
  • LifeExp: average life expectancy for the country in years
  • InfantSurvival: proportion of those surviving to one year or more
  • Under5Survival: proportion of those surviving to five years or more
  • TBFree: proportion of the population without TB.
  • PropMD: proportion of the population who are MDs
  • PropRN: proportion of the population who are RNs
  • PersExp: mean personal expenditures on healthcare in US dollars at average exchange rate
  • GovtExp: mean government expenditures per capita on healthcare, US dollars at average exchange rate
  • TotExp: sum of personal and government expenditures.

Question 1:

Provide a scatterplot of LifeExp~TotExp, and run simple linear regression. Do not transform the variables. Provide and interpret the F statistics, R^2, standard error,and p-values only. Discuss whether the assumptions of simple linear regression met.

df<-read.csv("https://raw.githubusercontent.com/deepasharma06/Data-605/main/who.csv")
head(df)
##               Country LifeExp InfantSurvival Under5Survival  TBFree      PropMD
## 1         Afghanistan      42          0.835          0.743 0.99769 0.000228841
## 2             Albania      71          0.985          0.983 0.99974 0.001143127
## 3             Algeria      71          0.967          0.962 0.99944 0.001060478
## 4             Andorra      82          0.997          0.996 0.99983 0.003297297
## 5              Angola      41          0.846          0.740 0.99656 0.000070400
## 6 Antigua and Barbuda      73          0.990          0.989 0.99991 0.000142857
##        PropRN PersExp GovtExp TotExp
## 1 0.000572294      20      92    112
## 2 0.004614439     169    3128   3297
## 3 0.002091362     108    5184   5292
## 4 0.003500000    2589  169725 172314
## 5 0.001146162      36    1620   1656
## 6 0.002773810     503   12543  13046

Check missing calues for each columns

print(colSums(is.na(df)))
##        Country        LifeExp InfantSurvival Under5Survival         TBFree 
##              0              0              0              0              0 
##         PropMD         PropRN        PersExp        GovtExp         TotExp 
##              0              0              0              0              0
str(df)
## 'data.frame':    190 obs. of  10 variables:
##  $ Country       : chr  "Afghanistan" "Albania" "Algeria" "Andorra" ...
##  $ LifeExp       : int  42 71 71 82 41 73 75 69 82 80 ...
##  $ InfantSurvival: num  0.835 0.985 0.967 0.997 0.846 0.99 0.986 0.979 0.995 0.996 ...
##  $ Under5Survival: num  0.743 0.983 0.962 0.996 0.74 0.989 0.983 0.976 0.994 0.996 ...
##  $ TBFree        : num  0.998 1 0.999 1 0.997 ...
##  $ PropMD        : num  2.29e-04 1.14e-03 1.06e-03 3.30e-03 7.04e-05 ...
##  $ PropRN        : num  0.000572 0.004614 0.002091 0.0035 0.001146 ...
##  $ PersExp       : int  20 169 108 2589 36 503 484 88 3181 3788 ...
##  $ GovtExp       : int  92 3128 5184 169725 1620 12543 19170 1856 187616 189354 ...
##  $ TotExp        : int  112 3297 5292 172314 1656 13046 19654 1944 190797 193142 ...

Simple linear Regression:

Scatter plot:

ggplot(df, aes(x = TotExp, y = LifeExp)) +
    geom_point() +
    labs(title = "LifeExp ~ TotExp",
         x = "TotExp", y = "LifeExp")

ggplot(df, aes(x = TotExp, y = LifeExp)) +
    geom_point() +
    geom_smooth(method = lm, se = FALSE) + 
    labs(title = "LifeExp ~ TotExp",
         x = "TotExp", y = "LifeExp")

The assumptions of simple linear regression are not met because the relationship appears curvilinear based on the scatterplot.

Model1:

Provide and interpret the F statistics, R^2, standard error,and p-values only. Discuss whether the assumptions of simple linear regression met.

lm<-lm(LifeExp~TotExp, data = df)
#lm
summary(lm)
## 
## Call:
## lm(formula = LifeExp ~ TotExp, data = df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -24.764  -4.778   3.154   7.116  13.292 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 6.475e+01  7.535e-01  85.933  < 2e-16 ***
## TotExp      6.297e-05  7.795e-06   8.079 7.71e-14 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 9.371 on 188 degrees of freedom
## Multiple R-squared:  0.2577, Adjusted R-squared:  0.2537 
## F-statistic: 65.26 on 1 and 188 DF,  p-value: 7.714e-14

y^=64.75+0.00006297x

F statistics: The F-statistic is more appropriate when you’re looking at multiple variables, but here it is 65.26. That is large, and it is associated with a p-value of 0.00000000000007714, which is very small. So there is a statistically significant relationship between TotExp and LifeExp.

R^2: value is 0.2537, which indicates 25.37% of the variation in LifeExp can be explained by TotExp.

Standard error: The standard error, 0.000007795, is roughly 8 times lower than the coefficient, 0.00006297, which is in line with what we want to see from a good model.

ggplot(lm, aes(x = .fitted, y = .resid)) +
    geom_point() +
    geom_hline(yintercept = 0) +
    labs(title = "Residual vs. Fitted Values",
         x = "Fitted Values", y= "Residuals")

Normal QQ plot:

# define residuals
res <- resid(lm)

#create Q-Q plot for residuals
qqnorm(res)

#add a straight diagonal line to the plot
qqline(res, col = "red")

Question 2:

Raise life expectancy to the 4.6 power (i.e., LifeExp^4.6). Raise total expenditures to the 0.06 power (nearly a log transform, TotExp^.06). Plot LifeExp^4.6 as a function of TotExp^.06, and
re-run the simple regression model using the transformed variables. Provide and interpret the F statistics, R^2, standard error, and p-values. Which model is “better?”

Scatter plot of Transform data:

df <- df %>%
    mutate(TotExp_to_0.06 = TotExp^0.06,
           LifeExp_to_4.6 = LifeExp^4.6)

ggplot(df, aes(x = TotExp_to_0.06, y = LifeExp_to_4.6)) +
    geom_point() +
    labs(title = "LifeExp^4.6 ~ TotExp^0.06",
         x = "TotExp^0.06", y = "LifeExp^4.6")

Add line to the plot

ggplot(df, aes(x = TotExp_to_0.06, y = LifeExp_to_4.6)) +
    geom_point() +
    geom_smooth(method = lm, se = FALSE) + 
    labs(title = "LifeExp^4.6 ~ TotExp^0.06",
         x = "TotExp", y = "LifeExp")

Model 2:

lm2 <- lm(LifeExp_to_4.6 ~ TotExp_to_0.06,data = df)
lm2
## 
## Call:
## lm(formula = LifeExp_to_4.6 ~ TotExp_to_0.06, data = df)
## 
## Coefficients:
##    (Intercept)  TotExp_to_0.06  
##     -736527909       620060216
summary(lm2)
## 
## Call:
## lm(formula = LifeExp_to_4.6 ~ TotExp_to_0.06, data = df)
## 
## Residuals:
##        Min         1Q     Median         3Q        Max 
## -308616089  -53978977   13697187   59139231  211951764 
## 
## Coefficients:
##                  Estimate Std. Error t value Pr(>|t|)    
## (Intercept)    -736527910   46817945  -15.73   <2e-16 ***
## TotExp_to_0.06  620060216   27518940   22.53   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 90490000 on 188 degrees of freedom
## Multiple R-squared:  0.7298, Adjusted R-squared:  0.7283 
## F-statistic: 507.7 on 1 and 188 DF,  p-value: < 2.2e-16

y^=−736527909+620060216x

Model2 yields an F-statistic of 507.7, with the same degrees of freedom as the model1. Additionally, the p-value is even more favorable. Moreover, the R2 value of 0.7298 is considerably superior to the model1, making the transformed model the better choice.

Finally, the standard error is a relatively small percentage of the coefficient.

The model using the transformations is better because the relationship appears clearly linear from the scatterplot after the transformations.

ggplot(lm2, aes(x = .fitted, y = .resid)) +
    geom_point() +
    geom_hline(yintercept = 0) +
    labs(title = "Residual vs. Fitted Values",
         x = "Fitted Values", y= "Residuals")

plotting the residuals vs. the fitted values shows us that the residuals are uniformly scattered above and below zero; there is no obvious pattern, so errors are independent, and variance is equal

Normal QQ plot:

# define residuals
res <- resid(lm2)

#create Q-Q plot for residuals
qqnorm(res)

#add a straight diagonal line to the plot
qqline(res, col = "red")

Question 3:

Using the results from 3, forecast life expectancy when TotExp^.06 =1.5. Then forecast life expectancy when TotExp^.06=2.5.

a <- 1.5
forecast1 <- round((-736527909 + 620060216 * a)^(1/4.6), 1)
forecast1
## [1] 63.3

LifeExp=63.3

Then forecast life expectancy when TotExp.06=2.5

b <- 2.5
forecast2 <- round((-736527909 + 620060216 * b)^(1/4.6), 1)
forecast2
## [1] 86.5

LifeExp=86.5

Question 4:

Build the following multiple regression model and interpret the F Statistics, R^2, standard error, and p-values. How good is the model?

LifeExp = b0+b1 x PropMd + b2 x TotExp +b3 x PropMD x TotExp

lm3 <- lm(LifeExp ~ PropMD * TotExp, data = df)
lm3
## 
## Call:
## lm(formula = LifeExp ~ PropMD * TotExp, data = df)
## 
## Coefficients:
##   (Intercept)         PropMD         TotExp  PropMD:TotExp  
##     6.277e+01      1.497e+03      7.233e-05     -6.026e-03
summary(lm3)
## 
## Call:
## lm(formula = LifeExp ~ PropMD * TotExp, data = df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -27.320  -4.132   2.098   6.540  13.074 
## 
## Coefficients:
##                 Estimate Std. Error t value Pr(>|t|)    
## (Intercept)    6.277e+01  7.956e-01  78.899  < 2e-16 ***
## PropMD         1.497e+03  2.788e+02   5.371 2.32e-07 ***
## TotExp         7.233e-05  8.982e-06   8.053 9.39e-14 ***
## PropMD:TotExp -6.026e-03  1.472e-03  -4.093 6.35e-05 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 8.765 on 186 degrees of freedom
## Multiple R-squared:  0.3574, Adjusted R-squared:  0.3471 
## F-statistic: 34.49 on 3 and 186 DF,  p-value: < 2.2e-16

The F-statistic is 34.49, which is large, and it is associated with a p-value of 0.00000000000000022, which is very low, So there is a statistically significant relationship between PropMD, TotEx and LifeExp.

The Adjusted R2 value is 0.3471, which indicates 34.71% of the variation in LifeExp can be explained by PropMD, TotExp, and their interaction term.

Standard error is a relatively small percentage of the coefficient.

Normal QQ plot:

# define residuals
res <- resid(lm3)

#create Q-Q plot for residuals
qqnorm(res)

#add a straight diagonal line to the plot
qqline(res, col = "red")

Question 5:

Forecast LifeExp when PropMD=.03 and TotExp = 14. Does this forecast seem realistic? Why or why not?

x <- 0.03
z <- 14
forecast3 <- 62.77 + (1497 * x) + (0.00007233 * z) - (0.006026 * x * z)
forecast3
## [1] 107.6785

LifeExp = 107.6785

This doesn’t seem realistic beacuse it’s difficult to predict values beyond your data with any model, and the max LifeExp in our data is 83.The reason for this is that we utilized a large value for PropMD.