{r setup, include=FALSE} knitr::opts_chunk$set(echo = TRUE)
R Markdown
library(RCurl)
library(readr)
library(ggplot2)
library(dplyr)Load the data
who<-read.csv("https://raw.githubusercontent.com/maliat-hossain/FileProcessing/main/who.csv")Exercise 1
Provide a scatterplot of LifeExp~TotExp, and run simple linear regression. Do not transform the variables. Provide and interpret the F statistics, R2, standard error,and p-values only. Discuss whether the assumptions of simple linear regression met.
F statistic: 65.26 on 1 and 188 DF, P-value: 7.714e-14, given the small pvalue, which is much below 0.05 indicates the model has some level of validity.
Multiple R2: 0.2577, Adjusted R2: 0.2537 - The model only accounts for roughly 25% of the data’s variation.
Residual standard error: In our example, the total expenditure required can deviate from the true regression line by approximately 15.3795867, on average.
Are the assumptions of simple linear regression met? No, the conditions are not met.
Linearity: The relationship between X and the mean of Y is not linear. Based on the Residuals vs. Fitted plot, the the red line exhibits a quadratic relationship and is not linear.
Homoscedasticity: The variance of residual is not the same for any value of X. The Scale-Location plot shows the residuals are not spread equally along the ranges of predictor.
Independence: Observations are not independent of each other. Upon examining the Residuals vs. Fitted plot, we can see a correlation between the variables.
Normality: For any fixed value of X, Y is not normally distributed. The nearly normal residual condition doesn’t seem to be met based on the histogram of residuals shown below show the residuals are heavily left skew.
who.lm <- lm(LifeExp~TotExp, data=who)
summary(who.lm)ggplot(who, aes(x =TotExp , y = LifeExp)) +
geom_point()+
geom_smooth(method = "lm", formula = y ~ x, se = FALSE)+
ggtitle("Plot of Life Expectancy by Total Expenditure") +
xlab("Total Expenditure") + ylab("Life Expectancy")par(mfrow=c(2,2)) #prints out two rows, two columns of plots
plot(who.lm)par(mfrow=c(1,1))
hist(who.lm$residuals)Exercise 2
Raise life expectancy to the 4.6 power (exponential increase) (i.e., LifeExp4.6). Raise total expenditures to the 0.06 power (exponential decrease) (nearly a log transform, TotExp.06). Plot LifeExp4.6 as a function of TotExp.06, and re-run the simple regression model using the transformed variables. Provide and interpret the F statistics, R2, standard error, and p-values. Which model is “better?”
F-statistic: 507.7 on 1 and 188 DF, p-value: < 2.2e-16, given the small pvalue, which is much below 0.05 indicates the model has some level of validity.
Multiple R-squared: 0.7298, Adjusted R-squared: 0.7283 - The model only accounts for roughly 73% of the data’s variation.
Residual standard error: In our example, the total expenditure required can deviate from the true regression line by approximately 90490000, on average.
Are the assumptions of simple linear regression met? Yes, conditions are met.
Linearity: The relationship between X and the mean of Y is linear. Based on the Residuals vs. Fitted plot, the the red line exhibits an almost linear relationship.
Homoscedasticity: The variance of residual is the same for any value of X.
The Scale-Location plot shows the residuals are almost spread equally along the ranges of predictor.
Independence: Observations are independent of each other.
Upon examining the Residuals vs. Fitted plot, we can see the there is no correlation between the points, and the red line is fairly flat.
Normality: For any fixed value of X, Y is normally distributed.
The nearly normal residual condition is closely met, although the distribution is slightly left skewed.
Which model is “better?”
Of the models in exercise 1 and 2, the model of exercise 2 is better based solely on the statistics and criteria, this is not to say the model is best fit overall model for the data.
who2 <- who %>%
mutate(LifeExp2 = LifeExp^4.6,
TotExp2 = TotExp^.06)
who.lm2 <- lm(LifeExp2~TotExp2, data=who2)
summary(who.lm2)ggplot(who2, aes(x =TotExp2 , y = LifeExp2)) +
geom_point()+
geom_smooth(method = "lm", formula = y ~ x, se = FALSE, col="red")+
ggtitle("Plot of Life Expectancy by Total Expenditure") +
xlab("Total Expenditure ^0.06") + ylab("Life Expectancy ^4.6")par(mfrow=c(2,2)) #prints out two rows, two columns of plots
plot(who.lm2)par(mfrow=c(1,1))
hist(who.lm2$residuals)Exercise 3
Using the results from 2, forecast life expectancy when \(\operatorname{Tot} E x p^{06}=1.5\). Then forecast life expectancy when \(\operatorname{Tot} E x p^{06}=2.5\)
Life expectancy when \(T o t E x p^{06}=1.5\) is approximately \(63.3\)
#y = 620060216x - 736527910
x = 1.5
y <- (-736527910 + 620060216*x)
y(y)^(1/4.6)Life expectancy when TotExp.06=2.5 is approximately 86.5
x = 2.5
y <- (-736527910 + 620060216*x)
y(y)^(1/4.6)4. Build the following multiple regression model and interpret the F Statistics, \(R^{2}\), standard error, and p-values. How good is the model?
###LifeExp \(=b_{0}+b_{1} \times\) PropMd \(+b_{2} \times\) TotExp \(+b_{3} \times\) PropMD \(\times\) TotExp ###F-statistic: \(34.49\) on 3 and 186 DF, p-value: \(<2.2 \mathrm{e}-16\), given the small pvalue, which is much below \(0.05\) indicates the model has some level of validity. ###Multiple R-squared: \(0.3574\), Adjusted R-squared: \(0.3471\) - The model only accounts for roughly \(36 \%\) of the data’s variation. ## Residual standard error: In our example, the total expenditure required can deviate from the true regression line by approximately \(8.765\), on average. ## Are the assumptions of simple linear regression met? No, conditions are not met. ### 1. Linearity: The relationship between \(X\) and the mean of \(Y\) is not linear. Based on the Residuals vs. Fitted plot, the the red line exhibits a quadratic relationship and is not linear. ### 2. Homoscedasticity: The variance of residual is not the same for any value of \(X\). The Scale-Location plot shows the residuals are not spread equally along the ranges of predictor. ###3. Independence: Observations are not independent of each other. Upon examining the Residuals vs. Fitted plot, we can see a correlation between the variables. ###4. Multivariate Normality: The nearly normal residual condition doesn’t seem to be met based on the histogram of residuals shown below show the residuals are heavily left skew. ## How good is the model? The model is not that great given the criteria mentioned above.
who.lm3 <- lm(LifeExp~TotExp + PropMD + PropMD * TotExp, data=who)
summary(who.lm3)par(mfrow=c(2,2))
plot(who.lm3)par(mfrow=c(1,1))
# residuals histogram
hist(who.lm3$residuals,
xlab = "Residuals", ylab = "",
main = "Histogram of Residuals Distribution")###Exercise 5 Forecast \(L i f e E x p\) when \(\operatorname{Prop} M D=.03\) and \(\operatorname{Tot} E x p=14\). Does this forecast seem realistic? Why or why not? The forecasted life expectancy using the linear model from exercise 4 is \(107.7\). This forecast is unrealistic with the actual data because the maximum life expectancy from the WHO data is 83 and the mean of all life expectancies is \(67.4\).
b0 <- 6.277 * 10^1
b1 <- 1.497 * 10^3
b2 <- 7.233 * 10^-5
b3 <- -6.026 * 10^-3
x1 <- .03 #PropMD
x2 <- 14 #TotExp
y = b0 + (b1 * x1) + (b2 * x2) + (b3 * x1 * x2)
y\[ \text { LifeExp }=b_{0}+b_{1} \times \text { PropMd }+b_{2} \times \text { TotExp }+b_{3} \times \text { PropMD } \times \text { TotExp } \]
max(who$LifeExp)mean(who$LifeExp)