Correlation is a statistical technique that can show whether and how strongly pairs (strength) of variables are related. For example, height and weight are related; taller people tend to be heavier than shorter people.
The coefficient of determination is the proportion of the variance (or variability) in the dependent variable that is predictable from the independent variable(s).
Regression analysis is a set of statistical processes for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or ‘predictors’). More specifically, regression analysis helps one understand how the typical value of the dependent variable changes when any one of the independent variables is varied, while the other independent variables are held fixed.
The existence of a statistical relation between the response variable Y and an explanatory or predictor variable X does not imply in any way that Y depends causally on X.
Pearson correlation coefficient is a measure of the strength of linear association between two quantitative variables X and Y.
The correlation coefficient will be between -1 and +1 (inclusive).
r > 0 – positive or direct linear association
r < 0 – negative or indirect or inverse linear association
r = 0 – no linear association
r = 1 – perfect positive linear association
Residuals vs Fitted: is used to check the assumptions of linearity. If the residuals are spread equally around a horizontal line without distinct patterns (red line is approximately horizontal at zero), that is a good indication of having a linear relationship.
Normal Q-Q: is used to check the normality of residuals assumption. If the majority of the residuals follow the straight dashed line, then the assumption is fulfilled.
Scale-Location: is used to check the homoscedasticity of residuals (equal variance of residuals). If the residuals are spread randomly and the see a horizontal line with equally (randomly) spread points, then the assumption is fulfilled.
Residuals vs Leverage: is used to identify any influential value in our dataset. Influential values are extreme values that might influence the regression results when included or excluded from the analysis. Look for cases outside of a dashed line.
## [1] 0.9762662
##
## Pearson's product-moment correlation
##
## data: x and y
## t = 12.75, df = 8, p-value = 1.349e-06
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## 0.8996224 0.9945560
## sample estimates:
## cor
## 0.9762662
This is better. Allows you to talk about statistical significant relationship between the variables.
By default it uses 95% confidence level.
##
## Pearson's product-moment correlation
##
## data: x and y
## t = 12.75, df = 8, p-value = 1.349e-06
## alternative hypothesis: true correlation is not equal to 0
## 90 percent confidence interval:
## 0.9200475 0.9930968
## sample estimates:
## cor
## 0.9762662
## [1] 0.9530957
##
## Call:
## lm(formula = y ~ x)
##
## Residuals:
## Min 1Q Median 3Q Max
## -13.074 -4.403 -1.607 5.719 14.834
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.6979 5.9520 0.789 0.453
## x 1.9705 0.1545 12.750 1.35e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 9.022 on 8 degrees of freedom
## Multiple R-squared: 0.9531, Adjusted R-squared: 0.9472
## F-statistic: 162.6 on 1 and 8 DF, p-value: 1.349e-06
## (Intercept) x
## 4.697852 1.970478
The intercept is 4.70
The slope is 1.97
## 2.5 % 97.5 %
## (Intercept) -9.027533 18.423236
## x 1.614088 2.326867
The intercept is 4.70 (95% CI: -9.03, 18.42)
The slope is 1.97 (95% CI: 1.61, 2.33)
## 1
## 34.25502
## fit lwr upr
## 1 34.25502 24.86499 43.64505
Residuals vs Fitted: is used to check the assumptions of linearity. If the residuals are spread equally around a horizontal line without distinct patterns (red line is approximately horizontal at zero), that is a good indication of having a linear relationship.
Normal Q-Q: is used to check the normality of residuals assumption. If the majority of the residuals follow the straight dashed line, then the assumption is fulfilled.
Scale-Location: is used to check the homoscedasticity of residuals (equal variance of residuals). If the residuals are spread randomly and the see a horizontal line with equally (randomly) spread points, then the assumption is fulfilled.
Residuals vs Leverage: is used to identify any influential value in our dataset. Influential values are extreme values that might influence the regression results when included or excluded from the analysis. Look for cases outside of a dashed line.
##
## Pearson's product-moment correlation
##
## data: wn$ScoreSub1 and wn$FamilyMonthlyInc
## t = 2.2703, df = 448, p-value = 0.02366
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## 0.01435481 0.19714661
## sample estimates:
## cor
## 0.1066518
## Age FamilyMonthlyInc ScoreSub1 ScoreSub2
## Age 1.00000000 -0.043594688 0.02500791 -0.03567968
## FamilyMonthlyInc -0.04359469 1.000000000 0.10665176 -0.02403096
## ScoreSub1 0.02500791 0.106651759 1.00000000 -0.04958413
## ScoreSub2 -0.03567968 -0.024030962 -0.04958413 1.00000000
## ScoreSub3 0.05129655 -0.003516893 0.01919580 0.05149042
## ScoreSub3
## Age 0.051296546
## FamilyMonthlyInc -0.003516893
## ScoreSub1 0.019195800
## ScoreSub2 0.051490416
## ScoreSub3 1.000000000
##
## Call:
## lm(formula = wn$FamilyMonthlyInc ~ wn$ScoreSub1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -4024.0 -1830.8 -109.9 1549.4 6533.9
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3058.07 942.49 3.245 0.00126 **
## wn$ScoreSub1 25.39 11.18 2.270 0.02366 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2227 on 448 degrees of freedom
## Multiple R-squared: 0.01137, Adjusted R-squared: 0.009168
## F-statistic: 5.154 on 1 and 448 DF, p-value: 0.02366
## (Intercept) wn$ScoreSub1
## 3058.06803 25.38689
## 2.5 % 97.5 %
## (Intercept) 1205.813118 4910.32295
## wn$ScoreSub1 3.411261 47.36252
There is a weak positive significant linear relationship between the score and monthly income (r(448) = 0.11, p = 0.0237),
Approximately 2% of the variability in monthly income is accounted for or explained by score.
From the R ouput, the estimated regression line equation can be written as follow: monthly income = 3058.07 + 25.39*score
For intercept, if score is equal to zero, we expect monthly income of 3058.07 (see units)
For slope, a unit (one) increase in score will cause a unit increase in monthly income by 25.39.
Is just an extension of simple linear regression (SLR).
The multiple linear regression analysis allows the prediction of one (dependent) variable from several other (independent) variables.
The first thing to think about is what predictor (or independent) variables to enter into the model.
A great deal of care should be taken in selecting predictors for a model because the estimates of the regression coefficients depend upon the variables in the model.
The predictors included and the way in which they are entered into the model can have a great impact. Do not select hundreds of random predictors, bung them all into a regression analysis and hope for the best.
Remember the quote from George E. Box - “All models are wrong. some are useful.”
R-square increases whenever we add more variables.
So, we prefer adjusted R-square - it is adjusted for the “size” of the model.
If two models fit the data equally well, the simplest (the smaller model) is the best (the better).
Using the Focus dataset posted on Canvas.
The data is approximately normal from the graph.
##
## Call:
## lm(formula = Focus$GPA ~ Focus$AGE + Focus$CREDITS)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.29806 -0.39124 0.01604 0.33502 1.09542
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.317654 0.387577 11.140 < 2e-16 ***
## Focus$AGE -0.084106 0.020922 -4.020 8.28e-05 ***
## Focus$CREDITS 0.005602 0.001262 4.438 1.51e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.4917 on 197 degrees of freedom
## Multiple R-squared: 0.1008, Adjusted R-squared: 0.09165
## F-statistic: 11.04 on 2 and 197 DF, p-value: 2.858e-05
##
## Call:
## lm(formula = Focus$GPA ~ factor(Focus$SEX) + Focus$AGE + Focus$CREDITS)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.21681 -0.37086 0.00697 0.32877 1.17585
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.356240 0.386381 11.274 < 2e-16 ***
## factor(Focus$SEX)M -0.120611 0.070774 -1.704 0.0899 .
## Focus$AGE -0.084202 0.020821 -4.044 7.55e-05 ***
## Focus$CREDITS 0.005786 0.001261 4.589 7.95e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.4893 on 196 degrees of freedom
## Multiple R-squared: 0.1139, Adjusted R-squared: 0.1003
## F-statistic: 8.398 on 3 and 196 DF, p-value: 2.801e-05
# Changing reference category (from Females to Males)
Focus$SEX <- relevel(factor(Focus$SEX), ref = "M")
mlr1<- lm(Focus$GPA ~ factor(Focus$SEX) + Focus$AGE + Focus$CREDITS)
summary(mlr1)##
## Call:
## lm(formula = Focus$GPA ~ factor(Focus$SEX) + Focus$AGE + Focus$CREDITS)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.21681 -0.37086 0.00697 0.32877 1.17585
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.235630 0.388708 10.897 < 2e-16 ***
## factor(Focus$SEX)F 0.120611 0.070774 1.704 0.0899 .
## Focus$AGE -0.084202 0.020821 -4.044 7.55e-05 ***
## Focus$CREDITS 0.005786 0.001261 4.589 7.95e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.4893 on 196 degrees of freedom
## Multiple R-squared: 0.1139, Adjusted R-squared: 0.1003
## F-statistic: 8.398 on 3 and 196 DF, p-value: 2.801e-05
##
## Call:
## lm(formula = Focus$GPA ~ factor(Focus$SEX) + Focus$AGE + Focus$CREDITS)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.21681 -0.37086 0.00697 0.32877 1.17585
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.235630 0.388708 10.897 < 2e-16 ***
## factor(Focus$SEX)F 0.120611 0.070774 1.704 0.0899 .
## Focus$AGE -0.084202 0.020821 -4.044 7.55e-05 ***
## Focus$CREDITS 0.005786 0.001261 4.589 7.95e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.4893 on 196 degrees of freedom
## Multiple R-squared: 0.1139, Adjusted R-squared: 0.1003
## F-statistic: 8.398 on 3 and 196 DF, p-value: 2.801e-05
## 2.5 % 97.5 %
## (Intercept) 3.469041839 5.002217423
## factor(Focus$SEX)F -0.018965819 0.260186897
## Focus$AGE -0.125264445 -0.043139145
## Focus$CREDITS 0.003299627 0.008273339
## 2.5 % 97.5 %
## (Intercept) 3.469041839 5.002217423
## factor(Focus$SEX)F -0.018965819 0.260186897
## Focus$AGE -0.125264445 -0.043139145
## Focus$CREDITS 0.003299627 0.008273339
## 2.5 % 97.5 %
## (Intercept) 3.469041839 5.002217423
## factor(Focus$SEX)F -0.018965819 0.260186897
## Focus$AGE -0.125264445 -0.043139145
## Focus$CREDITS 0.003299627 0.008273339
## 5 % 95 %
## (Intercept) 3.593224896 4.878034366
## factor(Focus$SEX)F 0.003644793 0.237576285
## Focus$AGE -0.118612519 -0.049791071
## Focus$CREDITS 0.003702484 0.007870482
## factor(Focus$SEX) Focus$AGE Focus$CREDITS
## 1.011989 1.686409 1.698870
# Calling Data from a Package
data("Boston", package = "MASS")
# library(olsrr)
#View variables in Boston Dataset
names(Boston) ## [1] "crim" "zn" "indus" "chas" "nox" "rm" "age"
## [8] "dis" "rad" "tax" "ptratio" "black" "lstat" "medv"
##
## Call:
## lm(formula = medv ~ ., data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -15.595 -2.730 -0.518 1.777 26.199
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.646e+01 5.103e+00 7.144 3.28e-12 ***
## crim -1.080e-01 3.286e-02 -3.287 0.001087 **
## zn 4.642e-02 1.373e-02 3.382 0.000778 ***
## indus 2.056e-02 6.150e-02 0.334 0.738288
## chas 2.687e+00 8.616e-01 3.118 0.001925 **
## nox -1.777e+01 3.820e+00 -4.651 4.25e-06 ***
## rm 3.810e+00 4.179e-01 9.116 < 2e-16 ***
## age 6.922e-04 1.321e-02 0.052 0.958229
## dis -1.476e+00 1.995e-01 -7.398 6.01e-13 ***
## rad 3.060e-01 6.635e-02 4.613 5.07e-06 ***
## tax -1.233e-02 3.760e-03 -3.280 0.001112 **
## ptratio -9.527e-01 1.308e-01 -7.283 1.31e-12 ***
## black 9.312e-03 2.686e-03 3.467 0.000573 ***
## lstat -5.248e-01 5.072e-02 -10.347 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.745 on 492 degrees of freedom
## Multiple R-squared: 0.7406, Adjusted R-squared: 0.7338
## F-statistic: 108.1 on 13 and 492 DF, p-value: < 2.2e-16
## crim zn indus chas nox rm age dis
## 1.792192 2.298758 3.991596 1.073995 4.393720 1.933744 3.100826 3.955945
## rad tax ptratio black lstat
## 7.484496 9.008554 1.799084 1.348521 2.941491
## crim zn indus chas nox rm age dis
## 1.791940 2.184240 3.226015 1.058220 4.369271 1.923075 3.098044 3.954446
## rad ptratio black lstat
## 2.837494 1.788839 1.347564 2.940800
Therefore we can safely assume that linearity assumption is not violated.
Normality assumption is violated.
Equal variance assumption is violated.
Logistic regression, also called a logit model, is used to model categorical (usually dichotomous or binary) outcome variables.
In the logit model the log odds of the outcome is modeled as a linear combination of the predictor variables.
Before fitting a model to a dataset, logistic regression makes the following assumptions:
Logistic regression assumes that the response variable only takes on two possible outcomes. If there are more than two possible outcomes, you will need to perform ordinal regression instead.
Logistic regression assumes that there exists a linear relationship between each explanatory variable and the logit of the response variable. Although this analysis does not require the dependent and independent variables to be related linearly, it requires that the independent variables are linearly related to the log odds.
Finally, logistic regression typically requires a large sample size. A general guideline is that you need at minimum of 10 cases with the least frequent outcome for each independent variable in your model.
## admit gre gpa rank id
## 1 0 380 3.61 3 1
## 2 1 660 3.67 3 2
## 3 1 800 4.00 1 3
## 4 1 640 3.19 4 4
## 5 0 520 2.93 4 5
## 6 1 760 3.00 2 6
##
## 0 1
## 273 127
##
## 1 2 3 4
## 61 151 121 67
##
## 1 2 3 4
## 0 28 97 93 55
## 1 33 54 28 12
## Binary$rank
## Binary$admit 1 2 3 4
## 0 28 97 93 55
## 1 33 54 28 12
Two-way contingency table of categorical outcome and predictors we want to make sure there are not 0 cells.
Most of the variables can be investigated using cross tabulations with the dependent variable admit.
Another reason for the cross tabulation is to identify categories with small frequencies as this can cause problems with the logistic regression procedure. For building contingency tables and testing relationships between categorical variables use Chi Squared tests.
##
## Pearson's Chi-squared test
##
## data: Binary$admit and Binary$rank
## X-squared = 25.242, df = 3, p-value = 1.374e-05
Compute chi-square test to know the association between the variables.
There is a statistically significant association between admit and rank.
The code below estimates a logistic regression model using the glm (generalized linear model) function.
Convert rank to a factor to indicate that rank should be treated as a categorical variable.
model <- glm(Binary$admit ~ Binary$gre + Binary$gpa + factor(Binary$rank), family = "binomial")
summary(model)##
## Call:
## glm(formula = Binary$admit ~ Binary$gre + Binary$gpa + factor(Binary$rank),
## family = "binomial")
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -1.6268 -0.8662 -0.6388 1.1490 2.0790
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -3.989979 1.139951 -3.500 0.000465 ***
## Binary$gre 0.002264 0.001094 2.070 0.038465 *
## Binary$gpa 0.804038 0.331819 2.423 0.015388 *
## factor(Binary$rank)2 -0.675443 0.316490 -2.134 0.032829 *
## factor(Binary$rank)3 -1.340204 0.345306 -3.881 0.000104 ***
## factor(Binary$rank)4 -1.551464 0.417832 -3.713 0.000205 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 499.98 on 399 degrees of freedom
## Residual deviance: 458.52 on 394 degrees of freedom
## AIC: 470.52
##
## Number of Fisher Scoring iterations: 4
We can use the confint function to obtain confidence intervals for the coefficient estimates.
Note that for logistic models, confidence intervals are based on the profiled log-likelihood function. We can also get CIs based on just the standard errors by using the default method.
## Waiting for profiling to be done...
## 2.5 % 97.5 %
## (Intercept) -6.2716202334 -1.792547080
## Binary$gre 0.0001375921 0.004435874
## Binary$gpa 0.1602959439 1.464142727
## factor(Binary$rank)2 -1.3008888002 -0.056745722
## factor(Binary$rank)3 -2.0276713127 -0.670372346
## factor(Binary$rank)4 -2.4000265384 -0.753542605
## 2.5 % 97.5 %
## (Intercept) -6.2242418514 -1.755716295
## Binary$gre 0.0001202298 0.004408622
## Binary$gpa 0.1536836760 1.454391423
## factor(Binary$rank)2 -1.2957512650 -0.055134591
## factor(Binary$rank)3 -2.0169920597 -0.663415773
## factor(Binary$rank)4 -2.3703986294 -0.732528724
Logistic regression models were developed to help examine the impact of covariates on both outcomes.
Odds Ratio represent an increase (if significantly greater than 1) and decrease (if significantly less than 1) from the baseline hazard. Odds ratio of one (1) indicates no difference between the selected variable and the corresponding outcome.
Holding all other variables constant, some important relationships between the response variable and the associated covariates were statistically significant.
## Waiting for profiling to be done...
## OR 2.5 % 97.5 %
## (Intercept) 0.0185001 0.001889165 0.1665354
## Binary$gre 1.0022670 1.000137602 1.0044457
## Binary$gpa 2.2345448 1.173858216 4.3238349
## factor(Binary$rank)2 0.5089310 0.272289674 0.9448343
## factor(Binary$rank)3 0.2617923 0.131641717 0.5115181
## factor(Binary$rank)4 0.2119375 0.090715546 0.4706961
Suppose that we are interested in the factors that contributes to Obesity.
The outcome (response) variable is binary (0 = No and 1 = Yes).
## Experience Income Location Switch Perception Days IQ Stress College FC
## 1 16 no no yes decreased 5 85 17 No No
## 2 13 no no yes decreased 13 95 25 No No
## 3 40 yes no no decreased 7 110 12 No No
## 4 31 no no no decreased 20 120 18 No No
## 5 32 no yes no decreased 11 105 13 Yes No
## 6 46 no no no decreased 16 100 27 No No
## Marital.Status Sex Job.Status Obese Age LogAge
## 1 Married Male Full No 27 1.431364
## 2 Married Male Full No 47 1.672098
## 3 Married Male Full No 48 1.681241
## 4 Married Male Full No 64 1.806180
## 5 Married Male Part No 48 1.681241
## 6 Married Male Full No 58 1.763428
# library(plyr)
FC_Edited$Obese_cat <- revalue(FC_Edited$Obese, c("No"="0", "Yes"="1"))
head(FC_Edited)## Experience Income Location Switch Perception Days IQ Stress College FC
## 1 16 no no yes decreased 5 85 17 No No
## 2 13 no no yes decreased 13 95 25 No No
## 3 40 yes no no decreased 7 110 12 No No
## 4 31 no no no decreased 20 120 18 No No
## 5 32 no yes no decreased 11 105 13 Yes No
## 6 46 no no no decreased 16 100 27 No No
## Marital.Status Sex Job.Status Obese Age LogAge Obese_cat
## 1 Married Male Full No 27 1.431364 0
## 2 Married Male Full No 47 1.672098 0
## 3 Married Male Full No 48 1.681241 0
## 4 Married Male Full No 64 1.806180 0
## 5 Married Male Part No 48 1.681241 0
## 6 Married Male Full No 58 1.763428 0
##
## No Yes
## 131 68
##
## 0 1
## 131 68
##
## Pearson's Chi-squared test with Yates' continuity correction
##
## data: table(FC_Edited$Obese_cat, FC_Edited$Sex)
## X-squared = 67.772, df = 1, p-value < 2.2e-16
## Warning in chisq.test(table(FC_Edited$Obese_cat, FC_Edited$Marital.Status)):
## Chi-squared approximation may be incorrect
##
## Pearson's Chi-squared test
##
## data: table(FC_Edited$Obese_cat, FC_Edited$Marital.Status)
## X-squared = 1.927, df = 2, p-value = 0.3816
## FC_Edited$Sex
## FC_Edited$Obese Female Male
## No 83 48
## Yes 1 67
##
## Pearson's Chi-squared test with Yates' continuity correction
##
## data: table(FC_Edited$Obese, FC_Edited$Sex)
## X-squared = 67.772, df = 1, p-value < 2.2e-16
##
## Pearson's Chi-squared test
##
## data: table(FC_Edited$Obese, FC_Edited$Sex)
## X-squared = 70.286, df = 1, p-value < 2.2e-16
model1 <- glm(factor(Obese_cat) ~ factor(Sex), data = FC_Edited, family = "binomial")
summary(model1)##
## Call:
## glm(formula = factor(Obese_cat) ~ factor(Sex), family = "binomial",
## data = FC_Edited)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -1.3219 -0.1548 -0.1548 1.0395 2.9769
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -4.419 1.006 -4.392 1.12e-05 ***
## factor(Sex)Male 4.752 1.024 4.643 3.44e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 255.58 on 198 degrees of freedom
## Residual deviance: 167.12 on 197 degrees of freedom
## AIC: 171.12
##
## Number of Fisher Scoring iterations: 7
model2 <- glm(factor(Obese_cat) ~ factor(Sex) + factor(Marital.Status),
data = FC_Edited, family = "binomial")
summary(model2)##
## Call:
## glm(formula = factor(Obese_cat) ~ factor(Sex) + factor(Marital.Status),
## family = "binomial", data = FC_Edited)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -1.7427 -0.2349 -0.1390 1.0885 3.0477
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -4.6346 1.0243 -4.524 6.06e-06 ***
## factor(Sex)Male 4.8473 1.0332 4.691 2.71e-06 ***
## factor(Marital.Status)Separate 1.0443 1.0396 1.005 0.315
## factor(Marital.Status)Single 1.0585 0.7563 1.399 0.162
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 255.58 on 198 degrees of freedom
## Residual deviance: 163.96 on 195 degrees of freedom
## AIC: 171.96
##
## Number of Fisher Scoring iterations: 7
model3 <- glm(factor(Obese_cat) ~ factor(Sex) + factor(Marital.Status) + Stress,
data = FC_Edited, family = "binomial")
summary(model3)##
## Call:
## glm(formula = factor(Obese_cat) ~ factor(Sex) + factor(Marital.Status) +
## Stress, family = "binomial", data = FC_Edited)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -2.0136 -0.2481 -0.1268 0.8401 2.8666
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -3.70064 1.09149 -3.390 0.000698 ***
## factor(Sex)Male 4.72849 1.03543 4.567 4.95e-06 ***
## factor(Marital.Status)Separate 0.87648 1.05368 0.832 0.405503
## factor(Marital.Status)Single 1.30580 0.78198 1.670 0.094945 .
## Stress -0.05595 0.02517 -2.223 0.026219 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 255.58 on 198 degrees of freedom
## Residual deviance: 158.75 on 194 degrees of freedom
## AIC: 168.75
##
## Number of Fisher Scoring iterations: 7
Check AIC values from all the competing models. Model with lower AIC is preferred.
Model 3 is the best competing model.
##
## Call:
## glm(formula = factor(Obese_cat) ~ factor(Sex) + factor(Marital.Status) +
## Stress, family = "binomial", data = FC_Edited)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -2.0136 -0.2481 -0.1268 0.8401 2.8666
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -3.70064 1.09149 -3.390 0.000698 ***
## factor(Sex)Male 4.72849 1.03543 4.567 4.95e-06 ***
## factor(Marital.Status)Separate 0.87648 1.05368 0.832 0.405503
## factor(Marital.Status)Single 1.30580 0.78198 1.670 0.094945 .
## Stress -0.05595 0.02517 -2.223 0.026219 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 255.58 on 198 degrees of freedom
## Residual deviance: 158.75 on 194 degrees of freedom
## AIC: 168.75
##
## Number of Fisher Scoring iterations: 7
Both stress and sex (with reference group females) are statistically significant.
Both marital status single and separate (with reference group married) are not statistically significant at 0.05 level of significance.
However, marital status single (with reference group married) is statistically significant at 10% level of significance.
## 2.5 % 97.5 %
## (Intercept) -5.8399196 -1.561355809
## factor(Sex)Male 2.6990929 6.757890337
## factor(Marital.Status)Separate -1.1886879 2.941657414
## factor(Marital.Status)Single -0.2268448 2.838451758
## Stress -0.1052740 -0.006618739
## Waiting for profiling to be done...
## OR 2.5 % 97.5 %
## (Intercept) 0.02470777 0.001283817 0.1412557
## factor(Sex)Male 113.12479826 23.103969985 2054.8394463
## factor(Marital.Status)Separate 2.40243969 0.359460256 26.5982718
## factor(Marital.Status)Single 3.69065325 0.885282035 20.5110786
## Stress 0.94558983 0.898370076 0.9922267
## OR 2.5 % 97.5 %
## (Intercept) 0.02470777 0.002909077 0.2098514
## factor(Sex)Male 113.12479826 14.866240441 860.8242300
## factor(Marital.Status)Separate 2.40243969 0.304620694 18.9472237
## factor(Marital.Status)Single 3.69065325 0.797044469 17.0892867
## Stress 0.94558983 0.900077832 0.9934031
Since female was used as the reference group, males had significantly higher odds of being obese compared to females (OR = 113.13, 95% CI=14.87-860.24).
Compared to those married, both those who are separated (OR = 2.40, 95% CI=0.30-18.95) and single (OR = 3.69, 95% CI=0.80-17.09) had insignificantly greater odds of being obese.
In terms of percent change, we can say that the odds for those who are separated are 140% higher than the odds for those who are married. Equivalently, the odds for those who are separated are 269% higher than the odds for those who are married.
##
## FALSE TRUE
## No 98 33
## Yes 13 55
The classification table shows us that approximately 77% (=(98+55)/199) of the fitted values were correctly classified.
In terms of accuracy that means that our model predicts way better than just a random guess, that is, the chance of predicting correctly for obese or not by random is 50% so our model is more accurate than random guessing.
Talk about Sensitivity and Specificity, see link https://bookdown.org/brianmachut/uofm_analytics_r_hw_sol_2/logreg.html
This t-test is designed to compare means of same variable between two groups.
Description: The independent-samples t-test compares the means of two independent samples. Samples are independent if there is no relationship between them (that is, one group does not influence the other).
# Reading data into R
fresh <- c(10.2,10.5,10.3,10.8,9.8,10.6,10.7,10.2,10.0,10.6)
stored <- c(9.8,9.6,10.1,10.2,10.1,9.7,9.5,9.6,9.8,9.9)
# Re-format Data
Potency <- c(10.2,10.5,10.3,10.8,9.8,10.6,10.7,10.2,10.0,10.6,
9.8,9.6,10.1,10.2,10.1,9.7,9.5,9.6,9.8,9.9)
Group <- c(1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2)##
## Shapiro-Wilk normality test
##
## data: Potency
## W = 0.95248, p-value = 0.4064
##
## F test to compare two variances
##
## data: fresh and stored
## F = 1.8061, num df = 9, denom df = 9, p-value = 0.3917
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
## 0.4486201 7.2715173
## sample estimates:
## ratio of variances
## 1.806142
## [1] 0.3233505
## [1] 0.2406011
##
## Two Sample t-test
##
## data: Potency by Group
## t = 4.2368, df = 18, p-value = 0.0004959
## alternative hypothesis: true difference in means between group 1 and group 2 is not equal to 0
## 95 percent confidence interval:
## 0.2722297 0.8077703
## sample estimates:
## mean in group 1 mean in group 2
## 10.37 9.83
##
## Welch Two Sample t-test
##
## data: Potency by Group
## t = 4.2368, df = 16.628, p-value = 0.000581
## alternative hypothesis: true difference in means between group 1 and group 2 is not equal to 0
## 90 percent confidence interval:
## 0.3179948 0.7620052
## sample estimates:
## mean in group 1 mean in group 2
## 10.37 9.83
## Warning in wilcox.test.default(x = DATA[[1L]], y = DATA[[2L]], ...): cannot
## compute exact p-value with ties
##
## Wilcoxon rank sum test with continuity correction
##
## data: Potency by Group
## W = 91, p-value = 0.00211
## alternative hypothesis: true location shift is not equal to 0
The Wilcoxon-Mann-Whitney test is a non-parametric analog to the independent samples t-test and can be used when you do not assume that the dependent variable is a normally distributed.
The results suggest that there is a statistically significant mean potency difference between the two groups (fresh and stored) (W = 91, p = 0.00211).
## COMMIT AGE SEX SCHTYPE SCHLEVEL SALARY CLASSSIZE RESOURCES AUTONOMY CLIMATE
## 1 38 30 1 1 1 42.28 26 5 12 12
## 2 10 30 1 1 3 40.15 21 5 11 9
## 3 30 26 1 1 1 41.83 28 7 9 10
## 4 10 27 1 1 3 37.00 22 7 5 10
## 5 16 24 1 1 1 39.03 25 7 10 11
## 6 38 30 1 1 1 40.79 23 5 10 9
## SUPPORT
## 1 12
## 2 11
## 3 12
## 4 8
## 5 9
## 6 13
##
## Shapiro-Wilk normality test
##
## data: Handout2$COMMIT
## W = 0.99047, p-value = 0.4027
ggplot(data = Handout2,
aes(x = factor(SEX), y = COMMIT)) +
geom_boxplot() +
labs(title = "Commitment Scores versus Sex",
x = "Sex", y = "Commitment Scores")##
## F test to compare two variances
##
## data: Handout2$COMMIT by Handout2$SEX
## F = 0.86064, num df = 78, denom df = 71, p-value = 0.5164
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
## 0.5429501 1.3574243
## sample estimates:
## ratio of variances
## 0.8606385
##
## Two Sample t-test
##
## data: Handout2$COMMIT by Handout2$SEX
## t = -2.0004, df = 149, p-value = 0.04728
## alternative hypothesis: true difference in means between group 1 and group 2 is not equal to 0
## 95 percent confidence interval:
## -9.87413819 -0.06046097
## sample estimates:
## mean in group 1 mean in group 2
## 37.94937 42.91667
Comments from Model 2