Read the data in R.
data<-read.csv("binary.csv")
head(data)
## admit gre gpa rank
## 1 0 380 3.61 3
## 2 1 660 3.67 3
## 3 1 800 4.00 1
## 4 1 640 3.19 4
## 5 0 520 2.93 4
## 6 1 760 3.00 2
str(data)
## 'data.frame': 400 obs. of 4 variables:
## $ admit: int 0 1 1 1 0 1 1 0 1 0 ...
## $ gre : int 380 660 800 640 520 760 560 400 540 700 ...
## $ gpa : num 3.61 3.67 4 3.19 2.93 3 2.98 3.08 3.39 3.92 ...
## $ rank : int 3 3 1 4 4 2 1 2 3 2 ...
Getting the data summary
summary(data)
## admit gre gpa rank
## Min. :0.0000 Min. :220.0 Min. :2.260 Min. :1.000
## 1st Qu.:0.0000 1st Qu.:520.0 1st Qu.:3.130 1st Qu.:2.000
## Median :0.0000 Median :580.0 Median :3.395 Median :2.000
## Mean :0.3175 Mean :587.7 Mean :3.390 Mean :2.485
## 3rd Qu.:1.0000 3rd Qu.:660.0 3rd Qu.:3.670 3rd Qu.:3.000
## Max. :1.0000 Max. :800.0 Max. :4.000 Max. :4.000
There are no missing values in the data set.
sum(is.na(data))
## [1] 0
The respose variable is admit. It has two levels: 0 and 1, The observations are distributed as below:
table(data$admit)
##
## 0 1
## 273 127
Distribution of admissions across the ranks
xtabs(~admit+rank,data=data)
## rank
## admit 1 2 3 4
## 0 28 97 93 55
## 1 33 54 28 12
Since the response variable is Binary, we run Logistic Regression.
Logistic Regression Model
logit_model<-glm(admit~.,data=data,family = "binomial")
summary(logit_model)
##
## Call:
## glm(formula = admit ~ ., family = "binomial", data = data)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -3.449548 1.132846 -3.045 0.00233 **
## gre 0.002294 0.001092 2.101 0.03564 *
## gpa 0.777014 0.327484 2.373 0.01766 *
## rank -0.560031 0.127137 -4.405 1.06e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 499.98 on 399 degrees of freedom
## Residual deviance: 459.44 on 396 degrees of freedom
## AIC: 467.44
##
## Number of Fisher Scoring iterations: 4
Logistic Regression Model Diagnostics
1. Test for linearity
Generate Predicted values
probabilities <- predict(logit_model, type = "response")
predicted.classes <- ifelse(probabilities > 0.5, "pos", "neg")
head(predicted.classes)
## 1 2 3 4 5 6
## "neg" "neg" "pos" "neg" "neg" "neg"
Bind the logit values to the data
library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.4 ✔ readr 2.1.4
## ✔ forcats 1.0.0 ✔ stringr 1.5.1
## ✔ ggplot2 3.4.4 ✔ tibble 3.2.1
## ✔ lubridate 1.9.3 ✔ tidyr 1.3.0
## ✔ purrr 1.0.2
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
data <- data %>%
mutate(logit = log(probabilities/(1-probabilities))) %>%
gather(key = "predictors", value = "predictor.value", -logit)
Create the scatter plots:
ggplot(data, aes(logit, predictor.value))+
geom_point(size = 0.5, alpha = 0.5) +
geom_smooth(method = "loess") +
theme_bw() +
facet_wrap(~predictors, scales = "free_y")
## `geom_smooth()` using formula = 'y ~ x'
The plots show that there exists linear relationship between the
dependent and the independent variables: gre, gpa, and rank.
‘rank’ is categorical with 4 levels as shown by the plots.
2. Test for Influential values
plot(logit_model, which = 4, id.n = 3)
The plot shows top three most influencial points in the model.To check
for the potential outliers, we calculate the standardised residuals and
Cook’s distance as below, using augment() function from ‘broom’
package.
# Extract model results
library(broom)
model.data <- augment(logit_model) %>%
mutate(index = 1:n())
Now, the following are the top three most influential points in the dataset.
model.data %>% top_n(3, .cooksd)
## # A tibble: 3 × 11
## admit gre gpa rank .fitted .resid .hat .sigma .cooksd .std.resid index
## <int> <int> <dbl> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <int>
## 1 1 580 2.86 4 -2.14 2.12 0.00904 1.07 0.0195 2.13 156
## 2 1 400 3.23 4 -2.26 2.17 0.00845 1.07 0.0206 2.18 198
## 3 1 300 2.84 2 -1.67 1.92 0.0158 1.07 0.0218 1.94 316
Plot the standardized residuals:
ggplot(model.data, aes(index, .std.resid)) +
geom_point(aes(color = admit), alpha = .6) +
theme_bw()
Filter out potential outliers:
model.data %>%
filter(abs(.std.resid) > 3)
## # A tibble: 0 × 11
## # ℹ 11 variables: admit <int>, gre <int>, gpa <dbl>, rank <int>, .fitted <dbl>,
## # .resid <dbl>, .hat <dbl>, .sigma <dbl>, .cooksd <dbl>, .std.resid <dbl>,
## # index <int>
There are no potential outliers in our dataset
3. Test for Multicollinearlity
library(car)
## Loading required package: carData
##
## Attaching package: 'car'
## The following object is masked from 'package:dplyr':
##
## recode
## The following object is masked from 'package:purrr':
##
## some
vif(logit_model)
## gre gpa rank
## 1.135657 1.134708 1.004034
Variance Inflation Factors (VIF) associated with all the independent variaables are less than 5. Hence, there is no multicollinearity
Results suummary
Logistic regression of admit on gre, gpa and position showed that the independent variables jointly significantly predict the response variable, admit, p<0.0001.
All the variables significantly independently predicts admit at 5% significance level, p-values<0.05.