Read the data in R.

data<-read.csv("binary.csv")

head(data)
##   admit gre  gpa rank
## 1     0 380 3.61    3
## 2     1 660 3.67    3
## 3     1 800 4.00    1
## 4     1 640 3.19    4
## 5     0 520 2.93    4
## 6     1 760 3.00    2
str(data)
## 'data.frame':    400 obs. of  4 variables:
##  $ admit: int  0 1 1 1 0 1 1 0 1 0 ...
##  $ gre  : int  380 660 800 640 520 760 560 400 540 700 ...
##  $ gpa  : num  3.61 3.67 4 3.19 2.93 3 2.98 3.08 3.39 3.92 ...
##  $ rank : int  3 3 1 4 4 2 1 2 3 2 ...

Getting the data summary

summary(data)
##      admit             gre             gpa             rank      
##  Min.   :0.0000   Min.   :220.0   Min.   :2.260   Min.   :1.000  
##  1st Qu.:0.0000   1st Qu.:520.0   1st Qu.:3.130   1st Qu.:2.000  
##  Median :0.0000   Median :580.0   Median :3.395   Median :2.000  
##  Mean   :0.3175   Mean   :587.7   Mean   :3.390   Mean   :2.485  
##  3rd Qu.:1.0000   3rd Qu.:660.0   3rd Qu.:3.670   3rd Qu.:3.000  
##  Max.   :1.0000   Max.   :800.0   Max.   :4.000   Max.   :4.000

There are no missing values in the data set.

sum(is.na(data))
## [1] 0

The respose variable is admit. It has two levels: 0 and 1, The observations are distributed as below:

table(data$admit)
## 
##   0   1 
## 273 127

Distribution of admissions across the ranks

xtabs(~admit+rank,data=data)
##      rank
## admit  1  2  3  4
##     0 28 97 93 55
##     1 33 54 28 12

Since the response variable is Binary, we run Logistic Regression.


Logistic Regression Model

logit_model<-glm(admit~.,data=data,family = "binomial")
summary(logit_model)
## 
## Call:
## glm(formula = admit ~ ., family = "binomial", data = data)
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)    
## (Intercept) -3.449548   1.132846  -3.045  0.00233 ** 
## gre          0.002294   0.001092   2.101  0.03564 *  
## gpa          0.777014   0.327484   2.373  0.01766 *  
## rank        -0.560031   0.127137  -4.405 1.06e-05 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 499.98  on 399  degrees of freedom
## Residual deviance: 459.44  on 396  degrees of freedom
## AIC: 467.44
## 
## Number of Fisher Scoring iterations: 4

Logistic Regression Model Diagnostics


1. Test for linearity
Generate Predicted values

probabilities <- predict(logit_model, type = "response")
predicted.classes <- ifelse(probabilities > 0.5, "pos", "neg")
head(predicted.classes)
##     1     2     3     4     5     6 
## "neg" "neg" "pos" "neg" "neg" "neg"

Bind the logit values to the data

library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.1.4     ✔ readr     2.1.4
## ✔ forcats   1.0.0     ✔ stringr   1.5.1
## ✔ ggplot2   3.4.4     ✔ tibble    3.2.1
## ✔ lubridate 1.9.3     ✔ tidyr     1.3.0
## ✔ purrr     1.0.2     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
data <- data %>%
  mutate(logit = log(probabilities/(1-probabilities))) %>%
  gather(key = "predictors", value = "predictor.value", -logit)

Create the scatter plots:

ggplot(data, aes(logit, predictor.value))+
  geom_point(size = 0.5, alpha = 0.5) +
  geom_smooth(method = "loess") + 
  theme_bw() + 
  facet_wrap(~predictors, scales = "free_y")
## `geom_smooth()` using formula = 'y ~ x'


The plots show that there exists linear relationship between the dependent and the independent variables: gre, gpa, and rank. ‘rank’ is categorical with 4 levels as shown by the plots.


2. Test for Influential values

plot(logit_model, which = 4, id.n = 3)

The plot shows top three most influencial points in the model.To check for the potential outliers, we calculate the standardised residuals and Cook’s distance as below, using augment() function from ‘broom’ package.

# Extract model results
library(broom)
model.data <- augment(logit_model) %>% 
  mutate(index = 1:n()) 

Now, the following are the top three most influential points in the dataset.

model.data %>% top_n(3, .cooksd)
## # A tibble: 3 × 11
##   admit   gre   gpa  rank .fitted .resid    .hat .sigma .cooksd .std.resid index
##   <int> <int> <dbl> <int>   <dbl>  <dbl>   <dbl>  <dbl>   <dbl>      <dbl> <int>
## 1     1   580  2.86     4   -2.14   2.12 0.00904   1.07  0.0195       2.13   156
## 2     1   400  3.23     4   -2.26   2.17 0.00845   1.07  0.0206       2.18   198
## 3     1   300  2.84     2   -1.67   1.92 0.0158    1.07  0.0218       1.94   316

Plot the standardized residuals:

ggplot(model.data, aes(index, .std.resid)) + 
  geom_point(aes(color = admit), alpha = .6) +
  theme_bw()

Filter out potential outliers:

model.data %>% 
  filter(abs(.std.resid) > 3)
## # A tibble: 0 × 11
## # ℹ 11 variables: admit <int>, gre <int>, gpa <dbl>, rank <int>, .fitted <dbl>,
## #   .resid <dbl>, .hat <dbl>, .sigma <dbl>, .cooksd <dbl>, .std.resid <dbl>,
## #   index <int>

There are no potential outliers in our dataset


3. Test for Multicollinearlity

library(car)
## Loading required package: carData
## 
## Attaching package: 'car'
## The following object is masked from 'package:dplyr':
## 
##     recode
## The following object is masked from 'package:purrr':
## 
##     some
vif(logit_model)
##      gre      gpa     rank 
## 1.135657 1.134708 1.004034

Variance Inflation Factors (VIF) associated with all the independent variaables are less than 5. Hence, there is no multicollinearity


Results suummary

Logistic regression of admit on gre, gpa and position showed that the independent variables jointly significantly predict the response variable, admit, p<0.0001.


All the variables significantly independently predicts admit at 5% significance level, p-values<0.05.