In this micro project, we’ll use Decision Trees, and later, Random Forests to classify schools as Private or Public based off their features. We’ll be using the College dataframe from the ISLR library.

Setup

library(ggplot2)
library(ISLR)
library(dplyr)
library(caTools)
library(rpart)

df <- College
head(College)
##                              Private Apps Accept Enroll Top10perc
## Abilene Christian University     Yes 1660   1232    721        23
## Adelphi University               Yes 2186   1924    512        16
## Adrian College                   Yes 1428   1097    336        22
## Agnes Scott College              Yes  417    349    137        60
## Alaska Pacific University        Yes  193    146     55        16
## Albertson College                Yes  587    479    158        38
##                              Top25perc F.Undergrad P.Undergrad Outstate
## Abilene Christian University        52        2885         537     7440
## Adelphi University                  29        2683        1227    12280
## Adrian College                      50        1036          99    11250
## Agnes Scott College                 89         510          63    12960
## Alaska Pacific University           44         249         869     7560
## Albertson College                   62         678          41    13500
##                              Room.Board Books Personal PhD Terminal
## Abilene Christian University       3300   450     2200  70       78
## Adelphi University                 6450   750     1500  29       30
## Adrian College                     3750   400     1165  53       66
## Agnes Scott College                5450   450      875  92       97
## Alaska Pacific University          4120   800     1500  76       72
## Albertson College                  3335   500      675  67       73
##                              S.F.Ratio perc.alumni Expend Grad.Rate
## Abilene Christian University      18.1          12   7041        60
## Adelphi University                12.2          16  10527        56
## Adrian College                    12.9          30   8735        54
## Agnes Scott College                7.7          37  19016        59
## Alaska Pacific University         11.9           2  10922        15
## Albertson College                  9.4          11   9727        55

EDA

Some basic data exploration before we begin building our models.

** Scatterplot of graduation rate vs room and boarding costs. **

ggplot(df,aes(Room.Board,Grad.Rate)) + geom_point(aes(color=Private))

It makes sense to see that Private colleges have higher Room and boarding costs.

** Histogram of full time undergrad students, colored by private. **

ggplot(df,aes(F.Undergrad)) + geom_histogram(aes(fill=Private),color="black",bins=50)

** Histogram of graduation rate, colored by private. **

ggplot(df,aes(Grad.Rate)) + geom_histogram(aes(fill=Private),color="black",bins=50)

There’s a college with a graduation rate above 100%. Lets find that out and fix it.

subset(df,Grad.Rate>100)
##                   Private Apps Accept Enroll Top10perc Top25perc
## Cazenovia College     Yes 3847   3433    527         9        35
##                   F.Undergrad P.Undergrad Outstate Room.Board Books
## Cazenovia College        1010          12     9384       4840   600
##                   Personal PhD Terminal S.F.Ratio perc.alumni Expend
## Cazenovia College      500  22       47      14.3          20   7697
##                   Grad.Rate
## Cazenovia College       118

Fix:

df['Cazenovia College','Grad.Rate'] <- 100

Train test split

set.seed(101)

sample = sample.split(df$Private,SplitRatio=0.7)
train = subset(df,sample==T)
test = subset(df,sample==F)

Decision Tree

Building the model and making predictions.

tree <- rpart(Private ~ . ,method="class",data=train)
tree.preds <- predict(tree,test)
head(tree.preds)
##                                                  No       Yes
## Adrian College                          0.003311258 0.9966887
## Alfred University                       0.003311258 0.9966887
## Allegheny College                       0.003311258 0.9966887
## Allentown Coll. of St. Francis de Sales 0.003311258 0.9966887
## Alma College                            0.003311258 0.9966887
## Amherst College                         0.003311258 0.9966887

Creating a column “Private”, with the variables Yes/No to indicate if the college is private or not, to match our original dataframe so that we can compare the results easily.

tree.preds <- as.data.frame(tree.preds)
tree.preds$Private <- ifelse(tree.preds$Yes > 0.5,"Yes","No")

Checking out our confusion matrix.

table(tree.preds$Private,test$Private)
##      
##        No Yes
##   No   57   9
##   Yes   7 160

Visualizing the tree.

library(rpart.plot)
prp(tree)

Random Forest

Loading the library:

library(randomForest)

Building the model:

rf.model <- randomForest(Private ~ ., data=train,importance=TRUE)

Making predictions:

p <- predict(rf.model,test)
table(p,test$Private)
##      
## p      No Yes
##   No   58   7
##   Yes   6 162

We see that the model has performed a bit better than the decision tree.