Kathleen_Livingstone_Hw7
Kathleen Livingstone
2022-08-24
library(rpart.plot)## Loading required package: rpartdelays.df <- read.csv("C:\\Users\\klivingstone\\Documents\\FlightDelays.csv")
t(t(names(delays.df))) # Look at your selected column names.## [,1]
## [1,] "CRS_DEP_TIME"
## [2,] "CARRIER"
## [3,] "DEP_TIME"
## [4,] "DEST"
## [5,] "DISTANCE"
## [6,] "FL_DATE"
## [7,] "FL_NUM"
## [8,] "ORIGIN"
## [9,] "Weather"
## [10,] "DAY_WEEK"
## [11,] "DAY_OF_MONTH"
## [12,] "TAIL_NUM"
## [13,] "Flight.Status"delays.df <- delays.df[, -c(3, 6, 7, 12)] # Select your variables.
lapply(delays.df, class)## $CRS_DEP_TIME
## [1] "integer"
##
## $CARRIER
## [1] "character"
##
## $DEST
## [1] "character"
##
## $DISTANCE
## [1] "integer"
##
## $ORIGIN
## [1] "character"
##
## $Weather
## [1] "integer"
##
## $DAY_WEEK
## [1] "integer"
##
## $DAY_OF_MONTH
## [1] "integer"
##
## $Flight.Status
## [1] "character"delays.df$DAY_WEEK <- as.factor(delays.df$DAY_WEEK)
delays.df$CRS_DEP_TIME <- cut(delays.df$CRS_DEP_TIME/100, 8)
set.seed(201)
train.index <- sample(c(1:dim(delays.df)[1]), dim(delays.df)[1]*0.6)
valid.index <- setdiff(c(1:dim(delays.df)[1]), train.index)
train.df <- delays.df[train.index, ]
valid.df <- delays.df[valid.index, ]Part A
Fit a classification tree to the flight delay variable using all the relevant predictors. Do not include DEP_TIME (actual departure time) in the model because it is unknown at the time of prediction (unless we are generating our predictions of delays after the plane takes off, which is unlikely). Use a pruned tree with a maximum of 8 levels, setting cp = 0.001. Express the resulting tree as a set of rules.
delays.ct <- rpart(Flight.Status ~ .,
data = train.df[,-8],
method = "class",
maxdepth = 8,
cp = 0.001, model = TRUE)
#pruned tree
pfit<- prune(delays.ct,
cp = delays.ct$cptable[which.min(delays.ct$cptable[,"xerror"]),"CP"])
prp(pfit)
Part B
If you needed to fly between DCA and EWR on a Monday at 7:00 AM, would you be able to use this tree? What other information would you need? Is it available in practice? What information is redundant?
Part C
Fit the same tree as in (Part A), this time excluding the Weather predictor. Display both the pruned and unpruned tree. You will find that the pruned tree contains a single terminal node. How is the pruned tree used for classification? What is the rule for classifying? Examine the unpruned tree below. What are the top three predictors according to this tree?
delays.ct <- rpart(Flight.Status ~ .,
data = train.df[,-c(6,8)],
method = "class",
maxdepth = 8,
cp = 0.001, model = TRUE)
pfit<- prune(delays.ct,
cp = delays.ct$cptable[which.min(delays.ct$cptable[,"xerror"]),"CP"])
prp(pfit)
prp(delays.ct)
delays.ct## n= 1320
##
## node), split, n, loss, yval, (yprob)
## * denotes terminal node
##
## 1) root 1320 240 ontime (0.18181818 0.81818182)
## 2) CARRIER=CO,DH,MQ,RU 815 202 ontime (0.24785276 0.75214724)
## 4) DAY_WEEK=1,2,5,7 458 145 ontime (0.31659389 0.68340611)
## 8) CRS_DEP_TIME=(13.6,15.6],(17.5,19.4] 119 51 ontime (0.42857143 0.57142857)
## 16) DEST=EWR 57 28 ontime (0.49122807 0.50877193)
## 32) CARRIER=DH 8 3 delayed (0.62500000 0.37500000) *
## 33) CARRIER=CO,RU 49 23 ontime (0.46938776 0.53061224)
## 66) DAY_WEEK=1,2,7 34 17 delayed (0.50000000 0.50000000)
## 132) CRS_DEP_TIME=(17.5,19.4] 13 5 delayed (0.61538462 0.38461538) *
## 133) CRS_DEP_TIME=(13.6,15.6] 21 9 ontime (0.42857143 0.57142857)
## 266) ORIGIN=DCA 7 3 delayed (0.57142857 0.42857143) *
## 267) ORIGIN=BWI,IAD 14 5 ontime (0.35714286 0.64285714) *
## 67) DAY_WEEK=5 15 6 ontime (0.40000000 0.60000000) *
## 17) DEST=JFK,LGA 62 23 ontime (0.37096774 0.62903226)
## 34) CRS_DEP_TIME=(17.5,19.4] 17 8 delayed (0.52941176 0.47058824) *
## 35) CRS_DEP_TIME=(13.6,15.6] 45 14 ontime (0.31111111 0.68888889)
## 70) DISTANCE>=228.5 9 4 delayed (0.55555556 0.44444444) *
## 71) DISTANCE< 228.5 36 9 ontime (0.25000000 0.75000000) *
## 9) CRS_DEP_TIME=(5.98,7.91],(7.91,9.82],(9.82,11.7],(11.7,13.6],(15.6,17.5],(19.4,21.3] 339 94 ontime (0.27728614 0.72271386)
## 18) CRS_DEP_TIME=(5.98,7.91],(9.82,11.7],(11.7,13.6],(15.6,17.5],(19.4,21.3] 289 87 ontime (0.30103806 0.69896194)
## 36) ORIGIN=BWI,IAD 180 62 ontime (0.34444444 0.65555556)
## 72) CRS_DEP_TIME=(5.98,7.91],(19.4,21.3] 59 25 ontime (0.42372881 0.57627119)
## 144) DISTANCE>=191 52 25 ontime (0.48076923 0.51923077)
## 288) CRS_DEP_TIME=(5.98,7.91] 20 8 delayed (0.60000000 0.40000000) *
## 289) CRS_DEP_TIME=(19.4,21.3] 32 13 ontime (0.40625000 0.59375000) *
## 145) DISTANCE< 191 7 0 ontime (0.00000000 1.00000000) *
## 73) CRS_DEP_TIME=(9.82,11.7],(11.7,13.6],(15.6,17.5] 121 37 ontime (0.30578512 0.69421488) *
## 37) ORIGIN=DCA 109 25 ontime (0.22935780 0.77064220) *
## 19) CRS_DEP_TIME=(7.91,9.82] 50 7 ontime (0.14000000 0.86000000) *
## 5) DAY_WEEK=3,4,6 357 57 ontime (0.15966387 0.84033613)
## 10) DEST=LGA 92 23 ontime (0.25000000 0.75000000)
## 20) CRS_DEP_TIME=(5.98,7.91],(13.6,15.6],(15.6,17.5],(17.5,19.4],(19.4,21.3] 52 18 ontime (0.34615385 0.65384615)
## 40) DAY_WEEK=3 23 10 ontime (0.43478261 0.56521739)
## 80) CRS_DEP_TIME=(13.6,15.6],(17.5,19.4],(19.4,21.3] 11 5 delayed (0.54545455 0.45454545) *
## 81) CRS_DEP_TIME=(5.98,7.91],(15.6,17.5] 12 4 ontime (0.33333333 0.66666667) *
## 41) DAY_WEEK=4,6 29 8 ontime (0.27586207 0.72413793) *
## 21) CRS_DEP_TIME=(7.91,9.82],(9.82,11.7],(11.7,13.6] 40 5 ontime (0.12500000 0.87500000) *
## 11) DEST=EWR,JFK 265 34 ontime (0.12830189 0.87169811) *
## 3) CARRIER=DL,OH,UA,US 505 38 ontime (0.07524752 0.92475248)
## 6) DEST=JFK 29 8 ontime (0.27586207 0.72413793)
## 12) DAY_WEEK=2,7 7 3 delayed (0.57142857 0.42857143) *
## 13) DAY_WEEK=1,3,4,5,6 22 4 ontime (0.18181818 0.81818182) *
## 7) DEST=LGA 476 30 ontime (0.06302521 0.93697479) *Note that the echo = FALSE parameter was added to the
code chunk to prevent printing of the R code that generated the
plot.