Homework #4

install.packages("mlbench")

## Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.6'
## (as 'lib' is unspecified)

library(mlbench)

3.1a- Using visualizations, explore the predictor variables to understand their distributions as well as the relationships between predictors.

data(Glass)

str(Glass)

## 'data.frame':    214 obs. of  10 variables:
##  $ RI  : num  1.52 1.52 1.52 1.52 1.52 ...
##  $ Na  : num  13.6 13.9 13.5 13.2 13.3 ...
##  $ Mg  : num  4.49 3.6 3.55 3.69 3.62 3.61 3.6 3.61 3.58 3.6 ...
##  $ Al  : num  1.1 1.36 1.54 1.29 1.24 1.62 1.14 1.05 1.37 1.36 ...
##  $ Si  : num  71.8 72.7 73 72.6 73.1 ...
##  $ K   : num  0.06 0.48 0.39 0.57 0.55 0.64 0.58 0.57 0.56 0.57 ...
##  $ Ca  : num  8.75 7.83 7.78 8.22 8.07 8.07 8.17 8.24 8.3 8.4 ...
##  $ Ba  : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Fe  : num  0 0 0 0 0 0.26 0 0 0 0.11 ...
##  $ Type: Factor w/ 6 levels "1","2","3","5",..: 1 1 1 1 1 1 1 1 1 1 ...

par(mfrow = c(3, 3))

for(i in 1:9) {
  hist(Glass[, i],
       main = names(Glass)[i],
       xlab = names(Glass)[i])
}

pairs(Glass[, 1:9])

The histograms show that several predictors are not normally distributed. K, Ca, Ba, and Fe are especially right-skewed, with most values near the lower end and a few much larger values. The scatterplot matrix also shows relationships between some of the predictors, including a positive relationship between RI and Ca.

3.1b- Do there appear to be any outliers in the data? Are any predictors skewed?

boxplot(Glass[, 1:9],
        main = "Glass Predictors")

install.packages("e1071")

## Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.6'
## (as 'lib' is unspecified)

library(e1071)

sapply(Glass[, 1:9], skewness)

##         RI         Na         Mg         Al         Si          K         Ca 
##  1.6027151  0.4478343 -1.1364523  0.8946104 -0.7202392  6.4600889  2.0184463 
##         Ba         Fe 
##  3.3686800  1.7298107

The boxplot shows several possible outliers, especially for Na, K, Ca, Ba, and Fe. The skewness values also show that several predictors are skewed. K is the most strongly skewed, followed by Ba and Ca.

3.1c- Are there any relevant transformations of one or more predictors that might improve the classification model?

Glass2 <- Glass

Glass2$K <- log1p(Glass2$K)
Glass2$Ca <- log1p(Glass2$Ca)
Glass2$Ba <- log1p(Glass2$Ba)

par(mfrow = c(1, 3))

hist(Glass2$K, main = "K", xlab = "K")
hist(Glass2$Ca, main = "Ca", xlab = "Ca")
hist(Glass2$Ba, main = "Ba", xlab = "Ba")

skewness(Glass2$K)

## [1] 1.95035

skewness(Glass2$Ca)

## [1] 1.152118

skewness(Glass2$Ba)

## [1] 2.688565

A log transformation reduced the skewness of K, Ca, and Ba. K decreased from 6.46 to 1.95, Ca decreased from 2.02 to 1.15, and Ba decreased from 3.37 to 2.69. This suggests that a log transformation could be useful for these predictors before building a classification model.

3.2a- Investigate the frequency distributions for the categorical predictors. Are any of the distributions degenerate in the ways discussed earlier in this chapter?

data(Soybean)

dim(Soybean)

## [1] 683  36

str(Soybean)

## 'data.frame':    683 obs. of  36 variables:
##  $ Class          : Factor w/ 19 levels "2-4-d-injury",..: 11 11 11 11 11 11 11 11 11 11 ...
##  $ date           : Factor w/ 7 levels "0","1","2","3",..: 7 5 4 4 7 6 6 5 7 5 ...
##  $ plant.stand    : Ord.factor w/ 2 levels "0"<"1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ precip         : Ord.factor w/ 3 levels "0"<"1"<"2": 3 3 3 3 3 3 3 3 3 3 ...
##  $ temp           : Ord.factor w/ 3 levels "0"<"1"<"2": 2 2 2 2 2 2 2 2 2 2 ...
##  $ hail           : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 2 1 1 ...
##  $ crop.hist      : Factor w/ 4 levels "0","1","2","3": 2 3 2 2 3 4 3 2 4 3 ...
##  $ area.dam       : Factor w/ 4 levels "0","1","2","3": 2 1 1 1 1 1 1 1 1 1 ...
##  $ sever          : Factor w/ 3 levels "0","1","2": 2 3 3 3 2 2 2 2 2 3 ...
##  $ seed.tmt       : Factor w/ 3 levels "0","1","2": 1 2 2 1 1 1 2 1 2 1 ...
##  $ germ           : Ord.factor w/ 3 levels "0"<"1"<"2": 1 2 3 2 3 2 1 3 2 3 ...
##  $ plant.growth   : Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 2 ...
##  $ leaves         : Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 2 ...
##  $ leaf.halo      : Factor w/ 3 levels "0","1","2": 1 1 1 1 1 1 1 1 1 1 ...
##  $ leaf.marg      : Factor w/ 3 levels "0","1","2": 3 3 3 3 3 3 3 3 3 3 ...
##  $ leaf.size      : Ord.factor w/ 3 levels "0"<"1"<"2": 3 3 3 3 3 3 3 3 3 3 ...
##  $ leaf.shread    : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ leaf.malf      : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ leaf.mild      : Factor w/ 3 levels "0","1","2": 1 1 1 1 1 1 1 1 1 1 ...
##  $ stem           : Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 2 ...
##  $ lodging        : Factor w/ 2 levels "0","1": 2 1 1 1 1 1 2 1 1 1 ...
##  $ stem.cankers   : Factor w/ 4 levels "0","1","2","3": 4 4 4 4 4 4 4 4 4 4 ...
##  $ canker.lesion  : Factor w/ 4 levels "0","1","2","3": 2 2 1 1 2 1 2 2 2 2 ...
##  $ fruiting.bodies: Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 2 ...
##  $ ext.decay      : Factor w/ 3 levels "0","1","2": 2 2 2 2 2 2 2 2 2 2 ...
##  $ mycelium       : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ int.discolor   : Factor w/ 3 levels "0","1","2": 1 1 1 1 1 1 1 1 1 1 ...
##  $ sclerotia      : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ fruit.pods     : Factor w/ 4 levels "0","1","2","3": 1 1 1 1 1 1 1 1 1 1 ...
##  $ fruit.spots    : Factor w/ 4 levels "0","1","2","4": 4 4 4 4 4 4 4 4 4 4 ...
##  $ seed           : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ mold.growth    : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ seed.discolor  : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ seed.size      : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ shriveling     : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ roots          : Factor w/ 3 levels "0","1","2": 1 1 1 1 1 1 1 1 1 1 ...

for(i in 2:ncol(Soybean)) {
  print(names(Soybean)[i])
  print(table(Soybean[, i], useNA = "ifany"))
}

## [1] "date"
## 
##    0    1    2    3    4    5    6 <NA> 
##   26   75   93  118  131  149   90    1 
## [1] "plant.stand"
## 
##    0    1 <NA> 
##  354  293   36 
## [1] "precip"
## 
##    0    1    2 <NA> 
##   74  112  459   38 
## [1] "temp"
## 
##    0    1    2 <NA> 
##   80  374  199   30 
## [1] "hail"
## 
##    0    1 <NA> 
##  435  127  121 
## [1] "crop.hist"
## 
##    0    1    2    3 <NA> 
##   65  165  219  218   16 
## [1] "area.dam"
## 
##    0    1    2    3 <NA> 
##  123  227  145  187    1 
## [1] "sever"
## 
##    0    1    2 <NA> 
##  195  322   45  121 
## [1] "seed.tmt"
## 
##    0    1    2 <NA> 
##  305  222   35  121 
## [1] "germ"
## 
##    0    1    2 <NA> 
##  165  213  193  112 
## [1] "plant.growth"
## 
##    0    1 <NA> 
##  441  226   16 
## [1] "leaves"
## 
##   0   1 
##  77 606 
## [1] "leaf.halo"
## 
##    0    1    2 <NA> 
##  221   36  342   84 
## [1] "leaf.marg"
## 
##    0    1    2 <NA> 
##  357   21  221   84 
## [1] "leaf.size"
## 
##    0    1    2 <NA> 
##   51  327  221   84 
## [1] "leaf.shread"
## 
##    0    1 <NA> 
##  487   96  100 
## [1] "leaf.malf"
## 
##    0    1 <NA> 
##  554   45   84 
## [1] "leaf.mild"
## 
##    0    1    2 <NA> 
##  535   20   20  108 
## [1] "stem"
## 
##    0    1 <NA> 
##  296  371   16 
## [1] "lodging"
## 
##    0    1 <NA> 
##  520   42  121 
## [1] "stem.cankers"
## 
##    0    1    2    3 <NA> 
##  379   39   36  191   38 
## [1] "canker.lesion"
## 
##    0    1    2    3 <NA> 
##  320   83  177   65   38 
## [1] "fruiting.bodies"
## 
##    0    1 <NA> 
##  473  104  106 
## [1] "ext.decay"
## 
##    0    1    2 <NA> 
##  497  135   13   38 
## [1] "mycelium"
## 
##    0    1 <NA> 
##  639    6   38 
## [1] "int.discolor"
## 
##    0    1    2 <NA> 
##  581   44   20   38 
## [1] "sclerotia"
## 
##    0    1 <NA> 
##  625   20   38 
## [1] "fruit.pods"
## 
##    0    1    2    3 <NA> 
##  407  130   14   48   84 
## [1] "fruit.spots"
## 
##    0    1    2    4 <NA> 
##  345   75   57  100  106 
## [1] "seed"
## 
##    0    1 <NA> 
##  476  115   92 
## [1] "mold.growth"
## 
##    0    1 <NA> 
##  524   67   92 
## [1] "seed.discolor"
## 
##    0    1 <NA> 
##  513   64  106 
## [1] "seed.size"
## 
##    0    1 <NA> 
##  532   59   92 
## [1] "shriveling"
## 
##    0    1 <NA> 
##  539   38  106 
## [1] "roots"
## 
##    0    1    2 <NA> 
##  551   86   15   31

Some of the categorical predictors have levels that occur very rarely. For example, mycelium has only 6 observations in level 1, and some other variables also have levels with very few observations. These rare levels could create dummy variables with very little variation, which may cause problems when building a predictive model. Several predictors also contain missing values.

install.packages("caret")

## Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.6'
## (as 'lib' is unspecified)

library(caret)

## Loading required package: ggplot2

## 
## Attaching package: 'ggplot2'

## The following object is masked from 'package:e1071':
## 
##     element

## Loading required package: lattice

nzv <- nearZeroVar(Soybean[, -1], saveMetrics = TRUE)

nzv

nzv[nzv$nzv == TRUE, ]

The near-zero variance analysis identified three predictors: leaf.mild, mycelium, and sclerotia. These predictors have highly unbalanced frequencies, so they may provide limited information for a predictive model.

3.2b Roughly 18 % of the data are missing. Are there particular predictors that are more likely to be missing? Is the pattern of missing data related to the classes?

sort(colSums(is.na(Soybean)), decreasing = TRUE)

##            hail           sever        seed.tmt         lodging            germ 
##             121             121             121             121             112 
##       leaf.mild fruiting.bodies     fruit.spots   seed.discolor      shriveling 
##             108             106             106             106             106 
##     leaf.shread            seed     mold.growth       seed.size       leaf.halo 
##             100              92              92              92              84 
##       leaf.marg       leaf.size       leaf.malf      fruit.pods          precip 
##              84              84              84              84              38 
##    stem.cankers   canker.lesion       ext.decay        mycelium    int.discolor 
##              38              38              38              38              38 
##       sclerotia     plant.stand           roots            temp       crop.hist 
##              38              36              31              30              16 
##    plant.growth            stem            date        area.dam           Class 
##              16              16               1               1               0 
##          leaves 
##               0

missing_by_class <- aggregate(
  is.na(Soybean[, -1]),
  by = list(Class = Soybean$Class),
  FUN = sum
)

missing_by_class

Some predictors are more likely to be missing than others. Hail, sever, seed.tmt, and lodging have the most missing values, with 121 each. The missing-by-class results also show that missing values are concentrated in certain disease classes, while other classes have few or no missing values. This suggests that the pattern of missing data is related to the classes.

3.2c Develop a strategy for handling missing data, either by eliminating predictors or imputation.

I would use imputation rather than removing all predictors with missing values because many predictors contain missing data. Since the predictors are categorical, I would replace missing values with the most common category for each predictor. This would allow the observations and predictors to remain in the data for modeling.