DATA_624_Homework_4

Exercise 3.1.

The UC Irvine Machine Learning Repository contains a data set related to glass identification. The data consist of 214 glass samples labeled as one of seven class categories. There are nine predictors, including the refractive index and percentages of eight elements: Na, Mg, Al, Si, K, Ca, Ba, and Fe.

library(mlbench)
data(Glass)
str(Glass)

'data.frame':   214 obs. of  10 variables:
 $ RI  : num  1.52 1.52 1.52 1.52 1.52 ...
 $ Na  : num  13.6 13.9 13.5 13.2 13.3 ...
 $ Mg  : num  4.49 3.6 3.55 3.69 3.62 3.61 3.6 3.61 3.58 3.6 ...
 $ Al  : num  1.1 1.36 1.54 1.29 1.24 1.62 1.14 1.05 1.37 1.36 ...
 $ Si  : num  71.8 72.7 73 72.6 73.1 ...
 $ K   : num  0.06 0.48 0.39 0.57 0.55 0.64 0.58 0.57 0.56 0.57 ...
 $ Ca  : num  8.75 7.83 7.78 8.22 8.07 8.07 8.17 8.24 8.3 8.4 ...
 $ Ba  : num  0 0 0 0 0 0 0 0 0 0 ...
 $ Fe  : num  0 0 0 0 0 0.26 0 0 0 0.11 ...
 $ Type: Factor w/ 6 levels "1","2","3","5",..: 1 1 1 1 1 1 1 1 1 1 ...

(a) Using visualizations, explore the predictor variables to understand their distributions as well as the relationships between predictors.

library(tidyverse)

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.2.0     ✔ readr     2.2.0
✔ forcats   1.0.1     ✔ stringr   1.6.0
✔ ggplot2   4.0.2     ✔ tibble    3.3.1
✔ lubridate 1.9.5     ✔ tidyr     1.3.2
✔ purrr     1.2.1     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

glass_long <- Glass |>
  pivot_longer(cols = -Type, names_to = "predictor", values_to = "value")

ggplot(glass_long, aes(x = value)) +
  geom_histogram(bins = 30, fill = "steelblue", color = "black") +
  facet_wrap(~ predictor, scales = "free") +
  labs(title = "Glass Predictor Variables")

Looking at the histograms: Al looks slightly skewed right, Ba looks to be mostly absent from the samples with few having small amounts, Ca is skewed right, Fe is also absent from most samples, K is skewed right with the majority around 1, Mg is bimodal with a significant amount having 0, Na is skewed right centering around 13, RI is skewed right centering around 1.517, Si is skewed left centering around 73.

library(corrplot)

corrplot 0.95 loaded

cor_matrix <- cor(Glass[, 1:9])
corrplot(cor_matrix, method = "color")

Looking at the correlation plots, there seems to be a strong positive correlation between RI and Ca, a positive correlation between Al and Ba, and a positive correlation between Na and Ba. There is a sizable negative correlation between RI and Si, a negative correlation between RI and Al, a negative correlation between Mg and Ca, a negative correlation between Mg and Ba, a negative correlation between Mg and Al. Everything else looks to have weak correlation.

(b) Do there appear to be any outliers in the data? Are any predictors skewed?

ggplot(glass_long, aes(x = predictor, y = value)) +
  geom_boxplot(fill = "steelblue", outlier.color = "red") +
  facet_wrap(~ predictor, scales = "free_y") +
  labs(title = "Glass Predictor Variables")

The boxplots show outliers for almost all of the plots. There are positive and negative outliers for Al, Ca, Na, RI, and Si. There are positive outliers for Ba, K, and Fe. Mg is the only boxplot that doesn’t have any outliers. Al, Ba, Ca, Fe, K, Na, RI are skewed right, whereas Si is slightly skewed left. Mg is bimodal with a peak at 0 and another around 3.5.

(c) Are there any relevant transformations of one or more predictors that might improve the classification model?

We could possibly use a log transformation for Ba, Fe, and maybe K since they’re mostly zero and heavily skewed right in order to improve the model. We could also look into using a Box-Cox transformation although it would require adding a constant since Box-Cox requires positive values.

Exercise 3.2

The soybean data can also be found at the UC Irvine Machine Learning Repository. Data were collected to predict disease in 683 soybeans. The 35 predictors are mostly categorical and include information on the environmental conditions (e.g., temperature, precipitation) and plant conditions (e.g., left spots, mold growth). The outcome labels consist of 19 distinct classes.

library(mlbench)
data(Soybean)
str(Soybean)

'data.frame':   683 obs. of  36 variables:
 $ Class          : Factor w/ 19 levels "2-4-d-injury",..: 11 11 11 11 11 11 11 11 11 11 ...
 $ date           : Factor w/ 7 levels "0","1","2","3",..: 7 5 4 4 7 6 6 5 7 5 ...
 $ plant.stand    : Ord.factor w/ 2 levels "0"<"1": 1 1 1 1 1 1 1 1 1 1 ...
 $ precip         : Ord.factor w/ 3 levels "0"<"1"<"2": 3 3 3 3 3 3 3 3 3 3 ...
 $ temp           : Ord.factor w/ 3 levels "0"<"1"<"2": 2 2 2 2 2 2 2 2 2 2 ...
 $ hail           : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 2 1 1 ...
 $ crop.hist      : Factor w/ 4 levels "0","1","2","3": 2 3 2 2 3 4 3 2 4 3 ...
 $ area.dam       : Factor w/ 4 levels "0","1","2","3": 2 1 1 1 1 1 1 1 1 1 ...
 $ sever          : Factor w/ 3 levels "0","1","2": 2 3 3 3 2 2 2 2 2 3 ...
 $ seed.tmt       : Factor w/ 3 levels "0","1","2": 1 2 2 1 1 1 2 1 2 1 ...
 $ germ           : Ord.factor w/ 3 levels "0"<"1"<"2": 1 2 3 2 3 2 1 3 2 3 ...
 $ plant.growth   : Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 2 ...
 $ leaves         : Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 2 ...
 $ leaf.halo      : Factor w/ 3 levels "0","1","2": 1 1 1 1 1 1 1 1 1 1 ...
 $ leaf.marg      : Factor w/ 3 levels "0","1","2": 3 3 3 3 3 3 3 3 3 3 ...
 $ leaf.size      : Ord.factor w/ 3 levels "0"<"1"<"2": 3 3 3 3 3 3 3 3 3 3 ...
 $ leaf.shread    : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
 $ leaf.malf      : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
 $ leaf.mild      : Factor w/ 3 levels "0","1","2": 1 1 1 1 1 1 1 1 1 1 ...
 $ stem           : Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 2 ...
 $ lodging        : Factor w/ 2 levels "0","1": 2 1 1 1 1 1 2 1 1 1 ...
 $ stem.cankers   : Factor w/ 4 levels "0","1","2","3": 4 4 4 4 4 4 4 4 4 4 ...
 $ canker.lesion  : Factor w/ 4 levels "0","1","2","3": 2 2 1 1 2 1 2 2 2 2 ...
 $ fruiting.bodies: Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 2 ...
 $ ext.decay      : Factor w/ 3 levels "0","1","2": 2 2 2 2 2 2 2 2 2 2 ...
 $ mycelium       : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
 $ int.discolor   : Factor w/ 3 levels "0","1","2": 1 1 1 1 1 1 1 1 1 1 ...
 $ sclerotia      : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
 $ fruit.pods     : Factor w/ 4 levels "0","1","2","3": 1 1 1 1 1 1 1 1 1 1 ...
 $ fruit.spots    : Factor w/ 4 levels "0","1","2","4": 4 4 4 4 4 4 4 4 4 4 ...
 $ seed           : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
 $ mold.growth    : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
 $ seed.discolor  : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
 $ seed.size      : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
 $ shriveling     : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
 $ roots          : Factor w/ 3 levels "0","1","2": 1 1 1 1 1 1 1 1 1 1 ...

## See ?Soybean for details

(a) Investigate the frequency distributions for the categorical predictors. Are any of the distributions degenerate in the ways discussed earlier in this chapter?

soybean_char <- Soybean |>
   mutate(across(-Class, as.character))

soybean_long <- soybean_char |>
  pivot_longer(cols = -Class, names_to = "predictor", values_to = "value")

ggplot(soybean_long, aes(x = value)) +
  geom_bar(fill = "steelblue", color = "black") +
  facet_wrap(~ predictor, scales = "free") +
  labs(title = "Soybean Predictor Variables")

Looking at the bar charts, there are a few degenerate distributions that are mostly zero like int.discolor, mycelium, and scelotia. There are also a few that are also mostly zero besides the NA values like leaf.malf, leaf.mild, and lodging.

(b) Roughly 18 % of the data are missing. Are there particular predictors that are more likely to be missing? Is the pattern of missing data related to the classes?

missing_by_predictor <- sort(colMeans(is.na(Soybean)), decreasing = TRUE)
missing_by_predictor

           hail           sever        seed.tmt         lodging            germ 
    0.177159590     0.177159590     0.177159590     0.177159590     0.163982430 
      leaf.mild fruiting.bodies     fruit.spots   seed.discolor      shriveling 
    0.158125915     0.155197657     0.155197657     0.155197657     0.155197657 
    leaf.shread            seed     mold.growth       seed.size       leaf.halo 
    0.146412884     0.134699854     0.134699854     0.134699854     0.122986823 
      leaf.marg       leaf.size       leaf.malf      fruit.pods          precip 
    0.122986823     0.122986823     0.122986823     0.122986823     0.055636896 
   stem.cankers   canker.lesion       ext.decay        mycelium    int.discolor 
    0.055636896     0.055636896     0.055636896     0.055636896     0.055636896 
      sclerotia     plant.stand           roots            temp       crop.hist 
    0.055636896     0.052708638     0.045387994     0.043923865     0.023426061 
   plant.growth            stem            date        area.dam           Class 
    0.023426061     0.023426061     0.001464129     0.001464129     0.000000000 
         leaves 
    0.000000000 

Some of the predictors like hail, sever, seed.tmt, lodging are more likely to be missing. There are also a couple predictors that don’t have any missing data like leaves and Class.

missing_by_class <- Soybean |>
  mutate(total_missing = rowSums(is.na(pick(everything())))) |>
  group_by(Class) |>
  summarize(avg_missing_per_row = mean(total_missing)) |>
  arrange(desc(avg_missing_per_row))

missing_by_class

# A tibble: 19 × 2
   Class                       avg_missing_per_row
   <fct>                                     <dbl>
 1 2-4-d-injury                               28.1
 2 cyst-nematode                              24  
 3 herbicide-injury                           20  
 4 phytophthora-rot                           13.8
 5 diaporthe-pod-&-stem-blight                11.8
 6 alternarialeaf-spot                         0  
 7 anthracnose                                 0  
 8 bacterial-blight                            0  
 9 bacterial-pustule                           0  
10 brown-spot                                  0  
11 brown-stem-rot                              0  
12 charcoal-rot                                0  
13 diaporthe-stem-canker                       0  
14 downy-mildew                                0  
15 frog-eye-leaf-spot                          0  
16 phyllosticta-leaf-spot                      0  
17 powdery-mildew                              0  
18 purple-seed-stain                           0  
19 rhizoctonia-root-rot                        0  

The pattern of missing data looks to be related to class since a majority of the missing data is tied to only five classes. These are the 2-4-d-injury, cyst-nematode, herbicide-injury, phytophthora-rot, and diaporthe-pod-&-stem-blight.

(c) Develop a strategy for handling missing data, either by eliminating predictors or imputation.

Since five of the classes have a majority of the missing data, we could consider removing these data points but that can be risky due to loss of the other valuable data. I would instead try imputing the missing data with zero first and see how that goes. After that, I would try another option, which would be to use the KKN algorithm to impute the values.