Homework 4

3.1) The UC Irvine Machine Learning Repository contains a data set related to glass identification. The data consist of 214 glass samples labeled as one of seven class categories. There are nine predictors, including the refractive index and percentages of eight elements: Na, Mg, Al, Si, K, Ca, Ba, and Fe.

1. Using visualizations, explore the predictor variables to understand their distributions as well as the relationships between predictors.

2. Do there appear to be any outliers in the data? Are any predictors skewed?

According to the Boxplots, we can see outliers in the following predictor variables: RI, Na, Al, Si, K, Ca, Ba, and Fe. Some have many outliers, while others have a few. According to the Density Plots, we can see Mg is left-skewed. Ri and Ca are slightly right-skewed. K, Ba, and Fe are highly right-skewed. Na, Al, and Si are negligible skewed.

##         RI         Na         Mg         Al         Si          K         Ca 
##  1.6027151  0.4478343 -1.1364523  0.8946104 -0.7202392  6.4600889  2.0184463 
##         Ba         Fe 
##  3.3686800  1.7298107

3. Are there any relevant transformations of one or more predictors that might improve the classification model?

From the BoxCosTrans function, we can determine log transformation could be applicable to Mg, K, Ba, and Fe. Ri can be inverse squared, Ai can be square rooted, and Si can be squared.

## Loading required package: lattice

## 
## Attaching package: 'caret'

## The following object is masked from 'package:purrr':
## 
##     lift

## Warning: `funs()` was deprecated in dplyr 0.8.0.
## ℹ Please use a list of either functions or lambdas:
## 
## # Simple named list: list(mean = mean, median = median)
## 
## # Auto named with `tibble::lst()`: tibble::lst(mean, median)
## 
## # Using lambdas list(~ mean(., trim = .2), ~ median(., na.rm = TRUE))
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

##   RI   Na Mg  Al Si  K   Ca Ba Fe
## 1 -2 -0.1 NA 0.5  2 NA -1.1 NA NA

3.2) The soybean data can also be found at the UC Irvine Machine Learning Repository. Data were collected to predict disease in 683 soybeans. The 35 predictors are mostly categorical and include information on the environmental conditions and plant conditions. The outcome labels consist of 19 distinct classes.

1. Investigate the frequency distributions for the categorical predictors. Are any of the distributions degenerate in the ways discussed earlier in this chapter?

mycelium and sclerotia show strong signs of degenerate properties. leaf.malf, lodging, leaf.mild, and shriveling show some degenerate properties.

## Warning: `aes_string()` was deprecated in ggplot2 3.0.0.
## ℹ Please use tidy evaluation idioms with `aes()`.
## ℹ See also `vignette("ggplot2-in-packages")` for more information.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

## [[1]]

## 
## [[2]]

## 
## [[3]]

## 
## [[4]]

## 
## [[5]]

## 
## [[6]]

## 
## [[7]]

## 
## [[8]]

## 
## [[9]]

## 
## [[10]]

## 
## [[11]]

## 
## [[12]]

## 
## [[13]]

## 
## [[14]]

## 
## [[15]]

## 
## [[16]]

## 
## [[17]]

## 
## [[18]]

## 
## [[19]]

## 
## [[20]]

## 
## [[21]]

## 
## [[22]]

## 
## [[23]]

## 
## [[24]]

## 
## [[25]]

## 
## [[26]]

## 
## [[27]]

## 
## [[28]]

## 
## [[29]]

## 
## [[30]]

## 
## [[31]]

## 
## [[32]]

## 
## [[33]]

## 
## [[34]]

## 
## [[35]]

## 
## [[36]]

2. Roughly 18% of the data are missing. Are there particular predictors that are more likely to be missing? Is the pattern of missing data related to the classes?

It seems sever, seed.tmt, lodging, and hail are more likely to be missing.

## Warning: Returning more (or less) than 1 row per `summarise()` group was deprecated in
## dplyr 1.1.0.
## ℹ Please use `reframe()` instead.
## ℹ When switching from `summarise()` to `reframe()`, remember that `reframe()`
##   always returns an ungrouped data frame and adjust accordingly.
## ℹ The deprecated feature was likely used in the dplyr package.
##   Please report the issue at <https://github.com/tidyverse/dplyr/issues>.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

We can see that only the phytophthora-rot, diaporthe-pod-&-stem-blight, cyst-nematode, 2-4-d-injury, and herbicide-injury classes have missing data.

## # A tibble: 5 × 2
##   Class                       miss_diff
##   <fct>                           <dbl>
## 1 phytophthora-rot                0.773
## 2 diaporthe-pod-&-stem-blight     1    
## 3 cyst-nematode                   1    
## 4 2-4-d-injury                    1    
## 5 herbicide-injury                1

3. Develop a strategy for handling missing data, either by eliminating predictors or imputation.

Seeing as the classes derived from part b are the classes causing significant impact from missing data, removing these classes will be best to clean the data.