Group 2 Members

Danielle Cortez, Daniel Cho, David Tu, Angelica Rivera

Goal of pcasimple

Principal Component Analysis is an extremely useful concept that is covered in STAT 580 that allows users to greatly simplify high-dimensional data without losing much information. However, this concept is complex and hard to learn, especially for students who have never seen it before. This R Package is meant to simplify and streamline the PCA process to assist in the teaching and learning of PCA. The functions of this package translates PCA into an ordered process that will be easier to follow.

pima_diabetes <- pima_diabetes[,-9]

2D Plots

plot2D() produces a 2D scatter plot of the data, which can be customized via its arguments.

The function plot2D() has three arguments:

1. data: a dataframe containing two variables
2. type: a character argument changing the orientation of the plot
   • “original”: the axes of the scatter plot are the original variables
   • “PC”: the axes of the scatter plot are the principal components
3. arrows: a logical argument indicating whether to include the principal components as arrows on the plot

If the requirements of these arguments are not met, then an error message will print to indicate what needs to be changed.

pima_scale <- scale(pima_diabetes)
plot2D(pima_scale)
#> [1] "Data should contain two variables."
#> NULL

pima_subset <- subset(pima_scale, select = c("Glucose", "BloodPressure"))
plot2D(pima_subset)

plot2D(pima_subset, type = "PC")

plot2D(pima_subset, arrows = TRUE)

plot2D(pima_subset, type = "PC", arrows = TRUE)

3D Plots

plot3D() produces a 3D interactive scatter plot of the data using the rgl package, which can be customized via its arguments.

The function plot3D() has three arguments:

1. data: a dataframe containing three variables
2. type: a character argument changing the orientation of the plot
   • “original”: the axes of the scatter plot are the original variables
   • “PC”: the axes of the scatter plot are the principal components
3. arrows: a logical argument indicating whether to include the principal components as arrows on the plot

If the requirements of these arguments are not met, then an error message will print to indicate what needs to be changed.

pima_subset_3 <- subset(pima_scale, select = c("Glucose", "BloodPressure", "BMI"))

plot3D(pima_subset_3)

plot3D(pima_subset_3, type = "PC")

plot3D(pima_subset_3, arrows = TRUE)

plot3D(pima_subset_3, type = "PC", arrows = TRUE)

Limitations of 2D and 3D Plots

Similar Functions from STAT 580

• Base R
  • plot()
  • arrows()
• rgl
  • plot3d()

Data and Scale

The function pcvalues() requires that the data frame is standardized. This function has a scale argument that allows users to standardize their data by setting scale = TRUE, where the default setting is set to scale = FALSE for users who have already standardized their data.

Integer

The digits argument changes the decimal digits of the output values.

Value

The value argument allows users to change the output of values as eigenvectors, loadings, eigenvalues, or accumulated proportion of variance explained of the principal components. A matrix of eigenvectors can be produced with the argument value = “eigenvectors”.

pcvalues(data = pima_diabetes, scale = TRUE, value = "eigenvectors")

prcomp(scale(pima_diabetes))
#> Standard deviations (1, .., p=8):
#> [1] 1.4471973 1.3157546 1.0147068 0.9356971 0.8731234 0.8262133 0.6479322
#> [8] 0.6359733
#> 
#> Rotation (n x k) = (8 x 8):
#>                                 PC1        PC2         PC3         PC4
#> Pregnancies              -0.1284321  0.5937858 -0.01308692  0.08069115
#> Glucose                  -0.3930826  0.1740291  0.46792282 -0.40432871
#> BloodPressure            -0.3600026  0.1838921 -0.53549442  0.05598649
#> SkinThickness            -0.4398243 -0.3319653 -0.23767380  0.03797608
#> Insulin                  -0.4350262 -0.2507811  0.33670893 -0.34994376
#> BMI                      -0.4519413 -0.1009598 -0.36186463  0.05364595
#> DiabetesPedigreeFunction -0.2706114 -0.1220690  0.43318905  0.83368010
#> Age                      -0.1980271  0.6205885  0.07524755  0.07120060
#>                                 PC5          PC6         PC7          PC8
#> Pregnancies              -0.4756057  0.193598168  0.58879003  0.117840984
#> Glucose                   0.4663280  0.094161756  0.06015291  0.450355256
#> BloodPressure             0.3279531 -0.634115895  0.19211793 -0.011295538
#> SkinThickness            -0.4878621  0.009589438 -0.28221253  0.566283799
#> Insulin                  -0.3469348 -0.270650609  0.13200992 -0.548621381
#> BMI                       0.2532038  0.685372179  0.03536644 -0.341517637
#> DiabetesPedigreeFunction  0.1198105 -0.085784088  0.08609107 -0.008258731
#> Age                      -0.1092900 -0.033357170 -0.71208542 -0.211661979

We can observe that the row names are our variables, and the column names are the principal components. The eigenvector values are the orthogonal transformation of the covariance matrix, where it represents the direction or projection of the data to a principal component.

\[Loadings = eigenvector * \sqrt{eigenvalue}\] The loadings values can be obtained by multiplying the eigenvector by the squared root of the eigenvalue of its respective principal component. This process “loads” the eigenvector with magnitude or variance, where the loading value represents the covariability of each variable with the principal component. This process is simplified with the argument value = “loadings”.

pcvalues(data = pima_diabetes, scale = TRUE, value = "loadings")

Eigenvalues are obtained by taking the sum of each squared loadings of their respective component. These values represent the amount of variance that can be explained by each principal component. The eigenvalues can be produced with the argument value = “eigenvalues”.

pcvalues(data = pima_diabetes, scale = TRUE, value = "eigenvalues")

However, these eigenvalues can be more meaningful by getting the ratio of each eigenvalue to the sum of every eigenvalue. This will give us the percentage of variance explained of each principal component in respect to the percentage of total variance explained of every principal component. In addition, the argument value = “variance” will output the cumulative percentage of variance explained.

pcvalues(data = pima_diabetes, scale = TRUE, value = "variance")

Data and Scale

The function screeplot() requires that the data frame is standardized. This function has a scale argument that allows users to standardize their data by setting scale = TRUE, where the default setting is set to scale = FALSE for users who have already standardized their data.

Type

The type argument allows users to change the type of screeplot to display eigenvalues or accumulated proportion of variance explained of the principal components. The user can display eigenvalues by setting type = “eigenvalue” or display the accumulated proportion of variance explained by setting type = “cumulative”. The default setting is set to cumulative since that is more intuitively understood.

screeplot(pima_diabetes, type = "eigenvalue", scale = T)

The eigenvalues shown above represent the total amount of variance that can be explained by its principal component. We can observe that the first three components have the highest eigenvalues, where the third component marks the point of the largest drop. This “elbow” joint typically marks where there will be diminishing returns in selecting more principal components. Let’s take a look at the accumulated proportion of variance explained to see if there is anything different.

screeplot(pima_diabetes, type = "cumulative", scale = T)

We can see that if we selected only the first three components, the accumulated proportion of variance explained would be less than 60%. There are many different opinions on the the criteria of choosing the number of components as it depends on the context of the research. However, this function allows flexibility in setting their thresholds for their own needs with the varexplain argument.

Varexplain

The varexplain argument is supplementary to the type = “cumulative” argument as it allows users to change the threshold of the total variance explained that is shown by a dashed, red line. This will aid in visualizing the number of principal components that are below and above the threshold. The default setting is set to 0.80, and is bounded between 0 and 1.

screeplot(pima_diabetes, varexplain = 0.90, type = "cumulative", scale = T)

After selecting components

Based on the 90 percent threshold, we would select the first six principal components and remove the last two. This process reduces dimensionality by eliminating the data points of the variables that were in the principal components that explained the least amount of variance. It is important to note that we are not eliminating variables as a whole, but extracting the more important parts of the variables into a smaller space or less dimensions as defined in the selection of the number of principal components.

Users can go further by using their principal components as predictors for their linear regression models, also known as principle component regression. However, we will not cover how to perform principle component regression, but this link will provide more information.

Principal Component Analysis is a very useful tool in reducing dimensionality but the topic can get very complex. Our goal for this package was to streamline the process to assist users in gaining a deeper understanding of the topic with simplified functions. This article is a “One-Stop Shop” for PCA that provides additional information on the topic.