PCA-EFA-CFA
Heru Wiryanto
August 4, 2016
Principal Components and Factor Analysis
This section covers principal components and factor analysis. The later includes both exploratory and confirmatory methods.
Principal Components The princomp( ) function produces an unrotated principal component analysis.
## Loading required package: MASS## Loading required package: boot##
## Attaching package: 'boot'## The following object is masked from 'package:psych':
##
## logit## Loading required package: lattice##
## Attaching package: 'lattice'## The following object is masked from 'package:boot':
##
## melanoma##
## Attaching package: 'nFactors'## The following object is masked from 'package:lattice':
##
## parallel## Importance of components:
## Comp.1 Comp.2 Comp.3 Comp.4
## Standard deviation 2.1385578 1.8580281 0.98093470 0.76226030
## Proportion of Variance 0.3811191 0.2876890 0.08018607 0.04842006
## Cumulative Proportion 0.3811191 0.6688082 0.74899422 0.79741429
## Comp.5 Comp.6 Comp.7 Comp.8
## Standard deviation 0.72112170 0.63713131 0.57525023 0.56287676
## Proportion of Variance 0.04333471 0.03382803 0.02757607 0.02640252
## Cumulative Proportion 0.84074900 0.87457702 0.90215309 0.92855561
## Comp.9 Comp.10 Comp.11 Comp.12
## Standard deviation 0.5269887 0.46601099 0.45145129 0.39829766
## Proportion of Variance 0.0231431 0.01809719 0.01698402 0.01322009
## Cumulative Proportion 0.9516987 0.96979589 0.98677991 1.00000000SCREE PLOT DAN BIPLOT
##
## Loadings:
## Comp.1 Comp.2 Comp.3 Comp.4 Comp.5 Comp.6 Comp.7 Comp.8 Comp.9
## ags1 -0.107 -0.397 -0.275 0.345 -0.345 0.658 0.175 -0.167
## ags2 -0.107 -0.357 0.592 -0.114 0.240 -0.531 -0.242 0.295
## ags3 -0.387 0.138 -0.105 -0.145 -0.463 0.572 -0.120
## ags4 -0.369 0.675 0.336 0.248 0.205 0.289
## ags5 -0.400 -0.129 0.651 0.221 -0.488 -0.176 0.169
## ags6 -0.127 -0.393 -0.298 -0.617 -0.187
## ags7 -0.112 -0.440 -0.329 -0.219 -0.121
## ags8 -0.390 0.122 0.109 -0.435 0.327 -0.225 0.295
## ags9 -0.401 -0.277 -0.452 -0.609
## ags10 -0.417 0.105 0.269 -0.288
## ags11 -0.390 0.134 -0.406 -0.228 -0.108 0.188 0.290
## ags12 -0.126 -0.359 0.575 -0.194 0.412 0.337 -0.443
## Comp.10 Comp.11 Comp.12
## ags1 0.135
## ags2
## ags3 0.485
## ags4 -0.239 -0.199
## ags5 0.210
## ags6 0.513 -0.197
## ags7 -0.744 0.235
## ags8 0.125 0.312 -0.524
## ags9 -0.289 -0.278
## ags10 0.173 0.292 0.719
## ags11 -0.251 -0.577 0.282
## ags12
##
## Comp.1 Comp.2 Comp.3 Comp.4 Comp.5 Comp.6 Comp.7 Comp.8
## SS loadings 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000
## Proportion Var 0.083 0.083 0.083 0.083 0.083 0.083 0.083 0.083
## Cumulative Var 0.083 0.167 0.250 0.333 0.417 0.500 0.583 0.667
## Comp.9 Comp.10 Comp.11 Comp.12
## SS loadings 1.000 1.000 1.000 1.000
## Proportion Var 0.083 0.083 0.083 0.083
## Cumulative Var 0.750 0.833 0.917 1.000
PRINCIPAL AXIS FACTOR ANALYSIS
- Varimax Rotated Principal Components
- retaining 5 components
## Warning: In princomp.default(Mydata, nfactors = 3, rotation = "varimax") :
## extra arguments 'nfactors', 'rotation' will be disregarded## Call:
## princomp(x = Mydata, nfactors = 3, rotation = "varimax")
##
## Standard deviations:
## Comp.1 Comp.2 Comp.3 Comp.4 Comp.5 Comp.6 Comp.7
## 3.1726301 2.2499565 1.2607626 0.9999420 0.8912621 0.8310057 0.7975506
## Comp.8 Comp.9 Comp.10 Comp.11 Comp.12
## 0.7108109 0.6779607 0.6324754 0.5788580 0.4832414
##
## 12 variables and 324 observations.mydata can be a raw data matrix or a covariance matrix. Pairwise deletion of missing data is used. rotate can “none”, “varimax”, “quatimax”, “promax”, “oblimin”, “simplimax”, or “cluster” .
Exploratory Factor Analysis
- The factanal( ) function produces maximum likelihood factor analysis.
- Maximum Likelihood Factor Analysis
- entering raw data and extracting 3 factors,
- with varimax rotation
##
## Call:
## factanal(x = Mydata, factors = 3, rotation = "varimax")
##
## Uniquenesses:
## ags1 ags2 ags3 ags4 ags5 ags6 ags7 ags8 ags9 ags10 ags11 ags12
## 0.51 0.41 0.31 0.43 0.54 0.38 0.10 0.28 0.27 0.19 0.29 0.22
##
## Loadings:
## Factor1 Factor2 Factor3
## ags3 0.83
## ags4 0.75
## ags8 0.84
## ags9 0.85
## ags10 0.90
## ags11 0.84
## ags1 0.66
## ags5 0.61
## ags6 0.77
## ags7 0.94
## ags2 0.36 0.68
## ags12 0.35 0.81
##
## Factor1 Factor2 Factor3
## SS loadings 4.20 2.55 1.31
## Proportion Var 0.35 0.21 0.11
## Cumulative Var 0.35 0.56 0.67
##
## Test of the hypothesis that 3 factors are sufficient.
## The chi square statistic is 113.42 on 33 degrees of freedom.
## The p-value is 9.39e-11
- The rotation= options include “varimax”, “promax”, and “none”. Add the option scores=“regression” or “Bartlett” to produce factor scores. Use the covmat= option to enter a correlation or covariance matrix directly. If entering a covariance matrix, include the option n.obs=. The factor.pa( ) function in the psych package offers a number of factor analysis related functions, including principal axis factoring.
Determine Number of Factors to Extract
A crucial decision in exploratory factor analysis is how many factors to extract. The nFactors package offer a suite of functions to aid in this decision. Details on this methodology can be found in a PowerPoint presentation by Raiche, Riopel, and Blais. Of course, any factor solution must be interpretable to be useful. 
Going Further
The FactoMineR package offers a large number of additional functions for exploratory factor analysis. This includes the use of both quantitative and qualitative variables, as well as the inclusion of supplimentary variables and observations. Here is an example of the types of graphs that you can create with this package.
PCA Variable Factor Map 
