Supervised and Unsupervised Statistical Learning

• In empirical sciences we try to learn from data
• Statistics provides tools for doing so
• We either know the type (eg. healthy/diseased) of our samples and want to describe the types
  • supervised methods
• Or, we do not know the types of our samples and want to discover or predict
  • unsupervised methods

• Define Multivariate Analysis
• Recognize the value of Principal Components Analysis and Multidimensional Scaling
• Be introduced to Linear Discriminant Analysis and Logistic Regression
• Examples of how the free statistical software R might help
• Do check this software out, and try the code enclosed

• Two quotations in The Elements of Statistical Learning
  • “We are drowning in information and starving for knowledge” (Rutherford D. Roger)
  • “The quiet statisticians have changed our world;not by discovering new facts or technical developments, but by changing the ways that we reason, experiment and form our opinions” (Ian Hacking)

• MVA: observation and analysis of more than one outcome variable
• Enables us to work with dependencies between variables
• Distinguish from multivariable, as in multivariable regression
  • one outcome, several explanatory variables

• This famous iris data set gives the measurements in centimeters of the variables sepal length (SL) and width (SW) and petal length (PL) and width (PW), respectively, for 50 flowers from each of 3 species of iris. The species are Iris setosa, versicolor, and virginica.
• The first few lines look like the following

   SL  SW  PL  PW Species
1 5.1 3.5 1.4 0.2  setosa
2 4.9 3.0 1.4 0.2  setosa
3 4.7 3.2 1.3 0.2  setosa
4 4.6 3.1 1.5 0.2  setosa
5 5.0 3.6 1.4 0.2  setosa
6 5.4 3.9 1.7 0.4  setosa

• R is a free software environment for statistical computing and graphics.
• A Graphical User Interface called Rstudio makes it easier (but not always easy) to use
• Suggestion:
  • Install R
  • Install the lastest version of RStudio

### load packages with data and tools that we need ###
library("SAGx")
library("multtest")
data(golub)
dim(golub) # 3051 rows (probes) and 38 columns (samples)

[1] 3051   38

head(golub[, 1:4], n = 5) ### show four columns and five rows ###

         [,1]     [,2]     [,3]     [,4]
[1,] -1.45769 -1.39420 -1.42779 -1.40715
[2,] -0.75161 -1.26278 -0.09052 -0.99596
[3,]  0.45695 -0.09654  0.90325 -0.07194
[4,]  3.13533  0.21415  2.08754  2.23467
[5,]  2.76569 -1.27045  1.60433  1.53182

table(golub.cl)  ### the vector goub.cl gives the class of samples ###

golub.cl
 0  1 
27 11 

• There are 3051 probes (variables)
• The 38 samples fall into two groups of sizes 27 (ALL) and 11 (AML)
• The article found a way to discriminate between ALL and AML
• But the dataset is too large to be analysed manually. What can we do ?

• Show the data in two dimensions with the samples colour coded: ALL = yellow, AML = blue
• The original data has dimension 3051 (= number of probes)
• How do we reduce the number of dimensione to two ?
• We could select two probes, skip the rest, and plot the data
• But that would waste a lot of information…

• The basic idea is to combine the given dimensions (3051) into new ones such that the first new one picks up as much information as possible
• The second new dimension will then optimaly capture the remaining information, and at the same time be independent (orthogonal to) of the first
• And so forth to define principal components PC1, PC2, etc.

• Actually, there cannot be more than \( min(3051, 38) = 38 \) independent dimensions in the data (This follows from Linear Algebra Theory, beyond the scope of this presentation)
• Dimensions where there is no variation provide no information
• Dimensions that change a lot provide information
• The first principal component (PC) is found by drawing a line in space in direction of the largest variation

• PCA does not make use of class information
  • It is unsupervised
  • It allows us to discover patterns in data
  • If we want to model differences between healthy and diseased we need a supervised method

• Turn the dataset so that variables are in columns
• Subtract mean and divide by standard deviation to make variables comparable
  • Centre and scale
  • For instance the measurement units (Fahrenheit or Celsius) should not change conclusions

Display summary information about the first three PCs

round(summary(pca.golub)$importance[,1:3], digits = 3)

                          PC1    PC2    PC3
Standard deviation     21.796 16.933 14.283
Proportion of Variance  0.156  0.094  0.067
Cumulative Proportion   0.156  0.250  0.317

• 1. Why MVA ?
  • a) Modern high-throughput technologies require special statistical methods
  • b) The signal-to-noise ratio increases after reduction of dimensions
  • c) Correlated data can only be assessed through MVA
• 2. Which is true regarding PCA
  • a) PCA requires normal distribution
  • b) PCA tend to keep the gross features of data
  • c) PCA is a method for testing hypotheses

• 1. What is the percentage of variance explained in Fig. 1 ?
  • a) 15%
  • b) 25%
  • c) 40%
• 2. Which is true regarding PCA
  • a) PCA assumes that variation equals information
  • b) PCA can find as many dimensions as you like
  • c) The first dimension is the most important

The mapping of observarions to the plane could use

• Correlations, or other forms of measures of similarity
• Dissimilarities. But the two are related
  • Example : Dissimilarity = 1- Similarity

• Distances

### Road distances in Europe (km) ###
### Create a distance matrix from a distance object ###
distances <- as.matrix(eurodist)
### Show the first four rows and columns ###
head(distances[,1:4], n = 4)

          Athens Barcelona Brussels Calais
Athens         0      3313     2963   3175
Barcelona   3313         0     1318   1326
Brussels    2963      1318        0    204
Calais      3175      1326      204      0

A matrix is set of numbers laid out in rows and columns

We may visualise the eurodist data with MDS

The R function cmdscale performs MDS

euro.mds <- cmdscale(eurodist)
### Use the first two dimensions ###
Dim1 <- euro.mds [,1]
Dim2 <- euro.mds [,2]

• Take two DNA sequences:
  • ATGCGTTAAATTGGCG CCGAATAT
  • ATG TTTCGATAGGCGTCTGAATAT

• Supervised method that finds dimensions that separated given groups (e.g. Diseased - Healthy) as well as possble
• Works best for normal distribution
• This gives a model to classify new observations
• Diagnostic tool
• Predicts the a new object as belonging to the class which is closest in terms of a weighted distance (Mahalanobis)

• Benchmark classifier
• Related to Neural Networks

• Risk of overfitting
  • Results may fit our dataset quite well
  • But our data may not be representative
• Model has to be validated in external data (new cohort, study, experiment)

• 1. What is similar between PCA and LDA ?
  • a) Both assume normal distribution
  • b) Both may reduce the number of dimensions
  • c) Neither
• 2. What is fundamentally different in terms of purpose ?
  • a) PCA does not assume any particular distribution
  • b) LDA assumes normal distribution
  • c) LDA assumes knowledge of the category of each observation

• 3. What is the difference between LDA and LR
  • a) LDA assumes normal distribution
  • b) LR models probability while LDA uses distance
  • c) LR assumes two groups

• R
  • Bioconductor
• SPSS
• SAS
• Excel (some)
• Matlab
• Qlucore Omics Explorer

• Methods fall into supervised and unsupervised
• PCA offers a model free overview using correlation (unsupervised)
• MDS translates general distances into a map (unsupervised)
• LDA build a predictor using known class label, assuming normal distribution , and classifies by the distance to class centroids (supervised)
• Logistic regression models the probability of belonging to one group rather than another by assuming a linear relationship between logodds and predictive variables (supervised)

• Talk to a Statistician before you start your research
• See how others in your field have presented data
  • Put emphasis on high impact publications when you look for inspiration

• Distance. Euclidean distance from \( (0,0) \) to \( (x, y) \) is \( \sqrt{x^{2}+y^{2}} \).

• How to install Bioconductor
  • Follow instructions here
  • To install an individual package use the function biocLite in the BiocInstaller package. E.g. biocLite(“SAGx”)