Singular Value Decomposition (SVD) is a form of matrix analysis that leads to a low-dimensional representation of a high-dimensional matrix. It allows us to easily eliminate the less important parts of generated matrix representation so as to produce an approximate representation. Of course the fewer the dimensions we choose, the less accurate will be the approximation.

Let M be an m × n matrix, and let the rank of M be r. Then we can find matrices U, d, and V with the following properties:

  1. U is an m × r column-orthonormal matrix ; that is, each of its columns is a unit vector and the dot product of any two columns is 0.

  2. V is an n × r column-orthonormal matrix. Note that we always use V in its transposed form, so it is the rows of t(V) that are orthonormal.

  3. d is a diagonal matrix; that is, all elements not on the main diagonal are 0. The elements of d are called the singular values of M.

We will perform SVD of an image and check its outcomes with reduced dimensions.

setwd("/Users/Tanmay/Desktop/R wd")
suppressWarnings(library(raster))
## Loading required package: sp
img<-'bike.png'
rasterimg<-raster(img)
img.flip<-flip(rasterimg, direction = "y")
rasterimg<-t(as.matrix(img.flip))
dim(rasterimg)
## [1] 400 289

The above code basically retrieves the raster layer of the image as the matrix of pixel values. Our original image:

image(rasterimg, col = grey(seq(0, 1, length = 256)))

Let us perform SVD on our image:

svdimg<-svd(rasterimg)
U<-svdimg$u
d<-diag(svdimg$d)
V<-svdimg$v
U[1:5, 1:5]
##             [,1]       [,2]         [,3]       [,4]     [,5]
## [1,] -0.09338453 -0.1359345 -0.001644688 0.02165999 0.017338
## [2,] -0.09338453 -0.1359345 -0.001644688 0.02165999 0.017338
## [3,] -0.09338453 -0.1359345 -0.001644688 0.02165999 0.017338
## [4,] -0.09338453 -0.1359345 -0.001644688 0.02165999 0.017338
## [5,] -0.09338453 -0.1359345 -0.001644688 0.02165999 0.017338
d[1:5, 1:5]
##          [,1]     [,2]     [,3]     [,4]    [,5]
## [1,] 43970.65    0.000    0.000    0.000    0.00
## [2,]     0.00 9969.766    0.000    0.000    0.00
## [3,]     0.00    0.000 6865.042    0.000    0.00
## [4,]     0.00    0.000    0.000 5599.518    0.00
## [5,]     0.00    0.000    0.000    0.000 5481.48
V[1:5, 1:5]
##             [,1]       [,2]          [,3]       [,4]        [,5]
## [1,] -0.07399926 0.04353212 -0.0242489962 0.02242174 -0.01895288
## [2,] -0.07136224 0.03821192  0.0003149156 0.03776942  0.01753529
## [3,] -0.06962591 0.03135863  0.0170534925 0.04514267  0.04393905
## [4,] -0.06862487 0.02537345  0.0266872627 0.04840377  0.05562120
## [5,] -0.06798304 0.02135297  0.0331777535 0.04722144  0.06156191

The best way to reduce the dimensionality of the three matrices is to set the smallest of the singular values to zero. If we set the s smallest singular values to 0, then we can also eliminate the corresponding s columns of U and V.

plot(svdimg$d)

Now, lets check our image by retaining only the 1st singular value:

U1 <- as.matrix(U[, 1])
d1 <- as.matrix(d[1, 1])
V1 <- as.matrix(V[, 1])

img1 <- U1 %*% d1 %*% t(V1)
image(img1, col = grey(seq(0, 1, length = 256)))

Not Cool! Lets check with first 10 singular values:

U10 <- as.matrix(U[, 1:10])
d10 <- as.matrix(d[1:10, 1:10])
V10 <- as.matrix(V[, 1:10])

img10 <- U10 %*% d10 %*% t(V10)
image(img10, col = grey(seq(0, 1, length = 256)))

Now, we can see some pattern of our image. Lets go for first 50 singular values:

U50 <- as.matrix(U[, 1:50])
d50 <- as.matrix(d[1:50, 1:50])
V50 <- as.matrix(V[, 1:50])

img50 <- U50 %*% d50 %*% t(V50)
image(img50, col = grey(seq(0, 1, length = 256)))

There we go. Although we are selecting first 50 singular values out of total 289, we are getting quite good approximation for our image.