USL Project - Clustering
Kamil Laskowski
Introduction
In this project, I aim to analyze two images of the “Pałac Na Wodzie” in Łazienki Królewskie, Warsaw Poland – one taken during the day and the other at night. Using image clustering, I will extract the most prominent colors from both images and verify whether the clustering results align with our expectations. Specifically, I want to check if the clustering will return lighter colors for the daytime image and darker colors for the nighttime image.
To perform the clustering, I will use the CLARA method, as it is well-suited for handling large datasets and provides reliable results for partitioning data. To determine the optimal number of clusters (i.e., the number of distinct colors to extract), I will evaluate the Average Silhouette score, which will help assess the quality of the clustering and guide the decision on how many colors to extract from each image.
Loading libraries
library(jpeg)
library(rasterImage)
library(cluster)
library(ggplot2)
library(gridExtra)
library(mclust)Loading pictures
lazienki_dayimage<-readJPEG("lazienki_dzien.jpg")
lazienki_nightimage<-readJPEG("lazienki_noc.jpg")Images are loaded now to RStudio
plot(1, type="n")
rasterImage(lazienki_dayimage, 0.6, 0.6, 1.4, 1.4)
The daytime image of the “Pałac Na Wodzie” in Łazienki Królewskie, Warsaw Poland features bright, vibrant colors. The sky is light blue, the surrounding trees are a mix of light green and dark green. The palace is in white and beige colours and is the biggest object in the photo. The algorithm might create distinct clusters for the sky, trees, water, and palace, reflecting their unique color properties.
plot(1, type="n")
rasterImage(lazienki_nightimage, 0.6, 0.6, 1.4, 1.4)
The nighttime image of the “Pałac Na Wodzie” in Łazienki Królewskie, Warsaw Poland presents a much darker and moodier atmosphere. The sky is deep blue, almost black. The surrounding trees appear in various shades of dark green, nearly black. The palace is still in white and beige colours, but in much darker shade. The water is in similar colour to the sky. Clustering will likely focus on these darker, cooler tones. The method may group the dark blues and blacks from the sky and trees, while the grayish tones from the palace and water will form separate clusters.
Entry analysis and data frame creation
class(lazienki_dayimage)## [1] "array"class(lazienki_nightimage)## [1] "array"dmday<-dim(lazienki_dayimage)
dmday## [1] 810 1920 3dmnight<-dim(lazienki_nightimage)
dmnight## [1] 662 1000 3Here I checked if the pictures were uploaded correctly - they are in array type, which is a 3 dimensional matrix with two dimensions showing the image’s height and width and third dimension representing the colour channels. Output summary:
The first image has dimensions: 810 (height) x 1920 (width) x 3 (color channels)
The second image has dimensions: 662 (height) x 1000 (width) x 3 (color channels)
Now, when I know that my dataset is uploaded correctly, I am creating two datasets of RGB colors, where: - x and y store pixel positions - r.value, g.value, b.value store red, green and blue colour intensity value
rgbday<-data.frame(x=rep(1:dmday[2],
each=dmday[1]),
y=rep(dmday[1]:1, dmday[2]),
r.value=as.vector(lazienki_dayimage[,,1]),
g.value=as.vector(lazienki_dayimage[,,2]),
b.value=as.vector(lazienki_dayimage[,,3]))
head(rgbday)## x y r.value g.value b.value
## 1 1 810 0.3254902 0.6392157 0.9254902
## 2 1 809 0.2784314 0.5921569 0.8784314
## 3 1 808 0.3372549 0.6431373 0.9333333
## 4 1 807 0.2705882 0.5686275 0.8666667
## 5 1 806 0.3568627 0.6352941 0.9568627
## 6 1 805 0.3764706 0.6274510 0.9725490rgbnight<-data.frame(x=rep(1:dmnight[2],
each=dmnight[1]),
y=rep(dmnight[1]:1, dmnight[2]),
r.value=as.vector(lazienki_nightimage[,,1]),
g.value=as.vector(lazienki_nightimage[,,2]),
b.value=as.vector(lazienki_nightimage[,,3]))
head(rgbnight)## x y r.value g.value b.value
## 1 1 662 0.000000000 0.000000000 0.007843137
## 2 1 661 0.000000000 0.000000000 0.007843137
## 3 1 660 0.000000000 0.000000000 0.007843137
## 4 1 659 0.003921569 0.003921569 0.011764706
## 5 1 658 0.003921569 0.003921569 0.011764706
## 6 1 657 0.007843137 0.007843137 0.015686275I also inspect head of the created datasets in order to check if it was done correctly
par(mfrow = c(1, 2))
plot(y~x, data=rgbday, main="Pałac Na Wodzie - Day",
col=rgb(rgbday[c("r.value", "g.value", "b.value")]), asp=1, pch=".")
plot(y~x, data=rgbnight, main="Pałac Na Wodzie - Night",
col=rgb(rgbnight[c("r.value", "g.value", "b.value")]), asp=1, pch=".")
par(mfrow = c(1, 1))Once again, displaying the pictures, but now from dataset
Clustering with CLARA
Why am I using CLARA? CLARA is chosen for its efficiency and scalability when handling large datasets, such as images. By sampling subsets of the data, it performs clustering efficiently without sacrificing reliability, making it well-suited for image analysis. To determine the optimal number of clusters, we use the Silhouette Index, which takes values ranging from -1 to 1. A higher value indicates better-defined clusters, where data points are well-matched to their own cluster and poorly matched to others. A value close to 1 suggests that the clustering is well-separated, while a value near -1 indicates poor clustering, where points may be assigned to the wrong cluster.
nday<-c()
# number of clusters to consider
for (i in 1:10) {
clday<-clara(rgbday[, c("r.value", "g.value", "b.value")], i)
# saving silhouette to vector
nday[i]<-clday$silinfo$avg.width
}
nnight<-c()
# number of clusters to consider
for (i in 1:10) {
clnight<-clara(rgbnight[, c("r.value", "g.value", "b.value")], i)
# saving silhouette to vector
nnight[i]<-clnight$silinfo$avg.width
}Optimal number of clusters - Day
plot(nday, type='l', main="Optimal number of clusters", xlab="Number of clusters", ylab="Average silhouette", col="blue")
points(nday, pch=21, bg="navyblue")
abline(h=(1:30)*5/100, lty=3, col="grey50")
Optimal number of clusters - Night
plot(nnight, type='l', main="Optimal number of clusters", xlab="Number of clusters", ylab="Average silhouette", col="blue")
points(nnight, pch=21, bg="navyblue")
abline(h=(1:30)*5/100, lty=3, col="grey50")
# Silhouette information, for 3 clusters
claraday3<-clara(rgbday[,3:5], 3)
plot(silhouette(claraday3))
# Silhouette information, for 6 clusters
claranight6<-clara(rgbnight[,3:5], 6)
plot(silhouette(claranight6))
As we can see, the optimal number of clusters for the daytime image is three, while for the nighttime image, it is six. The average Silhouette width for both is approximately 0.7 (0.69 for the day and 0.71 for the night), indicating good clustering quality. These values suggest that the clusters are well-separated and appropriately represent the dominant colours in each image.
coloursday3<-rgb(claraday3$medoids[claraday3$clustering, ])
coloursnight6<-rgb(claranight6$medoids[claranight6$clustering, ])Daytime picture with 3 dominant colours
plot(rgbday$y~rgbday$x, col=coloursday3, pch=".", cex=2, asp=1, main="Pałac Na Wodzie - Day - 3 colours")
Nighttime picture with 6 dominant colours
plot(rgbnight$y~rgbnight$x, col=coloursnight6, pch=".", cex=2, asp=1, main="Pałac Na Wodzie - Night - 6 colours")
As we can see, the daytime picture with only 3 dominant colors is much brighter compared to the nighttime picture with 6 dominant colors. Additionally, the clustering on the nighttime picture appears to capture more details, likely due to the higher number of clusters.
#Day
dominantColoursday <- as.data.frame(table(coloursday3))
dominantColoursday$Percentage <- round((dominantColoursday$Freq / sum(dominantColoursday$Freq))*100,2)
dominantColoursday$colours <- as.character(dominantColoursday$colours)
day_data <- data.frame(
colors = factor(dominantColoursday$colours, levels = dominantColoursday$colours),
distribution = dominantColoursday$Percentage
)
plot_day <- ggplot(day_data, aes(x = "", y = distribution, fill = colors)) +
geom_bar(stat = "identity", width = 1, color = "black") +
geom_text(aes(label = paste0(distribution, "%")),
position = position_stack(vjust = 0.5),
color = "white", size = 4, fontface = "bold") +
coord_polar("y") +
theme_void() +
theme(
plot.title = element_text(size = 16, face = "bold", hjust = 0.5),
legend.position = "bottom",
legend.title = element_blank()
) +
scale_fill_manual(values = dominantColoursday$colours) +
ggtitle("Main Colors - Day")
#Night
dominantColoursnight <- as.data.frame(table(coloursnight6))
dominantColoursnight$Percentage <- round((dominantColoursnight$Freq / sum(dominantColoursnight$Freq))*100,2)
dominantColoursnight$colours <- as.character(dominantColoursnight$colours)
night_data <- data.frame(
colors = factor(dominantColoursnight$colours, levels = dominantColoursnight$colours),
distribution = dominantColoursnight$Percentage
)
plot_night <- ggplot(night_data, aes(x = "", y = distribution, fill = colors)) +
geom_bar(stat = "identity", width = 1, color = "black") +
geom_text(aes(label = paste0(distribution, "%")),
position = position_stack(vjust = 0.5),
color = "white", size = 4, fontface = "bold") +
coord_polar("y") +
theme_void() +
theme(
plot.title = element_text(size = 16, face = "bold", hjust = 0.5),
legend.position = "bottom",
legend.title = element_blank()
) +
scale_fill_manual(values = dominantColoursnight$colours) +
ggtitle("Main Colors - Night")
grid.arrange(plot_day, plot_night, ncol = 2)
Conclusion
As we can see, the color palette of the daytime image differs significantly from that of the nighttime image. The dominant color in both images is the color of the forest - green, although the tones are much darker in the nighttime image. The daytime image still includes some brighter colors, particularly for the water and the palace. In contrast, the nighttime image contains almost no bright colors, which aligns with our expectations. Overall, the clustering results closely match what we anticipated at the start of the project. CLARA performed excellently in identifying the dominant colors in each image.
BONUS 1
Let’s compare how the daytime image would look if we used the same number of clusters (6) as for the nighttime image. By applying 6 clusters to the daytime image, we can examine whether the clustering still captures the expected dominant colors. This comparison will allow us to see how the number of clusters affects the overall color distribution and whether it provides more detailed segmentation of the daytime scene.
# Silhouette information, for 6 clusters
claraday6<-clara(rgbday[,3:5], 6)
plot(silhouette(claraday6))
# Silhouette information, for 6 clusters
plot(silhouette(claranight6))
We can see that the average silhouette width dropped significantly to 0.55 (from 0.69) when we increased the number of clusters to 6. This means that the clustering became less well-defined, with the data points becoming less distinct within their assigned clusters.
coloursday6<-rgb(claraday6$medoids[claraday6$clustering, ])Daytime picture with 6 dominant colours
plot(rgbday$y~rgbday$x, col=coloursday6, pch=".", cex=2, asp=1, main="Pałac Na Wodzie - Day - 6 colours")
Nighttime picture with 6 dominant colours
plot(rgbnight$y~rgbnight$x, col=coloursnight6, pch=".", cex=2, asp=1, main="Pałac Na Wodzie - Night - 6 colours")
As we can see, the daytime picture with only 3 dominant colors is much brighter compared to the nighttime picture with 6 dominant colors. Additionally, the clustering on the nighttime picture appears to capture more details, likely due to the higher number of clusters.
#Day
dominantColoursday <- as.data.frame(table(coloursday6))
dominantColoursday$Percentage <- round((dominantColoursday$Freq / sum(dominantColoursday$Freq))*100,2)
dominantColoursday$colours <- as.character(dominantColoursday$colours)
day_data <- data.frame(
colors = factor(dominantColoursday$colours, levels = dominantColoursday$colours),
distribution = dominantColoursday$Percentage
)
plot_day <- ggplot(day_data, aes(x = "", y = distribution, fill = colors)) +
geom_bar(stat = "identity", width = 1, color = "black") +
geom_text(aes(label = paste0(distribution, "%")),
position = position_stack(vjust = 0.5),
color = "white", size = 4, fontface = "bold") +
coord_polar("y") +
theme_void() +
theme(
plot.title = element_text(size = 16, face = "bold", hjust = 0.5),
legend.position = "bottom",
legend.title = element_blank()
) +
scale_fill_manual(values = dominantColoursday$colours) +
ggtitle("Main Colors - Day")
grid.arrange(plot_day, plot_night, ncol = 2)
As we can see, there are even more bright dominant colors in the daytime image than before. The additional clusters capture more variations in the lighter shades, such as those from the sky, water, and surrounding objects, leading to the appearance of more bright colors. However, as indicated by the drop in the average silhouette width, this increased segmentation may not improve the overall clustering quality, as it reduces the distinctiveness of the color groups
BONUS 2 - NEW METHOD - GAUSSIAN MIXTURE MODEL
Inspiration: https://stackoverflow.com/questions/28923865/clustering-an-image-using-gaussian-mixture-models
To perform the clustering, I will use the Gaussian Mixture Model (GMM), a probabilistic algorithm for clustering that assumes the data is generated from a mixture of multiple Gaussian distributions. For this analysis, I will apply GMM to extract most dominant colours from the images, directly specifying the number of clusters - 3 for both day image night image (computation time constrain). Later on, I will create similar plots as in the first part of this project and I will compare the results.
#new datasets for GMM
rgbdayGMM<-data.frame(x=rep(1:dmday[2],
each=dmday[1]),
y=rep(dmday[1]:1, dmday[2]),
r.value=as.vector(lazienki_dayimage[,,1]),
g.value=as.vector(lazienki_dayimage[,,2]),
b.value=as.vector(lazienki_dayimage[,,3]))
rgbnightGMM<-data.frame(x=rep(1:dmnight[2],
each=dmnight[1]),
y=rep(dmnight[1]:1, dmnight[2]),
r.value=as.vector(lazienki_nightimage[,,1]),
g.value=as.vector(lazienki_nightimage[,,2]),
b.value=as.vector(lazienki_nightimage[,,3]))#GMM with 3 clusters for day and 3 clusters night
GMMday <- Mclust(rgbdayGMM[, c("r.value", "g.value", "b.value")], G = 3)
GMMnight <- Mclust(rgbnightGMM[, c("r.value", "g.value", "b.value")], G = 3)#Assigning GMM classification to rgb dataset
rgbdayGMM$cluster <- factor(GMMday$classification)
rgbnightGMM$cluster <- factor(GMMnight$classification)Daytime picture - GMM clustering results
plot(rgbdayGMM$y ~ rgbdayGMM$x, col=rgb(rgbdayGMM[rgbdayGMM$cluster, c("r.value", "g.value", "b.value")]),
pch=".", cex=2, asp=1, main="Pałac Na Wodzie - Day - GMM clustering")
Nighttime picture - GMM clustering results
plot(rgbnightGMM$y ~ rgbnightGMM$x, col=rgb(rgbnightGMM[rgbnightGMM$cluster, c("r.value", "g.value", "b.value")]),
pch=".", cex=2, asp=1, main="Pałac Na Wodzie - Night - GMM clustering")
#Creating a list of assigned colours to clusters for Daytime picture
coloursGMMday<-rgb(rgbdayGMM[rgbdayGMM$cluster, c("r.value", "g.value", "b.value")])
colourcountsDay <- table(coloursGMMday)
top3countsDayGMM <- sort(colourcountsDay, decreasing = TRUE)[1:3]dominantColoursdayGMM <- as.data.frame(top3countsDayGMM)
dominantColoursdayGMM$Percentage <- round((dominantColoursdayGMM$Freq / sum(dominantColoursdayGMM$Freq)) * 100, 2)
colors_dayGMM <- as.character(dominantColoursdayGMM$coloursGMMday)
day_dataGMM <- data.frame(
colors = factor(colors_dayGMM, levels = colors_dayGMM),
distribution = dominantColoursdayGMM$Percentage
)
plot_dayGMM <- ggplot(day_dataGMM, aes(x = "", y = distribution, fill = colors)) +
geom_bar(stat = "identity", width = 1, color = "black") +
geom_text(aes(label = paste0(distribution, "%")),
position = position_stack(vjust = 0.5),
color = "white", size = 4, fontface = "bold") +
coord_polar("y") +
theme_void() +
theme(
plot.title = element_text(size = 16, face = "bold", hjust = 0.5),
legend.position = "bottom",
legend.title = element_blank()
) +
scale_fill_manual(values = colors_dayGMM) +
ggtitle("Main Colors - Day")#Creating a list of assigned colours to clusters for Nighttime picture
coloursGMMnight<-rgb(rgbnightGMM[rgbnightGMM$cluster, c("r.value", "g.value", "b.value")])
colourcountsNight <- table(coloursGMMnight)
#only one colour found #000002
top1countsNightGMM <- sort(colourcountsNight, decreasing = TRUE)[1]dominantColoursnightGMM <- top1countsNightGMM
dominantColoursnightGMM$Percentage <- 100## Warning in dominantColoursnightGMM$Percentage <- 100: Przekształcenie wyrażenia
## po lewej strony w listęcolors_nightGMM <- "#000002"
night_dataGMM <- data.frame(
colors = factor(colors_nightGMM, levels = colors_nightGMM),
distribution = dominantColoursnightGMM$Percentage
)
plot_nightGMM <- ggplot(night_dataGMM, aes(x = "", y = distribution, fill = colors)) +
geom_bar(stat = "identity", width = 1, color = "black") +
geom_text(aes(label = paste0(distribution, "%")),
position = position_stack(vjust = 0.5),
color = "white", size = 4, fontface = "bold") +
coord_polar("y") +
theme_void() +
theme(
plot.title = element_text(size = 16, face = "bold", hjust = 0.5),
legend.position = "bottom",
legend.title = element_blank()
) +
scale_fill_manual(values = colors_nightGMM) +
ggtitle("Main Colors - Night")grid.arrange(plot_dayGMM, plot_nightGMM, ncol = 2)
Upon analyzing the results, it’s clear that the Gaussian Mixture Model (GMM) did not perform weel in clustering the images. For the daytime image, the primary color identified is blue, which is inaccurate when compared to the original image. In the case of the nighttime image, the clustering results show only black, which is somewhat accurate, but it lacks the diversity of other colors present in the scene. Initially, I thought I might be making a mistake, as the assigned colors didn’t seem to match the original images. However, after spending a lot of hours thoroughly checking and verifying the process, I think that the issue lies with the method itself. It simply doesn’t seem to work well for this particular case.
CLARA clustering method was able to capture a wider range of dominant colors, providing better results.