Introduction

Visual features such as colour play a important role in understanding fruit freshness. This project explores the use of unsupervised clustering techniques to analyse colour patterns in apples at two stages of rotenness: fresh and rotten. By applying image clustering techniques, we aim to segment the apples into colour based groups and observe how the colour distribution changes as the fruit rots.

This study uses Clustering Large Applications (CLARA), a k-medoids based clustering algorithm, to group pixels in images of fresh and rotten apples based on their RGB colour values. These insights may serve as a foundation for automated fruit quality assessment systems in the future.

Dataset

For this analysis, I used Apple Photo, which I divided into two separate images for further analysis. By working with these images, I can demonstrate how image clustering can be applied to categorise fruits based on visual features like colour and texture.

Analysis

Uploading useful packages.

library(jpeg)
library(cluster)

Loading apples images.

apple1 <- readJPEG("apple1.jpg")
apple2 <- readJPEG("apple2.jpg")

class(apple1)
## [1] "array"
class(apple2)
## [1] "array"

Showing images on plots.

## Ładowanie wymaganego pakietu: plotrix
plot(1, type = "n")
rasterImage(apple1, 0.6, 0.6, 1.4, 1.4)

plot(1, type = "n")
rasterImage(apple2, 0.6, 0.6, 1.4, 1.4)

Preparing images

Checking dimensions of the images.

dm_apple1 <- dim(apple1) 
dm_apple1
## [1] 706 725   3
dm_apple2 <- dim(apple2) 
dm_apple2
## [1] 728 763   3

RGB format

Changing the format of the images (apple1, apple2) to RGB and representing it in the table.

rgb_apple1 <- data.frame(x = rep(1:dm_apple1[2], each = dm_apple1[1]),  y = rep(dm_apple1[1]:1, dm_apple1[2]), r.value = as.vector(apple1[,,1]),  g.value = as.vector(apple1[,,2]), b.value = as.vector(apple1[,,3]))
head(rgb_apple1)
##   x   y   r.value   g.value   b.value
## 1 1 706 0.9960784 0.9764706 0.9529412
## 2 1 705 0.9960784 0.9764706 0.9529412
## 3 1 704 0.9960784 0.9764706 0.9529412
## 4 1 703 0.9960784 0.9764706 0.9529412
## 5 1 702 0.9960784 0.9764706 0.9529412
## 6 1 701 0.9960784 0.9764706 0.9529412
dim(rgb_apple1)
## [1] 511850      5
rgb_apple2 <- data.frame(x = rep(1:dm_apple2[2], each = dm_apple2[1]),  y = rep(dm_apple2[1]:1, dm_apple2[2]), r.value = as.vector(apple2[,,1]),  g.value = as.vector(apple2[,,2]), b.value = as.vector(apple2[,,3]))
head(rgb_apple2)
##   x   y   r.value   g.value   b.value
## 1 1 728 0.9529412 0.7960784 0.6588235
## 2 1 727 0.9529412 0.7960784 0.6588235
## 3 1 726 0.9529412 0.7960784 0.6588235
## 4 1 725 0.9568627 0.8000000 0.6627451
## 5 1 724 0.9529412 0.8039216 0.6627451
## 6 1 723 0.9568627 0.8078431 0.6666667
dim(rgb_apple2)
## [1] 555464      5

Plotting images

plot(y ~ x, data = rgb_apple1, main = "Apple", col = rgb(rgb_apple1[c("r.value", "g.value", "b.value")]), asp = 1, pch = ".")

plot(y ~ x, data = rgb_apple2, main = "Rotten apple", col = rgb(rgb_apple2[c("r.value", "g.value", "b.value")]), asp = 1, pch = ".")

CLARA

Clustering Large Applications (CLARA) is a clustering algorithm based on k-medoids, designed to handle large datasets efficiently. It works by drawing multiple samples from the dataset, applying the Partitioning Around Medoids (PAM) algorithm to each sample, and selecting the best clustering result. I have decided to use this algorithm here, because it is efficient and well-suited for handling pixel based clustering in images.

Searching for optimal clusters

## Warning: pakiet 'cluster' został zbudowany w wersji R 4.4.2
n1<-c()

for (i in 1:10) {
  cl_apple1 <- clara(rgb_apple1[, c("r.value", "g.value", "b.value")], i)
  n1[i] <- cl_apple1$silinfo$avg.width
}

plot(n1, type = 'l', main = "Optimal number of clusters", xlab = "Number of clusters", ylab = "Average silhouette", col = "darkseagreen3")
points(n1, pch = 21, bg = "darkseagreen4")
abline(h = (1:30)*5/100, lty = 3, col = "grey50")
abline(v = (1:10), lty = 3, col = "grey50")

The optimal number of clusters for fresh apple is 2, because it has the highest average silhouette.

n2 <- c() 

for (i in 1:10) {
  cl_apple2 <- clara(rgb_apple2[, c("r.value", "g.value", "b.value")], i)
  n2[i] <- cl_apple2$silinfo$avg.width
}

plot(n2, type = 'l', main = "Optimal number of clusters", xlab = "Number of clusters", ylab = "Average silhouette", col = "darkseagreen3")
points(n2, pch = 21, bg = "darkseagreen4")
abline(h = (1:30)*5/100, lty = 3, col = "grey50")
abline(v = (1:10), lty = 3, col = "grey50")

The optimal number of clusters for rotten apple is 4, because it has the highest average silhouette.

Silhouette plot

clara_apple1 <- clara(rgb_apple1[, 3:5], 2) 
plot(silhouette(clara_apple1), col = "darkseagreen2")

clara_apple2 <- clara(rgb_apple2[, 3:5], 4) 
plot(silhouette(clara_apple2), col = "darkseagreen2")

colours_apple1 <- rgb(clara_apple1$medoids[clara_apple1$clustering, ])
colours_apple2 <- rgb(clara_apple2$medoids[clara_apple2$clustering, ])

Plotting the images based on the chosen number of clusters.

plot(rgb_apple1$y ~ rgb_apple1$x, col = colours_apple1, pch = ".", cex = 2, asp = 1, main = "2 colours")

plot(rgb_apple2$y ~ rgb_apple2$x, col = colours_apple2, pch = ".", cex = 2, asp = 1, main = "4 colours")

Checking the number of original colours on the images.

cols_orginal1 <- rgb(rgb_apple1[, 3:5])
head(cols_orginal1)
## [1] "#FEF9F3" "#FEF9F3" "#FEF9F3" "#FEF9F3" "#FEF9F3" "#FEF9F3"
length(unique(cols_orginal1))
## [1] 97152
cols_orginal2 <- rgb(rgb_apple2[, 3:5])
head(cols_orginal2)
## [1] "#F3CBA8" "#F3CBA8" "#F3CBA8" "#F4CCA9" "#F3CDA9" "#F4CEAA"
length(unique(cols_orginal2))
## [1] 98994

Plot of most used colours

cluster_colours1 <- rgb(clara_apple1$medoids[, 1], clara_apple1$medoids[, 2], clara_apple1$medoids[, 3])
cluster_counts1 <- table(clara_apple1$clustering)

barplot(cluster_counts1, main = "Pixel counts", col = cluster_colours1, xlab = "Cluster", ylab = "Pixel count")

cluster_colours2 <- rgb(clara_apple2$medoids[, 1], clara_apple2$medoids[, 2], clara_apple2$medoids[, 3])
cluster_counts2 <- table(clara_apple2$clustering)

barplot(cluster_counts2, main = "Pixel counts", col = cluster_colours2, xlab = "Cluster", ylab = "Pixel count")

Conclusion

The visual comparison between the fresh and rotten apple reveals some colour differences. While both images display colours similar to orange/red as their primary colour, the rotten apple have more colours. The fresh apple has a pallete of two colours, while the rotten apple is represented by more colours (as rotten piece is showing up). The CLARA clustering algorithm successfully captured these distinctions, pointing the key colour differences between fresh and rotten apple issue.