The purpose of this project is to analyse urban gminas in Poland based on their cultural acitivities and centers of culture. The data was collected from Statistics Poland and regards theatres, musical institutions, museums, libraries, centres of culture, clubs and community centres, cinemas and organisation of mass events. Firstly, dimensions of data will be reduced using the PCA method. Secondly, the the different clustering methods will be used (k means, PAM, hierarchical clustering) and their performance will be compared.
Dataset contains information on cultural centers and cultural events per 1000 people for each urban gmina in Poland in 2020. Data source is various tables from Statistics Poland. Dataset consists of 33 columns:
There are also columns which represent counts of courses organised by entity:
library(readxl)
data <- read_excel("C:/Users/karol/OneDrive/Pulpit/KULTURA GUS CLUSTERING/Actualdata - Copy.xlsx")
Let’s have an overview on what does the data look like:
summary(data)
## Nazwa libraries libraryservicepoints massevents
## Length:302 Min. :0.06903 Min. :0.00000 Min. : 0.000
## Class :character 1st Qu.:0.37213 1st Qu.:0.00000 1st Qu.: 1.359
## Mode :character Median :0.48097 Median :0.00000 Median : 2.595
## Mean :0.53593 Mean :0.01788 Mean : 3.945
## 3rd Qu.:0.59923 3rd Qu.:0.00000 3rd Qu.: 4.575
## Max. :3.38983 Max. :0.58887 Max. :59.100
## artisticgroups circles cinemas cinemascreens
## Min. :0.0000 Min. :0.0000 Min. :0.00000 Min. :0.00000
## 1st Qu.:0.1190 1st Qu.:0.2095 1st Qu.:0.00000 1st Qu.:0.00000
## Median :0.2911 Median :0.4403 Median :0.01896 Median :0.03759
## Mean :0.4232 Mean :0.6405 Mean :0.02640 Mean :0.04639
## 3rd Qu.:0.5863 3rd Qu.:0.8153 3rd Qu.:0.03850 3rd Qu.:0.07446
## Max. :4.3053 Max. :7.7121 Max. :0.30864 Max. :0.30864
## screenings museums choir philharmonic
## Min. : 0.00 Min. :0.00000 Min. :0.000e+00 Min. :0.0000000
## 1st Qu.: 0.00 1st Qu.:0.00000 1st Qu.:0.000e+00 1st Qu.:0.0000000
## Median : 11.51 Median :0.02334 Median :0.000e+00 Median :0.0000000
## Mean : 26.76 Mean :0.03978 Mean :5.813e-05 Mean :0.0004673
## 3rd Qu.: 43.90 3rd Qu.:0.04852 3rd Qu.:0.000e+00 3rd Qu.:0.0000000
## Max. :271.34 Max. :0.66667 Max. :8.698e-03 Max. :0.0163854
## symphony concertschoir concertsphilharmonic
## Min. :0.0000000 Min. :0.000e+00 Min. :0.00000
## 1st Qu.:0.0000000 1st Qu.:0.000e+00 1st Qu.:0.00000
## Median :0.0000000 Median :0.000e+00 Median :0.00000
## Mean :0.0004335 Mean :9.832e-06 Mean :0.02056
## 3rd Qu.:0.0000000 3rd Qu.:0.000e+00 3rd Qu.:0.00000
## Max. :0.0301841 Max. :2.969e-03 Max. :1.02339
## concertssymphony dramatheater entertainingtheater operatheater
## Min. :0.000000 Min. :0.000000 Min. :0.0000000 Min. :0.000e+00
## 1st Qu.:0.000000 1st Qu.:0.000000 1st Qu.:0.0000000 1st Qu.:0.000e+00
## Median :0.000000 Median :0.000000 Median :0.0000000 Median :0.000e+00
## Mean :0.005248 Mean :0.001029 Mean :0.0002388 Mean :8.302e-05
## 3rd Qu.:0.000000 3rd Qu.:0.000000 3rd Qu.:0.0000000 3rd Qu.:0.000e+00
## Max. :0.422530 Max. :0.038655 Max. :0.0301841 Max. :6.501e-03
## puppettheater dramaperf entertainingperf operaperf
## Min. :0.0000000 Min. :0.0000 Min. :0.00000 Min. :0.000000
## 1st Qu.:0.0000000 1st Qu.:0.0000 1st Qu.:0.00000 1st Qu.:0.000000
## Median :0.0000000 Median :0.0000 Median :0.00000 Median :0.000000
## Mean :0.0005064 Mean :0.1138 Mean :0.01186 Mean :0.004113
## 3rd Qu.:0.0000000 3rd Qu.:0.0000 3rd Qu.:0.00000 3rd Qu.:0.000000
## Max. :0.0227118 Max. :4.1747 Max. :1.45524 Max. :0.396542
## pupperperf foreignlanguages plastic instrument
## Min. :0.00000 Min. :0.00000 Min. :0.000000 Min. :0.00000
## 1st Qu.:0.00000 1st Qu.:0.00000 1st Qu.:0.000000 1st Qu.:0.00000
## Median :0.00000 Median :0.00000 Median :0.000000 Median :0.00000
## Mean :0.03703 Mean :0.02243 Mean :0.006837 Mean :0.01964
## 3rd Qu.:0.00000 3rd Qu.:0.00000 3rd Qu.:0.000000 3rd Qu.:0.00000
## Max. :1.36428 Max. :1.78947 Max. :0.323625 Max. :0.50955
## practicalknowledge dancing computer art
## Min. :0.000000 Min. :0.00000 Min. :0.000000 Min. :0.000000
## 1st Qu.:0.000000 1st Qu.:0.00000 1st Qu.:0.000000 1st Qu.:0.000000
## Median :0.000000 Median :0.00000 Median :0.000000 Median :0.000000
## Mean :0.001617 Mean :0.01466 Mean :0.007639 Mean :0.001919
## 3rd Qu.:0.000000 3rd Qu.:0.00000 3rd Qu.:0.000000 3rd Qu.:0.000000
## Max. :0.069396 Max. :0.33501 Max. :0.361446 Max. :0.090827
## otherworkshops centresofculture
## Min. :0.00000 Min. :0.0000
## 1st Qu.:0.00000 1st Qu.:0.3000
## Median :0.00000 Median :0.5000
## Mean :0.00673 Mean :0.7523
## 3rd Qu.:0.00000 3rd Qu.:0.8000
## Max. :0.50744 Max. :5.2000
head(data)
## # A tibble: 6 Ă— 33
## Nazwa libraries libraryservicepoints massevents artisticgroups circles cinemas
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 Alek… 0.510 0 3.31 0.170 1.10 0
## 2 Augu… 0.304 0 5.95 0.440 0.507 0.0338
## 3 Bart… 0.491 0.0892 2.72 0.0892 0 0
## 4 Będz… 0.429 0 0.519 0.197 0.197 0.0179
## 5 Bełc… 0.513 0 2.80 0.568 0.623 0.0367
## 6 Biał… 0.828 0.0180 1.15 0.324 0.324 0.0360
## # ℹ 26 more variables: cinemascreens <dbl>, screenings <dbl>, museums <dbl>,
## # choir <dbl>, philharmonic <dbl>, symphony <dbl>, concertschoir <dbl>,
## # concertsphilharmonic <dbl>, concertssymphony <dbl>, dramatheater <dbl>,
## # entertainingtheater <dbl>, operatheater <dbl>, puppettheater <dbl>,
## # dramaperf <dbl>, entertainingperf <dbl>, operaperf <dbl>, pupperperf <dbl>,
## # foreignlanguages <dbl>, plastic <dbl>, instrument <dbl>,
## # practicalknowledge <dbl>, dancing <dbl>, computer <dbl>, art <dbl>, …
Fortunately there are no missing observations in the dataset:
anyNA(data)
## [1] FALSE
Converting to a data frame & deleting column with names for a furhter analysis:
library(dplyr)
##
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
##
## filter, lag
## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, union
culture <- as.data.frame(data[2:33])
Checking dimensions:
dim(culture)
## [1] 302 32
There are 302 rows and 32 dimensions.
It seems that the data is suitable for PCA dimension reduction technique. There is a high number of dimensions, so reducing it would benefit the performance of the model. All of data is of numeric type. PCA can be sensitive to skewed data and outliers, therefore, these problems will be tackled first.
culture<-as.data.frame(culture)
for(i in 1:32){
hist(culture$i, main=colnames(culture[i]), xlab = colnames(culture[i]))}
The data is right-skewed, so before further analysis, it will be transformed by taking a square root of each observation:
#for(i in 1:ncol(culture)){
# culture[,i]<-culture[,i]+1
#}
culture<-sqrt(culture)
There are a lot of outliers in the data, but upon further investigation it seems that all the values are correct, so they will not be tackled in any way:
vars <- c(colnames(culture))
Outliers <- c()
for(i in vars){
max <- quantile(culture[,i], 0.75) + (IQR(culture[,i]) * 1.5 )
min <- quantile(culture[,i], 0.25) - (IQR(culture[,i]) * 1.5 )
idx <- which(culture[,i] < min | culture[,i] > max)
print(paste(i, length(idx), sep=' ')) # printing variable and number of potential outliers
Outliers <- c(Outliers, idx)
}
## [1] "libraries 13"
## [1] "libraryservicepoints 65"
## [1] "massevents 12"
## [1] "artisticgroups 5"
## [1] "circles 9"
## [1] "cinemas 2"
## [1] "cinemascreens 0"
## [1] "screenings 0"
## [1] "museums 5"
## [1] "choir 5"
## [1] "philharmonic 24"
## [1] "symphony 14"
## [1] "concertschoir 1"
## [1] "concertsphilharmonic 23"
## [1] "concertssymphony 11"
## [1] "dramatheater 36"
## [1] "entertainingtheater 11"
## [1] "operatheater 10"
## [1] "puppettheater 22"
## [1] "dramaperf 40"
## [1] "entertainingperf 12"
## [1] "operaperf 10"
## [1] "pupperperf 22"
## [1] "foreignlanguages 64"
## [1] "plastic 39"
## [1] "instrument 61"
## [1] "practicalknowledge 19"
## [1] "dancing 62"
## [1] "computer 34"
## [1] "art 20"
## [1] "otherworkshops 32"
## [1] "centresofculture 28"
The next step is to visualize the correletion matrix. Some of the features are highly positively correlated. There are no features with a high negative correlation.
library(corrplot)
## corrplot 0.92 loaded
culture_matrix <- data.matrix(culture)
corr <- cor(culture_matrix)
corrplot(corr,method = "number",number.cex = 0.3, tl.cex = 0.4, tl.col="blue")
There are two measures, which can be checked before conducting PCA, namely Kaiser-Meyer-Olkin (KMO) measure and Bartlett’s test. KMO measure equal to 0.68 is an acceptable score, and it suggest that it might be suitable to perform PCA on the data. P-value equal to 0 in the Bartlett’s test supports the use of PCA.
library("psych")
KMO(corr)
## Kaiser-Meyer-Olkin factor adequacy
## Call: KMO(r = corr)
## Overall MSA = 0.68
## MSA for each item =
## libraries libraryservicepoints massevents
## 0.56 0.38 0.75
## artisticgroups circles cinemas
## 0.78 0.77 0.46
## cinemascreens screenings museums
## 0.58 0.65 0.74
## choir philharmonic symphony
## 0.65 0.78 0.66
## concertschoir concertsphilharmonic concertssymphony
## 0.64 0.74 0.57
## dramatheater entertainingtheater operatheater
## 0.75 0.61 0.62
## puppettheater dramaperf entertainingperf
## 0.72 0.75 0.63
## operaperf pupperperf foreignlanguages
## 0.62 0.73 0.79
## plastic instrument practicalknowledge
## 0.77 0.72 0.71
## dancing computer art
## 0.75 0.69 0.59
## otherworkshops centresofculture
## 0.83 0.78
cortest.bartlett(corr, n = 302)
## $chisq
## [1] 6555.311
##
## $p.value
## [1] 0
##
## $df
## [1] 496
For PCA it is important to scale and center the data. Proportion of variance shows variance explained by each principal component. In this case, the first principal component explains almost 20% of the variance, the second 8,9%, etc.
data.pca <- prcomp(culture, center=TRUE, scale=TRUE)
summary(data.pca)
## Importance of components:
## PC1 PC2 PC3 PC4 PC5 PC6 PC7
## Standard deviation 2.5039 1.68927 1.53949 1.51953 1.37533 1.22883 1.12494
## Proportion of Variance 0.1959 0.08918 0.07406 0.07216 0.05911 0.04719 0.03955
## Cumulative Proportion 0.1959 0.28511 0.35917 0.43132 0.49043 0.53762 0.57717
## PC8 PC9 PC10 PC11 PC12 PC13 PC14
## Standard deviation 1.07108 1.04144 1.0322 1.01404 0.95877 0.94762 0.92289
## Proportion of Variance 0.03585 0.03389 0.0333 0.03213 0.02873 0.02806 0.02662
## Cumulative Proportion 0.61302 0.64691 0.6802 0.71234 0.74107 0.76913 0.79575
## PC15 PC16 PC17 PC18 PC19 PC20 PC21
## Standard deviation 0.90318 0.88852 0.79769 0.77303 0.75318 0.7462 0.70867
## Proportion of Variance 0.02549 0.02467 0.01988 0.01867 0.01773 0.0174 0.01569
## Cumulative Proportion 0.82124 0.84591 0.86580 0.88447 0.90220 0.9196 0.93529
## PC22 PC23 PC24 PC25 PC26 PC27 PC28
## Standard deviation 0.69305 0.6325 0.61019 0.49869 0.45281 0.38271 0.3247
## Proportion of Variance 0.01501 0.0125 0.01164 0.00777 0.00641 0.00458 0.0033
## Cumulative Proportion 0.95030 0.9628 0.97444 0.98221 0.98862 0.99319 0.9965
## PC29 PC30 PC31 PC32
## Standard deviation 0.23655 0.17949 0.13472 0.07768
## Proportion of Variance 0.00175 0.00101 0.00057 0.00019
## Cumulative Proportion 0.99824 0.99924 0.99981 1.00000
Eigenvalues that are higher than one represent optimal components. In that case the first 11 components are valuable.
eigen(corr)$values
## [1] 6.26976059 2.85362059 2.37002358 2.30897428 1.89153856 1.51002339
## [7] 1.26549754 1.14721422 1.08460082 1.06546908 1.02828328 0.91923679
## [13] 0.89797499 0.85171929 0.81573444 0.78947423 0.63630533 0.59757019
## [19] 0.56728581 0.55674540 0.50221384 0.48032379 0.40006721 0.37233544
## [25] 0.24869524 0.20503946 0.14647071 0.10544532 0.05595599 0.03221641
## [31] 0.01815005 0.00603415
Another way of choosing the right number of components is by using the scree plot, which shows the percentage of variance explained by each component.
library(factoextra)
## Loading required package: ggplot2
##
## Attaching package: 'ggplot2'
## The following objects are masked from 'package:psych':
##
## %+%, alpha
## Welcome! Want to learn more? See two factoextra-related books at https://goo.gl/ve3WBa
fviz_eig(data.pca, addlabels = TRUE, ncp=15)
It is also possible to choose variables which will explain a constant percentage of variance. In that case it’s possible to explain 70% of variance using the first 11 components, or 80% of variance using 14 components.
plot(summary(data.pca)$importance[3,])
abline(h = 0.8, col = "pink")
abline(h = 0.7, col = "lightblue")
index1 <- which.min(abs(summary(data.pca)$importance[3,] - 0.81))
index2 <- which.min(abs(summary(data.pca)$importance[3,] - 0.71))
text(x = index1, y = 0.80, labels = index1, pos = 4, col = "red")
text(x = index2, y = 0.70, labels = index2, pos = 4, col = "blue")
Plotting principal components allows to see the contribution of each variable to the selected components. The red line on below graphs shows which variable contribute significantly to the component. Each component consists of different variables, for the first component the most valueable variables are: dramaperf, dramatheater, puppetperf, puppettheater, concertsphilharmonic, philharmonic, operatheater, screenings, operaperf, cinemascreens, entertainingtheater, entertainingperf.
PC_plot <- list()
for (i in 1:13) {
PC_plot[[i]] <- fviz_contrib(data.pca, choice = "var", axes = i)
}
PC_plot[[1]]
PC_plot[[2]]
PC_plot[[3]]
PC_plot[[4]]
PC_plot[[5]]
PC_plot[[6]]
PC_plot[[7]]
PC_plot[[8]]
PC_plot[[9]]
PC_plot[[10]]
PC_plot[[11]]
According to the eigenvalues, the most significant are first 11 components. They represent around 70% of the variance in data. These components are going to be transformed into a new data frame in order to proceed with the clustering.
culture_pca <- data.frame(data.pca$x[,1:11])
str(culture_pca)
## 'data.frame': 302 obs. of 11 variables:
## $ PC1 : num -1.174 1.682 -1.529 2.258 -0.457 ...
## $ PC2 : num -1.128 -3.968 0.856 0.491 0.716 ...
## $ PC3 : num 1.422 0.749 1.824 0.182 -1.411 ...
## $ PC4 : num 0.557 -1.964 1.215 3.575 -0.685 ...
## $ PC5 : num -0.255 3.133 -0.561 -0.376 0.669 ...
## $ PC6 : num 0.8902 4.4385 0.0791 -0.5569 -0.8073 ...
## $ PC7 : num 0.2728 1.7085 0.0288 0.2217 -0.0188 ...
## $ PC8 : num -1.114 -3.756 0.239 1.235 -0.442 ...
## $ PC9 : num -0.71 -3.691 -1.597 -0.851 0.14 ...
## $ PC10: num -0.236 -0.205 0.442 -0.969 -0.279 ...
## $ PC11: num -0.00548 0.49311 -0.26344 -1.27586 -0.24991 ...
Based on the Hopkins statistic it is possible to conclude that the dataset has a clustered structure.
library(clustertend)
## Package `clustertend` is deprecated. Use package `hopkins` instead.
library(factoextra)
get_clust_tendency(culture_pca,10, graph=TRUE)
## $hopkins_stat
## [1] 0.8871433
##
## $plot
The data is numerical and continuous, which makes it suitable for k-means clustering. Based on the plot below it is clear that the optimal number of clusters for k-means clustering is 2.
fviz_nbclust(culture_pca, FUNcluster = kmeans, method = "silhouette") + theme_classic()
Alternative method of selecting the number of clusters is the Within-Cluster Sum of Squares. Based on the elbow rule it looks like it would be beneficial to the analysis to compare k-means with 2 and 3 clusters.
fviz_nbclust(culture_pca, FUNcluster = kmeans, method = "wss")
For the k-means with 2 clusters, they overlap with each other. This can be seen on the graph, but also by a silhouette width which is close to 0 for the 1st cluster. This suggests that it may be worth to use a different clustering algorithm, like for example hierarchical clustering. Total within sum of squares value seems to be quite large, as expected it decreases when the number of clusters increase.
## cluster size ave.sil.width
## 1 1 32 0.05
## 2 2 270 0.59
ss_culture_km <- culture_km$tot.withinss
print(ss_culture_km)
## [1] 5309.871
With 3 clusters, the average silhouette width decreases. The clusters overlap with each other.
## cluster size ave.sil.width
## 1 1 33 0.01
## 2 2 161 0.28
## 3 3 108 0.19
ss_culture_km <- culture_km3$tot.withinss
print(ss_culture_km)
## [1] 4701.633
The next method of clustering, which will be applied is PAM (due to the small number of observations it is not worth to apply CLARA). Based on the graphs below, it can be concluded that PAM alogorithm should be compared for a case with 3 and 4 clusters.
b <- fviz_nbclust(culture_pca, FUNcluster = cluster::pam, method = "silhouette") + theme_classic()
plot(b)
fviz_nbclust(culture_pca, FUNcluster = cluster::pam, method = "wss")
For a PAM algorithm with 3 clusters, the average silhouette is equal to 19.3. The clusters overlap with each other.
culture_pam3 <- eclust(culture_pca, k=3, FUNcluster="pam", hc_metric="pearson", graph=FALSE)
e <- fviz_silhouette(culture_pam3)
## cluster size ave.sil.width
## 1 1 107 0.22
## 2 2 166 0.23
## 3 3 29 0.11
f <- fviz_cluster(culture_pam3, data=culture_pca)
gridExtra::grid.arrange(e,f, ncol=2)
For PAM algorithm with 4 clusters, average silhouette value decreased. The silhouette value of cluster 2 is negative, which suggests that some values might have been assigned to the wrong cluster.
culture_pam4 <- eclust(culture_pca, k=4, FUNcluster="pam", hc_metric="pearson", graph=FALSE)
g <- fviz_silhouette(culture_pam4)
## cluster size ave.sil.width
## 1 1 94 0.19
## 2 2 87 -0.07
## 3 3 29 0.11
## 4 4 92 0.23
h <- fviz_cluster(culture_pam4, data=culture_pca)
gridExtra::grid.arrange(g,h, ncol=2)
The rule of elbow was used in order to determine number of clusters for hierarchical clustering algorithm. It can be suspected that the optimal number of clusters is equal to 2 or 3. Average silhouette is larger for 2 clusters.
fviz_nbclust(culture_pca, FUN = hcut, method = "wss")
fviz_nbclust(culture_pca, FUN = hcut, method = "silhouette")
Dendrogram of complete linkage clustering suggests that most of the values are assigned to only one cluster.
library(factoextra)
culture_c <- eclust(culture_pca, k=2, FUNcluster="hclust", hc_metric="euclidean", hc_method="complete")
## Warning: The `<scale>` argument of `guides()` cannot be `FALSE`. Use "none" instead as
## of ggplot2 3.3.4.
## ℹ The deprecated feature was likely used in the factoextra package.
## Please report the issue at <https://github.com/kassambara/factoextra/issues>.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.
plot(culture_c, cex=0.5, hang=-1)
rect.hclust(culture_c, k=3, border='blue')
The average silhouette for this method is high, but almost all of the observations were assigned to the first cluster.
library(cluster)
sil <- silhouette(culture_c$cluster, dist(culture_pca))
a<-fviz_silhouette(sil)
## cluster size ave.sil.width
## 1 1 301 0.71
## 2 2 1 0.00
b <- fviz_cluster(culture_c, data=culture_pca)
gridExtra::grid.arrange(a,b, ncol=2)
table(culture_c$cluster)
##
## 1 2
## 301 1
It’s interesting to see what observation has been assigned to the second cluster.
library(dplyr)
cluster_c <- cutree(culture_c, k = 2)
data<-data %>% mutate(cluster_c = cluster_c)
cluster_2_data <- data %>%
filter(cluster_c == 2) %>%
select(Nazwa, cluster_c)
# Print the filtered data
print(cluster_2_data)
## # A tibble: 1 Ă— 2
## Nazwa cluster_c
## <chr> <int>
## 1 Wrocław (1) 2
Ward’s minimum variance method might produce less overlapping clusters. For 2 clusters, one of them is again much bigger than the other, but the difference isn’t as signifant as in the complete linkage method.
culture_w <- eclust(culture_pca, k=2, FUNcluster="hclust", hc_metric="euclidean", hc_method = "ward.D")
plot(culture_w, cex=0.6, hang=-1, main = "Dendrogram of HAC")
rect.hclust(culture_w, k=2, border='red')
One of the clusters has a negative silhouette value, which means that some of the observations should have been assigned differently.
sil <- silhouette(culture_w$cluster, dist(culture_pca))
a<-fviz_silhouette(sil)
## cluster size ave.sil.width
## 1 1 262 0.58
## 2 2 40 -0.03
b <- fviz_cluster(culture_w, data=culture_pca)
gridExtra::grid.arrange(a,b, ncol=2)
Let’s check the observations that were assigned into a smaller cluster. It seems that these are mostly the bigger cities, including the capitals of voivodeships, which is not surprising, as the cultural life is probably more vivid in those cities.
library(dplyr)
cluster <- cutree(culture_w, k = 2)
mutate(data, cluster = cluster)
## # A tibble: 302 Ă— 35
## Nazwa libraries libraryservicepoints massevents artisticgroups circles
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 Aleksandrów… 0.510 0 3.31 0.170 1.10
## 2 AugustĂłw (1) 0.304 0 5.95 0.440 0.507
## 3 Bartoszyce … 0.491 0.0892 2.72 0.0892 0
## 4 Będzin (1) 0.429 0 0.519 0.197 0.197
## 5 Bełchatów (… 0.513 0 2.80 0.568 0.623
## 6 Biała Podla… 0.828 0.0180 1.15 0.324 0.324
## 7 Białogard (… 0.473 0 6.37 0.215 0.602
## 8 Białystok (… 0.373 0.00339 1.76 0.143 0.255
## 9 Bielawa (1) 0.311 0 7.87 0.621 0.242
## 10 Bielsk Podl… 0.566 0 5.74 0.808 1.21
## # ℹ 292 more rows
## # ℹ 29 more variables: cinemas <dbl>, cinemascreens <dbl>, screenings <dbl>,
## # museums <dbl>, choir <dbl>, philharmonic <dbl>, symphony <dbl>,
## # concertschoir <dbl>, concertsphilharmonic <dbl>, concertssymphony <dbl>,
## # dramatheater <dbl>, entertainingtheater <dbl>, operatheater <dbl>,
## # puppettheater <dbl>, dramaperf <dbl>, entertainingperf <dbl>,
## # operaperf <dbl>, pupperperf <dbl>, foreignlanguages <dbl>, plastic <dbl>, …
data %>% filter(cluster == "2") %>%
select(Nazwa, cluster)
## Warning: Using an external vector in selections was deprecated in tidyselect 1.1.0.
## ℹ Please use `all_of()` or `any_of()` instead.
## # Was:
## data %>% select(cluster)
##
## # Now:
## data %>% select(all_of(cluster))
##
## See <https://tidyselect.r-lib.org/reference/faq-external-vector.html>.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.
## # A tibble: 40 Ă— 2
## Nazwa libraries
## <chr> <dbl>
## 1 Będzin (1) 0.429
## 2 Białystok (1) 0.373
## 3 Bielsko-Biała (1) 0.578
## 4 Bydgoszcz (1) 0.301
## 5 Bytom (1) 0.202
## 6 ChorzĂłw (1) 0.258
## 7 Częstochowa (1) 0.360
## 8 ElblÄ…g (1) 0.450
## 9 Gdańsk (1) 0.366
## 10 Gdynia (1) 0.358
## # ℹ 30 more rows
The last method used in this study is average linkage clustering. It produces very similar results to the complete linkage method.
culture_a <- eclust(culture_pca, k=2, FUNcluster="hclust", hc_metric="euclidean", hc_method="average")
plot(culture_a, cex=0.5, hang=-1)
rect.hclust(culture_a, k=3, border='blue')
sil <- silhouette(culture_a$cluster, dist(culture_pca))
a<-fviz_silhouette(sil)
## cluster size ave.sil.width
## 1 1 301 0.71
## 2 2 1 0.00
b <- fviz_cluster(culture_a, data=culture_pca)
gridExtra::grid.arrange(a,b, ncol=2)
The main goal of this project was to decrease the number of dimensions in the dataset using the PCA method and compare different clustering algorithms on the reduced dataset. The initial dataset was reduced to 11 dimensions out of 32. The optimal clustering method is subjective, however, in terms of average silhouette width it seems that hierarchical clustering with the usage of Ward’s minimum variance method produces the most optimal clusters.
GUS, Bank Danych Lokalnych. Retrieved from https://bdl.stat.gov.pl/bdl/dane/podgrup/temat