Countries vary widely in terms of income, Internet use, education, health, employment characteristics, and access to basic services. While these factors are connected, countries with similar income levels can still look quite different in other ways. By using clustering techniques, I can explore how these factors relate to each other without relying on predefined development categories.
The main question here is whether countries can be grouped based on their Internet use and socioeconomic characteristics in 2023. I also want to see how much the results change when different indicators or clustering methods are used. For this, I compare K-means, Partitioning Around Medoids (PAM), and Hierarchical clustering, looking at how clearly the groups are separated, how similar the results are, and how stable the clusters remain. Another thing worth checking is whether the countries actually form distinct groups or simply fall along a continuous development gradient. Since Internet use is the only direct digital indicator included, the results are meant to describe differences within the sample, not to rank countries officially or suggest that one factor causes another.
For this project, I use 12 different indicators from the World Bank’s World Development Indicators (WDI) database. The data covers the years 2022 to 2024 and includes 217 countries and territories, excluding regional and income-group aggregates. The indicators I look at cover areas like income, Internet use, health, education, jobs, urbanization, infrastructure, trade, investment, and basic services.
The metadata file starts with a duplicate of the data export, followed by a table with details about each indicator. Since some of the metadata entries are messy, I double-check things like the series codes, source names, definitions, and units before deciding which indicators to use. All the numbers I analyze come directly from this data extract.
#load data and locate the separate metadata table
wdi_raw<-read.csv("data/WDI_Data_2022_2024.csv",na.strings=c("..",""),check.names=FALSE)
metadata_lines<-readLines("data/WDI_Series_Metadata.csv",warn=FALSE,encoding="latin1")
wdi_raw<-subset(wdi_raw,!is.na(`Country Code`)&`Country Code`!=""&!is.na(`Series Code`)&`Series Code`!="")
metadata_start<-grep("^Code,License Type,Indicator Name,Short definition,",metadata_lines)
stopifnot(length(metadata_start)==1)
#allow irregular trailing fields while retaining the first metadata fields
wdi_metadata<-read.csv(text=paste(metadata_lines[metadata_start:length(metadata_lines)],collapse="\n"),skip=1,header=FALSE,fill=TRUE,col.names=paste0("V",1:18),stringsAsFactors=FALSE)
The table summarizes the units of 12 candidate indicators. The code also checks that each code matches its official series name and has a nonempty short definition and unit field in the supplied WDI metadata. These are candidate indicators, not the final clustering inputs. In particular, gross secondary enrollment may exceed 100% because it counts students of all ages relative to the official age group.
indicator_labels<-c("NY.GDP.PCAP.PP.KD"="GDP per capita (PPP)","IT.NET.USER.ZS"="Internet users","SP.DYN.LE00.IN"="Life expectancy","SE.SEC.ENRR"="Secondary enrollment",
"SL.EMP.TOTL.SP.ZS"="Employment ratio","SP.URB.TOTL.IN.ZS"="Urban population","EG.ELC.ACCS.ZS"="Electricity access","NE.TRD.GNFS.ZS"="Trade",
"NE.GDI.TOTL.ZS"="Capital formation","SL.EMP.VULN.ZS"="Vulnerable employment","IT.NET.BBND.P2"="Fixed broadband","SH.STA.BASS.ZS"="Basic sanitation")
indicator_units<-c("NY.GDP.PCAP.PP.KD"="Constant 2021 international dollars","IT.NET.USER.ZS"="% of population","SP.DYN.LE00.IN"="Years","SE.SEC.ENRR"="% gross enrollment",
"SL.EMP.TOTL.SP.ZS"="% of population aged 15+","SP.URB.TOTL.IN.ZS"="% of total population","EG.ELC.ACCS.ZS"="% of population","NE.TRD.GNFS.ZS"="% of GDP",
"NE.GDI.TOTL.ZS"="% of GDP","SL.EMP.VULN.ZS"="% of total employment","IT.NET.BBND.P2"="Subscriptions per 100 people","SH.STA.BASS.ZS"="% of population")
indicator_cols<-names(indicator_labels)
meta_check<-wdi_metadata[match(indicator_cols,wdi_metadata$V1),]
source_names<-wdi_raw$`Series Name`[match(indicator_cols,wdi_raw$`Series Code`)]
stopifnot(all(meta_check$V1==indicator_cols),all(meta_check$V3==source_names),all(nzchar(trimws(meta_check$V4))),all(nzchar(trimws(meta_check$V9))))
knitr::kable(data.frame(Indicator=unname(indicator_labels),Code=indicator_cols,Unit=unname(indicator_units)),row.names=FALSE,caption="The twelve candidate WDI indicators")
| Indicator | Code | Unit |
|---|---|---|
| GDP per capita (PPP) | NY.GDP.PCAP.PP.KD | Constant 2021 international dollars |
| Internet users | IT.NET.USER.ZS | % of population |
| Life expectancy | SP.DYN.LE00.IN | Years |
| Secondary enrollment | SE.SEC.ENRR | % gross enrollment |
| Employment ratio | SL.EMP.TOTL.SP.ZS | % of population aged 15+ |
| Urban population | SP.URB.TOTL.IN.ZS | % of total population |
| Electricity access | EG.ELC.ACCS.ZS | % of population |
| Trade | NE.TRD.GNFS.ZS | % of GDP |
| Capital formation | NE.GDI.TOTL.ZS | % of GDP |
| Vulnerable employment | SL.EMP.VULN.ZS | % of total employment |
| Fixed broadband | IT.NET.BBND.P2 | Subscriptions per 100 people |
| Basic sanitation | SH.STA.BASS.ZS | % of population |
The final model uses 1 year rather than mixing observations from different years. The following comparison uses the eight indicators ultimately retained in Section 3.5 to justify the year choice: 2023 preserves slightly more complete economies than 2022 and substantially more than 2024 (this is a data-coverage check, not a time-series analysis).
selected_vars<-c("NY.GDP.PCAP.PP.KD","IT.NET.USER.ZS","SP.DYN.LE00.IN","SE.SEC.ENRR","SL.EMP.TOTL.SP.ZS","SP.URB.TOTL.IN.ZS","SL.EMP.VULN.ZS","SH.STA.BASS.ZS")
coverage_by_year<-do.call(rbind,lapply(2022:2024,function(year){
x<-wdi_raw[c("Country Code","Series Code",paste0(year," [YR",year,"]"))]
names(x)[3]<-"Value"
wide<-reshape(x,idvar="Country Code",timevar="Series Code",direction="wide")
names(wide)<-sub("^Value\\.","",names(wide))
data.frame(Year=year,Complete_economies=sum(complete.cases(wide[selected_vars])))
}))
knitr::kable(coverage_by_year,caption="Complete-case coverage for the selected eight indicators")
| Year | Complete_economies |
|---|---|
| 2022 | 112 |
| 2023 | 113 |
| 2024 | 81 |
Internet use measures recent adoption, not affordability, digital skills, or service quality. So that, I believe the title and conclusions therefore refer specifically to Internet use and socioeconomic development.
The original export has one row for each country-indicator combination. For clustering, each economy needs one row and each indicator its own column. I first extract 2023 values and check that each country-indicator pair occurs only once.
#reshape country-indicator rows into one row per economy
wdi_2023<-wdi_raw[c("Country Name","Country Code","Series Code","2023 [YR2023]")]
names(wdi_2023)[4]<-"Value"
stopifnot(anyDuplicated(wdi_2023[c("Country Code","Series Code")])==0)
wdi_wide<-reshape(wdi_2023,idvar=c("Country Name","Country Code"),timevar="Series Code",direction="wide")
names(wdi_wide)<-sub("^Value\\.","",names(wdi_wide))
rownames(wdi_wide)<-NULL
stopifnot(all(indicator_cols%in%names(wdi_wide)))
The reshaped dataset contains 217 economies and 12 indicators, with country identifiers kept for subsequent interpretation (no duplicate country-indicator combinations were found).
Missingness matters because most distance-based clustering methods require a complete matrix. Discarding incomplete countries at the outset would restrict which development profiles can be studied, so coverage is assessed before reducing the number of variables.
missing_by_variable<-data.frame(Indicator=indicator_cols,Label=unname(indicator_labels[indicator_cols]),Missing=colSums(is.na(wdi_wide[indicator_cols])))
missing_by_variable$Available<-nrow(wdi_wide)-missing_by_variable$Missing
missing_by_variable$Coverage<-round(100*missing_by_variable$Available/nrow(wdi_wide),1)
missing_by_variable<-missing_by_variable[order(missing_by_variable$Coverage),]
knitr::kable(missing_by_variable[c("Label","Missing","Available","Coverage")],row.names=FALSE,caption="Indicator coverage in 2023 (%)")
| Label | Missing | Available | Coverage |
|---|---|---|---|
| Secondary enrollment | 70 | 147 | 67.7 |
| Fixed broadband | 48 | 169 | 77.9 |
| Capital formation | 43 | 174 | 80.2 |
| Trade | 41 | 176 | 81.1 |
| Internet users | 36 | 181 | 83.4 |
| Employment ratio | 33 | 184 | 84.8 |
| Vulnerable employment | 33 | 184 | 84.8 |
| Basic sanitation | 21 | 196 | 90.3 |
| GDP per capita (PPP) | 20 | 197 | 90.8 |
| Electricity access | 2 | 215 | 99.1 |
| Life expectancy | 0 | 217 | 100.0 |
| Urban population | 0 | 217 | 100.0 |
ggplot(missing_by_variable,aes(x=reorder(Label,Coverage),y=Coverage))+
geom_col(fill="steelblue")+geom_text(aes(label=paste0(Coverage,"%")),hjust=-.1,size=3)+
coord_flip(ylim=c(0,108))+labs(x=NULL,y="Economies with an available value (%)")+theme_minimal(base_size=11)
Figure 1. Coverage of WDI indicators across 217 economies in 2023.
Across the 12 indicators, 2257 of 2604 values (86.7%) are available. Secondary enrollment and fixed broadband have relatively low coverage, while life expectancy and urban population are complete. Only 97 economies have all 12 indicators. This makes sample size an explicit part of variable selection rather than a decision left until the final preprocessing step.
The indicators use different scales and have different distributions. Summary statistics and histograms help identify skewness and unusually concentrated values before deciding whether any transformations are justified. Missing observations are omitted within each calculation (no countries are removed).
#calculate skewness after excluding missing observations
skewness<-function(x){
x<-x[!is.na(x)]
if(length(x)<3||sd(x)==0)return(NA_real_)
mean((x-mean(x))^3)/mean((x-mean(x))^2)^(3/2)
}
descriptive_stats<-t(sapply(wdi_wide[indicator_cols],function(x)c(N=sum(!is.na(x)),Median=median(x,na.rm=TRUE),Mean=mean(x,na.rm=TRUE),
SD=sd(x,na.rm=TRUE),Min=min(x,na.rm=TRUE),Max=max(x,na.rm=TRUE),Skewness=skewness(x))))
descriptive_stats<-data.frame(Indicator=unname(indicator_labels[rownames(descriptive_stats)]),round(descriptive_stats,2),row.names=NULL)
knitr::kable(descriptive_stats,caption="Descriptive statistics for the twelve candidate indicators")
| Indicator | N | Median | Mean | SD | Min | Max | Skewness |
|---|---|---|---|---|---|---|---|
| GDP per capita (PPP) | 197 | 17685.86 | 27404.73 | 26991.49 | 1033.01 | 130296.84 | 1.48 |
| Internet users | 181 | 81.71 | 71.70 | 25.34 | 7.17 | 100.00 | -0.99 |
| Life expectancy | 217 | 74.55 | 73.79 | 7.12 | 54.46 | 86.37 | -0.41 |
| Secondary enrollment | 147 | 93.28 | 90.04 | 24.95 | 3.35 | 142.96 | -0.74 |
| Employment ratio | 184 | 58.16 | 57.04 | 11.97 | 21.33 | 87.44 | -0.31 |
| Urban population | 217 | 63.66 | 62.61 | 23.24 | 14.68 | 100.00 | -0.19 |
| Electricity access | 215 | 100.00 | 88.18 | 22.43 | 5.40 | 100.00 | -1.95 |
| Trade | 176 | 82.31 | 94.84 | 60.26 | 2.47 | 402.57 | 2.19 |
| Capital formation | 174 | 22.98 | 23.45 | 7.93 | -3.74 | 45.27 | -0.02 |
| Vulnerable employment | 184 | 29.33 | 35.18 | 25.73 | 0.44 | 90.11 | 0.55 |
| Fixed broadband | 169 | 16.07 | 18.52 | 15.85 | 0.00 | 55.88 | 0.37 |
| Basic sanitation | 196 | 93.75 | 80.47 | 25.89 | 10.02 | 100.00 | -1.29 |
dist_long<-data.frame(Indicator=factor(rep(unname(indicator_labels[indicator_cols]),each=nrow(wdi_wide)),levels=unname(indicator_labels[indicator_cols])),Value=unlist(wdi_wide[indicator_cols],use.names=FALSE))
ggplot(dist_long,aes(x=Value))+
geom_histogram(bins=22,fill="grey65",color="white",na.rm=TRUE)+facet_wrap(~Indicator,scales="free",ncol=2)+
labs(x="Indicator value",y="Number of economies")+theme_minimal(base_size=12)
Figure 2. Distributions of the twelve candidate indicators (different horizontal scales).
GDP/capita and trade have long right tails, while electricity access and sanitation are concentrated near the upper end. In a distance-based analysis, the shape of the distribution matters beyond the mean alone, and especially when a small number of observations are far from the majority.
The conventional 1.5 × IQR rule flags unusual values without automatically treating them as errors. For an indicator \(x\), the fences are \(Q_1-1.5\,\mathrm{IQR}\) and \(Q_3+1.5\,\mathrm{IQR}\), where \(\mathrm{IQR}=Q_3-Q_1\). Extreme values can be economically meaningful, so I retain them unless there is evidence of a data problem.
#flag extreme observations without automatically removing them
outlier_summary<-do.call(rbind,lapply(indicator_cols,function(v){
x<-wdi_wide[[v]]
q<-quantile(x,c(.25,.75),na.rm=TRUE,names=FALSE)
limits<-q+c(-1,1)*1.5*(q[2]-q[1])
data.frame(Code=v,Indicator=unname(indicator_labels[v]),Outliers=sum(x<limits[1]|x>limits[2],na.rm=TRUE),Available=sum(!is.na(x)),Lower=limits[1],Upper=limits[2])
}))
outlier_summary$Percent<-round(100*outlier_summary$Outliers/outlier_summary$Available,1)
outlier_summary<-outlier_summary[order(-outlier_summary$Outliers),]
knitr::kable(outlier_summary[c("Indicator","Outliers","Available","Percent")],row.names=FALSE,caption="Potential outliers identified by the IQR rule")
| Indicator | Outliers | Available | Percent |
|---|---|---|---|
| Electricity access | 39 | 215 | 18.1 |
| Secondary enrollment | 13 | 147 | 8.8 |
| Basic sanitation | 13 | 196 | 6.6 |
| Capital formation | 12 | 174 | 6.9 |
| GDP per capita (PPP) | 7 | 197 | 3.6 |
| Trade | 7 | 176 | 4.0 |
| Employment ratio | 5 | 184 | 2.7 |
| Internet users | 2 | 181 | 1.1 |
| Life expectancy | 0 | 217 | 0.0 |
| Urban population | 0 | 217 | 0.0 |
| Vulnerable employment | 0 | 184 | 0.0 |
| Fixed broadband | 0 | 169 | 0.0 |
The greatest number of IQR flags appears in electricity access, largely because many economies are close to universal coverage while others are not. High GDP and trade values occur in economies (Luxembourg and Singapore). The flagged values are retained; the main adjustment made later is a log transformation of GDP/capita, whose scale would otherwise strongly affect distances.
Several indicators describe related aspects of development. When highly correlated variables are included together, they may give the same underlying pattern greater influence in Euclidean distance (even after standardization). Spearman correlation is used because many distributions are skewed and relationships may be monotonic rather than linear. The initial matrix uses available observation pairs, while the final sample is checked separately to avoid relying on correlations calculated from different sets of economies.
#use available pairs to inspect relationships before choosing the final sample
cor_matrix<-cor(wdi_wide[indicator_cols],method="spearman",use="pairwise.complete.obs")
cor_n<-crossprod(!is.na(as.matrix(wdi_wide[indicator_cols])))
idx<-which(upper.tri(cor_matrix)&abs(cor_matrix)>=.80,arr.ind=TRUE)
strong_pairs<-data.frame(Variable_1=unname(indicator_labels[rownames(cor_matrix)[idx[,1]]]),Variable_2=unname(indicator_labels[colnames(cor_matrix)[idx[,2]]]),Spearman=round(cor_matrix[idx],3),N=cor_n[idx])
strong_pairs<-strong_pairs[order(-abs(strong_pairs$Spearman)),]
knitr::kable(strong_pairs,row.names=FALSE,caption="Strong pairwise Spearman correlations (|rho| ≥ 0.80)")
| Variable_1 | Variable_2 | Spearman | N |
|---|---|---|---|
| GDP per capita (PPP) | Fixed broadband | 0.886 | 161 |
| GDP per capita (PPP) | Vulnerable employment | -0.885 | 172 |
| GDP per capita (PPP) | Internet users | 0.883 | 177 |
| GDP per capita (PPP) | Life expectancy | 0.872 | 197 |
| Internet users | Vulnerable employment | -0.852 | 165 |
| Life expectancy | Fixed broadband | 0.852 | 169 |
| Life expectancy | Basic sanitation | 0.845 | 196 |
| GDP per capita (PPP) | Basic sanitation | 0.830 | 179 |
| Internet users | Basic sanitation | 0.829 | 168 |
| Electricity access | Basic sanitation | 0.829 | 195 |
| Internet users | Life expectancy | 0.824 | 181 |
| Vulnerable employment | Basic sanitation | -0.824 | 172 |
| Life expectancy | Vulnerable employment | -0.809 | 184 |
| Internet users | Electricity access | 0.808 | 181 |
cor_long<-as.data.frame(as.table(cor_matrix))
cor_long$Var1<-factor(unname(indicator_labels[as.character(cor_long$Var1)]),levels=unname(indicator_labels[indicator_cols]))
cor_long$Var2<-factor(unname(indicator_labels[as.character(cor_long$Var2)]),levels=rev(unname(indicator_labels[indicator_cols])))
ggplot(cor_long,aes(x=Var1,y=Var2,fill=Freq))+
geom_tile(color="white",linewidth=.2)+scale_fill_gradient2(low="blue",mid="white",high="red",midpoint=0,limits=c(-1,1),name="Spearman")+
labs(x=NULL,y=NULL)+coord_equal()+theme_minimal(base_size=9)+theme(axis.text.x=element_text(angle=55,hjust=1),panel.grid=element_blank())
Figure 3. Pairwise Spearman correlations among the twelve candidate indicators.
I kept 8 indicators to balance distinct development dimensions against the number of complete observations. Fixed broadband and electricity access add comparatively limited information or coverage; trade and capital formation are less central to the intended profiles. Secondary enrollment remains despite its missingness because education adds a meaningful dimension. Correlation alone doesnt show that 2 indicators are interchangeable: the stronger test is whether removing a correlated indicator materially changes the resulting partition (Section 7.4).
sample_comparison<-data.frame(Scenario=c("All 12 indicators","Selected 8 indicators","7 indicators (without education)"),
Complete=c(sum(complete.cases(wdi_wide[indicator_cols])),sum(complete.cases(wdi_wide[selected_vars])),
sum(complete.cases(wdi_wide[setdiff(selected_vars,"SE.SEC.ENRR")]))))
knitr::kable(sample_comparison,caption="Sample-size trade-off in indicator selection")
| Scenario | Complete |
|---|---|
| All 12 indicators | 97 |
| Selected 8 indicators | 113 |
| 7 indicators (without education) | 153 |
Using all 12 indicators would leave only 97 economies, compared with 113 when using the selected 8. If I remove education, the sample increases to 153 economies, but I would lose an important part of socioeconomic development. To see whether this decision affects the results, I will check how the clusters change without education in Section 7.4.
The analytical sample includes economies with complete values for all 8 retained indicators. GDP/capita is log-transformed to reduce its long right tail, followed by standardization of every input. The transformations are \(x_{\mathrm{GDP}}^{*}=\ln(x_{\mathrm{GDP}})\) and \(z_{ij}=(x_{ij}^{*}-\bar{x}_j^{*})/s_j\), using the transformed GDP and original values of the other variables. Euclidean distance is then \(d(i,l)=\sqrt{\sum_{j=1}^{8}(z_{ij}-z_{lj})^2}\). Economy names and codes stay outside the distance matrix.
#keep identifiers separate from the variables used for distances
wdi_complete<-wdi_wide[complete.cases(wdi_wide[selected_vars]),c("Country Name","Country Code",selected_vars)]
rownames(wdi_complete)<-NULL
country_info<-wdi_complete[c("Country Name","Country Code")]
clustering_data<-wdi_complete[selected_vars]
stopifnot(all(clustering_data[["NY.GDP.PCAP.PP.KD"]]>0))
#reduce the right skew of income before standardization
clustering_data[["NY.GDP.PCAP.PP.KD"]]<-log(clustering_data[["NY.GDP.PCAP.PP.KD"]])
names(clustering_data)[1]<-"log_GDP_PPP"
clustering_scaled<-scale(clustering_data)
rownames(clustering_scaled)<-country_info$`Country Code`
stopifnot(!anyNA(clustering_scaled),all(is.finite(clustering_scaled)))
gdp_log<-data.frame(Scale=rep(c("GDP per capita (raw)","Natural log of GDP per capita"),each=nrow(wdi_complete)),Value=c(wdi_complete[["NY.GDP.PCAP.PP.KD"]],clustering_data$log_GDP_PPP))
ggplot(gdp_log,aes(x=Value))+
geom_histogram(bins=20,fill="grey65",color="white")+facet_wrap(~Scale,scales="free",ncol=2)+
labs(x=NULL,y="Number of economies")+theme_minimal(base_size=10)
Figure 4. GDP per capita before and after the log transformation (same 113 economies).
The GDP skewness changes from 1.34 to -0.53 after the log transformation. The final matrix contains 113 economies and 8 standardized indicators. The extreme observations were not discarded merely because they were flagged by the IQR rule.
Complete-case selection has an important consequence: 104 economies are excluded. To check whether those exclusions may be systematic, the following table compares available GDP and Internet-use values for retained and excluded economies. (Note:” medians are computed only from observed values within each group).
wdi_wide$Sample<-ifelse(wdi_wide$`Country Code`%in%country_info$`Country Code`,"Retained","Excluded")
selection_comparison<-do.call(rbind,lapply(c("NY.GDP.PCAP.PP.KD","IT.NET.USER.ZS"),function(v){
do.call(rbind,lapply(c("Retained","Excluded"),function(group){
x<-wdi_wide[wdi_wide$Sample==group,v]
data.frame(Indicator=unname(indicator_labels[v]),Group=group,Observed=sum(!is.na(x)),Median=round(median(x,na.rm=TRUE),2))
}))
}))
knitr::kable(selection_comparison,caption="Observed development indicators in retained and excluded economies")
| Indicator | Group | Observed | Median |
|---|---|---|---|
| GDP per capita (PPP) | Retained | 113 | 21864.92 |
| GDP per capita (PPP) | Excluded | 84 | 13368.56 |
| Internet users | Retained | 113 | 85.79 |
| Internet users | Excluded | 68 | 71.40 |
After removing economies with missing values, the final sample contains 113 out of 217 economies, meaning 104 (47.9%) are excluded. The economies kept in the sample have higher observed median GDP and Internet use than those excluded, although these values are not available for every excluded economy. This suggests that the final sample may include more developed and better-documented economies. As a result, the clusters may not represent all economies equally well, and we cannot determine which groups the excluded economies would belong to. The correlations among the selected indicators are also worth checking in the final sample:
final_spearman<-cor(wdi_complete[selected_vars],method="spearman")
final_idx<-which(upper.tri(final_spearman)&abs(final_spearman)>=.80,arr.ind=TRUE)
final_pairs<-data.frame(Indicator_1=unname(indicator_labels[rownames(final_spearman)[final_idx[,1]]]),
Indicator_2=unname(indicator_labels[colnames(final_spearman)[final_idx[,2]]]),Spearman=round(final_spearman[final_idx],3))
knitr::kable(final_pairs,caption="Strong correlations remaining in the 113-economy sample")
| Indicator_1 | Indicator_2 | Spearman |
|---|---|---|
| GDP per capita (PPP) | Internet users | 0.864 |
| GDP per capita (PPP) | Life expectancy | 0.870 |
| GDP per capita (PPP) | Vulnerable employment | -0.856 |
| Internet users | Vulnerable employment | -0.807 |
| GDP per capita (PPP) | Basic sanitation | 0.824 |
| Life expectancy | Basic sanitation | 0.861 |
The final-sample correlations confirm that the development indicators are not independent. GDP/capita has strong associations with life expectancy (ρ = 0.87), Internet use (ρ = 0.864), and vulnerable employment (ρ = -0.856). These indicators describe different outcomes, but together they may emphasize a broad development gradient. Section 7.4 therefore tests whether removing life expectancy, one of the most strongly correlated measures, changes the two-cluster solution.
A clustering algorithm can partition even a continuous distribution. The Hopkins statistic and an ordered dissimilarity image (ODI) first assess whether the data depart from spatial randomness. For sampled observed-point nearest-neighbor distances \(w_i\) and uniformly generated-point nearest-neighbor distances \(u_i\), the Hopkins statistic used here is \[H=\frac{\sum_{i=1}^{m}u_i^{D}}{\sum_{i=1}^{m}u_i^{D}+\sum_{i=1}^{m}w_i^{D}},\qquad D=8.\] High values indicate nonuniform spatial structure, not necessarily discrete clusters.
hopkins_result<-factoextra::get_clust_tendency(clustering_scaled,n=25,graph=TRUE,seed=42)
hopkins_result$plot
Figure 5. Ordered dissimilarity image for the final standardized dataset.
#check whether the hopkins estimate depends on random sampling
hopkins_repeats<-sapply(1:20,function(i)factoextra::get_clust_tendency(clustering_scaled,n=25,graph=FALSE,seed=100+i)$hopkins_stat)
knitr::kable(data.frame(Original=round(hopkins_result$hopkins_stat,4),Repeated_median=round(median(hopkins_repeats),4),
Repeated_min=round(min(hopkins_repeats),4),Repeated_max=round(max(hopkins_repeats),4)),
caption="Hopkins statistic: original run and sensitivity to random sampling")
| Original | Repeated_median | Repeated_min | Repeated_max |
|---|---|---|---|
| 0.9994 | 0.994 | 0.9068 | 0.9996 |
The original Hopkins value is 0.9994, with the repeated results shown above. This high value suggests that the data are not uniformly distributed, but it does not necessarily mean there are clearly separated clusters. Correlated indicators, a continuous development gradient, and the choice of reference region may also affect the result. The ODI plot provides another view of the data structure, while the next step checks whether most of the variation follows a single continuous direction.
#use a one-dimensional projection only to inspect the development gradient
pc1_model<-prcomp(clustering_scaled,center=FALSE,scale.=FALSE)
pc1_share<-summary(pc1_model)$importance[2,1]
pc1_data<-data.frame(PC1=pc1_model$x[,1])
ggplot(pc1_data,aes(x=PC1))+
geom_histogram(aes(y=after_stat(density)),bins=18,fill="grey75",color="white")+
geom_density(color="steelblue",linewidth=.9)+labs(x="First principal component score",y="Density")+theme_minimal(base_size=11)
Figure 6. Distribution of the first principal component of the standardized indicators.
The first principal component explains 70.7% of total standardized variance. This supports the possibility of one dominant development gradient. However, a one-dimensional projection can hide structure in other dimensions and a histogram alone cannot establish the number of natural groups. (Note:Section 7.5 returns to this question using a continuous reference distribution with a similar correlation structure).
The Elbow method compares within-cluster dispersion, while average Silhouette width measures how well each economy fits within its assigned cluster relative to neighboring groups. They address different questions, so disagreement is informative rather than an error.
set.seed(42)
fviz_nbclust(clustering_scaled,kmeans,method="wss",k.max=10,nstart=25)
Figure 7. Elbow method for K-means, k = 1 to 10.
set.seed(42)
fviz_nbclust(clustering_scaled,kmeans,method="silhouette",k.max=10,nstart=25)
Figure 8. Average Silhouette widths for K-means, k = 2 to 10.
The Elbow method suggests approximately three clusters, while the Silhouette plot favors k = 2. Since the two criteria highlight different aspects of the data, both solutions are carried forward for comparison rather than selecting a cluster count from one graph alone.
K-means chooses cluster assignments \(C_1,\ldots,C_k\) and centroids \(\mu_1,\ldots,\mu_k\) by minimizing the within-cluster sum of squares. \[\mathrm{WCSS}=\sum_{g=1}^{k}\sum_{i\in C_g}\lVert z_i-\mu_g\rVert_2^2.\] Both \(k=2\) and \(k=3\) use 25 random starts to reduce sensitivity to initialization.
set.seed(42)
km2<-kmeans(clustering_scaled,centers=2,nstart=25)
km3<-kmeans(clustering_scaled,centers=3,nstart=25)
knitr::kable(data.frame(K=rep(c(2,3),c(2,3)),Cluster=c(seq_along(km2$size),seq_along(km3$size)),Economies=c(km2$size,km3$size)),caption="K-means cluster sizes")
| K | Cluster | Economies |
|---|---|---|
| 2 | 1 | 87 |
| 2 | 2 | 26 |
| 3 | 1 | 17 |
| 3 | 2 | 44 |
| 3 | 3 | 52 |
fviz_cluster(km2,data=clustering_scaled,geom="point",main="K-means: k = 2")
Figure 9. K-means with two clusters, displayed in a two-dimensional projection.
fviz_cluster(km3,data=clustering_scaled,geom="point",main="K-means: k = 3")
Figure 10. K-means with three clusters, displayed in a two-dimensional projection.
With k = 2, the sample divides into 87 and 26 economies. The three-cluster solution captures additional gradations but also introduces more complexity. These projections help us visualize the partitions; because the models were fitted in eight dimensions, apparent overlap or separation in 2 dimensions shouldnt be treated as a complete validation of the original clusters.
PAM selects an observed medoid \(m_g\) for each cluster and minimizes total dissimilarity, \(\sum_{g=1}^{k}\sum_{i\in C_g}d(z_i,m_g)\) . Unlike K-means centroids, medoids correspond to actual economies. Euclidean distance, standardized inputs, and the candidate cluster counts are kept consistent.
pam2<-cluster::pam(clustering_scaled,k=2,metric="euclidean")
pam3<-cluster::pam(clustering_scaled,k=3,metric="euclidean")
knitr::kable(data.frame(Solution=c("PAM, k = 2","PAM, k = 3"),Average_silhouette=round(c(pam2$silinfo$avg.width,pam3$silinfo$avg.width),3)),caption="PAM average Silhouette widths")
| Solution | Average_silhouette |
|---|---|
| PAM, k = 2 | 0.489 |
| PAM, k = 3 | 0.319 |
As we can see, PAM with k = 2 has a higher average Silhouette width than its three-cluster alternative. Rather than repeating the development profiles described for K-means, Section 7 examines whether the use of medoids changes which economies are grouped together.
Hierarchical clustering builds nested groups that can be cut at different levels. I compare single, complete, average, and Ward.D2 linkage using Euclidean distance. Ward.D2 merges the two groups producing the smallest increase in within-cluster sum of squares: \[\Delta(A,B)=\frac{|A|\,|B|}{|A|+|B|}\lVert\bar z_A-\bar z_B\rVert_2^2.\] This favors compact groups, while single linkage can suffer from chaining.
d<-dist(clustering_scaled)
methods<-c("single","complete","average","ward.D2")
hc_models<-setNames(lapply(methods,function(m)hclust(d,method=m)),methods)
hc_scores<-sapply(hc_models,function(h)sapply(2:3,function(k)mean(cluster::silhouette(cutree(h,k),d)[,3])))
hc_table<-data.frame(k=2:3,round(hc_scores,3),check.names=FALSE)
knitr::kable(hc_table,row.names=FALSE,caption="Hierarchical clustering: average Silhouette width")
| k | single | complete | average | ward.D2 |
|---|---|---|---|---|
| 2 | 0.437 | 0.506 | 0.508 | 0.520 |
| 3 | 0.361 | 0.315 | 0.486 | 0.329 |
plot(hc_models[["ward.D2"]],labels=FALSE,main="Ward.D2 dendrogram",xlab="",sub="")
rect.hclust(hc_models[["ward.D2"]],k=2,border=2:3)
Figure 11. Ward.D2 dendrogram with the two-cluster cut highlighted.
In this part, Ward.D2 achieves the highest average Silhouette width among the hierarchical two-cluster solutions, while average linkage performs well for 3 clusters. Silhouette alone is not decisive, however; cluster size, agreement with other methods, and stability under resampling must also be considered.
All partitions are evaluated using the same Euclidean distance matrix. If \(a(i)\) is the mean distance from economy \(i\) to its own cluster and \(b(i)\) is the smallest mean distance to another cluster then its Silhouette width is: \[s(i)=\frac{b(i)-a(i)}{\max\{a(i),b(i)\}}.\] The table also reports the percentage of negative silhouettes and the smallest group size, since a high mean alone can conceal tiny groups.
solutions<-list(Kmeans_2=km2$cluster,Kmeans_3=km3$cluster,PAM_2=pam2$clustering,PAM_3=pam3$clustering,
Ward_2=cutree(hc_models[["ward.D2"]],2),Ward_3=cutree(hc_models[["ward.D2"]],3),Average_3=cutree(hc_models[["average"]],3))
sil<-lapply(solutions,function(x)cluster::silhouette(x,d)[,3])
validation<-data.frame(Solution=names(solutions),Silhouette=round(sapply(sil,mean),3),NegativePct=round(100*sapply(sil,function(x)mean(x<0)),1),MinCluster=sapply(solutions,function(x)min(table(x))))
validation<-validation[order(-validation$Silhouette),]
knitr::kable(validation,row.names=FALSE,caption="Comparison of clustering solutions")
| Solution | Silhouette | NegativePct | MinCluster |
|---|---|---|---|
| Ward_2 | 0.520 | 4.4 | 15 |
| Kmeans_2 | 0.492 | 1.8 | 26 |
| PAM_2 | 0.489 | 1.8 | 26 |
| Average_3 | 0.486 | 6.2 | 1 |
| Kmeans_3 | 0.341 | 2.7 | 17 |
| Ward_3 | 0.329 | 9.7 | 15 |
| PAM_3 | 0.319 | 8.0 | 20 |
Among the two-cluster solutions, Ward.D2 has the highest average Silhouette width (0.52), followed by K-means (0.492) and PAM (0.489). However, some three-cluster solutions contain very small groups and the higher Silhouette score for Ward.D2 doesn’t imply stronger agreement with the partitional methods.
ari<-c(Kmeans_PAM=mclust::adjustedRandIndex(km2$cluster,pam2$clustering),
Kmeans_Ward=mclust::adjustedRandIndex(km2$cluster,solutions$Ward_2),PAM_Ward=mclust::adjustedRandIndex(pam2$clustering,solutions$Ward_2),
Kmeans3_PAM3=mclust::adjustedRandIndex(km3$cluster,pam3$clustering),Kmeans3_Average3=mclust::adjustedRandIndex(km3$cluster,solutions$Average_3))
knitr::kable(data.frame(Comparison=names(ari),ARI=round(ari,3)),row.names=FALSE,caption="Adjusted Rand Index across alternative partitions")
| Comparison | ARI |
|---|---|
| Kmeans_PAM | 0.924 |
| Kmeans_Ward | 0.582 |
| PAM_Ward | 0.582 |
| Kmeans3_PAM3 | 0.712 |
| Kmeans3_Average3 | 0.266 |
K-means and PAM with 2 groups have an ARI of 0.924, showing close agreement. The adjusted rand index corrects the raw agreement between pairs of economies for chance. With \(n_{ij}\) in the contingency table, row totals \(a_i\), column totals \(b_j\), and \(N\) economies, the formula is: \[\mathrm{ARI}=\frac{\sum_{ij}\binom{n_{ij}}{2}-E}{\frac12\left[\sum_i\binom{a_i}{2}+\sum_j\binom{b_j}{2}\right]-E},\quad E=\frac{\left(\sum_i\binom{a_i}{2}\right)\left(\sum_j\binom{b_j}{2}\right)}{\binom{N}{2}}.\] So, weaker agreement with Ward.D2 means the methods assign some economies differently, not that either partition is automatically incorrect.
Average scores hide observations near cluster boundaries. The Silhouette plot for K-means with k=2 shows both the overall pattern and economies whose assignments are less convincing.
km_sil<-cluster::silhouette(km2$cluster,d)
factoextra::fviz_silhouette(km_sil,label=FALSE,print.summary=FALSE)+labs(title="K-means (k = 2): individual Silhouette widths")
Figure 12. Individual Silhouette widths for the selected K-means partition.
low_sil<-data.frame(Country=country_info$`Country Name`,Cluster=km2$cluster,Silhouette=round(km_sil[,3],3))
low_sil<-low_sil[order(low_sil$Silhouette),]
knitr::kable(head(low_sil,10),row.names=FALSE,caption="Ten economies with the lowest Silhouette widths")
| Country | Cluster | Silhouette |
|---|---|---|
| Tajikistan | 2 | -0.055 |
| Honduras | 2 | -0.054 |
| Guatemala | 2 | 0.053 |
| Cambodia | 2 | 0.079 |
| Nepal | 2 | 0.099 |
| Tonga | 1 | 0.117 |
| Namibia | 2 | 0.120 |
| Sri Lanka | 1 | 0.122 |
| Bolivia | 1 | 0.126 |
| India | 2 | 0.134 |
There are 2 economies with negative Silhouette widths in the selected solution. Low or negative values suggest weaker membership relative to neighboring clusters, not proof that these economies have been misclassified. It’s worth noting when we’re looking at the 2 development profiles.
Bootstrap resampling assesses whether groups persist when observations change. The clusterwise Jaccard similarity compares an original group \(A\) with its best-matching resampled group \(B\): \[J(A,B)=\frac{|A\cap B|}{|A\cup B|}.\] I’m going to use 100 bootstrap replications for K-means, PAM, and Ward.D2. The numeric group labels need not align across methods, so comparisons concern the overall stability pattern rather than matching labels.
#compare cluster stability across methods using the same resampling setup
set.seed(42)
boot_km<-fpc::clusterboot(clustering_scaled,B=100,clustermethod=fpc::kmeansCBI,krange=2,runs=25,count=FALSE)
set.seed(42)
boot_pam<-fpc::clusterboot(clustering_scaled,B=100,clustermethod=fpc::claraCBI,k=2,usepam=TRUE,count=FALSE)
set.seed(42)
boot_ward<-fpc::clusterboot(clustering_scaled,B=100,clustermethod=fpc::hclustCBI,k=2,method="ward.D2",scaling=FALSE,count=FALSE)
boot_values<-rbind(Kmeans=boot_km$bootmean,PAM=boot_pam$bootmean,Ward_D2=boot_ward$bootmean)
colnames(boot_values)<-paste0("Group ",seq_len(ncol(boot_values)))
boot_long<-data.frame(Method=rep(rownames(boot_values),times=ncol(boot_values)),Group=rep(colnames(boot_values),each=nrow(boot_values)),Jaccard=as.vector(boot_values))
ggplot(boot_long,aes(x=Method,y=Jaccard,fill=Group))+
geom_col(position=position_dodge(width=.75),width=.65)+geom_text(aes(label=sprintf("%.3f",Jaccard)),position=position_dodge(width=.75),vjust=-.3,size=3)+
coord_cartesian(ylim=c(0,1.06))+labs(x=NULL,y="Mean Jaccard similarity",fill=NULL)+theme_minimal(base_size=11)
Figure 13. Clusterwise mean Jaccard similarities over 100 bootstrap replications.
The two K-means groups have mean Jaccard values of 0.976 and 0.927, compared with 0.738 and 0.902 for Ward.D2. Although Ward.D2 produces a higher average Silhouette width, one of its groups is less reproducible under bootstrap resampling. PAM provides a second, broadly stable solution with close agreement to K-means. Considering separation, stability, and agreement together, K-means with k = 2 keeps the preferred description, not a uniquely correct partition.
The sensitivity analysis checks how much the clusters change when certain indicators are removed. Secondary enrollment is tested using both the original 113 economies and a larger sample. Life expectancy, GDP per capita, and Internet use are also removed one at a time to see how they affect the results. To keep the comparisons fair, the same economies are used for each pair of models, and the remaining variables are standardized again before running K-means.
#apply the same transformations when gdp is included
prepare_vars<-function(df,vars){
x<-df[vars]
if("NY.GDP.PCAP.PP.KD"%in%vars){
stopifnot(all(x[["NY.GDP.PCAP.PP.KD"]]>0))
x[["NY.GDP.PCAP.PP.KD"]]<-log(x[["NY.GDP.PCAP.PP.KD"]])
}
scale(x)
}
seven_vars<-setdiff(selected_vars,"SE.SEC.ENRR")
wdi_seven<-wdi_wide[complete.cases(wdi_wide[seven_vars]),c("Country Name","Country Code",seven_vars)]
set.seed(42)
km7_same<-kmeans(prepare_vars(wdi_complete,seven_vars),centers=2,nstart=25)
set.seed(42)
km7_extended<-kmeans(prepare_vars(wdi_seven,seven_vars),centers=2,nstart=25)
#match the original economies before calculating ari
common_idx<-match(country_info$`Country Code`,wdi_seven$`Country Code`)
stopifnot(!anyNA(common_idx))
set.seed(42)
km_no_life<-kmeans(prepare_vars(wdi_complete,setdiff(selected_vars,"SP.DYN.LE00.IN")),centers=2,nstart=25)
set.seed(42)
km_no_gdp<-kmeans(prepare_vars(wdi_complete,setdiff(selected_vars,"NY.GDP.PCAP.PP.KD")),centers=2,nstart=25)
set.seed(42)
km_no_internet<-kmeans(prepare_vars(wdi_complete,setdiff(selected_vars,"IT.NET.USER.ZS")),centers=2,nstart=25)
robustness<-data.frame(Scenario=c("Baseline: 8 indicators","Without education: same sample","Without education: expanded sample","Without life expectancy","Without GDP","Without Internet use"),
Fitted_economies=c(nrow(wdi_complete),nrow(wdi_complete),nrow(wdi_seven),rep(nrow(wdi_complete),3)),
ARI_vs_baseline=c(1,mclust::adjustedRandIndex(km2$cluster,km7_same$cluster),
mclust::adjustedRandIndex(km2$cluster,km7_extended$cluster[common_idx]),
mclust::adjustedRandIndex(km2$cluster,km_no_life$cluster),
mclust::adjustedRandIndex(km2$cluster,km_no_gdp$cluster),
mclust::adjustedRandIndex(km2$cluster,km_no_internet$cluster)))
robustness$ARI_vs_baseline<-round(robustness$ARI_vs_baseline,3)
knitr::kable(robustness,row.names=FALSE,col.names=c("Scenario","Economies fitted","ARI vs. baseline"),caption="Sensitivity to missingness and indicator selection")
| Scenario | Economies fitted | ARI vs. baseline |
|---|---|---|
| Baseline: 8 indicators | 113 | 1.000 |
| Without education: same sample | 113 | 0.923 |
| Without education: expanded sample | 153 | 0.884 |
| Without life expectancy | 113 | 0.925 |
| Without GDP | 113 | 0.923 |
| Without Internet use | 113 | 0.887 |
Removing education gives ARIs of 0.923 on the original sample and 0.884 when using the larger sample. Removing life expectancy, GDP, and Internet use gives ARIs of 0.925, 0.923, and 0.887 respectively. These high values suggest that the two-group solution is fairly stable, although they do not prove that every indicator adds unique information. The larger-sample result also reflects changes in sample composition, not just the removal of education.
A dominant first principal component and a high Silhouette score do not always mean that clear clusters exist. To check this, I generate 100 multivariate normal datasets, each with 113 observations and the same eight-variable covariance matrix as the original data. Each dataset is standardized, then K-means (k=2) is applied to calculate Silhouette score. This provides a comparison with a smooth, correlated distribution, but doesn’t prove that exactly 2 natural clusters exist.
#compare observed separation with a smooth correlated reference
set.seed(1423)
null_cov<-cov(clustering_scaled)
null_sil<-replicate(100,{
x<-matrix(rnorm(length(clustering_scaled)),ncol=ncol(clustering_scaled))%*%chol(null_cov)
x<-scale(x)
fit<-kmeans(x,centers=2,nstart=25)
mean(cluster::silhouette(fit$cluster,dist(x))[,3])
})
observed_sil<-mean(km_sil[,3])
null_comparison<-data.frame(Observed=round(observed_sil,3),Null_mean=round(mean(null_sil),3),Null_95th_percentile=round(unname(quantile(null_sil,.95)),3),Exceedances=sum(null_sil>=observed_sil),Simulations=length(null_sil))
knitr::kable(null_comparison,row.names=FALSE,col.names=c("Observed","Reference mean","Reference 95th pct.","Reference >= observed","Simulations"),caption="K-means separation versus a correlated continuous reference")
| Observed | Reference mean | Reference 95th pct. | Reference >= observed | Simulations |
|---|---|---|---|---|
| 0.492 | 0.355 | 0.392 | 0 | 100 |
The observed average Silhouette is 0.492, compared with 0.355 for the simulated datasets. Only 0 out of 100 simulations reach the observed score. This suggests that the actual clusters are better separated than those in the Gaussian reference. However, the strong first principal component and overlap between groups suggest that the clusters should be seen as general patterns along a development gradient, rather than completely separate categories.
Once the partition has been selected, the original WDI units provide a clearer interpretation than standardized scores. The table shows medians which are less affected by the high-income tail, a second heatmap shows the standardized cluster centers used by K-means.
profile_data<-wdi_complete
profile_data$Cluster<-factor(km2$cluster)
cluster_profiles<-aggregate(profile_data[selected_vars],by=list(Cluster=profile_data$Cluster),FUN=median)
names(cluster_profiles)<-c("Cluster",unname(indicator_labels[selected_vars]))
cluster_profiles[-1]<-round(cluster_profiles[-1],2)
higher_id<-which.max(cluster_profiles[["GDP per capita (PPP)"]])
lower_id<-which.min(cluster_profiles[["GDP per capita (PPP)"]])
knitr::kable(cluster_profiles,caption="Median development indicators by cluster (original units)")
| Cluster | GDP per capita (PPP) | Internet users | Life expectancy | Secondary enrollment | Employment ratio | Urban population | Vulnerable employment | Basic sanitation |
|---|---|---|---|---|---|---|---|---|
| 1 | 34703.23 | 89.00 | 77.65 | 99.30 | 59.17 | 72.36 | 14.95 | 98.09 |
| 2 | 4388.54 | 38.82 | 66.08 | 54.47 | 58.52 | 39.72 | 69.07 | 47.17 |
centers_plot<-km2$centers
colnames(centers_plot)<-unname(indicator_labels[selected_vars])
centers_long<-as.data.frame(as.table(centers_plot))
names(centers_long)<-c("Cluster","Indicator","Z")
centers_long$Cluster<-paste("Cluster",centers_long$Cluster)
max_abs<-max(abs(centers_long$Z))
ggplot(centers_long,aes(x=Indicator,y=Cluster,fill=Z))+
geom_tile(color="white",linewidth=.6)+geom_text(aes(label=sprintf("%.2f",Z)),size=3)+
scale_fill_gradient2(low="blue",mid="white",high="red",midpoint=0,limits=c(-max_abs,max_abs),name="Z-score")+
labs(x=NULL,y=NULL)+theme_minimal(base_size=10)+theme(axis.text.x=element_text(angle=45,hjust=1),panel.grid=element_blank())
Figure 14. Standardized K-means cluster centers; zero denotes the sample mean.
The higher-income group has median Internet use of 89% vs 38.82% in the lower-income group. Vulnerable employment shows the opposite pattern (14.95% vs 69.07%) and while employment ratios are quite similar (59.17% vs 58.52%). This means that the groups differ more in vulnerable employment than in overall employment participation.
The heatmap shows the cluster centers but doesn’t capture differences between countries within each group. The boxplots below compare four percentage-based indicators to see how clearly the groups differ and where they overlap.
profile_focus<-c("IT.NET.USER.ZS","SE.SEC.ENRR","SL.EMP.VULN.ZS","SH.STA.BASS.ZS")
profile_long<-data.frame(Cluster=rep(paste("Cluster",profile_data$Cluster),times=length(profile_focus)),
Indicator=rep(unname(indicator_labels[profile_focus]),each=nrow(profile_data)),Value=unlist(profile_data[profile_focus],use.names=FALSE))
ggplot(profile_long,aes(x=Cluster,y=Value,fill=Cluster))+
geom_boxplot(width=.6,outlier.alpha=.45)+facet_wrap(~Indicator,scales="free_y",ncol=2)+
labs(x=NULL,y="Indicator value (%)")+theme_minimal(base_size=11)+theme(legend.position="none")
Figure 15. Within-cluster distributions for four key development indicators.
The boxplots show that countries within the same group can still differ quite a lot. Some overlap between groups is expected because the clustering is based on all 8 indicators together, not just 1 variable.
The cluster medians show the overall development pattern, but some economies may have higher or lower Internet use than their income would suggest. The scatterplot compares Internet use with log GDP per capita to explore these differences. The fitted line is only a descriptive reference and is not used to create or validate the clusters.
#identify economies furthest above and below the descriptive income trend
internet_fit<-lm(IT.NET.USER.ZS~log(NY.GDP.PCAP.PP.KD),data=profile_data)
profile_data$Internet_gap<-residuals(internet_fit)
plot_labels<-c(head(order(profile_data$Internet_gap,decreasing=TRUE),2),head(order(profile_data$Internet_gap),2))
ggplot(profile_data,aes(x=log(NY.GDP.PCAP.PP.KD),y=IT.NET.USER.ZS,color=Cluster))+
geom_point(size=2.2,alpha=.8)+geom_smooth(aes(group=1),method="lm",formula=y~x,se=FALSE,color="grey45",linewidth=.7)+
ggrepel::geom_text_repel(data=profile_data[plot_labels,],aes(label=`Country Name`),size=3,seed=42,max.overlaps=Inf,box.padding=.35,point.padding=.25,show.legend=FALSE)+
labs(x="Natural log of GDP per capita (PPP)",y="Internet users (% of population)",color="Cluster")+theme_minimal(base_size=11)
Figure 16. Internet use versus log GDP per capita, with K-means groups and selected deviations from the fitted trend.
Internet use increases with income, but there are some exceptions. We can see Kyrgyz Republic and Morocco have Internet use around 91%, despite having much lower GDP per capita than the richest economies. Meanwhile, Papua New Guinea and Chad fall below the fitted trend. These differences show that the cluster labels do not tell the whole story ( this comparison is only descriptive, since both GDP and Internet use were already included in the clustering).
To give a clearer picture of each cluster, the table below lists 5 economies closest to each K-means centroid. Distances are measured using all 8 standardized indicators. These economies are considered representative of their clusters, but they are not the best or worst performers on any single indicator.
#measure distances to the assigned centroid in standardized space
dist_center<-sqrt(rowSums((clustering_scaled-km2$centers[km2$cluster,])^2))
representatives<-data.frame(Cluster=km2$cluster,Country=profile_data$`Country Name`,Distance=round(dist_center,2))
representative_table<-do.call(rbind,lapply(split(representatives,representatives$Cluster),function(x)head(x[order(x$Distance),],5)))
rownames(representative_table)<-NULL
knitr::kable(representative_table,caption="Five representative economies per cluster")
| Cluster | Country | Distance |
|---|---|---|
| 1 | Czechia | 0.62 |
| 1 | Lithuania | 0.62 |
| 1 | Hungary | 0.67 |
| 1 | Latvia | 0.70 |
| 1 | Poland | 0.73 |
| 2 | Cameroon | 1.09 |
| 2 | Togo | 1.40 |
| 2 | Cote d’Ivoire | 1.41 |
| 2 | Mali | 1.46 |
| 2 | Senegal | 1.47 |
The economies closest to each centroid give an idea of what is typical in each group. In contrast, Tajikistan and Honduras have the two lowest Silhouette widths, meaning their cluster assignments are less clear. These cases show why the groups should not be treated as strict categories.
For clarity, the groups are named Higher Internet Use and Socioeconomic Development and Lower Internet Use and Socioeconomic Development. These labels describe differences within the sample.
By using 8 WDI indicators for 113 economies in 2023, the analysis identifies two broad development profiles. The main differences involve income, Internet use, health, sanitation, and vulnerable employment. The first principal component explains 70.7% of the variation, suggesting that the groups also reflect a wider development gradient.
Ward.D2 gets the highest average Silhouette score but K-means with k=2 shows better bootstrap stability and close agreement with PAM. The sensitivity checks also produce broadly similar groups when individual indicators are removed although some assignments change. The observed separation is stronger than the Gaussian reference, but this does not mean that two natural clusters exist.
Limitation: missing data, which excludes 104 out of 217 economies and may favor better-documented countries. Correlated indicators and the use of Internet use as the only direct digital measure also limit interpretation. Overall, the clusters provide a useful exploratory summary of socioeconomic development, not causal findings or official classifications.