Dimension Reduction
Ahmad Huseynov
2025-02-03
Introduction
Multidimensional scaling (MDS) is a dimensionality reduction technique that is used to project high-dimensional data onto a lower-dimensional space while preserving the pairwise distances between the data points as much as possible. MDS is based on the concept of distance and aims to find a projection of the data that minimizes the differences between the distances in the original space and the distances in the lower-dimensional space. https://www.geeksforgeeks.org/multidimensional-scaling-mds-using-scikit-learn/
Multidimensional scaling (MDS) is frequently used to represent intricate, high-dimensional datasets visually and to uncover patterns and relationships that might be obscured in the original space. This technique is versatile, applicable to various data types, including numerical, categorical, and mixed datasets. The implementation of MDS relies on numerical optimization algorithms, such as gradient descent or simulated annealing, which aim to minimize the discrepancy between distances in the original and reduced-dimensional spaces.
Data
This dataset contains player stats from the EA FC (FIFA) 24 football game, a continuation of FIFA 23. In addition to the names and positions of the players, the data also include different variables that affect the gameplay, such as sprint speed, shoot power, and passing.
Libraries:
library(dplyr)
library(ggplot2)
library(qgraph)
library(tidyr)
library(plotly)
library(smacof)- 1)We should start with loading the data, then create a subset of first 200 rows.
full_data <- read.csv("fc_24.csv")
data <- full_data[1:200, ]
head(data)## X name nation club position alternative.positions
## 1 0 Kylian Mbappé France Paris SG ST ST, LW, CF
## 2 2 Erling Haaland Norway Manchester City ST ST, CF
## 3 3 Kevin De Bruyne Belgium Manchester City CM CM, CAM
## 4 5 Lionel Messi Argentina Inter Miami CF CF CF, CAM, ST
## 5 7 Karim Benzema France Al Ittihad CF CF, ST
## 6 8 Thibaut Courtois Belgium Real Madrid GK GK
## age overall PAC SHO PAS DRI DEF PHY Acceleration Sprint.Speed Positioning
## 1 25 91 97 90 80 92 36 78 97 97 93
## 2 24 91 89 93 66 80 45 88 82 94 96
## 3 33 91 72 88 94 87 65 78 72 72 88
## 4 37 90 80 87 90 94 33 64 87 74 91
## 5 36 90 79 88 83 87 39 78 78 79 92
## 6 32 90 85 89 76 93 46 90 42 52 13
## Finishing Shot.Power Long.Shots Volleys Penalties Vision Crossing
## 1 94 90 83 84 84 83 78
## 2 96 94 86 90 84 74 47
## 3 85 92 92 83 83 95 95
## 4 89 83 90 86 75 92 83
## 5 91 87 81 88 85 90 75
## 6 14 57 17 12 27 44 14
## Free.Kick.Accuracy Short.Passing Long.Passing Curve Agility Balance Reactions
## 1 69 86 71 80 93 82 93
## 2 62 77 53 77 76 72 94
## 3 83 94 94 92 74 78 92
## 4 93 91 90 93 91 95 88
## 5 73 89 76 82 77 72 92
## 6 20 33 35 19 63 45 88
## Ball.Control Dribbling Composure Interceptions Heading.Accuracy Def.Awareness
## 1 92 93 88 38 73 26
## 2 82 79 87 43 83 38
## 3 92 86 88 66 55 66
## 4 93 96 96 40 60 20
## 5 91 87 90 39 90 43
## 6 23 13 66 15 13 20
## Standing.Tackle Sliding.Tackle Jumping Stamina Strength Aggression
## 1 34 32 88 88 77 64
## 2 47 29 93 76 93 87
## 3 70 53 72 88 74 75
## 4 35 24 71 70 68 44
## 5 24 18 85 82 82 63
## 6 18 16 68 38 70 23
## height weight preferred.foot weak.foot league
## 1 182CM / 6'0"FT 75KG / 165LBS Right 4 Ligue 1 Uber Eats
## 2 195CM / 6'5"FT 94KG / 207LBS Left 3 Premier League
## 3 181CM / 5'11"FT 75KG / 165LBS Right 5 Premier League
## 4 169CM / 5'7"FT 67KG / 148LBS Left 4 MLS
## 5 185CM / 6'1"FT 81KG / 179LBS Right 4 ROSHN Saudi League
## 6 199CM / 6'6"FT 96KG / 212LBS Left 3 LALIGA EA SPORTS
## att.work.rate def.work.rate skill.moves
## 1 High Low 5
## 2 High Medium 3
## 3 High Medium 4
## 4 Low Low 4
## 5 Medium Medium 4
## 6 Medium Medium 1
## url
## 1 https://www.ea.com/games/ea-sports-fc/ratings/player-ratings/kylian-mbappe/231747
## 2 https://www.ea.com/games/ea-sports-fc/ratings/player-ratings/erling-haaland/239085
## 3 https://www.ea.com/games/ea-sports-fc/ratings/player-ratings/kevin-de-bruyne/192985
## 4 https://www.ea.com/games/ea-sports-fc/ratings/player-ratings/lionel-messi/158023
## 5 https://www.ea.com/games/ea-sports-fc/ratings/player-ratings/karim-benzema/165153
## 6 https://www.ea.com/games/ea-sports-fc/ratings/player-ratings/thibaut-courtois/192119
## GK.Diving GK.Handling GK.Kicking GK.Positioning GK.Reflexes Gender
## 1 NA NA NA NA NA M
## 2 NA NA NA NA NA M
## 3 NA NA NA NA NA M
## 4 NA NA NA NA NA M
## 5 NA NA NA NA NA M
## 6 85 89 76 90 93 M- 2)Then, we create a new dataset with numeric data, which are located between the 8th and 43rd columns. A position column was also added for better modeling. As goalkeeper overalls are correlated mostly with goalkeeper attributes, we will exclude them.
data_numeric <- data %>%
select(position, 8:43) %>%
filter(position != "GK") %>%
drop_na()- 3)Now, we exclude the position column to be left only with the attributes. We then apply standardization to the attributes.
attribute <- data_numeric %>%
select(-position)
distance <- dist(scale(attribute), method = "euclidean")- 4)We will be applying MDS for reducing high-dimensional player attributes into a 2D space. Then for visualization, MDS should be converted into dataframe and position column should be added.
fifa <- mds(distance, ndim=2)
frame <- as.data.frame(fifa$conf)
colnames(frame) <- c("Dim1", "Dim2")
frame$position <- data_numeric$position- 5)Now that we have transformed the high-dimensional player data into 2D, we can add colors and visualize them using ggplot2. Similar positions are colored in the same color.
position_colors <- c(
"ST" = "red",
"CF" = "darkred",
"RW" = "green",
"LW" = "green",
"CAM" = "yellow",
"CM" = "violet",
"CDM" = "orange",
"RM" = "blue",
"LM" = "blue",
"CB" = "black",
"LB" = "green4",
"RB" = "green4",
"LWB" = "darkkhaki",
"RWB" = "darkkhaki")
ggplot(frame, aes(x = Dim1, y = Dim2, color = position)) +
geom_point(alpha = 2) +
scale_color_manual(values = position_colors) +
labs(title = "MDS for Position Correlation")
- 6)For a better understanding, we can use K-means. 3 selected as a number of clusters, as defenders, midfielders and attackers are the main position groups.
set.seed(123)
k <- 3
clusters <- kmeans(frame[, c("Dim1", "Dim2")], centers = k, nstart = 25)
frame$cluster <- as.factor(clusters$cluster)
ggplot(frame, aes(x = Dim1, y = Dim2, color = cluster)) +
geom_point(alpha = 2, size = 3) +
scale_color_manual(values = c("red", "blue", "green")) + # Adjust colors as needed
labs(title = "MDS of Player Positions") +
theme_minimal()
- 7)We can group positions after observations and create a function that determines the correlations between the attributes.
position_groups <- list(
Attack = c("ST", "CF", "LW", "RW", "RM"),
Midfield = c("CM", "CAM", "LB", "RB", "CDM"),
Defence = c("CB"))
correlation_by_position <- function(position_group) {
subset_data <- data_numeric %>%
filter(position %in% position_group) %>%
select(-position)
cor <- cor(subset_data, use = "complete.obs")
return(cor)}- 8)After creating correlation matrices for every position, we can visualize them using a qgraph.
cor_attack <- correlation_by_position(position_groups$Attack)
cor_midfield <- correlation_by_position(position_groups$Midfield)
cor_defence <- correlation_by_position(position_groups$Defence)
qgraph(cor_attack, shape="rectangle", posCol="darkgreen", negCol="darkmagenta")
qgraph(cor_midfield, shape="rectangle", posCol="darkgreen", negCol="darkmagenta")
qgraph(cor_defence, shape="rectangle", posCol="darkgreen", negCol="darkmagenta")
- 9)For better understanding and visualization, we will use interactive heatmaps.
heatmap_data_attack <- as.data.frame(as.table(cor_attack))
plot_ly(data = heatmap_data_attack, x = ~Var1, y = ~Var2, z = ~Freq, type = "heatmap",
colors = colorRamp(c("green", "white", "red")), colorbar = list(title = "Correlation")) %>%
layout(title = "Attacke Correlation",
xaxis = list(title = "Attributes", tickangle = 45),
yaxis = list(title = "Attributes"))heatmap_data_midfield <- as.data.frame(as.table(cor_midfield))
plot_ly(data = heatmap_data_midfield, x = ~Var1, y = ~Var2, z = ~Freq, type = "heatmap",
colors = colorRamp(c("green", "white", "red")), colorbar = list(title = "Correlation")) %>%
layout(title = "Midfield Correlation",
xaxis = list(title = "Attributes", tickangle = 45),
yaxis = list(title = "Attributes"))heatmap_data_defence <- as.data.frame(as.table(cor_defence))
plot_ly(data = heatmap_data_defence, x = ~Var1, y = ~Var2, z = ~Freq, type = "heatmap",
colors = colorRamp(c("green", "white", "red")), colorbar = list(title = "Correlation")) %>%
layout(title = "Defence Correlation",
xaxis = list(title = "Attributes", tickangle = 45),
yaxis = list(title = "Attributes"))- 10)To determine which attributes affect the overall, we create a matrix of highly correlated attributes with the overall. As a next step, we convert correlation matrix to a distance matrix and apply MDS (SMACOF).
high_correlation_attack <- names(which(cor_attack["overall", ] > 0.5))
high_correlation_attack <- setdiff(high_correlation_attack, "overall")
cor_high <- cor_attack[high_correlation_attack, high_correlation_attack]
dist_matrix_attack <- dist(t(cor_high))
mds_attack <- smacofSym(dist_matrix_attack, ndim = 2)
mds_drame_attack <- data.frame(X = mds_attack$conf[,1],
Y = mds_attack$conf[,2],
Attribute = rownames(mds_attack$conf))
plot_ly(data = mds_drame_attack, x = ~X, y = ~Y, text = ~Attribute, type = 'scatter', mode = 'markers+text',
marker = list(color = 'blue', size = 10), textposition = 'bottom') %>%
layout(
title = "MDS Attack - Overall (SMACOF)")high_correlation_midfield <- names(which(cor_midfield["overall", ] > 0.5))
high_correlation_midfield <- setdiff(high_correlation_midfield, "overall")
cor_high_mid <- cor_midfield[high_correlation_midfield, high_correlation_midfield]
dist_matrix_midfield <- dist(t(cor_high_mid))
mds_midfield <- smacofSym(dist_matrix_midfield, ndim = 2)
mds_frame_midfield <- data.frame(X = mds_midfield$conf[,1],
Y = mds_midfield$conf[,2],
Attribute = rownames(mds_midfield$conf))
plot_ly(data = mds_frame_midfield, x = ~X, y = ~Y, text = ~Attribute, type = 'scatter', mode = 'markers+text',
marker = list(color = 'blue', size = 10), textposition = 'bottom') %>%
layout(
title = "MDS Midfield - Overall (SMACOF)")high_correlation_defence <- names(which(cor_defence["overall", ] > 0.5))
high_correlation_defence <- setdiff(high_correlation_defence, "overall")
cor_high_def <- cor_defence[high_correlation_defence, high_correlation_defence]
dist_matrix_def <- dist(t(cor_high_def))
mds_defence <- smacofSym(dist_matrix_def, ndim = 2)
mds_frame_defence <- data.frame(X = mds_defence$conf[,1],
Y = mds_defence$conf[,2],
Attribute = rownames(mds_defence$conf))
plot_ly(data = mds_frame_defence, x = ~X, y = ~Y, text = ~Attribute, type = 'scatter', mode = 'markers+text',
marker = list(color = 'blue', size = 10), textposition = 'bottom') %>%
layout(
title = "MDS Defence - Overall (SMACOF)")Conclusion
The graphical representations clearly illustrate the attributes correlated with overall performance. As anticipated, for attacking players, the most significant attributes are finishing and shot-related statistics, while for defenders, tackling and strength appear to be more influential. The midfield position presents an intriguing case, as there is not a strong correlation between overall performance and specific attributes. This observation is logically consistent, as midfielders can be defense- or attack-oriented, and their overall performance can be influenced by a variety of attributes.
Sources: https://www.youtube.com/watch?v=xerW2TvZHbQ https://www.r-bloggers.com/2013/01/7-functions-to-do-metric-multidimensional-scaling-in-r/ https://www.r-bloggers.com/2019/06/introduction-to-interactive-graphics-in-r-with-plotly/