Exploring How the Emotional Tone of Music Has Evolved

Introduction

Throughout history, the music we create and consume has continued to evolve, reflecting shifts in society, technology, and human expression. In this analysis, I explore how key attributes - measured and stored by Spotify - have changed over time, revealing how different periods leave their mark on the music we listen to. By examining these trends, we can uncover the ways in which cultural moments, technological advancements, and societal shifts shape the sounds of each generation.

Dataset

The dataset that I’m using can be found on Kaggle. It includes a variety of variables, many of which I have not used, but can be viewed on the Kaggle page. Some of the variables that I elected to use include ReleaseDate (renamed to Year), Duration, Explicit, Popularity, Genre, Danceability, Energy, Loudness, Speechiness, Acousticness, Instrumentalness, Liveness, Valence, and Tempo. While some of these proved to be more interesting than others, this is the final list I used. Valence is perhaps the most important variable that I looked at, as it measures the emotion behind music. A lower number indicating that the song evokes more negative emotions, while a higher number evokes more positive soundscape. Many of these variables are simply measured on a 0-1 scale. The exceptions to those include Loudness (measured in dB go from -60 to 0), Duration is measured in miliseconds (Converted to seconds when relevant), Tempo (measured by beats per minute) can range anywhere from 0-200. For a more detailed explanation of each variable, you can view the Spotify dev page here.

setwd("C://Users//Robert//Desktop//Loyola//gb736")
library(ggplot2)
library(scales)
library(RColorBrewer)
library(plyr)
library(dplyr)
library(tidyr)
library(data.table)
library(ggthemes)
library(lubridate)
library(ggrepel)
library(plotly)
library(reshape2)
library(ggridges)
data <- "R_datafiles//spotifydatanew.csv"
df <- fread(data, na.strings=c(NA, ""))

Findings

df$Year <- ifelse(nchar(df$ReleaseDate) == 4, # In the dataset the years are stored differently (2012, 01-01-2012)
                  df$ReleaseDate,             # This code checks to see if the year has 4 characters (2012), if not
                  year(mdy(df$ReleaseDate)))  # It stores the Year as only the 4 digit year

Average Valence by Decade

valence_duration_5yrs <- df %>%
  mutate(five_yr_grp = floor(as.numeric(Year) / 5) * 5) %>% 
  group_by(five_yr_grp) %>%
  summarize(
    mean_valence = mean(Valence, na.rm = TRUE),
    mean_duration = mean(Duration/1000, na.rm = TRUE)   # Convert to minutes
  )

five_yr_xlabels = min(df$Year):max(df$Year) # Make labels

min_max_valence <- df %>%
  filter(Valence == min(Valence) | Year == max(Valence)) %>%
  data.frame()

ggplot(valence_duration_5yrs, aes(x = five_yr_grp, y = mean_valence, color = mean_duration)) +
  geom_line(size = 1) +
  geom_point(size = 2) +
  scale_color_gradient(low = "red", high = "green") + 
  labs(title = "Music Emotion Trends by 5 Year Intervals (1950–2020s)",
       x = "5 Year Period", 
       y = "Mean Valence", 
       color = "Mean Duration",
       caption = "*Valence is a measure of how positive or negative a song is. Songs with a higher value sound more positive.") +
  theme_minimal() + 
  theme(plot.title = element_text(hjust = 0.5),
        plot.caption = element_text(hjust = 0.2)) + 
  scale_x_continuous(breaks = seq(min(valence_duration_5yrs$five_yr_grp), 
                                  max(valence_duration_5yrs$five_yr_grp), 
                                  by = 10)) + 
  geom_point(data = min_max_valence, aes(x = Year, y = Valence), shape = 21, 
             size = 4, fill = 'red', color = 'red') + 
  geom_label_repel(data = valence_duration_5yrs %>%
                     filter(mean_valence == min(mean_valence) | mean_valence == max(mean_valence)), 
                   aes(label = paste("Valence:", round(mean_valence, 2), 
                                     "\nDuration:", round(mean_duration, 2), "secs")), 
                   box.padding = 1, point.padding = 1, size = 4, 
                   color = 'grey50', segment.color = 'darkblue')

This chart establishes a key trend in the emotional tone of music over time by tracking valence, a measure of how positive or negative a song feels. Since 1955, there has been a clear overall decline in valence, suggesting that popular music has become less “happy” on average. The most dramatic drop occurred between 1955 and 1965, followed by a more gradual decline with occasional peaks. Additionally, the data reveals a trend in song duration—while songs have generally become longer over time, recent years show a slight reversal, hinting at a shift in music consumption and production patterns. This trend provides a strong starting point for understanding how cultural and societal changes may be reflected in the music we listen to.

Duration by Decade

df$Decades <- as.factor(floor(as.numeric(df$Year) / 10) * 10)
levels <- c('2020', '2010', '2000', '1990', '1980', '1970', '1960', '1950') # Store decades
df$Decades <- factor(df$Decades, levels = levels) # Set order


ggplot(df, aes(x = Duration/1000, y = Decades, fill = Decades)) +
  geom_density_ridges(alpha = 0.7, scale = 1.2) +
  labs(title = "Ridge Plot of Song Duration Across Decades",
       x = "Song Duration (Seconds)",
       y = "Decades",
       fill = "Decade") + 
  theme_minimal() + 
  theme(plot.title = element_text(hjust = 0.5), 
      plot.title.position = "plot") + 
  scale_x_continuous(limits = c(75, 450)) 

This ridge plot reveals how song length has evolved over the decades. In the 1950s, there was significant variation in song length, with a long tail extending toward longer durations. Over time, particularly in the late 20th century, this variation widened before narrowing again in the 2000s. This shift coincides with the rise of streaming platforms, where artists and labels may have adjusted song lengths to maximize engagement and retention. By the 2010s and 2020s, song length has become more standardized, and even shorter, reflecting a shift toward a more formulaic approach in the digital age of social media and streaming.

Feature Heat Map

df$Decades <- as.numeric(df$Decades) # Change the year columns to be numeric
df$Year <- as.numeric(df$Year)

cormat_df <- df %>%
  select(Valence, Loudness, Energy, Duration, Danceability, Popularity, Liveness, Speechiness, Mode, Tempo, Instrumentalness, Year) %>%
  data.frame() # Select the variables I want to include in the heatmap

cormat <- cor(cormat_df, use = "complete.obs")  # Avoids missing values
cormat <- round(cormat, 2)  # Rounds values for readability

melted_cormat <- melt(cormat) # Creation of the correlation heatmap

ggplot(melted_cormat, aes(x = Var1, y = Var2, fill = value)) +
  geom_tile(color = "white") +  
  geom_text(aes(label=value)) +
  theme_minimal() +
  labs(x = "",
       y = "",
       title = "Correlation Heatmap of Music Features",
       fill = "Correlation") +
  theme(axis.text.x = element_text(angle = 45, hjust = 1)) + 
  theme(plot.title = element_text(hjust = 0.5)) + 
  scale_fill_continuous(low='white', high='red')

Building upon previous analyses of valence and song duration, this correlation heatmap provides a broader perspective on how other various features interact. Notably, energy and loudness exhibit a strong correlation (0.68), suggesting that louder songs tend to have higher energy. Similarly, duration shows a weaker relationship with most features, aligning with our earlier observation that song length has slowly evolved independently over time. Importantly, valence has a slight negative correlation with year (-0.2), reinforcing our initial finding that music has become less positive over time. This heatmap helps us understand these interconnected relationships, setting the stage for deeper exploration of how different elements shape the music we listen to, and how they change over time.

Modern Trends in Tempo (Pre and Post-COVID)

df_covid <- df %>%
  select(Year, Loudness, Energy, Valence, Danceability, Duration, Popularity, Tempo) %>%
  filter(Year >= 2015) %>%
  mutate(Covid = case_when(
    Year <= 2019 ~ "Pre-Covid",
    Year >= 2020 ~ "Post-Covid")) %>%
  group_by(Covid) 

mean_values_covid <- df_covid %>%
  group_by(Year) %>%
  mutate(mean_valence = mean(Valence)) %>%
  mutate(mean_energy = mean(Energy)) %>%
  mutate(mean_duration = mean(Duration)) %>%
  mutate(mean_loudness = mean(Loudness)) %>%
  mutate(mean_tempo = mean(Tempo)) %>%
  mutate(mean_danceability = mean(Danceability)) %>%
  mutate(mean_duration = mean(Duration)/1000)

mean_values_covid$Covid <- factor(mean_values_covid$Covid, levels = c("Pre-Covid", "Post-Covid"))
mean_values_covid$Year <- as.numeric(mean_values_covid$Year)
breaks <- c(seq(min(mean_values_covid$Year), max(mean_values_covid$Year), by = 2))

ggplot(mean_values_covid, aes(x = Year, y = mean_tempo, color = mean_loudness)) +
  geom_point(size = 5, alpha = 0.8) +
  geom_vline(xintercept = 2020, linetype = "solid", color = 'black') + 
  theme_minimal() +
  labs(title = "Tempo Acceleration Pre vs. Post-COVID",
       x = "Year",
       y = "Tempo (BPM)",
       color = "Mean Loudness") +
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_x_continuous(breaks = breaks) + 
  scale_color_distiller(palette = "Oranges", direction=1)

This chart illustrates the increasing tempo of songs over time, with a notable spike around 2020. Before COVID, tempo remained relatively steady, even showing a slight decline. The post-pandemic rise in tempo can be linked to a demand for more uplifting music following a period of isolation. Additionally, the emergence of fast-paced genres like hyperpop and house music—popular on social media—may have contributed to this trend. The difference between the tempo of songs Pre-Covid and Post-Covid are statistically significant. The specifics can be seen in the “Statistics” tab.

Interestingly, loudness appears to correlate more strongly with time, as shown in the heatmap, rather than with tempo. This supports the idea that post-pandemic music has become both louder and more energetic, potentially serving as a means of uplifting society during hardship.

First Dip in Valence (Vietnam War Era)

df_vietnam <- df %>%
  mutate(vietnam = case_when(
    Year <= 1964 ~ "Pre-Vietnam War",
    Year >= 1965 & Year <= 1975 ~ "During Vietnam War",
    Year >= 1976 & Year <= 1984 ~ "Post-Vietnam War",
    TRUE ~ NA_character_
  )) %>%
  drop_na(vietnam) %>%  # Removes rows where vietnam is NA
  mutate(vietnam = factor(vietnam, levels = c("Pre-Vietnam War", "During Vietnam War", "Post-Vietnam War")))  # Set as factor with ordered levels

mean_values <- df_vietnam %>%
  group_by(vietnam) %>%
  summarise(mean_valence = mean(Valence))

ggplot(df_vietnam, aes(x = vietnam, y = Valence, fill = vietnam)) +
  geom_violin(trim = FALSE, alpha = 0.7) + 
  geom_boxplot(width = 0.1, alpha = 0.5) +
  stat_summary(fun = mean, geom = "point", shape = 23, size = 2, color = "black", fill = "yellow") + 
  geom_label(data = mean_values, aes(x = vietnam, y = mean_valence, label = round(mean_valence, 2)), 
             fill = "white", color = "black", vjust = -1, size = 3, fontface = "bold") + 
  labs(title = "Valence Distribution by Vietnam War Era", 
       x = "Era", 
       y = "Valence", 
       caption = "*Valence is a measure of how positive or negative a song is. Songs with a higher value sound more positive.",
       fill = "Era") +
  theme_minimal() +
  theme(plot.title = element_text(hjust = 0.5),
        plot.caption = element_text(hjust = 0.1))

This violin plot expands on the initial observation of valence decline by breaking it down into three distinct periods: before, during, and after the Vietnam War. A violin plot combines a box plot and a density plot, showing both the spread of data and where values are concentrated. The width of each section represents the frequency of songs with a given valence score, while the white dot marks the median.

Before the war (Pre-1964), songs had an average valence of 0.74, with a wider top section indicating that many songs were highly positive. During the war (1965–1975), valence dropped to 0.64, and the distribution became more even, meaning fewer songs had extremely high positivity. This could suggest a shift toward more serious or politically charged music. In the post-war period (1976–1984), valence rose slightly to 0.68, and the shape of the violin plot shows a concentration of higher-valence songs, though not returning to pre-war levels. This visualization helps illustrate how historical events may have influenced the emotional tone and intent of music, with wartime music showing a broader distribution and a dip in positivity, followed by a gradual rebound in the years after. While these differences are relatively small, they are statistically significant. The specific statistics can be viewed in the “Statistics” tab.

Statistics

t.test(Tempo ~ Covid, data = df_covid)

## 
##  Welch Two Sample t-test
## 
## data:  Tempo by Covid
## t = 2.4419, df = 1150.6, p-value = 0.01476
## alternative hypothesis: true difference in means between group Post-Covid and group Pre-Covid is not equal to 0
## 95 percent confidence interval:
##  0.5897935 5.4124302
## sample estimates:
## mean in group Post-Covid  mean in group Pre-Covid 
##                 121.7363                 118.7352

anova <- aov(Valence ~ vietnam, data = df_vietnam)
summary(anova)

##               Df Sum Sq Mean Sq F value   Pr(>F)    
## vietnam        2   1.55  0.7737   14.92 3.82e-07 ***
## Residuals   1576  81.74  0.0519                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

tukey_result = TukeyHSD(anova)
tukey_result$vietnam

##                                            diff          lwr         upr
## During Vietnam War-Pre-Vietnam War  -0.10096439 -0.145456915 -0.05647187
## Post-Vietnam War-Pre-Vietnam War    -0.06587473 -0.110308047 -0.02144142
## Post-Vietnam War-During Vietnam War  0.03508966  0.006501166  0.06367815
##                                            p adj
## During Vietnam War-Pre-Vietnam War  3.488428e-07
## Post-Vietnam War-Pre-Vietnam War    1.504911e-03
## Post-Vietnam War-During Vietnam War 1.125452e-02

The t-test results indicate a statistically significant difference in Tempo between songs released Pre- and Post-Covid (p = 0.01476 < 0.05). This suggests that the average Tempo of songs changed after Covid.

For the Valence during war periods, the ANOVA test (F = 14.92, p < 0.05) indicates a statistically significant difference among the Pre, During, and Post-Vietnam War groups. Post-hoc comparisons show that all group differences are statistically significant, with the largest difference observed between the Pre and During-Vietnam War periods.

Conclusion

I first want to discuss two of the variables that were left out of this analysis due to limitations in their interpretability or practicality. Popularity was initially considered but proved unsuitable since Spotify’s data reflects current-day streaming habits rather than historical popularity. While a separate study could explore how older songs resurface in popularity today, it was not relevant to this analysis. Genre was interesting but difficult to work with as songs often had multiple genre labels, leading to redundancy and an impractically large dataset. Additionally, the overlap of sub-genres made it challenging to create meaningful groupings without losing their distinct characteristics.

Ultimately, this analysis explored broad trends in music’s emotional tone, duration, and tempo, shedding light on how these characteristics have evolved over time. While some variables presented challenges in integration, they open the door for topics to study in a separate analysis. The patterns identified in this analysis suggest meaningful shifts in musical expression, potentially influenced by cultural, technological, or societal changes. Further exploration could examine genre-specific trends, listener preferences, or additional external factors such as specific shifts in music production technology, changes in audience demographics, and the continuously growing impact of social media and virality on music consumption.