Introduction

There are many different factors that help a musical become popular. Catchy songs, cool visuals, and a star-studded cast are a few examples. But is there a way to tell if a musical will be popular before it even opens on Broadway? Can you predict if a musical will be good based on the sentiment of the lyrics? For my text analysis project, I will be analyzing the most streamed musical soundtracks to see if their message appears to be more positive or negative and determine if sentiment affected their popularity.

I decided to focused on most streamed soundtracks instead of most performances or most Tony awards for a few reasons. First, focusing on streams allows me to not consider dialogue. I could not get access to the scripts for all musicals, which means I could not analyze the sentiment of the words spoken. Streams on Spotify is the only metric that allows me to consider just lyrics as opposed to lyrics and dialogue. Second, streaming allows me to focus on the popularity of musicals without considering if listeners have seen the show. Why are people drawn to these musicals even if they don’t have access to what is being said between songs? I can analyze the lyrics to see if lyric sentiment plays a role in popularity.

I will be using the top four most streamed musical soundtracks of the 2010s according to Playbill for this analysis. The link to this list can be found here: https://www.playbill.com/article/the-10-most-streamed-cast-albums-of-2019-and-of-the-decade. According to this source, the most streamed soundtracks of the 2010s are Hamilton, Dear Evan Hansen, Wicked, and Be More Chill.

Before I begin my analysis, here are the packages I will be using in my code.

library(tidyverse)
library(tidytext)
library(ggthemes)
library(genius)

Hamilton

Hamilton tells the story of one of America’s founding fathers, Alexander Hamilton. Written by Lin-Manuel Miranda, the show has won numerous awards, including 11 Tonys. The show also has three different tours, four locations outside of Broadway, and a professional recording on Disney Plus. To learn more about the musical, here is the Wikipedia link: https://en.wikipedia.org/wiki/Hamilton_(musical).

Though I mentioned I did not have access to the scripts, Hamilton is the exception to this rule. Almost of the words in the show are sung, which means I can analyze almost the entire show. The first thing I did was load the lyrics from Genius.

hamilton <- genius_album("Lin-Manuel Miranda", "Hamilton: An American Musical (Original Broadway Cast Recording)")

From there, I removed any stop words, such as “you” and “I,” and one syllable words like “la” and “ah”. I just wanted to focus on any words that had sentiment attached, hence why I removed those words. To get a sense of what the sentiment calculations would predict, I looked at the top 15 words sung in Hamilton.

hamilton %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>% 
  filter(!word %in% c("da","em", "whoa")) %>% 
  group_by(word) %>% 
  count() %>% 
  arrange(desc(n)) %>% 
  head(15)
## Joining, by = "word"
## # A tibble: 15 x 2
## # Groups:   word [15]
##    word          n
##    <chr>     <int>
##  1 wait         81
##  2 time         79
##  3 hamilton     77
##  4 hey          71
##  5 burr         63
##  6 shot         58
##  7 sir          56
##  8 alexander    50
##  9 gonna        38
## 10 president    38
## 11 rise         37
## 12 world        36
## 13 satisfied    35
## 14 story        35
## 15 alive        34

The top 15 words appeared to be mostly neutral, so I suspected the sentiment could go either way. I then moved on to calculate the sentiment using the bing and afinn lexicons.

Lexicon Calculations

Bing classifies words as positive or negative, so I used this lexicon to analyze how many positive and how many negative words are sung throughout Hamilton.

hamilton %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>%
  inner_join(get_sentiments("bing")) -> hamilton_bing
## Joining, by = "word"
## Joining, by = "word"
hamilton_bing %>% 
  count(sentiment, sort = TRUE)
## # A tibble: 2 x 2
##   sentiment     n
##   <chr>     <int>
## 1 negative    620
## 2 positive    530

As you can see, more negative words are sung than positive ones, which made me believe Hamilton would be a negative musical. Since bing does not assign value to the words, I could not tell how positive or how negative the words were. In order to say that a musical was positive or negative, I used a lexicon that would be give me values: the afinn lexicon.

Afinn not only classifies words as positive and negative, but it also assigns words a value. The values range from -5 to 5, with positive values indicating positive words and vice versa. While the words included in the two lexicons are not the exact same, there was enough overlap for me to conclude if the musical was positive or negative. I then graphed the sentiment values to see which values were the most common.

hamilton %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>%
  inner_join(get_sentiments("afinn")) -> hamilton_afinn
## Joining, by = "word"
## Joining, by = "word"
hamilton_afinn %>% 
  count(value, sort = TRUE) %>% 
  ggplot(aes(reorder(value, -n), n)) + geom_col() + theme_minimal()

The most common value was -2, which was somewhat surprising to me. I had expected a negative value to be the most popular since there were more negative words. However, I thought -1 would be the most common value. I knew the extreme values would be uncommon, but I had figured the values closest to 0, like 1 and -1, to occur more often.

In order to compare the musicals to one another, I found the average value of the sentiment values. I used this value to see how positive or how negative the lyrics are.

mean(hamilton_afinn$value)
## [1] -0.05334539

The average did come out negative, which means Hamilton would be classified as a negative musical. However, the low value shows that Hamilton is probably not going to be the most negative musical on this list.

Dear Evan Hansen

Dear Evan Hansen is about a teenager named Evan Hansen who deals with social anxiety. Through a series of misunderstandings, Evan is believed to be the best friend of a classmate who had died by suicide. He decides to go along with the lies until he starts to feel like he is living a double life. To learn more about the musical, here is the Wikipedia link: https://en.wikipedia.org/wiki/Dear_Evan_Hansen I am more familiar with this show than Hamilton, so I had some assumptions before I started analyzing the lyrics. I repeated the process I had stated above when examining Hamilton, which started with getting the lyrics.

dearevan <- genius_album("Original Broadway Cast of Dear Evan Hansen", "Dear Evan Hansen (Original Broadway Cast Recording)")

After getting the lyrics, I removed the stop words and one syllable words. Dear Evan Hansen had different one syllable words than Hamilton, hence why the two filters are not the same.

dearevan %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>%
  filter(!word %in% "ah") %>% 
  group_by(word) %>% 
  count() %>%
  arrange(desc(n)) %>% 
  head(15)
## Joining, by = "word"
## # A tibble: 15 x 2
## # Groups:   word [15]
##    word          n
##    <chr>     <int>
##  1 disappear    27
##  2 deserves     19
##  3 forever      19
##  4 found        19
##  5 world        19
##  6 connor       17
##  7 day          15
##  8 sun          14
##  9 evan         13
## 10 waving       12
## 11 perfect      11
## 12 step         11
## 13 start        10
## 14 hansen        9
## 15 hear          9

Dear Evan Hansen appeared to be more negative than Hamilton based on this list. I did not find this observation surprising since Dear Evan Hansen deals with harder topics like mental health and suicide. However, dealing with negative topics does not mean all the lyrics are super negative. I had to prove the musical was negative from a lyrical standpoint and not just a plot standpoint.

Lexicon Calculations

In order to prove my assumption about Dear Evan Hansen being a negative musical, I first counted the positive and negative words using the bing lexicon.

dearevan %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>%
  inner_join(get_sentiments("bing")) -> dearevan_bing
## Joining, by = "word"
## Joining, by = "word"
dearevan_bing %>% 
  count(sentiment, sort = TRUE)
## # A tibble: 2 x 2
##   sentiment     n
##   <chr>     <int>
## 1 negative    140
## 2 positive     90

As expected, Dear Evan Hansen had more negative words. The show also has the fewest sentimental lyrics of all the shows I analyzed, so I wondered if fewer words would impact the score. I then moved on to analyze the lyrics in the afinn lexicon:

dearevan %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>%
  inner_join(get_sentiments("afinn")) -> dearevan_afinn
## Joining, by = "word"
## Joining, by = "word"
dearevan_afinn %>% 
  count(value, sort = TRUE) %>% 
  ggplot(aes(reorder(value, -n), n)) + geom_col() + theme_minimal()

A negative value was the most common value again, but this time it was -1. I also found it interesting that 1 was not the second most common value. With Hamilton, the corresponding values came in pairs. You can see this trend in Dear Evan Hansen with some of the values too. In every other instance, the positive value is more common than the negative one. Because of that observation, I started questioning if Dear Evan Hansen would be perceived as a positive or negative musical. I decided to get the average sentiment score to determine if my earlier assumptions were correct.

mean(dearevan_afinn$value)
## [1] 0.1134752

Despite having more negative words and dealing with serious topics, Dear Evan Hansen received a positive score. One reason I theorized this result is because the lexicons cannot analyze context. Take the song “Sincerely, Me” from the show. The word sincerely is said seven times receives a score of 2 according to afinn. However, sincerely is used as a way to sign off on emails. Because I could not provide context, the lexicons assumed sincerely was used as an endearing term each time. I assumed this trend occurred in multiple instances throughout the show, but this one was the easiest to pinpoint.

While the lyrics of Dear Evan Hansen may have a positive sentiment, I would not classify the show itself as a positive musical. Having seen the show live, there are definitely scenes on stage that are meant to make the audience cry. Therefore, I would say Dear Evan Hansen would be a negative musical with positive lyrics. The show may seem negative on the outside, but there will be an inspiring message by the end.

Wicked

Wicked is a re-imagined tale of The Wizard of Oz. Long before Dorothy comes to Oz, Glinda the Good Witch and the Wicked Witch of the West were classmates at Shiz University. Although in those days, the Wicked Witch went by Elphabea. The two become an unlikely pair throughout their time together at school until Elphabea becomes “wicked.” At that point, Glinda struggles to stay good or stay true to her friend. To learn more about the musical, here is the Wikipedia link: https://en.wikipedia.org/wiki/Wicked_(musical) Like the previous two musicals, I started by getting the lyrics from Genius.

wicked <- genius_album("Original Broadway Cast of Wicked", "Wicked(Original Broadway Cast Recording)")

Unlike the previous two musicals, Wicked had no one syllable words that interfered with the data. I just removed the stop words this time.

wicked %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>% 
  group_by(word) %>% 
  count() %>%
  arrange(desc(n)) %>% 
  head(15)
## Joining, by = "word"
## # A tibble: 15 x 2
## # Groups:   word [15]
##    word          n
##    <chr>     <int>
##  1 wicked       30
##  2 wizard       25
##  3 wonderful    23
##  4 life         21
##  5 loathing     20
##  6 oz           20
##  7 happy        18
##  8 day          16
##  9 goodness     16
## 10 hope         16
## 11 nessa        15
## 12 girl         14
## 13 elphaba      13
## 14 glinda       13
## 15 happier      13

Wicked appears to have the most sentimental words out of the lists I had analyzed so far. There were words that were on both ends of the spectrum, so this musical could go either way.

Lexicon Calculations

To get a better sense of which direction Wicked would lean, I counted the positive and negative words.

wicked %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>%
  inner_join(get_sentiments("bing")) -> wicked_bing
## Joining, by = "word"
## Joining, by = "word"
wicked_bing %>% 
  count(sentiment, sort = TRUE)
## # A tibble: 2 x 2
##   sentiment     n
##   <chr>     <int>
## 1 negative    241
## 2 positive    225

Wicked did have more negative words, but the difference was only 19 words. More negative words also does not automatically mean it’s a negative musical, as seen with Dear Evan Hansen. I needed to get a sense of the sentiment value to conclude if Wicked was more positive or negative.

wicked %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>%
  inner_join(get_sentiments("afinn")) -> wicked_afinn
## Joining, by = "word"
## Joining, by = "word"
wicked_afinn %>% 
  count(value, sort = TRUE) %>% 
  ggplot(aes(reorder(value, -n), n)) + geom_col()

Like the previous two shows, the most common value was negative. However, Wicked appears to have the most words with a positive value. The lowest negative value is -3 while the highest positive value is 4. I predicted Wicked would have a positive score, so I calculated the average value to confirm my assumptions.

mean(wicked_afinn$value)
## [1] 0.2759382

As expected, Wicked is a positive musical. Having this show in person too, I agree with this conclusion. While there are some sad moments, there are way more happy ones throughout the show. The score is twice as high as Dear Evan Hansen’s score as well, indicating the show is more positive at least from a lyric standpoint.

Be More Chill

Be More Chill tells the story about a high school junior named Jeremy who wants to fit in with his peers. When a popular boy named Rich tells Jeremy about his method to become popular, Jeremy accepts his offer to follow down the same path. The problem is Rich took a pill called a “Super Quantum Unit Intel Processor,” or SQUIP for short. Now Jeremy is faced with a challenge: listen to his SQUIP or stay true to himself. To learn more about the musical, here is the Wikipedia link: https://en.wikipedia.org/wiki/Be_More_Chill_(musical)

Unlike the previous three musicals, I will not be analyzing the Original Broadway Cast Recording. There is an OBC soundtrack for Be More Chill from its run on Broadway in 2019. However, the musical actually premiered four years earlier at the Two River Theater in New Jersey in 2015. The popularity of this soundtrack is what allowed Be More Chill to get to Broadway. Even though some changes were made to the OBC soundtrack, I chose to stick with the Two River soundtrack since this version was more popular among fans.

bmc <- genius_album("Be More Chill Ensemble", "Be More Chill(Original Cast Recording)")

After removing the stop words, I removed the most common one syllable words as well as a few filler words. I wanted to try and get as many words with sentiment as possible since I know the least about this show.

bmc %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>%
  filter(!word %in% c("ah", "na", "la", "wanna", "woah", "gotta")) %>% 
  group_by(word) %>% 
  count() %>% 
  arrange(desc(n)) %>% 
  head(15)
## Joining, by = "word"
## # A tibble: 15 x 2
## # Groups:   word [15]
##    word          n
##    <chr>     <int>
##  1 jeremy       37
##  2 upgrade      36
##  3 yeah         36
##  4 michael      29
##  5 rich         28
##  6 christine    23
##  7 fire         22
##  8 guy          22
##  9 set          21
## 10 love         19
## 11 burned       18
## 12 wait         18
## 13 time         17
## 14 hey          16
## 15 knock        16

The words appeared to be mostly neutral. Be More Chill had the most character names in the list, which I was tempted to remove. Since I had not removed any names from the last three musicals, I kept them in to keep my analysis consistent. Since the words seemed the most neutral and I am not familiar with all of the songs, I had no assumptions about how the musical would be rated.

Lexicon Calculations

I counted all the positive and negative words to see if I could get a better sense of how the musical would be rated.

bmc %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>%
  inner_join(get_sentiments("bing")) -> bmc_bing
## Joining, by = "word"
## Joining, by = "word"
bmc_bing %>% 
  count(sentiment, sort = TRUE)
## # A tibble: 2 x 2
##   sentiment     n
##   <chr>     <int>
## 1 negative    308
## 2 positive    172

Be More Chill did have more negative words, as did the previous three musicals I analyzed. However, there were almost twice as many negative words as there were positive ones. The difference was also the largest of the four musicals. I started to suspect Be More Chill would end up being classified as a negative musical. I needed to evaluate the sentiment values to confirm my suspicions.

bmc %>% 
  unnest_tokens(word,lyric) %>% 
  anti_join(stop_words) %>%
  inner_join(get_sentiments("afinn")) -> bmc_afinn
## Joining, by = "word"
## Joining, by = "word"
bmc_afinn %>% 
  count(value, sort = TRUE) %>% 
  ggplot(aes(reorder(value, -n), n)) + geom_col()

As expected, the most common value was negative. Unlike the previous musicals, the negative value was almost always more common than the positive one. Since both lexicons indicated Be More Chill is more negative, I predicted the average sentiment would end up being negative as well.

mean(bmc_afinn$value)
## [1] -0.1478261

As expected, this version of Be More Chill is more negative. I am unaware of what lyric changes were made to the OBC soundtrack, so I do not know if the score would be the same if I analyzed that one. I would assume the changes are minor enough to keep the average value negative, so I would classify Be More Chill as a negative musical.

Conclusion

After reviewing these four soundtracks, I can say that positivity and negativity does not effect a musical’s popularity on streaming platforms. There were a mix of positive and negative musicals. However, I will admit my analysis could not be as thorough as I had hoped. I could not provide context for lyrics, which would have been helpful for some shows. My context for each show also varied wildly, and that definitely impacted my analysis. I could more into detail for some shows to explain trends I saw, whereas I had to make assumptions based on reading the Wikipedia page for others. Regardless of my obstacles, I would say lyric sentiment is not a major factor when determining a musical’s success. As long as the songs are good and the story is compelling, any musical will be bound to have success on Spotify.