Update 10/04/2021 I’ve updated this analysis to include evermore, and the ‘From the Vault’ tracks from the Fearless re-release. I’ve renamed Fearless to Fearless (TV) because we stan artists owning their own work.

With the surprise release of folklore, I decided it was time I did some text analysis on Taylor Swift lyrics. She’s lauded for her songwriting skills, so let’s take a look into what she likes to write about. Stick folklore on, grab a cuppa and let’s have a look.

If you want to have a go at this yourself, I’ve included the code (if you’re a complete newbie, this was done in a program called R Studio). If you don’t, you can just ignore the code - you absolutely don’t need to understand it to understand what’s going on here.

Resources & library

If you want to learn more or have a go yourself, here are a couple of resources that helped me.

And here are the packages you will need. For the actual lyrics, I created a CSV with the lyrics in - if you want it, drop me a tweet. But, if you click on the ‘Data visualisation…’ link above, they grabbed the lyrics via the Genius API.

library(tidytext)
library(tidyverse)
library(stm)
library(quanteda)
library(wordcloud)
library(reshape2)
library(ggplot2)
library(geometry)
library(Rtsne)
library(rsvd)
library(syuzhet)
library(scales)
## 
## Attaching package: 'scales'
## The following object is masked from 'package:syuzhet':
## 
##     rescale
## The following object is masked from 'package:purrr':
## 
##     discard
## The following object is masked from 'package:readr':
## 
##     col_factor

What is sentiment and emotion analysis?

Sentiment covers whether text is positive or negative, and emotion looks at the emotions (happy, sad, angry, etc) in text. To do this, we’re using a sentiment dictionary which assigns a sentiment and emotion to each word. We input the lyrics, and we can look at what the most commonly used positive words are, or which songs are considered to be the angriest.

Colour palette

I made a colour palette based off main colours used on Taylor Swift’s album covers. For folklore, I went a little bit extra and have set up a colour for each of the eight coloured vinyl records that are currently on her website. At some point, I will get around to putting this colour palette up somewhere for people to download but if you want it in the meantime, tweet me and I’ll send you the code.

Datacleaning

The first thing we need to do is remove stop words. These are words such as ‘yeah’, ‘oh’, etc. We can do that by importing a dataset of stop words. However, it’s always worth checking the output as there’s usually a word or two that needs to be excluded manually that is missed by the stop words data set. The table below is scrollable, by the way.

data(stop_words)

tidy_lyrics <- Lyrics %>%
   unnest_tokens(word, Lyrics) %>% 
   anti_join(stop_words, by=c("word"="word")) 

# Look at the most popular words
tidy_lyrics %>%
  count(word, sort = TRUE) %>%
  kable(align = "c")%>%
  kable_styling(bootstrap_options = c("striped", "condensed","responsive", "bordered")) %>%
  row_spec(0, background = tswift_cols("selftitledblue"), align = "c")%>%
  add_header_above(c("Most popular words in Taylor Swift songs"= 2), bold = TRUE)  %>% 
  scroll_box(width = "500px", height = "400px")
Most popular words in Taylor Swift songs
word n
love 272
time 254
baby 180
wanna 163
ooh 149
stay 120
yeah 119
gonna 111
night 102
ah 101
bad 87
home 85
girl 83
eyes 79
feel 76
shake 73
call 66
break 65
life 64
red 63
heart 62
day 61
remember 61
uh 61
leave 59
mind 57
dancing 52
ha 52
forever 51
smile 51
beautiful 49
hey 49
friends 48
mad 48
car 47
hope 47
hate 46
hands 45
mine 45
walk 45
door 44
fall 44
lost 44
hand 43
head 43
left 43
run 43
lights 42
waiting 42
feeling 41
daylight 40
talk 40
wrong 39
light 38
meet 38
miss 38
should’ve 38
town 38
woods 38
york 38
watch 37
dreams 36
dress 36
rain 36
whoa 36
world 36
hold 35
coming 34
people 34
play 34
street 34
dark 33
kiss 33
met 33
nice 33
bye 32
hear 32
trouble 32
wait 32
forget 31
live 31
blue 30
darling 30
lose 30
body 28
fight 28
hard 28
heard 28
late 28
song 28
fly 27
loved 27
phone 27
times 27
honey 26
someday 26
soul 26
touch 26
change 25
dead 25
fine 25
follow 25
found 25
gotta 25
grow 25
story 25
drive 24
fun 24
save 24
twenty 24
crazy 23
dance 23
ground 23
guess 23
stop 23
boy 22
cool 22
fake 22
happy 22
jump 22
makes 22
starlight 22
summer 22
true 22
anymore 21
dream 21
friend 21
front 21
getaway 21
lips 21
sad 21
shine 21
tied 21
window 21
wonderland 21
alright 20
babe 20
breathe 20
died 20
games 20
house 20
stand 20
blood 19
bout 19
cold 19
eye 19
hair 19
morning 19
past 19
picture 19
road 19
standing 19
til 19
tonight 19
walked 19
would’ve 19
clean 18
close 18
days 18
delicate 18
girls 18
god 18
middle 18
mmm 18
pain 18
perfectly 18
sun 18
wishing 18
begin 17
burn 17
city 17
deep 17
easy 17
ey 17
finally 17
line 17
moment 17
perfect 17
read 17
real 17
shame 17
single 17
sky 17
stood 17
thinking 17
told 17
walking 17
words 17
beat 16
begging 16
belong 16
bet 16
cornelia 16
cut 16
endgame 16
feels 16
free 16
goodbye 16
held 16
laughing 16
lucky 16
missing 16
pretty 16
tired 16
bless 15
die 15
em 15
fancy 15
happiness 15
hurt 15
list 15
party 15
plans 15
screaming 15
sick 15
simple 15
stars 15
trust 15
white 15
worse 15
bed 14
black 14
boys 14
caught 14
favorite 14
feet 14
laugh 14
sing 14
stupid 14
taking 14
tears 14
watched 14
worst 14
worth 14
alive 13
bought 13
breath 13
bright 13
burning 13
called 13
cry 13
december 13
drunk 13
eh 13
fell 13
floor 13
learned 13
losing 13
lot 13
magic 13
nights 13
rains 13
reason 13
rest 13
signs 13
string 13
undone 13
wondering 13
burned 12
changed 12
could’ve 12
cruel 12
eeh 12
fire 12
gold 12
golden 12
gorgeous 12
hide 12
hit 12
insane 12
loving 12
moved 12
paint 12
paper 12
pretend 12
prove 12
reputation 12
running 12
rush 12
school 12
sit 12
speak 12
team 12
usin 12
wake 12
walls 12
win 12
write 12
blame 11
broke 11
catch 11
damn 11
drug 11
evermore 11
funny 11
green 11
looked 11
lover 11
marvelous 11
memories 11
mm 11
passed 11
pouring 11
promise 11
pulled 11
ready 11
silence 11
sleep 11
slow 11
sparks 11
step 11
takes 11
talking 11
throw 11
waitin 11
water 11
wine 11
woman 11
american 10
arms 10
blind 10
brought 10
busy 10
closure 10
cried 10
crying 10
daydream 10
doin 10
faith 10
fast 10
fifteen 10
figured 10
forgot 10
game 10
garden 10
hang 10
heaven 10
huh 10
inside 10
invisible 10
living 10
lord 10
mess 10
million 10
months 10
nobody’s 10
scared 10
scene 10
starts 10
style 10
wide 10
wild 10
wildest 10
word 10
worship 10
ya 10
afraid 9
bar 9
battle 9
champagne 9
chance 9
covered 9
cross 9
cuts 9
death 9
dressed 9
drop 9
giving 9
half 9
heartbreak 9
holding 9
killing 9
la 9
lead 9
letter 9
london 9
lookin 9
loud 9
low 9
mistakes 9
money 9
movie 9
page 9
peace 9
pictures 9
rings 9
rumors 9
shaking 9
shit 9
short 9
sign 9
space 9
started 9
stayed 9
strong 9
swear 9
sweet 9
threw 9
truck 9
a.m 8
brand 8
brave 8
breakin 8
bring 8
calm 8
care 8
cheeks 8
comin 8
dorothea 8
falling 8
fearless 8
fighting 8
film 8
flames 8
flashback 8
flew 8
flying 8
haunt 8
heels 8
hoping 8
innocent 8
island 8
jeans 8
kid 8
kill 8
king 8
knees 8
leads 8
letting 8
move 8
pieces 8
playing 8
queen 8
quiet 8
rose 8
round 8
scar 8
shined 8
singing 8
sittin 8
sitting 8
spinning 8
spot 8
standin 8
strike 8
telling 8
thousand 8
till 8
train 8
treacherous 8
tryna 8
understand 8
warm 8
wear 8
wreck 8
actress 7
air 7
believed 7
bone 7
broken 7
choice 7
clothes 7
colder 7
conversation 7
dear 7
drowning 7
faded 7
fate 7
fears 7
forgiveness 7
freedom 7
glass 7
grace 7
gray 7
hallelujah 7
haunted 7
kingdom 7
leaving 7
lie 7
lipstick 7
lived 7
lives 7
lonely 7
loves 7
marry 7
nothin 7
palm 7
pay 7
pick 7
players 7
ran 7
realized 7
ring 7
rocks 7
roll 7
rules 7
seat 7
shining 7
someone’s 7
songs 7
spend 7
stairs 7
start 7
storm 7
superman 7
thinkin 7
tight 7
truth 7
tryin 7
view 7
wall 7
warning 7
wasted 7
wearing 7
whisper 7
why’d 7
wind 7
amount 6
angel 6
august 6
baby’s 6
band 6
breathin 6
brighter 6
casually 6
chasing 6
chill 6
color 6
counting 6
crime 6
crowd 6
daughter 6
dim 6
drivin 6
dying 6
enchanted 6
existed 6
false 6
family 6
faster 6
frames 6
frozen 6
goddamn 6
grown 6
hallway 6
heartbeat 6
hearts 6
hell 6
hometown 6
honestly 6
hoo 6
hung 6
hurts 6
joke 6
kids 6
kinda 6
lies 6
likes 6
listen 6
lock 6
love’s 6
lovers 6
lovin 6
lying 6
mcgraw 6
motion 6
names 6
prince 6
pushed 6
radio 6
reasons 6
reckless 6
ride 6
ridin 6
romantics 6
romeo 6
scars 6
scream 6
screams 6
secret 6
secrets 6
shade 6
shape 6
shiny 6
shirt 6
sight 6
skies 6
skin 6
smiles 6
sound 6
stone 6
struck 6
tall 6
tim 6
tragic 6
trees 6
weekend 6
windows 6
ache 5
age 5
ahh 5
assume 5
attitude 5
beneath 5
betty 5
bigger 5
bones 5
breaking 5
breaks 5
burnin 5
careful 5
careless 5
castle 5
catching 5
child 5
church 5
colors 5
corner 5
crashing 5
crowded 5
crowds 5
cryin 5
daddy’s 5
desperately 5
dinner 5
drama 5
dreaming 5
drew 5
dropped 5
dust 5
father 5
fear 5
fit 5
fits 5
flowers 5
footsteps 5
forward 5
ghost 5
grab 5
grey 5
guitar 5
guys 5
hanging 5
happened 5
headlights 5
hole 5
horse 5
impossible 5
ivy 5
james 5
jumping 5
keeping 5
keys 5
knife 5
knocked 5
lasts 5
lyin 5
man’s 5
match 5
matter 5
messed 5
midnight 5
misery 5
missed 5
moments 5
moon 5
moves 5
music 5
na 5
note 5
ocean 5
park 5
pass 5
picking 5
played 5
pocket 5
pool 5
porch 5
pretenders 5
pride 5
princess 5
putting 5
rainy 5
realize 5
recall 5
reminds 5
revenge 5
ruining 5
saint 5
season 5
seventeen 5
ships 5
sidewalk 5
sinking 5
sleeping 5
slope 5
solve 5
somethin 5
spelling 5
spotlight 5
stare 5
starin 5
stealing 5
stephen 5
stories 5
straight 5
strange 5
stranger 5
strangers 5
sunset 5
superstar 5
supposed 5
swing 5
swore 5
talks 5
tea 5
tolerate 5
top 5
tossing 5
trace 5
twisted 5
understands 5
vow 5
watching 5
water’s 5
waves 5
weather 5
whispers 5
woke 5
yesterday 5
2 4
accidents 4
act 4
affair 4
afternoon 4
ago 4
angry 4
autumn 4
bedroom 4
blink 4
block 4
blues 4
boarded 4
bottle 4
bottles 4
bricks 4
brother 4
build 4
buttons 4
cab 4
café 4
calling 4
cardigan 4
carry 4
cars 4
chandelier 4
changing 4
check 4
christmas 4
closer 4
combat 4
confused 4
counted 4
cover 4
cowboy 4
crashed 4
cursing 4
darlin 4
date 4
dice 4
dirty 4
drag 4
dreamland 4
drink 4
este’s 4
eyed 4
fakers 4
familiar 4
feelin 4
figure 4
flash 4
flawless 4
furious 4
gate 4
glow 4
grew 4
grows 4
guy 4
haters 4
history 4
holy 4
hunters 4
hush 4
imagined 4
impress 4
indifference 4
internet 4
jet 4
john 4
kissin 4
kitchen 4
lakes 4
land 4
laughed 4
lay 4
leaves 4
lines 4
lip 4
locked 4
magical 4
mall 4
mark 4
meant 4
memory 4
mirror 4
mirrorball 4
mistake 4
month 4
mystery 4
neck 4
nines 4
notice 4
one’s 4
painted 4
parents 4
parties 4
patch 4
patience 4
pebbles 4
picked 4
pickup 4
piece 4
pining 4
pockets 4
power 4
pretended 4
price 4
pull 4
push 4
queens 4
quicker 4
rabbit 4
rebel 4
recognize 4
redneck 4
remind 4
riding 4
roof 4
roots 4
rosy 4
rough 4
screamed 4
screamin 4
screen 4
sense 4
set 4
setting 4
shattered 4
shotgun 4
sinks 4
sirens 4
skipping 4
slamming 4
sleeve 4
slipped 4
sneaking 4
snow 4
snuck 4
something’s 4
soundtrack 4
spending 4
staring 4
starring 4
stick 4
stray 4
stronger 4
sunshine 4
sympathy 4
tallest 4
taste 4
taylor 4
tear 4
tells 4
ten 4
throwing 4
tiptoes 4
tires 4
tis 4
toast 4
tragedy 4
tricks 4
trip 4
trusts 4
tuesday 4
tune 4
twist 4
untouchable 4
voice 4
war 4
wednesday 4
west 4
worry 4
wrap 4
wrapped 4
writing 4
16th 3
aah 3
afterglow 3
ahead 3
amen 3
americana 3
anthem 3
apartment 3
applause 3
archer 3
arm 3
asleep 3
avenue 3
awake 3
ball 3
bathroom 3
beach 3
bear 3
bedpost 3
bedsheets 3
beloved 3
bench 3
bent 3
birthday 3
blank 3
bleachers 3
blew 3
blushing 3
born 3
breakable 3
breathless 3
breeze 3
bulletproof 3
buy 3
canceled 3
card 3
carve 3
celebrated 3
centerfold 3
chain 3
chair 3
chase 3
chest 3
circus 3
classic 3
cleaning 3
cliffside 3
club 3
coaster 3
coat 3
concerned 3
count 3
crawling 3
crown 3
crush 3
cursed 3
cursin 3
dad 3
daddy 3
danced 3
darkest 3
dean 3
defending 3
defense 3
deserve 3
dies 3
disappointments 3
ditch 3
divide 3
dog 3
doors 3
double 3
doubt 3
drawer 3
drawing 3
dreamed 3
dreamer 3
dreamin 3
drinkin 3
driving 3
dyin 3
easier 3
echoes 3
empty 3
enchanting 3
enemies 3
enjoy 3
exes 3
exile 3
fade 3
fades 3
falls 3
fantasy 3
farm 3
field 3
fightin 3
filled 3
fix 3
flashbacks 3
flaws 3
flickering 3
flight 3
floors 3
flush 3
flyin 3
folk 3
fool 3
football 3
forevermore 3
forgetting 3
foxes 3
fragile 3
freezing 3
fuck 3
ghosts 3
gimme 3
girlfriend 3
glisten 3
gon 3
gown 3
hall 3
handsome 3
harder 3
heartbreakers 3
hero 3
hiding 3
highgate 3
hips 3
hm 3
hmm 3
holdin 3
hollow 3
hundred 3
i’ma 3
idea 3
incredible 3
jealous 3
juliet 3
july 3
kings 3
kisses 3
knowing 3
ladies 3
learn 3
legged 3
lessons 3
locket 3
loose 3
louis 3
mama 3
mamas 3
mates 3
mattress 3
meetings 3
memorize 3
midnights 3
minds 3
minutes 3
mom’s 3
mother 3
motown 3
movies 3
moving 3
muse 3
mystified 3
nasty 3
national 3
nerve 3
news 3
nick 3
nothing’s 3
november 3
occasion 3
pacing 3
pack 3
pageant 3
pages 3
paradise 3
pause 3
pavement 3
peaceful 3
peaks 3
peculiar 3
pedestal 3
people’s 3
photograph 3
pinned 3
playground 3
poets 3
polite 3
praying 3
precipice 3
prey 3
promised 3
promises 3
proposition 3
proudly 3
question 3
quick 3
rare 3
reach 3
reaching 3
regret 3
remembered 3
rent 3
reputation’s 3
restaurant 3
revolution 3
revolves 3
ricochet 3
risk 3
roads 3
rolled 3
roller 3
romantic 3
row 3
ruin 3
rule 3
sadness 3
sat 3
score 3
scratches 3
sea 3
seein 3
serve 3
shadows 3
share 3
shot 3
shots 3
sink 3
sipped 3
skip 3
skirt 3
sleepin 3
sleepless 3
smiled 3
smoke 3
snaps 3
sparkling 3
speaking 3
special 3
spent 3
spite 3
stakes 3
stares 3
stayin 3
staying 3
stole 3
stolen 3
stones 3
stopped 3
stops 3
storming 3
straw 3
stream 3
streets 3
suddenly 3
surprises 3
swallowing 3
switch 3
table 3
tails 3
tangled 3
tapping 3
tattoo 3
teardrops 3
temper 3
tennessee 3
terribly 3
terrified 3
that’ll 3
there’ll 3
thin 3
thirty 3
thorns 3
touching 3
tough 3
toys 3
track 3
traffic 3
twin 3
tying 3
um 3
underneath 3
version 3
voted 3
waking 3
waters 3
week 3
willow 3
windermere 3
winter 3
wishes 3
witches 3
wonderful 3
wore 3
year’s 3
1 2
15 2
23 2
abigail 2
add 2
adore 2
affairs 2
aids 2
aisle 2
album 2
altar 2
angels 2
answer 2
anther 2
anticipating 2
anticipation 2
apology 2
architect 2
armor 2
art 2
awesome 2
axe 2
babies 2
bags 2
bait 2
balance 2
balcony 2
bandit 2
barbed 2
barefoot 2
bargin 2
bars 2
beating 2
beds 2
beg 2
beggin 2
bit 2
bitch 2
bitter 2
bleed 2
blow 2
bluff 2
blush 2
bond 2
book 2
books 2
borrowed 2
boss 2
boyish 2
bride 2
brittle 2
brother’s 2
brush 2
built 2
bullet 2
bury 2
business 2
butterflies 2
cages 2
calls 2
cameras 2
candle 2
captain 2
capture 2
carrying 2
cascade 2
centennial 2
chalk 2
chances 2
charming 2
cheer 2
chevy 2
chicks 2
choices 2
chose 2
cities 2
clandestine 2
class 2
claws 2
clever 2
climb 2
closets 2
cloth 2
clowns 2
clues 2
clung 2
coast 2
coastal 2
cobblestones 2
collected 2
complicated 2
complications 2
compliment 2
coney 2
confess 2
consequence 2
contrarian 2
control 2
cost 2
couch 2
counter 2
creek 2
crestfallen 2
cries 2
crooked 2
crossed 2
crossing 2
cruising 2
curious 2
curve 2
daddies 2
danger 2
dangerous 2
dappled 2
dare 2
decide 2
decided 2
deepest 2
delusion 2
demons 2
details 2
devil’s 2
devils 2
diamond 2
dignity 2
dirtiest 2
disappear 2
discovered 2
distance 2
dive 2
doll 2
doorstep 2
doorway 2
dragons 2
dreary 2
drinking 2
drives 2
drought 2
drum 2
dynasty 2
east 2
echoed 2
eighty 2
endings 2
epiphany 2
escape 2
exciting 2
expecting 2
expensive 2
fair 2
fairy 2
faithless 2
fakin 2
fallin 2
fashioned 2
fault 2
feelings 2
fences 2
fifty 2
fights 2
films 2
fireworks 2
flashed 2
flickers 2
flies 2
focus 2
folks 2
force 2
forgets 2
forgive 2
forgotten 2
fresh 2
frost 2
gates 2
gather 2
georgia 2
gettin 2
glaad 2
glad 2
glance 2
gleaming 2
glimpse 2
glitter 2
glorious 2
gloves 2
glowing 2
gowns 2
grocery 2
growin 2
guard 2
guilty 2
gun 2
hackney 2
happen 2
hardwood 2
heart’s 2
heat 2
heath 2
helmet 2
hindsight 2
hoax 2
holes 2
holiday 2
hollywood 2
homeland 2
honest 2
hopes 2
horses 2
hours 2
how’d 2
husband’s 2
hustling 2
ice 2
illicit 2
imagine 2
indentation 2
indigo 2
inez 2
inviting 2
jaguars 2
job 2
keepin 2
king’s 2
knee 2
lake 2
lame 2
landing 2
laughin 2
leading 2
leavin 2
led 2
lesson 2
letters 2
liars 2
lifetime 2
linger 2
listening 2
livin 2
lobby 2
lovely 2
lyrical 2
madness 2
magnetic 2
makin 2
marks 2
married 2
marryin 2
mascara 2
maserati 2
matches 2
mccartney 2
mend 2
merry 2
messy 2
miles 2
minute 2
miracle 2
miserable 2
mistress 2
mondays 2
monsters 2
mountains 2
mouth 2
mud 2
must’ve 2
nearest 2
necklace 2
nightlight 2
noose 2
obvious 2
ohh 2
oooh 2
overnight 2
owe 2
parade 2
parking 2
passing 2
path 2
pennies 2
perched 2
perfume 2
permanent 2
pickin 2
pillow 2
pink 2
plan 2
plane 2
planned 2
plastic 2
poisoned 2
poke 2
polaroids 2
portrait 2
pray 2
precedes 2
pretending 2
prizes 2
prom 2
psycho 2
pure 2
questions 2
range 2
reading 2
realizing 2
rear 2
rebekah 2
record 2
regrets 2
relief 2
religion’s 2
remarks 2
reputations 2
resist 2
rhyme 2
rich 2
ripped 2
roaring 2
rolling 2
roses 2
rovers 2
rude 2
rudely 2
runnin 2
runs 2
rushes 2
rushing 2
ruthless 2
sabotage 2
sacred 2
safe 2
scarf 2
scarlet 2
seal 2
searching 2
selling 2
send 2
september 2
settle 2
shades 2
shoe 2
shoes 2
shop 2
shoulders 2
shut 2
silent 2
sin 2
sister 2
sister’s 2
sixteen 2
skippin 2
slept 2
slick 2
slipping 2
smart 2
smarter 2
smell 2
smells 2
sneak 2
son 2
sounded 2
sounds 2
spell 2
spin 2
spirit 2
splashed 2
spring 2
stabbed 2
staircase 2
stairwell 2
star 2
starting 2
stella 2
stepping 2
steps 2
stopping 2
story’s 2
streak 2
streetlight 2
stuck 2
suit 2
summers 2
sunrise 2
surprise 2
survived 2
swaying 2
sweep 2
swept 2
switched 2
tables 2
takin 2
tale 2
taught 2
taxi 2
teach 2
tee 2
thigh 2
thirst 2
throwin 2
thrown 2
tightrope 2
timing 2
tongue 2
tonight’s 2
torture 2
tossed 2
touchin 2
tracks 2
traded 2
traitors 2
traveled 2
treat 2
tree 2
trend 2
trippin 2
tuck 2
tupelo 2
tweet 2
twenties 2
twinkling 2
twisting 2
type 2
unbelievable 2
underestimated 2
underlined 2
usual 2
uuh 2
vacant 2
veil 2
video 2
village 2
vintage 2
violence 2
vision 2
waited 2
wakes 2
walkin 2
walks 2
wallet 2
warn 2
warned 2
wave 2
wax 2
weakness 2
weapons 2
wears 2
weeks 2
whiskey 2
whispered 2
wife 2
winning 2
wins 2
wire 2
wise 2
wiser 2
women 2
won 2
wonders 2
woo 2
worlds 2
wound 2
wounded 2
wrote 2
yacht 2
16 1
20 1
3 1
45 1
4am 1
58 1
7 1
90 1
absent 1
absurd 1
accent 1
accident 1
account 1
accused 1
ace 1
achievement 1
achilles 1
aching 1
acid 1
acing 1
acted 1
acting 1
addressed 1
adjusting 1
admit 1
adventures 1
ages 1
aim 1
airplanes 1
airport 1
aligned 1
all’s 1
alley 1
alpha 1
amber 1
ambition 1
amnesia 1
andi 1
ane 1
apologies 1
applauded 1
arcade 1
argue 1
arrowhead 1
ash 1
ashes 1
assumptions 1
ate 1
attached 1
attack 1
auroras 1
avalanche 1
avoid 1
awhile 1
ayy 1
babylon 1
backlogged 1
backseat 1
backyard 1
bag 1
balancin 1
baller 1
ballet 1
bang 1
bare 1
barren 1
baseball 1
bass 1
basslines 1
bathe 1
bathtub 1
battered 1
battle’s 1
battles 1
battleships 1
beaches 1
beaten 1
beats 1
beauty 1
beer 1
beers 1
begged 1
beginning 1
beginnings 1
begins 1
bein 1
believer 1
believes 1
believing 1
bell 1
bells 1
belonged 1
belt 1
bets 1
bill 1
bills 1
bitches 1
blankets 1
blanks 1
blaze 1
bleached 1
bled 1
bleedin 1
bleeding 1
bleeds 1
blinding 1
blocked 1
bloodstain 1
bloody 1
blooms 1
blows 1
bluest 1
blur 1
blurry 1
board 1
boardwalk 1
boat 1
boating 1
boats 1
bobby 1
bold 1
bonnie 1
booked 1
boots 1
bore 1
bored 1
bother 1
bottoms 1
box 1
boxes 1
boxing 1
boyfriend 1
braced 1
brag 1
braids 1
brain 1
brakes 1
branches 1
breakdown 1
breakfast 1
breathed 1
breathing 1
bridesmaid 1
bridges 1
briefcase 1
brighten 1
brimstone 1
brixton 1
browns 1
bruise 1
bruised 1
bruising 1
building 1
bump 1
bumped 1
bunch 1
buried 1
burrowed 1
burst 1
burton 1
bus 1
bushes 1
busier 1
bustling 1
butt 1
buzzcut 1
byline 1
cabin 1
cabs 1
cage 1
cake 1
calamitous 1
callin 1
calmer 1
camden 1
camera 1
cancel 1
candlelight 1
candles 1
cannon 1
cannons 1
cap 1
captivated 1
cards 1
cat 1
cat’s 1
catchin 1
cats 1
celebrating 1
cell 1
chains 1
champion 1
champions 1
changer 1
chapter 1
chatter 1
cheat 1
cheats 1
cheek 1
chemistry 1
cherry 1
chess 1
childhood 1
children 1
chips 1
choose 1
chosen 1
circling 1
city’s 1
civility 1
ckin 1
classmates 1
classroom 1
cleaned 1
clearin 1
cliff 1
climbed 1
cloaks 1
clock 1
clones 1
closed 1
closet 1
clouds 1
clover 1
clyde 1
coats 1
coaxed 1
code 1
coffee 1
college 1
comb 1
comet 1
comfortable 1
commit 1
company 1
comparing 1
comparison 1
compasses 1
complained 1
complex 1
complication 1
con 1
condescending 1
condition 1
confessions 1
confetti 1
conquest 1
contest 1
convictions 1
cooler 1
cop 1
core 1
cornered 1
cory 1
cory’s 1
costs 1
countin 1
country 1
county 1
couple 1
courage 1
court 1
covers 1
cracks 1
crash 1
crashin 1
crawl 1
crazier 1
creaking 1
creaks 1
creepin 1
crept 1
crescent 1
crickets 1
crimes 1
crimson 1
critique 1
crook 1
crossword 1
crowns 1
cruelest 1
cruelty 1
crumbled 1
crumpled 1
crushed 1
crystal 1
cups 1
cure 1
curfew 1
current 1
currents 1
curse 1
curses 1
curtains 1
cutting 1
cuz 1
cycle 1
cynical 1
cynics 1
d.o.c 1
dagger 1
daggers 1
daisy 1
dalí 1
damsels 1
dared 1
daring 1
darkened 1
darker 1
darkness 1
dash 1
dated 1
dates 1
dating 1
dazzling 1
deadlines 1
deal 1
dealing 1
decade 1
deceiving 1
deck 1
defeat 1
defendin 1
defined 1
deny 1
depressed 1
desert 1
deserved 1
desk 1
desperate 1
devil 1
diamonds 1
difficult 1
digging 1
dignified 1
dimming 1
dimples 1
disappeared 1
disappearing 1
disapproves 1
disbelief 1
disco 1
disposition 1
distant 1
dividin 1
divorcée 1
doc 1
doctor 1
doctor’s 1
dollar 1
dollars 1
dollas 1
dolls 1
dom 1
dominoes 1
dominos 1
dope 1
dorm 1
doubted 1
downtown 1
drake 1
dramatic 1
draw 1
draws 1
dreamscapes 1
dreamt 1
dresser 1
dresses 1
dressin 1
dried 1
drinks 1
dropping 1
drops 1
drove 1
drown 1
dry 1
duchess 1
dude 1
dumb 1
dwarfs 1
dwindling 1
dyed 1
eagles 1
earned 1
earthquakes 1
ease 1
easily 1
echo 1
eclipsed 1
edge 1
edges 1
eighteen 1
electrified 1
elegies 1
elevator 1
embers 1
endearing 1
english 1
erase 1
erasing 1
escaped 1
este 1
eulogize 1
evergreen 1
everybody’s 1
everyday 1
everything’s 1
ew 1
excellent 1
exception 1
exchanged 1
excruciating 1
exhausting 1
exist 1
expect 1
expected 1
expert 1
expired 1
explain 1
expression 1
extra 1
eyelids 1
fading 1
failure 1
fairytale 1
faking 1
fame 1
fatal 1
fatefully 1
fathers 1
feast 1
fella 1
fence 1
ferociously 1
feud 1
fever 1
fields 1
figment 1
filling 1
final 1
finger 1
fingers 1
finish 1
firefly 1
firing 1
fist 1
fives 1
flame 1
flannel 1
flashes 1
flashin 1
flashing 1
flaw 1
flesh 1
flickered 1
flings 1
floats 1
flood 1
floods 1
flow 1
flowing 1
flushed 1
flutter 1
fogs 1
folklore 1
fond 1
footnotes 1
footprints 1
forbid 1
forcing 1
form 1
forts 1
fortune 1
fought 1
fourteen 1
fourth 1
foyer 1
frame 1
fray 1
freckle 1
freckles 1
freshman 1
friction 1
friday 1
frown 1
frustrated 1
frustrating 1
fucked 1
furniture 1
fury 1
future 1
g5 1
gain 1
gap 1
gardens 1
gatsby 1
gauche 1
gay 1
gaze 1
gentlemen 1
gestures 1
ghostly 1
gift 1
givin 1
glasses 1
gleam 1
glimmer 1
glory 1
gloss 1
godforsaken 1
goin 1
goodbye’s 1
goodnight 1
grabbed 1
graffiti 1
grand 1
granted 1
grass 1
grave 1
gravity 1
gravity’s 1
greed 1
greet 1
grief 1
grieving 1
grin 1
grind 1
grinning 1
grip 1
groceries 1
grooving 1
growing 1
grumbling 1
guarded 1
guessin 1
guessing 1
gunfight 1
guns 1
gym 1
habits 1
halfway 1
halls 1
halo 1
handful 1
handled 1
handshake 1
handwritten 1
harm 1
hatchets 1
haul 1
haze 1
headphones 1
heads 1
heal 1
healing 1
health 1
heartbeats 1
heartbreaks 1
heaven’s 1
heel 1
heir 1
heist 1
hella 1
helped 1
helpless 1
hero’s 1
heroes 1
hes 1
hesitation 1
highlight 1
highly 1
hipsters 1
hits 1
holds 1
holidays 1
homecoming 1
homeroom 1
honor 1
hood 1
hopeful 1
hopeless 1
hopelessly 1
horrified 1
hospital 1
hostage 1
hot 1
houses 1
hug 1
humiliation 1
humor 1
hundredth 1
hunting 1
hustled 1
icy 1
id 1
ide 1
ideas 1
idiotic 1
ifs 1
ignore 1
ignoring 1
illusion 1
impressed 1
incandescent 1
india 1
indie 1
indiscretions 1
indulgent 1
inescapable 1
infidelity 1
initial 1
injury 1
innocence 1
insecurities 1
insincere 1
insincerity 1
insisting 1
instantly 1
insult 1
insurance 1
insurmountable 1
integrity 1
intentions 1
interruptions 1
intertwined 1
intoxicating 1
iraq 1
iron 1
irrational 1
issue 1
issues 1
ist 1
jacket 1
jail 1
jailer 1
january 1
jealousy 1
jesus 1
jewelry 1
jewels 1
join 1
joint 1
joke’s 1
jokers 1
jokes 1
journey 1
judge 1
jumped 1
jumps 1
june 1
jungle 1
jury’s 1
kaleidoscope 1
karma 1
key 1
kidding 1
killed 1
killer 1
kissed 1
kissing 1
knelt 1
knives 1
knock 1
knuckles 1
l.a 1
lady 1
laid 1
language 1
lanterns 1
lark 1
lasted 1
laughs 1
laughter 1
lawless 1
lawn 1
leader 1
leak 1
legend 1
legs 1
leo 1
levi’s 1
liar 1
license 1
lied 1
lifeless 1
lifeline 1
lifetimes 1
lift 1
lighting 1
lightning 1
limbs 1
lime 1
lingering 1
liquor 1
listened 1
listenin 1
lit 1
locking 1
loneliness 1
longing 1
lookout 1
loses 1
loss 1
lots 1
loudest 1
lovestruck 1
luck 1
lullabies 1
lunch 1
lunchbox 1
lunching 1
lungs 1
lurking 1
luxury 1
machine 1
maddest 1
madhouse 1
madison 1
magazine 1
magazines 1
magic’s 1
magician 1
magnificently 1
mail 1
mama’s 1
mansion 1
maps 1
march 1
marching 1
marjorie 1
marked 1
market 1
masquerade 1
master 1
masterpiece 1
matinee 1
meaning 1
means 1
med 1
meetin 1
meets 1
melted 1
memorizing 1
mended 1
mending 1
mental 1
mercurial 1
merlot 1
message 1
methodist 1
midas 1
mindedly 1
mindless 1
mischief 1
miscommunications 1
misread 1
missile 1
mission 1
misty 1
models 1
mom 1
momma 1
mortified 1
mosaic 1
motel 1
mother’s 1
mothers 1
mouse 1
muddy 1
muffled 1
mural 1
mysterious 1
mystical 1
mythical 1
nails 1
naive 1
named 1
namedropping 1
napkin 1
narcissist 1
natural 1
neighbor 1
neighbor’s 1
nemeses 1
neon 1
nervously 1
nicer 1
niceties 1
nightfall 1
nightmare 1
nightstand 1
nikes 1
nineteen 1
ninth 1
noise 1
nonchalant 1
normal 1
nose 1
notch 1
notes 1
noticed 1
notices 1
nurses 1
oasis 1
obsessed 1
obsessin 1
obsession 1
obsessive 1
occurred 1
occurring 1
od 1
offending 1
office 1
og 1
oil 1
olive 1
opal 1
opinion 1
optimism 1
orange 1
organ 1
outdoor 1
outnumbered 1
outraged 1
outskirts 1
outta 1
overdose 1
overdramatic 1
overrate 1
owns 1
p.j.s 1
pace 1
paced 1
paces 1
pacin 1
packed 1
padding 1
paid 1
painting 1
palace 1
pane 1
papers 1
paralyzed 1
paranoia 1
passenger 1
passenger’s 1
passionate 1
passport 1
password 1
pastel 1
pastry 1
pathetic 1
patiently 1
patrón 1
pattern 1
peak 1
pen 1
pennsylvania 1
penny 1
persist 1
persona 1
perspective 1
peter 1
petty 1
phantom 1
phased 1
phones 1
photo 1
photographs 1
pierce 1
pin 1
pirate 1
pirates 1
piss 1
pitchforks 1
plaid 1
plain 1
planning 1
plates 1
playboy 1
playful 1
playin 1
plotted 1
podium 1
poems 1
poison 1
poker 1
polaroid 1
policy 1
polish 1
politics 1
pools 1
popcorn 1
potential 1
pounds 1
pour 1
pourin 1
pov 1
preacher 1
precede 1
prep 1
pressed 1
prettiest 1
previously 1
priceless 1
princesses 1
print’s 1
priorities 1
prison 1
prize 1
procedure 1
proof 1
proportion 1
prose 1
protect 1
proud 1
pub 1
pulling 1
pumpkin 1
punched 1
punished 1
purple 1
pushing 1
quicksand 1
quietly 1
raided 1
rainbow 1
rained 1
rainstorm 1
raking 1
ransom 1
ranting 1
rattling 1
reaches 1
rebounds 1
receipt 1
receipts 1
reconnected 1
records 1
reel 1
reflection 1
refrigerator 1
refused 1
regretting 1
regulars 1
reinvention 1
relate 1
related 1
religion 1
remembering 1
reminiscing 1
removed 1
rep 1
repeating 1
replace 1
replay 1
reply 1
reports 1
request 1
respect 1
respects 1
responsibility 1
restore 1
return 1
revelers 1
revolved 1
rewind 1
rhode 1
rice 1
rid 1
riddles 1
rides 1
rifle 1
rignon 1
riviera 1
robber 1
robbers 1
rock 1
rode 1
rodeo 1
rogue 1
role 1
rope 1
rosé 1
rugby 1
ruined 1
ruins 1
rust 1
rusting 1
saddest 1
sake 1
salt 1
saltbox 1
sang 1
sashay 1
saturn 1
saved 1
saving 1
sayin 1
scare 1
scarred 1
scathing 1
schemes 1
schoolyard 1
scooter 1
scoreboard 1
scorpion 1
scrap 1
scratch 1
screen’s 1
screw 1
search 1
searched 1
searchin 1
seasons 1
seats 1
secrecy 1
seek 1
sending 1
senior 1
sensual 1
separated 1
separating 1
septembers 1
sequin 1
shady 1
shakin 1
shameless 1
shaped 1
sharks 1
shatter 1
shelf 1
shifting 1
shimmering 1
shines 1
shiniest 1
shinx 1
ship 1
shipwrecked 1
shirts 1
shoot 1
shore 1
shoreditch 1
shoulder 1
shouted 1
shred 1
shy 1
si 1
sidelines 1
sights 1
signal 1
signals 1
signed 1
silhouette 1
similar 1
sincere 1
singer 1
singin 1
sir 1
sits 1
situation 1
sixties 1
size 1
skateboard 1
skeletons 1
skeptics 1
skirts 1
slammin 1
slates 1
sleaze 1
sleepers 1
sleeps 1
sleeves 1
sleight 1
slowly 1
smiling 1
smirk 1
smoking 1
smoothing 1
smug 1
snakes 1
sneakers 1
sneakin 1
snide 1
snotty 1
soak 1
sober 1
socal 1
soft 1
soho 1
somebody’s 1
sophistication 1
sorta 1
sparkin 1
speculate 1
speech 1
speeches 1
speechless 1
spells 1
spends 1
sphere 1
spilling 1
spinnin 1
spots 1
springsteen 1
sprung 1
square 1
squeeze 1
st 1
stage 1
stain 1
stained 1
stalk 1
stalls 1
standard 1
stands 1
stared 1
starry 1
stead 1
steel 1
steering 1
sting 1
stings 1
stitches 1
stoked 1
stomach 1
store 1
stormed 1
storms 1
stranded 1
strategized 1
streaming 1
strength 1
stressed 1
stressin 1
struggle 1
stuff 1
stumble 1
stumbled 1
suburban 1
summer’s 1
sunday 1
sunlit 1
sunny 1
surely 1
surface 1
surrounded 1
suspected 1
suspicious 1
swam 1
sweater 1
sweatshirt 1
sweetest 1
swerve 1
swift 1
swimming 1
swims 1
swindled 1
swings 1
switching 1
sword 1
swords 1
swung 1
ta 1
tab 1
takers 1
talked 1
tame 1
tan 1
tape 1
tapestry 1
tarnished 1
tasteful 1
tattoos 1
teaches 1
teal 1
teeth 1
temple 1
temptation 1
tend 1
tendency 1
tennis 1
tent 1
tents 1
terrible 1
terror 1
tests 1
thankin 1
thief 1
thieves 1
thing’s 1
thirteen 1
thread 1
threaten 1
thrill 1
throne 1
ticking 1
tide 1
tie 1
ties 1
tilted 1
tiny 1
tip 1
tlc 1
tomorrow’s 1
tore 1
toy 1
toyin 1
tractor 1
trained 1
trains 1
traits 1
trampled 1
translation 1
trapeze 1
trenches 1
trends 1
trick 1
tripped 1
tropez 1
trophies 1
trophy 1
trouble’s 1
troubling 1
trusted 1
tug 1
tunnel 1
tux 1
typical 1
ultraviolet 1
unaware 1
unbroken 1
understudy 1
unforgiven 1
uni 1
uninvited 1
united 1
universe 1
unlucky 1
unmoored 1
unnecessary 1
unresolved 1
upper 1
ups 1
upset 1
upstanding 1
upstate 1
urges 1
vacancy 1
vanished 1
vendetta 1
vending 1
verdict 1
views 1
villain 1
vine 1
violin 1
voices 1
vulture 1
wages 1
waist 1
waitress 1
waits 1
wandered 1
wantin 1
warmest 1
warpath 1
wary 1
washed 1
waste 1
wastin 1
wasting 1
watches 1
waving 1
weaker 1
wedding 1
weeds 1
weekends 1
weeping 1
weight 1
wendy 1
wheel 1
wheels 1
whippin 1
whisky 1
white’s 1
who’d 1
who’ll 1
wicked 1
widow 1
wield 1
wildfire 1
windshield 1
wings 1
wink 1
winless 1
wished 1
wishful 1
wishin 1
wisteria 1
witness 1
wives 1
wolves 1
wondered 1
wonderin 1
wondrous 1
wooden 1
wool 1
worn 1
worrying 1
wounds 1
wrinkle 1
yard 1
yay 1
yelling 1
yogurt 1

As suspected, we’ve got a couple of words to manually remove such as, “ooh”, “yeah”, “ha”, and “whoa”.

tidy_lyrics <- tidy_lyrics %>% 
  anti_join(stop_words, by=c("word"="word")) %>% 
  filter(word != "ooh") %>% 
  filter(word != "yeah") %>% 
  filter(word != "ah") %>% 
  filter(word != "uh") %>% 
  filter(word != "ha") %>%
  filter(word != "whoa") %>%
  filter(word != "eh") %>%
  filter(word != "hoo") %>%
  filter(word != "ey") %>%
  filter(word != "mmm") %>% 
  filter(word != "eeh") %>% 
  filter(word != "huh") %>% 
  filter(word != "na")

Positive & negative words

Something I find really interesting is to compare the most frequently used positive and negative words.

negvspos <- tidy_lyrics %>%
  inner_join(get_sentiments("bing")) %>%
  count(word, sentiment, sort = TRUE) %>%
  acast(word ~ sentiment, value.var = "n", fill = 0) %>%
  comparison.cloud(colors = c(tswift_cols("loverblue", "loverhotpink")),
                   max.words = 100, scale=c(3.5,0.50))
## Joining, by = "word"

 
Again, the word ‘love’ stands out here as a frequently used positive word. Other popular positive words are ‘beautiful’, ‘clean’, ‘magic’, and ‘darling’. When we look at the negative words, we can see words we’d associate with angry/unhappy songs, such as ‘bad’ (bad blood), ‘mad’ (madwoman), ‘trouble’ (I Knew You Were Trouble), etc. However, you may also notice the word ‘shake’ (Shake It Off) making an appearance, and this highlights one of the issues we have with sentiment dictionaries. The dictionary associates the word ‘shake’ with negative feelings, but in the context of the song Shake It Off, it’s used in a positive way.

Words important to Taylor Swift albums

The next thing we can look at is called ‘tf-idf’. tf-idf stands for term frequency-inverse document frequency. Each word is given a weight by dividing the number of times it’s used by the number of ‘things’ (in this case, it’s songs) that contain that word. So what it isn’t giving us, is a list of the most frequently used words.

tswift_tf_idf <- tidy_lyrics %>%
    count(Album, word, sort = TRUE) %>%
    bind_tf_idf(word, Album, n) %>%
    arrange(-tf_idf) %>%
    group_by(Album) %>%
    top_n(10) %>%
    ungroup
## Selecting by tf_idf
tswift_tf_idf %>%
    mutate(word = reorder_within(word, tf_idf, Album)) %>%
    ggplot(aes(word, tf_idf, fill = Album)) +
    geom_col(alpha = 0.8, show.legend = FALSE) +
    scale_fill_tswiftalbum() +
    facet_wrap(~ Album, scales = "free", ncol = 4) +
    scale_x_reordered() +
    coord_flip() +
    theme(strip.text=element_text(size=11)) +
    labs(x = NULL, y = "tf-idf",
         title = "Important words in Taylor Swift albums")

This is where I’m really proud of having developed a custom colour palette, so the bars are filled with colours associated with each album.

When we look through these plots, the words we’re seeing here are not surprising. If we start off with 1989, we can see ‘woods’ (Out of the Woods), ‘shake’ (Shake It Off), ‘blood’ (Bad Blood), ‘york’ (New York - note that ‘New’ doesn’t show up here), ‘style’ (Style), ‘wildest (Wildest Dreams). You might also notice the word ’bye’ is right at the top for Fearless (TV) and that’s probably due to the release of Bye Bye Baby.

Take a look through - does anything surprise you here? Are there words you instantly think of when you think of an album that aren’t showing up here? You might have noticed that ‘me’ isn’t showing up for Lover and that’s because it’s a stop word, so it’s been removed.

To me, this plot looks as expected. It’s included the kind of words I would think about when I think about each of these albums.

Hey kids, topic modelling is fun!

What’s a topic model? What we’re doing here is asking R to try and group words together into ‘topics’ of words that it think belongs together.

On the topic_model line of code, you’ll see that I’ve set K = 9. K is the number of topics you think will exist in a text - it’s very subjective. If you haven’t a clue how many topics there may be, you can set K = 0 and it’ll try to figure it out for you. In my experience, when I’m running lyric analysis K = 0 usually gives me a topic for almost every song. Since we know there are 9 albums, I’ve just set K = 9. What this means is we can figure out if a particular word is associated with belonging to a certain album.

tswift_dfm <- tidy_lyrics %>%
    count(Album, word, sort = TRUE) %>%
    cast_dfm(Album, word, n)

topic_model <- stm(tswift_dfm, K = 9, verbose = FALSE, init.type = "Spectral")
summary(topic_model)
## A topic model with 9 topics, 9 documents and a 2968 word dictionary.
## Topic 1 Top Words:
##       Highest Prob: love, shake, baby, woods, stay, gonna, york 
##       FREX: woods, wonderland, shake, style, york, blood, lovers 
##       Lift: ace, admit, aids, airplanes, amnesia, ane, anthem 
##       Score: shake, woods, ace, wonderland, york, blood, style 
## Topic 2 Top Words:
##       Highest Prob: time, red, stay, trouble, wanna, home, love 
##       FREX: trouble, starlight, red, lucky, bet, loving, twenty 
##       Lift: aah, chair, ditch, echoed, expecting, forgetting, hopes 
##       Score: trouble, starlight, 45, red, loving, lucky, eye 
## Topic 3 Top Words:
##       Highest Prob: wanna, should've, love, beautiful, song, baby, time 
##       FREX: should've, song, mcgraw, tim, burn, half, matter 
##       Lift: amen, chest, concerned, daddies, eighty, georgia, lake 
##       Score: andi, tim, mcgraw, matter, song, should've, truck 
## Topic 4 Top Words:
##       Highest Prob: wanna, time, baby, bad, call, hands, car 
##       FREX: getaway, endgame, gorgeous, usin, delicate, reputation, baby's 
##       Lift: bedpost, breeze, carve, cleaning, drinkin, flyin, getaway 
##       Score: 3, getaway, endgame, gorgeous, usin, delicate, reputation 
## Topic 5 Top Words:
##       Highest Prob: love, wanna, daylight, baby, street, walk, home 
##       FREX: daylight, cornelia, bless, worship, rings, calm, street 
##       Lift: daylight, 16th, 7, accent, accidents, afterglow, ages 
##       Score: daylight, 7, cornelia, bless, worship, rings, calm 
## Topic 6 Top Words:
##       Highest Prob: baby, time, bye, feel, love, fall, gonna 
##       FREX: bye, rains, jump, belong, hallelujah, fall, fifteen 
##       Lift: coaster, cursin, farm, fightin, i'ma, juliet, photograph 
##       Score: bye, absent, rains, belong, hallelujah, fifteen, shine 
## Topic 7 Top Words:
##       Highest Prob: time, love, mind, grow, meet, fly, someday 
##       FREX: grow, superman, sparks, december, someday, fly, enchanted 
##       Lift: applause, blushing, enchanting, gimme, mattress, people's, playground 
##       Score: 58, grow, sparks, superman, someday, december, enchanted 
## Topic 8 Top Words:
##       Highest Prob: time, love, mad, mine, call, hope, woman 
##       FREX: woman, august, breathin, betty, ruining, signs, pulled 
##       Lift: bedsheets, beloved, canceled, cliffside, defense, dies, exile 
##       Score: 16, woman, august, breathin, pulled, film, marvelous 
## Topic 9 Top Words:
##       Highest Prob: left, time, stay, love, died, hand, eyes 
##       FREX: evermore, closure, dorothea, wreck, goddamn, stone, alive 
##       Lift: bent, celebrated, centerfold, disappointments, double, flush, freezing 
##       Score: 20, evermore, happiness, alive, closure, prove, died
tswift_beta <- tidy(topic_model)

tswift_beta %>%
    group_by(topic) %>%
    top_n(10, beta) %>%
    ungroup() %>%
    mutate(topic = paste0("Topic ", topic),
           term = reorder_within(term, beta, topic)) %>%
    ggplot(aes(term, beta, fill = as.factor(topic))) +
    geom_col(alpha = 0.8, show.legend = FALSE) +
    scale_fill_tswiftfolk() +
    facet_wrap(~ topic, scales = "free_y",  ncol = 4) +
    coord_flip() +
    scale_x_reordered() +
    labs(x = NULL, y = expression(beta),
         title = "Words most likely to belong to a topic",
         subtitle = "Here, each topic appears to represent an album pretty well")

Each topic appears to be representing each album pretty well. Topic 1 looks like 1989, topic 2 is Red, topic 3 is self-titled, topic 4 is reputation, topic 5 is Lover, topic 6 is Fearless, topic 7 is folklore, topic 8 looks like Speak Now, and topic 9 seems to be evermore.

Emotions in Taylor Swift songs

We know our girl Taylor is a pro at evoking all the feels, so let’s take a look at some of the emotions that pop up in her songs. Quick note, we’re using the NRC dictionary, which has a list of emotions, so it isn’t going to give us literally every emotion ever.

lyrics <- as.character(tidy_lyrics)
lyrics_sentiment <- get_nrc_sentiment((lyrics))
sentimentscores <- data.frame(colSums(lyrics_sentiment[,]))   
names(sentimentscores) <- "Score"
sentimentscores <- cbind("sentiment" = rownames(sentimentscores), sentimentscores)

ggplot(sentimentscores, aes(sentiment, Score)) +
  geom_bar(aes(fill = sentiment), stat = "identity", show.legend = FALSE) +
  scale_fill_tswift() + 
  labs(x = "Emotion & sentiment", y = "Scores", title = "Emotion in Taylor Swift songs")

The highest scores are for positive and negative sentiment - not surprising because for every break-up or angry song T-Swift has, she also has some great ones about friendship, memories, and love. This plot shows us what we already know, which is that she covers a good range of emotions fairly equally.

I was curious about what was contributing to the fear emotion, so I took a look into that a bit more.

nrc_fear <- get_sentiments("nrc") %>% 
  filter(sentiment == "fear")

fear_words <- tidy_lyrics %>%
    inner_join(nrc_fear) %>%
    count(word, sort = TRUE)
## Joining, by = "word"
fear_words %>% 
  kable(align = "c") %>%
  kable_styling(bootstrap_options = c("striped", "condensed","responsive", "bordered")) %>%
  row_spec(0, background = tswift_cols("folkweeds"), color = "white", align = "c")%>%
  add_header_above(c("Words associated with fear in Taylor Swift songs"= 2), bold = TRUE)  %>% 
  scroll_box(width = "500px", height = "400px")
Words associated with fear in Taylor Swift songs
word n
bad 87
mad 48
hate 46
feeling 41
watch 37
lose 30
fight 28
change 25
crazy 23
god 18
pain 18
shame 17
missing 16
die 15
hurt 15
screaming 15
worse 15
cruel 12
fire 12
hide 12
insane 12
broke 11
worship 10
afraid 9
cross 9
death 9
killing 9
shaking 9
fearless 8
flying 8
haunt 8
kill 8
scar 8
treacherous 8
wreck 8
broken 7
haunted 7
lonely 7
marry 7
warning 7
dying 6
hell 6
reckless 6
scream 6
fear 5
ghost 5
hanging 5
misery 5
revenge 5
stealing 5
stranger 5
blues 4
combat 4
indifference 4
lines 4
rebel 4
sneaking 4
tragedy 4
war 4
worry 4
bear 3
concerned 3
cursed 3
defense 3
doubt 3
exile 3
fragile 3
hiding 3
nasty 3
precipice 3
prey 3
revolution 3
risk 3
ruin 3
rule 3
shot 3
armor 2
bait 2
beating 2
bitch 2
danger 2
dangerous 2
delusion 2
disappear 2
escape 2
force 2
forgotten 2
guard 2
gun 2
helmet 2
honest 2
illicit 2
madness 2
parade 2
poisoned 2
pray 2
ruthless 2
sabotage 2
sin 2
sneak 2
surprise 2
torture 2
treat 2
violence 2
warn 2
warned 2
wound 2
accident 1
accused 1
attack 1
avalanche 1
avoid 1
bang 1
battered 1
belt 1
bleeding 1
bloody 1
brimstone 1
buried 1
cannon 1
cliff 1
conquest 1
cop 1
court 1
crash 1
cruelty 1
crushed 1
curse 1
cutting 1
dagger 1
darkened 1
darkness 1
depressed 1
desert 1
devil 1
difficult 1
drown 1
erase 1
escaped 1
excruciating 1
failure 1
fatal 1
fever 1
flood 1
fury 1
ghostly 1
grave 1
harm 1
haze 1
helpless 1
hesitation 1
hood 1
hopeless 1
horrified 1
hospital 1
hostage 1
hunting 1
infidelity 1
injury 1
insurmountable 1
irrational 1
jail 1
jealousy 1
journey 1
jungle 1
lifeless 1
lightning 1
loneliness 1
loss 1
missile 1
mysterious 1
nightmare 1
paralyzed 1
paranoia 1
phantom 1
poison 1
prison 1
procedure 1
punished 1
ransom 1
rifle 1
robber 1
ruined 1
scare 1
scorpion 1
shady 1
shatter 1
shoot 1
stalk 1
sting 1
struggle 1
swerve 1
terrible 1
terror 1
thief 1
threaten 1
thrill 1
unlucky 1
vanished 1
vendetta 1
verdict 1
villain 1
vulture 1
wary 1
wasting 1
weight 1
wicked 1
wildfire 1
worrying 1

Ok, so when we look at the table it makes sense. It’s just that I hadn’t associated words like ‘bad’, ‘mad’, or ‘hate’ really with fear. Anger yes, fear? Not so much. Let’s take a look at words contributing to all emotions.

word_count <- tidy_lyrics %>% 
  count(Title)

lyric_counts <- tidy_lyrics %>%
left_join(word_count, by = "Title") %>%
rename(total_words=n)

lyric_sentiment <- tidy_lyrics %>%
inner_join(get_sentiments("nrc"), by="word")

Songs and emotions

Let’s go one step further and look at which songs are associated with each emotion. Any predictions?

lyric_sentiment %>%
  count(Title, sentiment,sort=TRUE) %>%
  group_by(sentiment) %>%
  top_n(n=5) %>%
  ggplot(aes((sub(Title, pattern = "(\\w{20}).*",replacement = "\\1.")), x=reorder(Title, n), y= n, fill= sentiment)) +
  geom_bar(stat="identity",show.legend = FALSE) +
  scale_fill_tswift() +
  facet_wrap(~sentiment, scales="free") +
  xlab("Emotions & sentiments") + ylab("Score")+
  ggtitle("Songs and the emotions & sentiments they're associated with") +
  coord_flip()
## Selecting by n

Blank Space shows up in four emotions; anger, fear, negative, and sadness. This surprised me a little because I consider it to be quite a fun song becuase it’s pretty sarcastic. What we’re seeing here is the dictionary taking the words literally; it’s not easy for sentiment dictionaries to understand sarcasm. Bad Blood is apparently her angriest song, though it also is linked to disgust, fear, negativity, and sadness. Damn, that bop packs a punch.

I really like that This Love is considered her most joyful and positive song. It’s such a nice, warm, fuzzy, song - one I always underrate as well, and then I hear it and remember how amazing it is.

When I updated this to include evermore and Fearless (TV) I was paying attention to find out if any of the new tracks made an appearance and Bye Bye Baby is the only one which shows up here - right a the top of ‘anticipation’. After listening to Bye Bye Baby again I decided to find out why it’s scoring so highly for anticipation because it doesn’t feel like a very anticipatory song to me.

nrc_anticipation <- get_sentiments("nrc") %>% 
  filter(sentiment == "anticipation")

ByeByeBaby <- subset(tidy_lyrics, Title == "Bye Bye Baby", select = Artist:word)

Bye_Anticipation <- ByeByeBaby %>%
    inner_join(nrc_anticipation) %>%
    count(word, sort = TRUE)
## Joining, by = "word"
ggplot(Bye_Anticipation, aes(word, n)) +
   geom_bar(stat="identity",show.legend = FALSE) +
   scale_fill_tswift() +
   xlab("Anticipation words") + ylab("Number of times use")+
  ggtitle("Anticipation words in Bye Bye Baby")

Okay, that wasn’t what I was expecting; two words are contributing to such a high ‘anticipation’ score; “bye” and “time”. I don’t personally associate the word “bye” with anticipation.

Emotions and albums

Let’s go even further and look at which albums are associated with emotions. I think this one will be harder because her albums generally have a mix of emotions. Even reputation, which is supposed to be the angriest and most pissed she’s ever been does have some happy songs on it.

lyric_sentiment %>%
  count(Album, sentiment,sort=TRUE) %>%
  group_by(sentiment) %>%
  top_n(n=5) %>%
  ggplot(aes((sub(Album, pattern = "(\\w{20}).*",replacement = "\\1.")), x=reorder(Album, n), y= n, fill= sentiment)) +
  geom_bar(stat="identity",show.legend = FALSE) +
  scale_fill_tswift() +
  facet_wrap(~sentiment, scales="free") +
  xlab("Emotions & sentiments") + ylab("Score")+
  ggtitle("Albums and the emotions & sentiments they're associated with") +
  coord_flip()
## Selecting by n

Ok, so Lover shows up in all of the emotions and sentiments. But, Lover is angry? That surprises me because on the face of it it’s a pretty happy album. When I think a little deeper, The Man, I Forgot That You Existed, Death By A Thousand Cuts aren’t super happy. I want to see what else the dictionary is picking up as anger, so let’s take a deeper dive.

nrc_anger <- get_sentiments("nrc") %>% 
  filter(sentiment == "anger")

loverlyrics <- subset(tidy_lyrics, Album == "Lover", select = Artist:word)

anger_words <- loverlyrics %>%
    inner_join(nrc_anger) %>%
    count(word, sort = TRUE)
## Joining, by = "word"
anger_words %>% 
  kable(align = "c") %>%
  kable_styling(bootstrap_options = c("striped", "condensed","responsive", "bordered")) %>%
  row_spec(0, background = tswift_cols("loverpalepink"), align = "c")%>%
  add_header_above(c("Angry words on Lover"= 2), bold = TRUE)  %>% 
  scroll_box(width = "500px", height = "400px")
Angry words on Lover
word n
bad 17
hate 12
lose 12
fight 9
cruel 7
death 6
feeling 5
combat 4
indifference 4
hurt 3
mad 3
bout 2
crazy 2
hit 2
killing 2
strike 2
attack 1
bitch 1
blame 1
boxing 1
conquest 1
cruelty 1
damn 1
delusion 1
depressed 1
deserve 1
devil 1
dying 1
fighting 1
force 1
hell 1
losing 1
punished 1
row 1
scar 1
scare 1
scream 1
screaming 1
spite 1
storm 1
suspicious 1
threaten 1
unlucky 1

Again, this makes sense now that we look into it. The word ‘bad’ pops up in Cruel Summer, The Man, Miss Americana, Death By A Thousand Cuts, and Soon You’ll Get Better. ‘Hate’ is in I Forgot That You Existed, The Archer, Paper Rings, and You Need To Calm Down. We have ‘lose’ in Cornelia Street and Afterglow; ‘fight’ in Miss Americana; ‘cruel’ in Cruel Summer, of course.

Going back to the plot above, I think this shows that each album contains a real mix of emotions - which is exactly what fans already know.

folklore vs evermore

Since folklore and evermore are sister albums, I thought it would be interesting to look at the similarity of words used in both albums. What we’re doing below is identifying words used in both albums and then creating a plot to visalise how they’re used. Words closer to the line are used equally in both albums, for example, we can see the word “love”, which makes complete sense. Words which are further away are used less frequently in both. The word “frozen” appears really close to the line, which surprised me. I instantly thought of “long limbs, frozen swims” from Marjorie on evermore but couldn’t remember where that word appeared on folklore. I had to search for it and the word ‘frozen’ is used in Hoax and The Lakes from folklore.

folkmore <- tidy_lyrics %>% 
  filter(Album == "folklore" | Album == "evermore")

FolkmoreFrequency <- folkmore %>%
  group_by(Album) %>% 
  count(word, sort = TRUE) %>% 
  left_join(folkmore %>% 
              group_by(Album) %>% 
              summarise(total = n())) %>% 
  mutate(freq = n/total)
## Joining, by = "Album"
FolkmoreFrequency <- FolkmoreFrequency %>% 
  select(Album, word, freq) %>% 
  spread(Album, freq) %>%
  arrange(folklore, evermore)

ggplot(FolkmoreFrequency, aes(folklore, evermore)) +
  geom_jitter(alpha = 0.1, size = 2.5, width = 0.25, height = 0.25) +
  geom_text(aes(label = word), check_overlap = TRUE, vjust = 1.5) +
  scale_x_log10(labels = percent_format()) +
  scale_y_log10(labels = percent_format()) +
  geom_abline(color = "red")
## Warning: Removed 1136 rows containing missing values (geom_point).
## Warning: Removed 1136 rows containing missing values (geom_text).

Did any of these results surprise you? If you have any questions, you can get in touch with me on Twitter [@rosie_baillie_](https://twitter.com/Rosie_Baillie_). There’s more analysis I’d like to do so I’ll update it as soon as I manage to get around to it.