AS3-1 Popularity of music records Group3

rm(list=ls(all=T))
options(digits=4, scipen=12)
library(dplyr); library(ggplot2)

---

Introduction

議題：使用歌曲的屬性，預測它會不會進入流行歌曲排行榜的前10名

學習重點：

• 依時間分割資料
• model formula 的寫法
• 高相關(共線性)自變數之間的選擇
• accuracy, sensitivity, specificity的實際意義
• 如何調整臨界機率來權衡：TFR/sensitivity vs. FPR/specificity

---

1 基本的資料處理 Understanding the Data

setwd("~/big data/unit3/code_data/data")

The working directory was changed to C:/Users/Administrator/Documents/big data/unit3/code_data/data inside a notebook chunk. The working directory will be reset when the chunk is finished running. Use the knitr root.dir option in the setup chunk to change the working directory for notebook chunks.

songs=read.csv("songs.csv")

【1.1】How many observations (songs) are from the year 2010?

nrow(songs[which(songs$year==2010),])

[1] 373

【1.2】How many songs does the dataset include for which the artist name is “Michael Jackson”?

nrow(songs[which(songs$artistname=="Michael Jackson"),])

[1] 18

【1.3】Which of these songs by Michael Jackson made it to the Top 10? Select all that apply.

as.character(songs[which(songs$artistname=="Michael Jackson" & songs$Top10==1),2])

[1] "You Rock My World" "You Are Not Alone" "Black or White"   
[4] "Remember the Time" "In The Closet"    

【1.4】(a) What are the values of timesignature that occur in our dataset? (b) Which timesignature value is the most frequent among songs in our dataset?

names(table(songs$timesignature))

[1] "0" "1" "3" "4" "5" "7"

names(which.max(table(songs$timesignature)))

[1] "4"

【1.5】 Which of the following songs has the highest tempo?

as.character(songs[which.max(songs$tempo),2])

[1] "Wanna Be Startin' Somethin'"

---

2 建立模型 Creating Our Prediction Model

【2.1 依時間分割資料】How many observations (songs) are in the training set?

SongsTrain=subset(songs,year<=2009)
SongsTest=subset(songs,year==2010)
nrow(SongsTrain)

[1] 7201

【2.2 建立模型、模型摘要】What is the value of the Akaike Information Criterion (AIC)?

nonvars = c("year", "songtitle", "artistname", "songID", "artistID")
SongsTrain = SongsTrain[ , !(names(SongsTrain) %in% nonvars) ]
SongsTest = SongsTest[ , !(names(SongsTest) %in% nonvars) ]
SongsLog1 = glm(Top10 ~ ., data=SongsTrain, family=binomial)
summary(SongsLog1)$aic

[1] 4827

【2.3 模型係數判讀】The LOWER or HIGHER our confidence about time signature, key and tempo, the more likely the song is to be in the Top 10

c("Ans:The higher our confidence about time signature, key and tempo, the more likely the song is to be in the Top 10 正确")

[1] "Ans:The higher our confidence about time signature, key and tempo, the more likely the song is to be in the Top 10 正确"

Warning message:
In strsplit(code, "\n", fixed = TRUE) :
  input string 1 is invalid in this locale

【2.4 進行推論】What does Model 1 suggest in terms of complexity?

c("Ans:Mainstream listeners tend to prefer less complex songs")

[1] "Ans:Mainstream listeners tend to prefer less complex songs"

【2.5 檢查異常係數】 (a) By inspecting the coefficient of the variable “loudness”, what does Model 1 suggest? (b) By inspecting the coefficient of the variable “energy”, do we draw the same conclusions as above?

c("Ans:Mainstream listeners prefer songs with heavy instrumentation")

[1] "Ans:Mainstream listeners prefer songs with heavy instrumentation"

---

3 處理共線性 Beware of Multicollinearity Issues!

【3.1 檢查相關係數】What is the correlation between loudness and energy in the training set?

cor(SongsTrain$loudness,SongsTrain$energy)

[1] 0.7399

【3.2 重新建立模型、檢查係數】Look at the summary of SongsLog2, and inspect the coefficient of the variable “energy”. What do you observe?

SongsLog2 = glm(Top10 ~ . - loudness, data=SongsTrain, family=binomial)
summary(SongsLog2)$coefficients["energy",1]

[1] 0.1813

【3.3 選擇模型】 do we make the same observation about the popularity of heavy instrumentation as we did with Model 2?

SongsLog3 = glm(Top10 ~ . - energy, data=SongsTrain, family=binomial)
summary(SongsLog3)$coefficients["loudness",1]

[1] 0.2306

c("Ans:YES")

[1] "Ans:YES"

---

4 驗證模型 Validating Our Model

【4.1 正確性】What is the accuracy of Model 3 on the test set, using a threshold of 0.45?

testPredict = predict(SongsLog3, newdata=SongsTest, type="response")
table(SongsTest$Top10, testPredict >= 0.45)

   
    FALSE TRUE
  0   309    5
  1    40   19

(309+19)/(309+5+40+19)

[1] 0.8794

【4.2 底線正確率】What would the accuracy of the baseline model be on the test set? ?

table(SongsTest$Top10)

  0   1 
314  59 

314/(314+59)

[1] 0.8418

【4.3 正確性 vs. 辨識率】How many songs does Model 3 correctly predict as Top 10 hits in 2010? How many non-hit songs does Model 3 predict will be Top 10 hits?

table(SongsTest$Top10, testPredict >= 0.45)[,2]

 0  1 
 5 19 

【Q】不能大幅度增加正確性的模型也會有用嗎？為甚麼？

Ans: 有用，因為辨識率比正確性重要，正確率的臨界值可以改動，但清楚的區分兩種狀況更能有助於進行決策。

【4.4 敏感性 & 明確性】What is the sensitivity and specificity of Model 3 on the test set, using a threshold of 0.45?

table(SongsTest$Top10, testPredict >= 0.45)

   
    FALSE TRUE
  0   309    5
  1    40   19

Warning message:
In strsplit(code, "\n", fixed = TRUE) :
  input string 1 is invalid in this locale

19/(19+40)

[1] 0.322

309/(309+5)

[1] 0.9841

【4.5 結論】What conclusions can you make about our model?

c("Ans:Model 3 provides conservative predictions, and predicts that a song will make it to the Top 10 very rarely. So while it detects less than half of the Top 10 songs, we can be very confident in the songs that it does predict to be Top 10 hits")

[1] "Ans:Model 3 provides conservative predictions, and predicts that a song will make it to the Top 10 very rarely. So while it detects less than half of the Top 10 songs, we can be very confident in the songs that it does predict to be Top 10 hits"

【Q】從這個結論我們學到什麼？

Ans: 辨識率可以幫助我們明確的區分兩部分狀況，除此之外，可以試當下情境需要，選擇你要比較關注敏感性或者明確性。像是ＭＩＴ的個案需求，就比較偏好明確性大於敏感性。 - - -