AS3-1 Popularity of music records

rm(list=ls(all=T))
options(digits=4, scipen=12)
library(dplyr); library(ggplot2)

---

Introduction

議題：使用歌曲的屬性，預測它會不會進入流行歌曲排行榜的前10名

學習重點：

• 依時間分割資料
• model formula 的寫法
• 高相關(共線性)自變數之間的選擇
• accuracy, sensitivity, specificity的實際意義
• 如何調整臨界機率來權衡：TFR/sensitivity vs. FPR/specificity

---

1 基本的資料處理 Understanding the Data

【1.1】How many observations (songs) are from the year 2010?

【1.2】How many songs does the dataset include for which the artist name is “Michael Jackson”?

【1.3】Which of these songs by Michael Jackson made it to the Top 10? Select all that apply.

【1.4】(a) What are the values of timesignature that occur in our dataset? (b) Which timesignature value is the most frequent among songs in our dataset?

【1.5】 Which of the following songs has the highest tempo?

---

2 建立模型 Creating Our Prediction Model

【2.1 依時間分割資料】How many observations (songs) are in the training set?

【2.2 建立模型、模型摘要】What is the value of the Akaike Information Criterion (AIC)?

【2.3 模型係數判讀】The LOWER or HIGHER our confidence about time signature, key and tempo, the more likely the song is to be in the Top 10

【2.4 進行推論】What does Model 1 suggest in terms of complexity?

【2.5 檢查異常係數】 (a) By inspecting the coefficient of the variable “loudness”, what does Model 1 suggest? (b) By inspecting the coefficient of the variable “energy”, do we draw the same conclusions as above?

---

3 處理共線性 Beware of Multicollinearity Issues!

【3.1 檢查相關係數】What is the correlation between loudness and energy in the training set?

【3.1 檢查相關係數】What is the correlation between loudness and energy in the training set?

【3.2 重新建立模型、檢查係數】Look at the summary of SongsLog2, and inspect the coefficient of the variable “energy”. What do you observe?

【3.3 選擇模型】 do we make the same observation about the popularity of heavy instrumentation as we did with Model 2?

---

4 驗證模型 Validating Our Model

【4.1 正確性】What is the accuracy of Model 3 on the test set, using a threshold of 0.45?

【4.2 底線正確率】What would the accuracy of the baseline model be on the test set? ?

【4.3 正確性 vs. 辨識率】How many songs does Model 3 correctly predict as Top 10 hits in 2010? How many non-hit songs does Model 3 predict will be Top 10 hits?

【Q】不能大幅度增加正確性的模型也會有用嗎？為甚麼？

【4.4 敏感性 & 明確性】What is the sensitivity and specificity of Model 3 on the test set, using a threshold of 0.45?

【4.5 結論】What conclusions can you make about our model?

【Q】從這個結論我們學到什麼？

---