Speed Up Your Data Analysis with data.table

Gentle introduction

Author

DSA_406_001_SP25_wk4_vmbaxi

Today’s Lesson: Efficient Big Data Analysis with data.table

Objectives

  • Understand why data.table is crucial for big data analysis

  • Learn the syntax and key features of data.table

  • Practice with real-world Spotify data

  • Compare performance with traditional methods

Why data.table?

data.table is designed for handling large datasets (1GB to 100GB) with:

  • Lightning-fast operations

  • Memory efficiency

  • Concise syntax

  • Enhanced features for data manipulation

Setting Up

First, let’s install and load our required packages:

# Install and load required packages
required_packages <- c("data.table", "tidyverse", "microbenchmark", "pryr")

# Install missing packages
for (pkg in required_packages) {
  if (!requireNamespace(pkg, quietly = TRUE)) {
    install.packages(pkg)
  }
}

# Load libraries individually
#library(data.table)
#library(tidyverse)
#library(microbenchmark) # For precise timing
#library(pryr) # For memory tracking

#Load libraries
lapply(required_packages, library, character.only = TRUE)
[[1]]
[1] "data.table" "stats"      "graphics"   "grDevices"  "utils"     
[6] "datasets"   "methods"    "base"      

[[2]]
 [1] "lubridate"  "forcats"    "stringr"    "dplyr"      "purrr"     
 [6] "readr"      "tidyr"      "tibble"     "ggplot2"    "tidyverse" 
[11] "data.table" "stats"      "graphics"   "grDevices"  "utils"     
[16] "datasets"   "methods"    "base"      

[[3]]
 [1] "microbenchmark" "lubridate"      "forcats"        "stringr"       
 [5] "dplyr"          "purrr"          "readr"          "tidyr"         
 [9] "tibble"         "ggplot2"        "tidyverse"      "data.table"    
[13] "stats"          "graphics"       "grDevices"      "utils"         
[17] "datasets"       "methods"        "base"          

[[4]]
 [1] "pryr"           "microbenchmark" "lubridate"      "forcats"       
 [5] "stringr"        "dplyr"          "purrr"          "readr"         
 [9] "tidyr"          "tibble"         "ggplot2"        "tidyverse"     
[13] "data.table"     "stats"          "graphics"       "grDevices"     
[17] "utils"          "datasets"       "methods"        "base"

Let’s read our Spotify data using data.table:

# Read the dataset using fread (much faster than read.csv() or read_csv())
#spotify_dt <- fread("https://docs.google.com/spreadsheets/d/e/2PACX-1vT9iNNoFknWSmgIglkiEXnmXkJplHHeEf9wPn07tvtOV-5eTh-MdYnLS3eyST_Bx2OXpd0GC9dw5gBt/pub?gid=1801845240&single=true&output=csv")

#read in teh data from data folder
spotify_dt <- fread("https://docs.google.com/spreadsheets/d/e/2PACX-1vT9iNNoFknWSmgIglkiEXnmXkJplHHeEf9wPn07tvtOV-5eTh-MdYnLS3eyST_Bx2OXpd0GC9dw5gBt/pub?gid=1801845240&single=true&output=csv")


spotify_csv <- read_csv("https://docs.google.com/spreadsheets/d/e/2PACX-1vT9iNNoFknWSmgIglkiEXnmXkJplHHeEf9wPn07tvtOV-5eTh-MdYnLS3eyST_Bx2OXpd0GC9dw5gBt/pub?gid=1801845240&single=true&output=csv")

Lets test it out with the fread function

# Let's measure memory usage
mem_before <- pryr::mem_used()
system.time(spotify_dt <- data.table::fread("https://docs.google.com/spreadsheets/d/e/2PACX-1vT9iNNoFknWSmgIglkiEXnmXkJplHHeEf9wPn07tvtOV-5eTh-MdYnLS3eyST_Bx2OXpd0GC9dw5gBt/pub?gid=1801845240&single=true&output=csv"))
   user  system elapsed 
   0.05    0.04    0.84 
mem_after_fread <- pryr::mem_used()
fread_memory <- mem_after_fread - mem_before

✅Now you try:

Now your turn try it out with the readr function read_csv

#comment and write your code below:
mem_before <- pryr::mem_used()
system.time(spotify_dt <- readr::read_csv("https://docs.google.com/spreadsheets/d/e/2PACX-1vT9iNNoFknWSmgIglkiEXnmXkJplHHeEf9wPn07tvtOV-5eTh-MdYnLS3eyST_Bx2OXpd0GC9dw5gBt/pub?gid=1801845240&single=true&output=csv"))
Rows: 9999 Columns: 30
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr  (11): track_name, artist_name_s, album_name, album_artist_name_s, album...
dbl  (16): disc_number, track_number, track_duration_ms, popularity, danceab...
lgl   (2): explicit, album_genres
dttm  (1): added_at

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
   user  system elapsed 
   0.22    0.30    0.98 
mem_after_fread <- pryr::mem_used()
fread_memory <- mem_after_fread - mem_before

When you have an existing data frame (like one from tidyverse or base R like spotify_csv), you can convert it to a data.table using setDT().

# Option 1: setDT() - converts in place (memory efficient)
#setDT(spotify_csv)  # Converts the existing object to data.table

# Option 2: as.data.table() - creates a copy (uses more memory)
spotify_dt <- as.data.table(spotify_csv)
  1. setDT() is more memory efficient because it modifies the existing object
  2. as.data.table() creates a copy, using twice the memory
  3. For big data, it’s best to read directly with fread() rather than converting
  4. The conversion only needs to happen if you start with a regular data frame

Here is a breakdown for comparison:

compare_methods <- function(file_path) {
 # Print initial memory state
 cat("Initial memory state:\n")
 print(gc())
 
 # Create container for results
 results <- list()
 
 # 1. fread method
 cat("\nTesting fread method...\n")
 mem_before <- pryr::mem_used()
 start_time <- Sys.time()
 dt1 <- data.table::fread(file_path)
 end_time <- Sys.time()
 mem_after <- pryr::mem_used()
 results$fread <- list(
   memory = (mem_after - mem_before) / 1024^2,  # Convert to MB
   time = end_time - start_time,
   size = format(object.size(dt1), units = "MB")
 )
 cat("fread object size:", results$fread$size, "\n")
 
 # 2. read_csv method
 cat("\nTesting read_csv method...\n")
 mem_before <- pryr::mem_used()
 start_time <- Sys.time()
 csv <- readr::read_csv(file_path)
 end_time <- Sys.time()
 mem_after <- pryr::mem_used()
 results$read_csv <- list(
   memory = (mem_after - mem_before) / 1024^2,
   time = end_time - start_time,
   size = format(object.size(csv), units = "MB")
 )
 cat("read_csv object size:", results$read_csv$size, "\n")
 
 # 3. setDT method
 cat("\nTesting setDT method...\n")
 mem_before <- pryr::mem_used()
 start_time <- Sys.time()
 csv_copy <- data.table::copy(csv)
 data.table::setDT(csv_copy)
 end_time <- Sys.time()
 mem_after <- pryr::mem_used()
 results$setDT <- list(
   memory = (mem_after - mem_before) / 1024^2,
   time = end_time - start_time,
   size = format(object.size(csv_copy), units = "MB")
 )
 cat("setDT object size:", results$setDT$size, "\n")
 
 # 4. as.data.table method
 cat("\nTesting as.data.table method...\n")
 mem_before <- pryr::mem_used()
 start_time <- Sys.time()
 dt2 <- data.table::as.data.table(csv)
 end_time <- Sys.time()
 mem_after <- pryr::mem_used()
 results$as_data_table <- list(
   memory = (mem_after - mem_before) / 1024^2,
   time = end_time - start_time,
   size = format(object.size(dt2), units = "MB")
 )
 cat("as.data.table object size:", results$as_data_table$size, "\n")
 
 # Final memory state
 cat("\nFinal memory state:\n")
 print(gc())
 
 # Create summary data.frame
 summary_df <- data.frame(
   method = names(results),
   memory_mb = sapply(results, function(x) round(x$memory, 2)),
   time_secs = sapply(results, function(x) round(as.numeric(x$time), 4)),
   final_size = sapply(results, function(x) x$size)
 )
 
 return(summary_df)
}

# Use the function
results <- compare_methods("https://docs.google.com/spreadsheets/d/e/2PACX-1vT9iNNoFknWSmgIglkiEXnmXkJplHHeEf9wPn07tvtOV-5eTh-MdYnLS3eyST_Bx2OXpd0GC9dw5gBt/pub?gid=1801845240&single=true&output=csv")
Initial memory state:
          used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 1126177 60.2    2361000 126.1  2361000 126.1
Vcells 2601167 19.9    8388608  64.0  3794908  29.0

Testing fread method...
fread object size: 5.5 Mb 

Testing read_csv method...
Rows: 9999 Columns: 30
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr  (11): track_name, artist_name_s, album_name, album_artist_name_s, album...
dbl  (16): disc_number, track_number, track_duration_ms, popularity, danceab...
lgl   (2): explicit, album_genres
dttm  (1): added_at

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
read_csv object size: 5.8 Mb 

Testing setDT method...
setDT object size: 5.8 Mb 

Testing as.data.table method...
as.data.table object size: 5.8 Mb 

Final memory state:
          used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 1132764 60.5    2361000 126.1  2361000 126.1
Vcells 3999368 30.6    8388608  64.0  4235351  32.4
print("\nFinal Results:")
[1] "\nFinal Results:"
print(results)
                     method memory_mb time_secs final_size
fread                 fread      4.12    0.9747     5.5 Mb
read_csv           read_csv      2.23    1.1264     5.8 Mb
setDT                 setDT      2.34    0.0012     5.8 Mb
as_data_table as_data_table      2.24    0.0007     5.8 Mb

🔍 Investigation Exercise:

Without using any external resources, what can you discover about this dataset using data.table methods?

💡Think about….

  • How many observations and variables are there?

  • What types of data are present?

# Use these data.table commands to gather clues:

# Clue 1: Basic structure and dimensions
spotify_dt[, .N]  # Number of rows
[1] 9999
ncol(spotify_dt)  # Number of columns
[1] 30
column_classes <- lapply(spotify_dt, class)
print(column_classes)
$track_name
[1] "character"

$artist_name_s
[1] "character"

$album_name
[1] "character"

$album_artist_name_s
[1] "character"

$album_release_date
[1] "character"

$disc_number
[1] "numeric"

$track_number
[1] "numeric"

$track_duration_ms
[1] "numeric"

$explicit
[1] "logical"

$popularity
[1] "numeric"

$isrc
[1] "character"

$added_by
[1] "character"

$added_at
[1] "POSIXct" "POSIXt" 

$artist_genres
[1] "character"

$danceability
[1] "numeric"

$energy
[1] "numeric"

$key
[1] "numeric"

$loudness
[1] "numeric"

$mode
[1] "numeric"

$speechiness
[1] "numeric"

$acousticness
[1] "numeric"

$instrumentalness
[1] "numeric"

$liveness
[1] "numeric"

$valence
[1] "numeric"

$tempo
[1] "numeric"

$time_signature
[1] "numeric"

$album_genres
[1] "logical"

$label
[1] "character"

$copyrights
[1] "character"

$main_genre
[1] "character"
# Clue 2: Column names and types
spotify_dt[, lapply(.SD, class)]  # Shows class of each column
   track_name artist_name_s album_name album_artist_name_s album_release_date
       <char>        <char>     <char>              <char>             <char>
1:  character     character  character           character          character
2:  character     character  character           character          character
   disc_number track_number track_duration_ms explicit popularity      isrc
        <char>       <char>            <char>   <char>     <char>    <char>
1:     numeric      numeric           numeric  logical    numeric character
2:     numeric      numeric           numeric  logical    numeric character
    added_by added_at artist_genres danceability  energy     key loudness
      <char>   <char>        <char>       <char>  <char>  <char>   <char>
1: character  POSIXct     character      numeric numeric numeric  numeric
2: character   POSIXt     character      numeric numeric numeric  numeric
      mode speechiness acousticness instrumentalness liveness valence   tempo
    <char>      <char>       <char>           <char>   <char>  <char>  <char>
1: numeric     numeric      numeric          numeric  numeric numeric numeric
2: numeric     numeric      numeric          numeric  numeric numeric numeric
   time_signature album_genres     label copyrights main_genre
           <char>       <char>    <char>     <char>     <char>
1:        numeric      logical character  character  character
2:        numeric      logical character  character  character
# Clue 3: look at data content
spotify_dt[1:5]
                                track_name      artist_name_s
                                    <char>             <char>
1: Justified & Ancient - Stand by the Jams            The KLF
2:         I Know You Want Me (Calle Ocho)            Pitbull
3:      From the Bottom of My Broken Heart     Britney Spears
4:        Apeman - 2014 Remastered Version          The Kinks
5:      You Can't Always Get What You Want The Rolling Stones
                                                               album_name
                                                                   <char>
1:                                                       Songs Collection
2:                                         Pitbull Starring In Rebelution
3:                         ...Baby One More Time (Digital Deluxe Version)
4: Lola vs. Powerman and the Moneygoround, Pt. One + Percy (Super Deluxe)
5:                                                           Let It Bleed
   album_artist_name_s album_release_date disc_number track_number
                <char>             <char>       <num>        <num>
1:             The KLF         1992-08-03           1            3
2:             Pitbull         2009-10-23           1            3
3:      Britney Spears         1999-01-12           1            6
4:           The Kinks         2014-10-20           1           11
5:  The Rolling Stones         1969-12-05           1            9
   track_duration_ms explicit popularity         isrc                 added_by
               <num>   <lgcl>      <num>       <char>                   <char>
1:            216270    FALSE          0 QMARG1760056 spotify:user:bradnumber1
2:            237120    FALSE         64 USJAY0900144 spotify:user:bradnumber1
3:            312533    FALSE         56 USJI19910455 spotify:user:bradnumber1
4:            233400    FALSE         42 GB5KW1499822 spotify:user:bradnumber1
5:            448720    FALSE          0 USA176910100 spotify:user:bradnumber1
              added_at
                <POSc>
1: 2020-03-05 09:20:39
2: 2021-08-08 09:26:31
3: 2021-08-08 09:26:31
4: 2021-08-08 09:26:31
5: 2021-08-08 09:26:31
                                                                                                             artist_genres
                                                                                                                    <char>
1:                                                                             acid house,ambient house,big beat,hip house
2:                                                                                             dance pop,miami hip hop,pop
3:                                                                                                           dance pop,pop
4: album rock,art rock,british invasion,classic rock,folk rock,glam rock,protopunk,psychedelic rock,rock,singer-songwriter
5:                                                                           album rock,british invasion,classic rock,rock
   danceability energy   key loudness  mode speechiness acousticness
          <num>  <num> <num>    <num> <num>       <num>        <num>
1:        0.617  0.872     8  -12.305     1      0.0480       0.0158
2:        0.825  0.743     2   -5.995     1      0.1490       0.0142
3:        0.677  0.665     7   -5.171     1      0.0305       0.5600
4:        0.683  0.728     9   -8.920     1      0.2590       0.5680
5:        0.319  0.627     0   -9.611     1      0.0687       0.6750
   instrumentalness liveness valence   tempo time_signature album_genres
              <num>    <num>   <num>   <num>          <num>       <lgcl>
1:         1.12e-01   0.4080   0.504 111.458              4           NA
2:         2.12e-05   0.2370   0.800 127.045              4           NA
3:         1.01e-06   0.3380   0.706  74.981              4           NA
4:         5.08e-05   0.0384   0.833  75.311              4           NA
5:         7.29e-05   0.2890   0.497  85.818              4           NA
                                 label
                                <char>
1:                 Jams Communications
2: Mr.305/Polo Grounds Music/J Records
3:                                Jive
4:                   Sanctuary Records
5:               Universal Music Group
                                                                                                   copyrights
                                                                                                       <char>
1:                                                       C 1992 Copyright Control, P 1992 Jams Communications
2:                                        P (P) 2009 RCA/JIVE Label Group, a unit of Sony Music Entertainment
3:                                                                             P (P) 1999 Zomba Recording LLC
4: C © 2014 Sanctuary Records Group Ltd., a BMG Company, P ℗ 2014 Sanctuary Records Group Ltd., a BMG Company
5:                                   C © 2002 ABKCO Music & Records Inc., P ℗ 2002 ABKCO Music & Records Inc.
         main_genre
             <char>
1: Dance/Electronic
2:              Pop
3:              Pop
4:             Rock
5:             Rock
# Clue 4: Unique values in categorical columns
spotify_dt[, lapply(.SD, function(x) uniqueN(x)), .SDcols = is.character]
   track_name artist_name_s album_name album_artist_name_s album_release_date
        <int>         <int>      <int>               <int>              <int>
1:       8259          4130       6636                3295               3333
    isrc added_by artist_genres label copyrights main_genre
   <int>    <int>         <int> <int>      <int>      <int>
1:  8949        1          2816  1467       5379         10

Data.table Operations - data.table Syntax: [ i, j, by ]

Now let’s explore data.table’s powerful syntax for data manipulation. Think of these operations as your detective’s toolkit:

  • i (WHERE): Your filter 🔍 (selects specific rows)(like filter() & selec())

  • j (SELECT): Your magnifying glass 🔬 (focuses on specific columns or computations) (like select() or summarize())

  • by (GROUP BY): Your filing system 📁 (organizes findings)(like group_by())

1.) Basic Operations

  1. Selecting Rows (i)

When you only specify i, data.table returns a data.table (like a data frame), not a vector.

# Select first 5 rows
spotify_dt[1:5]
                                track_name      artist_name_s
                                    <char>             <char>
1: Justified & Ancient - Stand by the Jams            The KLF
2:         I Know You Want Me (Calle Ocho)            Pitbull
3:      From the Bottom of My Broken Heart     Britney Spears
4:        Apeman - 2014 Remastered Version          The Kinks
5:      You Can't Always Get What You Want The Rolling Stones
                                                               album_name
                                                                   <char>
1:                                                       Songs Collection
2:                                         Pitbull Starring In Rebelution
3:                         ...Baby One More Time (Digital Deluxe Version)
4: Lola vs. Powerman and the Moneygoround, Pt. One + Percy (Super Deluxe)
5:                                                           Let It Bleed
   album_artist_name_s album_release_date disc_number track_number
                <char>             <char>       <num>        <num>
1:             The KLF         1992-08-03           1            3
2:             Pitbull         2009-10-23           1            3
3:      Britney Spears         1999-01-12           1            6
4:           The Kinks         2014-10-20           1           11
5:  The Rolling Stones         1969-12-05           1            9
   track_duration_ms explicit popularity         isrc                 added_by
               <num>   <lgcl>      <num>       <char>                   <char>
1:            216270    FALSE          0 QMARG1760056 spotify:user:bradnumber1
2:            237120    FALSE         64 USJAY0900144 spotify:user:bradnumber1
3:            312533    FALSE         56 USJI19910455 spotify:user:bradnumber1
4:            233400    FALSE         42 GB5KW1499822 spotify:user:bradnumber1
5:            448720    FALSE          0 USA176910100 spotify:user:bradnumber1
              added_at
                <POSc>
1: 2020-03-05 09:20:39
2: 2021-08-08 09:26:31
3: 2021-08-08 09:26:31
4: 2021-08-08 09:26:31
5: 2021-08-08 09:26:31
                                                                                                             artist_genres
                                                                                                                    <char>
1:                                                                             acid house,ambient house,big beat,hip house
2:                                                                                             dance pop,miami hip hop,pop
3:                                                                                                           dance pop,pop
4: album rock,art rock,british invasion,classic rock,folk rock,glam rock,protopunk,psychedelic rock,rock,singer-songwriter
5:                                                                           album rock,british invasion,classic rock,rock
   danceability energy   key loudness  mode speechiness acousticness
          <num>  <num> <num>    <num> <num>       <num>        <num>
1:        0.617  0.872     8  -12.305     1      0.0480       0.0158
2:        0.825  0.743     2   -5.995     1      0.1490       0.0142
3:        0.677  0.665     7   -5.171     1      0.0305       0.5600
4:        0.683  0.728     9   -8.920     1      0.2590       0.5680
5:        0.319  0.627     0   -9.611     1      0.0687       0.6750
   instrumentalness liveness valence   tempo time_signature album_genres
              <num>    <num>   <num>   <num>          <num>       <lgcl>
1:         1.12e-01   0.4080   0.504 111.458              4           NA
2:         2.12e-05   0.2370   0.800 127.045              4           NA
3:         1.01e-06   0.3380   0.706  74.981              4           NA
4:         5.08e-05   0.0384   0.833  75.311              4           NA
5:         7.29e-05   0.2890   0.497  85.818              4           NA
                                 label
                                <char>
1:                 Jams Communications
2: Mr.305/Polo Grounds Music/J Records
3:                                Jive
4:                   Sanctuary Records
5:               Universal Music Group
                                                                                                   copyrights
                                                                                                       <char>
1:                                                       C 1992 Copyright Control, P 1992 Jams Communications
2:                                        P (P) 2009 RCA/JIVE Label Group, a unit of Sony Music Entertainment
3:                                                                             P (P) 1999 Zomba Recording LLC
4: C © 2014 Sanctuary Records Group Ltd., a BMG Company, P ℗ 2014 Sanctuary Records Group Ltd., a BMG Company
5:                                   C © 2002 ABKCO Music & Records Inc., P ℗ 2002 ABKCO Music & Records Inc.
         main_genre
             <char>
1: Dance/Electronic
2:              Pop
3:              Pop
4:             Rock
5:             Rock
# Filter by popularity
spotify_dt[popularity > 75]
                                 track_name                       artist_name_s
                                     <char>                              <char>
   1:            Don't Stop - 2004 Remaster                       Fleetwood Mac
   2:       Eastside (with Halsey & Khalid)        benny blanco, Halsey, Khalid
   3:                      Here Without You                        3 Doors Down
   4:    We Will Rock You - Remastered 2011                               Queen
   5:                  Too Good At Goodbyes                           Sam Smith
  ---                                                                          
1013: B.O.T.A. (Baddest Of Them All) - Edit Eliza Rose, Interplanetary Criminal
1014:                   Push Up - Main Edit                              Creeds
1015:                    Baby Don't Hurt Me David Guetta, Anne-Marie, Coi Leray
1016:          Kernkraft 400 (A Better Day)                          Topic, A7S
1017:                               Lay Low                              Tiësto
                                  album_name
                                      <char>
   1:                                Rumours
   2:        Eastside (with Halsey & Khalid)
   3:                      Away From The Sun
   4:      News Of The World (2011 Remaster)
   5: The Thrill Of It All (Special Edition)
  ---                                       
1013:         B.O.T.A. (Baddest Of Them All)
1014:                    Push Up (Main Edit)
1015:                     Baby Don't Hurt Me
1016:           Kernkraft 400 (A Better Day)
1017:                                Lay Low
                      album_artist_name_s album_release_date disc_number
                                   <char>             <char>       <num>
   1:                       Fleetwood Mac         1977-02-04           1
   2:        benny blanco, Halsey, Khalid         2018-07-12           1
   3:                        3 Doors Down         2002-11-12           1
   4:                               Queen         1977-10-28           1
   5:                           Sam Smith         2017-11-03           1
  ---                                                                   
1013: Eliza Rose, Interplanetary Criminal         2022-08-12           1
1014:                              Creeds         2023-03-31           1
1015: David Guetta, Anne-Marie, Coi Leray         2023-04-06           1
1016:                          Topic, A7S         2022-06-17           1
1017:                              Tiësto         2023-01-06           1
      track_number track_duration_ms explicit popularity         isrc
             <num>             <num>   <lgcl>      <num>       <char>
   1:            4            193346    FALSE         79 USWB10400049
   2:            1            173799    FALSE         78 USUM71809132
   3:            6            238733    FALSE         80 USUR10200764
   4:            1            122066    FALSE         82 GBUM71029618
   5:            1            201000    FALSE         87 GBUM71704089
  ---                                                                
1013:            1            226626    FALSE         82 QZAKB2166405
1014:            1            139300    FALSE         86 DEE862300564
1015:            1            140017    FALSE         94 UKWLG2300016
1016:            1            165800    FALSE         79 DECE72201091
1017:            1            153442    FALSE         87 NLZ542202348
                      added_by            added_at
                        <char>              <POSc>
   1: spotify:user:bradnumber1 2022-08-31 00:08:18
   2: spotify:user:bradnumber1 2021-08-08 09:26:31
   3: spotify:user:bradnumber1 2021-08-08 09:26:31
   4: spotify:user:bradnumber1 2021-08-08 09:26:31
   5: spotify:user:bradnumber1 2021-08-08 09:26:31
  ---                                             
1013: spotify:user:bradnumber1 2023-07-09 01:59:43
1014: spotify:user:bradnumber1 2023-07-11 01:16:38
1015: spotify:user:bradnumber1 2023-07-11 05:37:45
1016: spotify:user:bradnumber1 2023-07-11 10:57:25
1017: spotify:user:bradnumber1 2023-07-18 22:06:36
                                                                       artist_genres
                                                                              <char>
   1:                              album rock,classic rock,rock,soft rock,yacht rock
   2:                        pop,electropop,etherpop,indie poptimism,pop,pop,pop r&b
   3:                                    alternative metal,nu metal,post-grunge,rock
   4:                                                    classic rock,glam rock,rock
   5:                                                                     pop,uk pop
  ---                                                                               
1013:                                                house,breaks,experimental house
1014:                                                                           <NA>
1015: big room,dance pop,edm,pop,pop dance,pop,new jersey underground rap,trap queen
1016:           german dance,pop dance,pop edm,uk dance,pop dance,scandipop,uk dance
1017:               big room,brostep,dutch edm,edm,house,pop dance,slap house,trance
      danceability energy   key loudness  mode speechiness acousticness
             <num>  <num> <num>    <num> <num>       <num>        <num>
   1:        0.671  0.710     9   -7.724     1      0.0356      0.03930
   2:        0.560  0.680     6   -7.648     0      0.3210      0.55500
   3:        0.557  0.533    10   -6.817     0      0.0252      0.04920
   4:        0.692  0.497     2   -7.316     1      0.1190      0.67600
   5:        0.681  0.372     5   -8.237     1      0.0432      0.64000
  ---                                                                  
1013:        0.736  0.906     0   -7.589     1      0.0480      0.16400
1014:        0.767  0.830     7   -8.780     1      0.2060      0.20900
1015:        0.602  0.910     7   -3.404     1      0.0308      0.00126
1016:        0.623  0.727    11   -5.570     0      0.0562      0.18400
1017:        0.534  0.855     1   -4.923     0      0.1830      0.06070
      instrumentalness liveness valence   tempo time_signature album_genres
                 <num>    <num>   <num>   <num>          <num>       <lgcl>
   1:         1.12e-05   0.0387   0.834 118.745              4           NA
   2:         0.00e+00   0.1160   0.319  89.391              4           NA
   3:         0.00e+00   0.2050   0.233 143.994              4           NA
   4:         0.00e+00   0.2590   0.475  81.308              4           NA
   5:         0.00e+00   0.1690   0.476  91.873              4           NA
  ---                                                                      
1013:         5.85e-01   0.1060   0.698 137.001              4           NA
1014:         8.36e-01   0.0582   0.187  75.023              4           NA
1015:         1.74e-04   0.1200   0.228 127.944              4           NA
1016:         2.02e-05   0.3090   0.400 125.975              4           NA
1017:         2.63e-04   0.3460   0.420 122.060              4           NA
                           label
                          <char>
   1:       Rhino/Warner Records
   2: Benny Blanco Solo Album PS
   3:          Universal Records
   4:                        EMI
   5:                    Capitol
  ---                           
1013:             Warner Records
1014:     Columbia/B1 Recordings
1015:              Parlophone UK
1016:                     Virgin
1017:            Musical Freedom
                                                                                                                                                                                    copyrights
                                                                                                                                                                                        <char>
   1:                                                                                                                               C © 2004 Warner Records Inc., P ℗ 2004 Warner Records Inc.
   2:                                                                                       C © 2018 Friends Keep Secrets/Interscope Records, P ℗ 2018 Friends Keep Secrets/Interscope Records
   3:                                                    C © 2002 Universal Records, a Division of UMG Recordings, Inc., P ℗ 2002 Universal Motown Records, a division of UMG Recordings, Inc.
   4: C © 2011 Queen Productions Ltd. under exclusive licence to Universal International Music BV, P ℗ 2011 Queen Productions Ltd. under exclusive licence to Universal International Music BV
   5:                                                                                                 C © 2017 Universal Music Operations Limited, P ℗ 2017 Universal Music Operations Limited
  ---                                                                                                                                                                                         
1013:                       C Under exclusive licence to Warner Music UK Limited, © 2022 One House X Limited, P Under exclusive licence to Warner Music UK Limited, ℗ 2022 One House X Limited
1014:                                       P (P) 2023 Rave Alert Records, under exclusive license to Ministry of Sound/Arista Records/B1 Recordings GmbH, a Sony Music Entertainment company.
1015:                                                       C Under license to Warner Music UK Limited, © 2023 What A DJ Ltd, P Under license to Warner Music UK Limited, ℗ 2023 What A DJ Ltd
1016:                                                         C © 2022 Topic, under exclusive license to Universal Music GmbH, P ℗ 2022 Topic, under exclusive license to Universal Music GmbH
1017:                                                                                                                 C © 2023 Musical Freedom Label Ltd., P ℗ 2023 Musical Freedom Label Ltd.
            main_genre
                <char>
   1:             Rock
   2:              Pop
   3:             Rock
   4:             Rock
   5:              Pop
  ---                 
1013: Dance/Electronic
1014:            Other
1015:              Pop
1016:              Pop
1017:              Pop
# Multiple conditions
spotify_dt[popularity > 75 & main_genre == "Rock"]
                             track_name               artist_name_s
                                 <char>                      <char>
  1:         Don't Stop - 2004 Remaster               Fleetwood Mac
  2:                   Here Without You                3 Doors Down
  3: We Will Rock You - Remastered 2011                       Queen
  4:                            My Hero                Foo Fighters
  5:                I Love Rock 'N Roll Joan Jett & the Blackhearts
 ---                                                               
230:                    One Step Closer                 Linkin Park
231:  Good Riddance (Time of Your Life)                   Green Day
232:                 When I Come Around                   Green Day
233:        Rebel Rebel - 2016 Remaster                 David Bowie
234:                           Rhiannon               Fleetwood Mac
                                  album_name         album_artist_name_s
                                      <char>                      <char>
  1:                                 Rumours               Fleetwood Mac
  2:                       Away From The Sun                3 Doors Down
  3:       News Of The World (2011 Remaster)                       Queen
  4:                The Colour And The Shape                Foo Fighters
  5: I Love Rock 'N' Roll (Expanded Edition) Joan Jett & the Blackhearts
 ---                                                                    
230:           Hybrid Theory (Bonus Edition)                 Linkin Park
231:                                  Nimrod                   Green Day
232:                                  Dookie                   Green Day
233:            Diamond Dogs (2016 Remaster)                 David Bowie
234:                           Fleetwood Mac               Fleetwood Mac
     album_release_date disc_number track_number track_duration_ms explicit
                 <char>       <num>        <num>             <num>   <lgcl>
  1:         1977-02-04           1            4            193346    FALSE
  2:         2002-11-12           1            6            238733    FALSE
  3:         1977-10-28           1            1            122066    FALSE
  4:         1997-05-20           1            7            260026    FALSE
  5:         1981-11-18           1            1            175173    FALSE
 ---                                                                       
230:         2000-10-24           1            2            157333    FALSE
231:         1997-10-14           1           17            153466     TRUE
232:         1994-02-01           1           10            178000    FALSE
233:         1974-05-24           1            6            274746    FALSE
234:         1975-07-11           1            4            252773    FALSE
     popularity         isrc                 added_by            added_at
          <num>       <char>                   <char>              <POSc>
  1:         79 USWB10400049 spotify:user:bradnumber1 2022-08-31 00:08:18
  2:         80 USUR10200764 spotify:user:bradnumber1 2021-08-08 09:26:31
  3:         82 GBUM71029618 spotify:user:bradnumber1 2021-08-08 09:26:31
  4:         79 USRW29600007 spotify:user:bradnumber1 2021-08-08 09:26:31
  5:         81 USBH18100118 spotify:user:bradnumber1 2021-08-08 09:26:31
 ---                                                                     
230:         83 USWB10002399 spotify:user:bradnumber1 2021-08-08 09:26:31
231:         82 USRE19700545 spotify:user:bradnumber1 2021-08-08 09:26:31
232:         80 USRE19900154 spotify:user:bradnumber1 2021-08-08 09:26:31
233:         76 USJT11600006 spotify:user:bradnumber1 2021-08-08 09:26:31
234:         77 USWB19900188 spotify:user:bradnumber1 2021-08-08 09:26:31
                                                                      artist_genres
                                                                             <char>
  1:                              album rock,classic rock,rock,soft rock,yacht rock
  2:                                    alternative metal,nu metal,post-grunge,rock
  3:                                                    classic rock,glam rock,rock
  4: alternative metal,alternative rock,modern rock,permanent wave,post-grunge,rock
  5:                                                                 glam punk,rock
 ---                                                                               
230:                          alternative metal,nu metal,post-grunge,rap metal,rock
231:                                           modern rock,permanent wave,punk,rock
232:                                           modern rock,permanent wave,punk,rock
233:                            art rock,classic rock,glam rock,permanent wave,rock
234:                              album rock,classic rock,rock,soft rock,yacht rock
     danceability energy   key loudness  mode speechiness acousticness
            <num>  <num> <num>    <num> <num>       <num>        <num>
  1:        0.671  0.710     9   -7.724     1      0.0356     0.039300
  2:        0.557  0.533    10   -6.817     0      0.0252     0.049200
  3:        0.692  0.497     2   -7.316     1      0.1190     0.676000
  4:        0.398  0.927     9   -4.212     1      0.0567     0.000062
  5:        0.535  0.716     4   -5.025     1      0.0431     0.326000
 ---                                                                  
230:        0.492  0.969     9   -4.419     1      0.0491     0.001400
231:        0.475  0.432     7   -7.844     1      0.0302     0.183000
232:        0.531  0.909    11   -4.236     1      0.0340     0.017500
233:        0.635  0.686     9  -15.648     1      0.0530     0.209000
234:        0.723  0.490     9  -14.745     0      0.0296     0.111000
     instrumentalness liveness valence   tempo time_signature album_genres
                <num>    <num>   <num>   <num>          <num>       <lgcl>
  1:         1.12e-05   0.0387   0.834 118.745              4           NA
  2:         0.00e+00   0.2050   0.233 143.994              4           NA
  3:         0.00e+00   0.2590   0.475  81.308              4           NA
  4:         1.77e-04   0.0575   0.311  77.106              4           NA
  5:         0.00e+00   0.6590   0.901  94.379              4           NA
 ---                                                                      
230:         1.32e-06   0.0787   0.538  95.136              4           NA
231:         0.00e+00   0.1610   0.648  94.605              4           NA
232:         0.00e+00   0.2310   0.814  97.546              4           NA
233:         1.76e-02   0.2820   0.464 125.988              4           NA
234:         9.46e-02   0.0923   0.795 129.012              4           NA
                    label
                   <char>
  1: Rhino/Warner Records
  2:    Universal Records
  3:                  EMI
  4:    RCA Records Label
  5:    Legacy Recordings
 ---                     
230:       Warner Records
231:              Reprise
232:              Reprise
233:        Parlophone UK
234:       Warner Records
                                                                                                                                                                                                 copyrights
                                                                                                                                                                                                     <char>
  1:                                                                                                                                             C © 2004 Warner Records Inc., P ℗ 2004 Warner Records Inc.
  2:                                                                  C © 2002 Universal Records, a Division of UMG Recordings, Inc., P ℗ 2002 Universal Motown Records, a division of UMG Recordings, Inc.
  3:               C © 2011 Queen Productions Ltd. under exclusive licence to Universal International Music BV, P ℗ 2011 Queen Productions Ltd. under exclusive licence to Universal International Music BV
  4:                                                                                                                                                                       P (P) 1997 Roswell Records, Inc.
  5:                                                                                                               P (P) 1981 Blackheart Records Group, under exclusive license to Sony Music Entertainment
 ---                                                                                                                                                                                                       
230:                                                                                                                                             C © 2000 Warner Records Inc., P ℗ 2000 Warner Records Inc.
231: C © 1997 Reprise Records for the U.S. and WEA International Inc. for the world outside of the U.S., P ℗ 1997 Reprise Records for the U.S. and WEA International Inc. for the world outside of the U.S.
232: C © 1994 Reprise Records for the U.S. and WEA International Inc. for the world outside of the U.S., P ℗ 1994 Reprise Records for the U.S. and WEA International Inc. for the world outside of the U.S.
233:               C © 2016 Jones/Tintoretto Entertainment Co. LLC., P ℗ 1974, 2016 Jones/Tintoretto Entertainment Co., LLC under exclusive license to Parlophone Records Ltd, a Warner Music Group Company
234:                                                                                                                                             C © 1975 Warner Records Inc., P ℗ 1975 Warner Records Inc.
     main_genre
         <char>
  1:       Rock
  2:       Rock
  3:       Rock
  4:       Rock
  5:       Rock
 ---           
230:       Rock
231:       Rock
232:       Rock
233:       Rock
234:       Rock
# Excluding certain values
spotify_dt[key != 10]
                                                      track_name
                                                          <char>
   1:                    Justified & Ancient - Stand by the Jams
   2:                            I Know You Want Me (Calle Ocho)
   3:                         From the Bottom of My Broken Heart
   4:                           Apeman - 2014 Remastered Version
   5:                         You Can't Always Get What You Want
  ---                                                           
9375:                               Kernkraft 400 (A Better Day)
9376:                               Never Say Never - Radio Edit
9377: Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
9378:                                                    Lay Low
9379:                                                Padam Padam
                     artist_name_s
                            <char>
   1:                      The KLF
   2:                      Pitbull
   3:               Britney Spears
   4:                    The Kinks
   5:           The Rolling Stones
  ---                             
9375:                   Topic, A7S
9376:                    Vandalism
9377: Spiller, Sophie Ellis-Bextor
9378:                       Tiësto
9379:                Kylie Minogue
                                                                  album_name
                                                                      <char>
   1:                                                       Songs Collection
   2:                                         Pitbull Starring In Rebelution
   3:                         ...Baby One More Time (Digital Deluxe Version)
   4: Lola vs. Powerman and the Moneygoround, Pt. One + Percy (Super Deluxe)
   5:                                                           Let It Bleed
  ---                                                                       
9375:                                           Kernkraft 400 (A Better Day)
9376:                                                        Never Say Never
9377:             Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
9378:                                                                Lay Low
9379:                                                            Padam Padam
      album_artist_name_s album_release_date disc_number track_number
                   <char>             <char>       <num>        <num>
   1:             The KLF         1992-08-03           1            3
   2:             Pitbull         2009-10-23           1            3
   3:      Britney Spears         1999-01-12           1            6
   4:           The Kinks         2014-10-20           1           11
   5:  The Rolling Stones         1969-12-05           1            9
  ---                                                                
9375:          Topic, A7S         2022-06-17           1            1
9376:           Vandalism         2005-10-24           1            1
9377:             Spiller         2000-08-14           1            1
9378:              Tiësto         2023-01-06           1            1
9379:       Kylie Minogue         2023-05-19           1            1
      track_duration_ms explicit popularity         isrc
                  <num>   <lgcl>      <num>       <char>
   1:            216270    FALSE          0 QMARG1760056
   2:            237120    FALSE         64 USJAY0900144
   3:            312533    FALSE         56 USJI19910455
   4:            233400    FALSE         42 GB5KW1499822
   5:            448720    FALSE          0 USA176910100
  ---                                                   
9375:            165800    FALSE         79 DECE72201091
9376:            176640    FALSE         17 AUVC00503711
9377:            227619    FALSE         62 GBCPZ0019728
9378:            153442    FALSE         87 NLZ542202348
9379:            166266    FALSE         69 GB5KW2301017
                      added_by            added_at
                        <char>              <POSc>
   1: spotify:user:bradnumber1 2020-03-05 09:20:39
   2: spotify:user:bradnumber1 2021-08-08 09:26:31
   3: spotify:user:bradnumber1 2021-08-08 09:26:31
   4: spotify:user:bradnumber1 2021-08-08 09:26:31
   5: spotify:user:bradnumber1 2021-08-08 09:26:31
  ---                                             
9375: spotify:user:bradnumber1 2023-07-11 10:57:25
9376: spotify:user:bradnumber1 2023-07-16 09:38:19
9377: spotify:user:bradnumber1 2023-07-16 09:39:17
9378: spotify:user:bradnumber1 2023-07-18 22:06:36
9379: spotify:user:bradnumber1 2023-07-25 11:57:02
                                                                                                                artist_genres
                                                                                                                       <char>
   1:                                                                             acid house,ambient house,big beat,hip house
   2:                                                                                             dance pop,miami hip hop,pop
   3:                                                                                                           dance pop,pop
   4: album rock,art rock,british invasion,classic rock,folk rock,glam rock,protopunk,psychedelic rock,rock,singer-songwriter
   5:                                                                           album rock,british invasion,classic rock,rock
  ---                                                                                                                        
9375:                                                    german dance,pop dance,pop edm,uk dance,pop dance,scandipop,uk dance
9376:                                                                                       australian dance,melbourne bounce
9377:                                                                  disco house,vocal house,dance pop,europop,new wave pop
9378:                                                        big room,brostep,dutch edm,edm,house,pop dance,slap house,trance
9379:                                                        australian dance,australian pop,dance pop,eurodance,new wave pop
      danceability energy   key loudness  mode speechiness acousticness
             <num>  <num> <num>    <num> <num>       <num>        <num>
   1:        0.617  0.872     8  -12.305     1      0.0480     0.015800
   2:        0.825  0.743     2   -5.995     1      0.1490     0.014200
   3:        0.677  0.665     7   -5.171     1      0.0305     0.560000
   4:        0.683  0.728     9   -8.920     1      0.2590     0.568000
   5:        0.319  0.627     0   -9.611     1      0.0687     0.675000
  ---                                                                  
9375:        0.623  0.727    11   -5.570     0      0.0562     0.184000
9376:        0.720  0.841     9   -6.373     1      0.0340     0.000354
9377:        0.719  0.806     9   -6.802     0      0.0389     0.000132
9378:        0.534  0.855     1   -4.923     0      0.1830     0.060700
9379:        0.744  0.620     5   -7.930     1      0.2460     0.214000
      instrumentalness liveness valence   tempo time_signature album_genres
                 <num>    <num>   <num>   <num>          <num>       <lgcl>
   1:         1.12e-01   0.4080   0.504 111.458              4           NA
   2:         2.12e-05   0.2370   0.800 127.045              4           NA
   3:         1.01e-06   0.3380   0.706  74.981              4           NA
   4:         5.08e-05   0.0384   0.833  75.311              4           NA
   5:         7.29e-05   0.2890   0.497  85.818              4           NA
  ---                                                                      
9375:         2.02e-05   0.3090   0.400 125.975              4           NA
9376:         1.12e-02   0.3380   0.767 130.978              4           NA
9377:         8.89e-02   0.3610   0.626 123.037              4           NA
9378:         2.63e-04   0.3460   0.420 122.060              4           NA
9379:         1.16e-03   0.1030   0.711 128.103              4           NA
                                    label
                                   <char>
   1:                 Jams Communications
   2: Mr.305/Polo Grounds Music/J Records
   3:                                Jive
   4:                   Sanctuary Records
   5:               Universal Music Group
  ---                                    
9375:                              Virgin
9376:                             Vicious
9377:                    Defected Records
9378:                     Musical Freedom
9379:                     Liberator Music
                                                                                                                                                                                    copyrights
                                                                                                                                                                                        <char>
   1:                                                                                                                                     C 1992 Copyright Control, P 1992 Jams Communications
   2:                                                                                                                      P (P) 2009 RCA/JIVE Label Group, a unit of Sony Music Entertainment
   3:                                                                                                                                                           P (P) 1999 Zomba Recording LLC
   4:                                                                               C © 2014 Sanctuary Records Group Ltd., a BMG Company, P ℗ 2014 Sanctuary Records Group Ltd., a BMG Company
   5:                                                                                                                 C © 2002 ABKCO Music & Records Inc., P ℗ 2002 ABKCO Music & Records Inc.
  ---                                                                                                                                                                                         
9375:                                                         C © 2022 Topic, under exclusive license to Universal Music GmbH, P ℗ 2022 Topic, under exclusive license to Universal Music GmbH
9376:                                                                       C 2005 Vicious, a division of Vicious Recordings Pty Ltd, P 2005 Vicious, a division of Vicious Recordings Pty Ltd
9377:                                                                                                                     C © 2021 Defected Records Limited, P ℗ 2021 Defected Records Limited
9378:                                                                                                                 C © 2023 Musical Freedom Label Ltd., P ℗ 2023 Musical Freedom Label Ltd.
9379: C 2023 Kylie Minogue/Darenote under exclusive license to BMG Rights Management (UK) Limited, P 2023 Kylie Minogue/Darenote under exclusive license to BMG Rights Management (UK) Limited
            main_genre
                <char>
   1: Dance/Electronic
   2:              Pop
   3:              Pop
   4:             Rock
   5:             Rock
  ---                 
9375:              Pop
9376: Dance/Electronic
9377:              Pop
9378:              Pop
9379:              Pop
# Select tracks by genre
spotify_dt[main_genre %in% c("Rock", "Pop")]
                                                      track_name
                                                          <char>
   1:                            I Know You Want Me (Calle Ocho)
   2:                         From the Bottom of My Broken Heart
   3:                           Apeman - 2014 Remastered Version
   4:                         You Can't Always Get What You Want
   5:                                 Don't Stop - 2004 Remaster
  ---                                                           
8103:                                         Baby Don't Hurt Me
8104:                               Kernkraft 400 (A Better Day)
8105: Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
8106:                                                    Lay Low
8107:                                                Padam Padam
                            artist_name_s
                                   <char>
   1:                             Pitbull
   2:                      Britney Spears
   3:                           The Kinks
   4:                  The Rolling Stones
   5:                       Fleetwood Mac
  ---                                    
8103: David Guetta, Anne-Marie, Coi Leray
8104:                          Topic, A7S
8105:        Spiller, Sophie Ellis-Bextor
8106:                              Tiësto
8107:                       Kylie Minogue
                                                                  album_name
                                                                      <char>
   1:                                         Pitbull Starring In Rebelution
   2:                         ...Baby One More Time (Digital Deluxe Version)
   3: Lola vs. Powerman and the Moneygoround, Pt. One + Percy (Super Deluxe)
   4:                                                           Let It Bleed
   5:                                                                Rumours
  ---                                                                       
8103:                                                     Baby Don't Hurt Me
8104:                                           Kernkraft 400 (A Better Day)
8105:             Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
8106:                                                                Lay Low
8107:                                                            Padam Padam
                      album_artist_name_s album_release_date disc_number
                                   <char>             <char>       <num>
   1:                             Pitbull         2009-10-23           1
   2:                      Britney Spears         1999-01-12           1
   3:                           The Kinks         2014-10-20           1
   4:                  The Rolling Stones         1969-12-05           1
   5:                       Fleetwood Mac         1977-02-04           1
  ---                                                                   
8103: David Guetta, Anne-Marie, Coi Leray         2023-04-06           1
8104:                          Topic, A7S         2022-06-17           1
8105:                             Spiller         2000-08-14           1
8106:                              Tiësto         2023-01-06           1
8107:                       Kylie Minogue         2023-05-19           1
      track_number track_duration_ms explicit popularity         isrc
             <num>             <num>   <lgcl>      <num>       <char>
   1:            3            237120    FALSE         64 USJAY0900144
   2:            6            312533    FALSE         56 USJI19910455
   3:           11            233400    FALSE         42 GB5KW1499822
   4:            9            448720    FALSE          0 USA176910100
   5:            4            193346    FALSE         79 USWB10400049
  ---                                                                
8103:            1            140017    FALSE         94 UKWLG2300016
8104:            1            165800    FALSE         79 DECE72201091
8105:            1            227619    FALSE         62 GBCPZ0019728
8106:            1            153442    FALSE         87 NLZ542202348
8107:            1            166266    FALSE         69 GB5KW2301017
                      added_by            added_at
                        <char>              <POSc>
   1: spotify:user:bradnumber1 2021-08-08 09:26:31
   2: spotify:user:bradnumber1 2021-08-08 09:26:31
   3: spotify:user:bradnumber1 2021-08-08 09:26:31
   4: spotify:user:bradnumber1 2021-08-08 09:26:31
   5: spotify:user:bradnumber1 2022-08-31 00:08:18
  ---                                             
8103: spotify:user:bradnumber1 2023-07-11 05:37:45
8104: spotify:user:bradnumber1 2023-07-11 10:57:25
8105: spotify:user:bradnumber1 2023-07-16 09:39:17
8106: spotify:user:bradnumber1 2023-07-18 22:06:36
8107: spotify:user:bradnumber1 2023-07-25 11:57:02
                                                                                                                artist_genres
                                                                                                                       <char>
   1:                                                                                             dance pop,miami hip hop,pop
   2:                                                                                                           dance pop,pop
   3: album rock,art rock,british invasion,classic rock,folk rock,glam rock,protopunk,psychedelic rock,rock,singer-songwriter
   4:                                                                           album rock,british invasion,classic rock,rock
   5:                                                                       album rock,classic rock,rock,soft rock,yacht rock
  ---                                                                                                                        
8103:                                          big room,dance pop,edm,pop,pop dance,pop,new jersey underground rap,trap queen
8104:                                                    german dance,pop dance,pop edm,uk dance,pop dance,scandipop,uk dance
8105:                                                                  disco house,vocal house,dance pop,europop,new wave pop
8106:                                                        big room,brostep,dutch edm,edm,house,pop dance,slap house,trance
8107:                                                        australian dance,australian pop,dance pop,eurodance,new wave pop
      danceability energy   key loudness  mode speechiness acousticness
             <num>  <num> <num>    <num> <num>       <num>        <num>
   1:        0.825  0.743     2   -5.995     1      0.1490     0.014200
   2:        0.677  0.665     7   -5.171     1      0.0305     0.560000
   3:        0.683  0.728     9   -8.920     1      0.2590     0.568000
   4:        0.319  0.627     0   -9.611     1      0.0687     0.675000
   5:        0.671  0.710     9   -7.724     1      0.0356     0.039300
  ---                                                                  
8103:        0.602  0.910     7   -3.404     1      0.0308     0.001260
8104:        0.623  0.727    11   -5.570     0      0.0562     0.184000
8105:        0.719  0.806     9   -6.802     0      0.0389     0.000132
8106:        0.534  0.855     1   -4.923     0      0.1830     0.060700
8107:        0.744  0.620     5   -7.930     1      0.2460     0.214000
      instrumentalness liveness valence   tempo time_signature album_genres
                 <num>    <num>   <num>   <num>          <num>       <lgcl>
   1:         2.12e-05   0.2370   0.800 127.045              4           NA
   2:         1.01e-06   0.3380   0.706  74.981              4           NA
   3:         5.08e-05   0.0384   0.833  75.311              4           NA
   4:         7.29e-05   0.2890   0.497  85.818              4           NA
   5:         1.12e-05   0.0387   0.834 118.745              4           NA
  ---                                                                      
8103:         1.74e-04   0.1200   0.228 127.944              4           NA
8104:         2.02e-05   0.3090   0.400 125.975              4           NA
8105:         8.89e-02   0.3610   0.626 123.037              4           NA
8106:         2.63e-04   0.3460   0.420 122.060              4           NA
8107:         1.16e-03   0.1030   0.711 128.103              4           NA
                                    label
                                   <char>
   1: Mr.305/Polo Grounds Music/J Records
   2:                                Jive
   3:                   Sanctuary Records
   4:               Universal Music Group
   5:                Rhino/Warner Records
  ---                                    
8103:                       Parlophone UK
8104:                              Virgin
8105:                    Defected Records
8106:                     Musical Freedom
8107:                     Liberator Music
                                                                                                                                                                                    copyrights
                                                                                                                                                                                        <char>
   1:                                                                                                                      P (P) 2009 RCA/JIVE Label Group, a unit of Sony Music Entertainment
   2:                                                                                                                                                           P (P) 1999 Zomba Recording LLC
   3:                                                                               C © 2014 Sanctuary Records Group Ltd., a BMG Company, P ℗ 2014 Sanctuary Records Group Ltd., a BMG Company
   4:                                                                                                                 C © 2002 ABKCO Music & Records Inc., P ℗ 2002 ABKCO Music & Records Inc.
   5:                                                                                                                               C © 2004 Warner Records Inc., P ℗ 2004 Warner Records Inc.
  ---                                                                                                                                                                                         
8103:                                                       C Under license to Warner Music UK Limited, © 2023 What A DJ Ltd, P Under license to Warner Music UK Limited, ℗ 2023 What A DJ Ltd
8104:                                                         C © 2022 Topic, under exclusive license to Universal Music GmbH, P ℗ 2022 Topic, under exclusive license to Universal Music GmbH
8105:                                                                                                                     C © 2021 Defected Records Limited, P ℗ 2021 Defected Records Limited
8106:                                                                                                                 C © 2023 Musical Freedom Label Ltd., P ℗ 2023 Musical Freedom Label Ltd.
8107: C 2023 Kylie Minogue/Darenote under exclusive license to BMG Rights Management (UK) Limited, P 2023 Kylie Minogue/Darenote under exclusive license to BMG Rights Management (UK) Limited
      main_genre
          <char>
   1:        Pop
   2:        Pop
   3:       Rock
   4:       Rock
   5:       Rock
  ---           
8103:        Pop
8104:        Pop
8105:        Pop
8106:        Pop
8107:        Pop

✅Now you try:

Selecting Rows (i)

#comment and write your code below:
#selecting 10 to 15 rows 
spotify_dt[1:10]
                                                 track_name
                                                     <char>
 1:                 Justified & Ancient - Stand by the Jams
 2:                         I Know You Want Me (Calle Ocho)
 3:                      From the Bottom of My Broken Heart
 4:                        Apeman - 2014 Remastered Version
 5:                      You Can't Always Get What You Want
 6:                              Don't Stop - 2004 Remaster
 7:                         Eastside (with Halsey & Khalid)
 8: Something About The Way You Look Tonight - Edit Version
 9:                                           Juke Box Hero
10:                                                   Mercy
                   artist_name_s
                          <char>
 1:                      The KLF
 2:                      Pitbull
 3:               Britney Spears
 4:                    The Kinks
 5:           The Rolling Stones
 6:                Fleetwood Mac
 7: benny blanco, Halsey, Khalid
 8:                   Elton John
 9:                    Foreigner
10:                 Shawn Mendes
                                                                album_name
                                                                    <char>
 1:                                                       Songs Collection
 2:                                         Pitbull Starring In Rebelution
 3:                         ...Baby One More Time (Digital Deluxe Version)
 4: Lola vs. Powerman and the Moneygoround, Pt. One + Percy (Super Deluxe)
 5:                                                           Let It Bleed
 6:                                                                Rumours
 7:                                        Eastside (with Halsey & Khalid)
 8:                          Candle In The Wind 1997 / Something About ...
 9:                                                           4 (Expanded)
10:                                                    Illuminate (Deluxe)
             album_artist_name_s album_release_date disc_number track_number
                          <char>             <char>       <num>        <num>
 1:                      The KLF         1992-08-03           1            3
 2:                      Pitbull         2009-10-23           1            3
 3:               Britney Spears         1999-01-12           1            6
 4:                    The Kinks         2014-10-20           1           11
 5:           The Rolling Stones         1969-12-05           1            9
 6:                Fleetwood Mac         1977-02-04           1            4
 7: benny blanco, Halsey, Khalid         2018-07-12           1            1
 8:                   Elton John         1997-01-01           1            1
 9:                    Foreigner               1981           1            2
10:                 Shawn Mendes         2016-09-23           1            2
    track_duration_ms explicit popularity         isrc                 added_by
                <num>   <lgcl>      <num>       <char>                   <char>
 1:            216270    FALSE          0 QMARG1760056 spotify:user:bradnumber1
 2:            237120    FALSE         64 USJAY0900144 spotify:user:bradnumber1
 3:            312533    FALSE         56 USJI19910455 spotify:user:bradnumber1
 4:            233400    FALSE         42 GB5KW1499822 spotify:user:bradnumber1
 5:            448720    FALSE          0 USA176910100 spotify:user:bradnumber1
 6:            193346    FALSE         79 USWB10400049 spotify:user:bradnumber1
 7:            173799    FALSE         78 USUM71809132 spotify:user:bradnumber1
 8:            240546    FALSE         61 GBAMS9700013 spotify:user:bradnumber1
 9:            259800    FALSE         74 USAT20803006 spotify:user:bradnumber1
10:            208733    FALSE          0 USUM71603531 spotify:user:bradnumber1
               added_at
                 <POSc>
 1: 2020-03-05 09:20:39
 2: 2021-08-08 09:26:31
 3: 2021-08-08 09:26:31
 4: 2021-08-08 09:26:31
 5: 2021-08-08 09:26:31
 6: 2022-08-31 00:08:18
 7: 2021-08-08 09:26:31
 8: 2021-08-08 09:26:31
 9: 2021-08-08 09:26:31
10: 2021-08-08 09:26:31
                                                                                                              artist_genres
                                                                                                                     <char>
 1:                                                                             acid house,ambient house,big beat,hip house
 2:                                                                                             dance pop,miami hip hop,pop
 3:                                                                                                           dance pop,pop
 4: album rock,art rock,british invasion,classic rock,folk rock,glam rock,protopunk,psychedelic rock,rock,singer-songwriter
 5:                                                                           album rock,british invasion,classic rock,rock
 6:                                                                       album rock,classic rock,rock,soft rock,yacht rock
 7:                                                                 pop,electropop,etherpop,indie poptimism,pop,pop,pop r&b
 8:                                                                                   glam rock,mellow gold,piano rock,rock
 9:                                  album rock,classic rock,glam metal,hard rock,heartland rock,mellow gold,rock,soft rock
10:                                                                                              canadian pop,pop,viral pop
    danceability energy   key loudness  mode speechiness acousticness
           <num>  <num> <num>    <num> <num>       <num>        <num>
 1:        0.617  0.872     8  -12.305     1      0.0480       0.0158
 2:        0.825  0.743     2   -5.995     1      0.1490       0.0142
 3:        0.677  0.665     7   -5.171     1      0.0305       0.5600
 4:        0.683  0.728     9   -8.920     1      0.2590       0.5680
 5:        0.319  0.627     0   -9.611     1      0.0687       0.6750
 6:        0.671  0.710     9   -7.724     1      0.0356       0.0393
 7:        0.560  0.680     6   -7.648     0      0.3210       0.5550
 8:        0.480  0.628     6   -7.643     1      0.0262       0.1740
 9:        0.357  0.653     9   -5.554     1      0.0654       0.0828
10:        0.562  0.681    11   -4.934     0      0.0871       0.1130
    instrumentalness liveness valence   tempo time_signature album_genres
               <num>    <num>   <num>   <num>          <num>       <lgcl>
 1:         1.12e-01   0.4080   0.504 111.458              4           NA
 2:         2.12e-05   0.2370   0.800 127.045              4           NA
 3:         1.01e-06   0.3380   0.706  74.981              4           NA
 4:         5.08e-05   0.0384   0.833  75.311              4           NA
 5:         7.29e-05   0.2890   0.497  85.818              4           NA
 6:         1.12e-05   0.0387   0.834 118.745              4           NA
 7:         0.00e+00   0.1160   0.319  89.391              4           NA
 8:         3.28e-05   0.0753   0.541 143.412              4           NA
 9:         0.00e+00   0.0844   0.522 176.647              4           NA
10:         0.00e+00   0.1100   0.357 148.064              4           NA
                                  label
                                 <char>
 1:                 Jams Communications
 2: Mr.305/Polo Grounds Music/J Records
 3:                                Jive
 4:                   Sanctuary Records
 5:               Universal Music Group
 6:                Rhino/Warner Records
 7:          Benny Blanco Solo Album PS
 8:                                 EMI
 9:                      Rhino Atlantic
10:               Universal Music Group
                                                                                                                                     copyrights
                                                                                                                                         <char>
 1:                                                                                        C 1992 Copyright Control, P 1992 Jams Communications
 2:                                                                         P (P) 2009 RCA/JIVE Label Group, a unit of Sony Music Entertainment
 3:                                                                                                              P (P) 1999 Zomba Recording LLC
 4:                                  C © 2014 Sanctuary Records Group Ltd., a BMG Company, P ℗ 2014 Sanctuary Records Group Ltd., a BMG Company
 5:                                                                    C © 2002 ABKCO Music & Records Inc., P ℗ 2002 ABKCO Music & Records Inc.
 6:                                                                                  C © 2004 Warner Records Inc., P ℗ 2004 Warner Records Inc.
 7:                                          C © 2018 Friends Keep Secrets/Interscope Records, P ℗ 2018 Friends Keep Secrets/Interscope Records
 8:                                                         C © 1997 Mercury Records Limited, P This Compilation ℗ 1997 Mercury Records Limited
 9: C © 2002 Atlantic Recording Corp., marketed by Rhino Entertainment Company, a Warner Music Group company, P ℗ 2002 Atlantic Recording Corp.
10:                    C © 2016 Island Records, a division of UMG Recordings, Inc., P ℗ 2016 Island Records, a division of UMG Recordings, Inc.
          main_genre
              <char>
 1: Dance/Electronic
 2:              Pop
 3:              Pop
 4:             Rock
 5:             Rock
 6:             Rock
 7:              Pop
 8:             Rock
 9:             Rock
10:              Pop
spotify_dt[popularity < 30 & main_genre == "pop"]
Empty data.table (0 rows and 30 cols): track_name,artist_name_s,album_name,album_artist_name_s,album_release_date,disc_number...

AS a reminder

Comparison Operators

  • == means “exactly equal to”
  • != means “not equal to”
  • > means “greater than”
  • < means “less than”
  • >= means “greater than or equal to”
  • <= means “less than or equal to”

Logical Operators

  • & means “AND” (both conditions must be true)
  • | means “OR” (at least one condition must be true)
  • ! means “NOT” (reverses the condition)

2.) Column Operations (j)

The j argument lets you select columns and perform computations:

spotify_dt[, .(track_name, artist_name_s, popularity)]

# Compute summary statistics

spotify_dt[, .(avg_popularity = mean(popularity),

max_energy = max(energy))]

# Multiple computations

spotify_dt[, .(

track_count = .N,

avg_duration = mean(track_duration_ms)/1000,

avg_popularity = mean(popularity)

)]

# Select specific columns
spotify_dt[, .(track_name, artist_name_s, popularity)]
                                                      track_name
                                                          <char>
   1:                    Justified & Ancient - Stand by the Jams
   2:                            I Know You Want Me (Calle Ocho)
   3:                         From the Bottom of My Broken Heart
   4:                           Apeman - 2014 Remastered Version
   5:                         You Can't Always Get What You Want
  ---                                                           
9995:                               Kernkraft 400 (A Better Day)
9996:                               Never Say Never - Radio Edit
9997: Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
9998:                                                    Lay Low
9999:                                                Padam Padam
                     artist_name_s popularity
                            <char>      <num>
   1:                      The KLF          0
   2:                      Pitbull         64
   3:               Britney Spears         56
   4:                    The Kinks         42
   5:           The Rolling Stones          0
  ---                                        
9995:                   Topic, A7S         79
9996:                    Vandalism         17
9997: Spiller, Sophie Ellis-Bextor         62
9998:                       Tiësto         87
9999:                Kylie Minogue         69
# Compute summary statistics
spotify_dt[, .(avg_popularity = mean(popularity),
              max_energy = max(energy))]
   avg_popularity max_energy
            <num>      <num>
1:       37.62466         NA
# Multiple computations
spotify_dt[, .(
  track_count = .N,
  avg_duration = mean(track_duration_ms)/1000,
  avg_popularity = mean(popularity)
)]
   track_count avg_duration avg_popularity
         <int>        <num>          <num>
1:        9999      224.815       37.62466
# Select specific columns
spotify_dt[, .(artist_name_s, track_name)]
                     artist_name_s
                            <char>
   1:                      The KLF
   2:                      Pitbull
   3:               Britney Spears
   4:                    The Kinks
   5:           The Rolling Stones
  ---                             
9995:                   Topic, A7S
9996:                    Vandalism
9997: Spiller, Sophie Ellis-Bextor
9998:                       Tiësto
9999:                Kylie Minogue
                                                      track_name
                                                          <char>
   1:                    Justified & Ancient - Stand by the Jams
   2:                            I Know You Want Me (Calle Ocho)
   3:                         From the Bottom of My Broken Heart
   4:                           Apeman - 2014 Remastered Version
   5:                         You Can't Always Get What You Want
  ---                                                           
9995:                               Kernkraft 400 (A Better Day)
9996:                               Never Say Never - Radio Edit
9997: Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
9998:                                                    Lay Low
9999:                                                Padam Padam
# Create new columns in output
spotify_dt[, .(
  song = track_name,
  artist = artist_name_s,
  popularity_score = popularity/100
)]
                                                            song
                                                          <char>
   1:                    Justified & Ancient - Stand by the Jams
   2:                            I Know You Want Me (Calle Ocho)
   3:                         From the Bottom of My Broken Heart
   4:                           Apeman - 2014 Remastered Version
   5:                         You Can't Always Get What You Want
  ---                                                           
9995:                               Kernkraft 400 (A Better Day)
9996:                               Never Say Never - Radio Edit
9997: Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
9998:                                                    Lay Low
9999:                                                Padam Padam
                            artist popularity_score
                            <char>            <num>
   1:                      The KLF             0.00
   2:                      Pitbull             0.64
   3:               Britney Spears             0.56
   4:                    The Kinks             0.42
   5:           The Rolling Stones             0.00
  ---                                              
9995:                   Topic, A7S             0.79
9996:                    Vandalism             0.17
9997: Spiller, Sophie Ellis-Bextor             0.62
9998:                       Tiësto             0.87
9999:                Kylie Minogue             0.69
# Calculate summary statistics
spotify_dt[, .(
  avg_popularity = mean(popularity),
  total_tracks = .N,
  avg_duration = mean(track_duration_ms)/1000  # in seconds
)]
   avg_popularity total_tracks avg_duration
            <num>        <int>        <num>
1:       37.62466         9999      224.815

✅Now you try:

Selecting and computing (i)

#comment and write your code below:
spotify_dt[, .(album_name, album_artist_name_s, loudness)]
                                                                  album_name
                                                                      <char>
   1:                                                       Songs Collection
   2:                                         Pitbull Starring In Rebelution
   3:                         ...Baby One More Time (Digital Deluxe Version)
   4: Lola vs. Powerman and the Moneygoround, Pt. One + Percy (Super Deluxe)
   5:                                                           Let It Bleed
  ---                                                                       
9995:                                           Kernkraft 400 (A Better Day)
9996:                                                        Never Say Never
9997:             Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
9998:                                                                Lay Low
9999:                                                            Padam Padam
      album_artist_name_s loudness
                   <char>    <num>
   1:             The KLF  -12.305
   2:             Pitbull   -5.995
   3:      Britney Spears   -5.171
   4:           The Kinks   -8.920
   5:  The Rolling Stones   -9.611
  ---                             
9995:          Topic, A7S   -5.570
9996:           Vandalism   -6.373
9997:             Spiller   -6.802
9998:              Tiësto   -4.923
9999:       Kylie Minogue   -7.930
# Multiple computations
spotify_dt[, .(
  track_count = .N,
  avg_duration = mean(track_duration_ms)/1000,
  avg_popularity = mean(popularity)
)]
   track_count avg_duration avg_popularity
         <int>        <num>          <num>
1:        9999      224.815       37.62466
  1. Grouping Operations (by)
# Group by genre
spotify_dt[, .(
  track_count = .N,
  avg_popularity = mean(popularity),
  avg_danceability = mean(danceability)
), by = main_genre]
                main_genre track_count avg_popularity avg_danceability
                    <char>       <int>          <num>            <num>
 1:       Dance/Electronic         254       32.76378        0.6779331
 2:                    Pop        5694       40.27116        0.6243319
 3:                   Rock        2413       37.57563        0.5480435
 4:                Hip Hop         214       32.50935        0.7607477
 5:                  Other         946       26.09831               NA
 6:               R&B/Soul         302       34.93709        0.6557980
 7:             Jazz/Blues          54       22.24074        0.5241852
 8:           Folk/Country          91       34.65934        0.5683407
 9:       Reggae/Caribbean          27       48.22222        0.7564815
10: Classical/Instrumental           4       14.75000        0.5580000
# Multiple grouping variables
spotify_dt[, .(
  track_count = .N,
  avg_popularity = mean(popularity)
), by = .(main_genre, explicit)]
                main_genre explicit track_count avg_popularity
                    <char>   <lgcl>       <int>          <num>
 1:       Dance/Electronic    FALSE         252       32.81746
 2:                    Pop    FALSE        5345       39.74275
 3:                   Rock    FALSE        2363       37.50994
 4:                Hip Hop    FALSE         132       31.15152
 5:                  Other    FALSE         924       25.98377
 6:               R&B/Soul    FALSE         301       35.05316
 7:                    Pop     TRUE         349       48.36390
 8:             Jazz/Blues    FALSE          54       22.24074
 9:                Hip Hop     TRUE          82       34.69512
10:           Folk/Country    FALSE          89       33.40449
11:       Reggae/Caribbean    FALSE          27       48.22222
12:                   Rock     TRUE          50       40.68000
13:                  Other     TRUE          22       30.90909
14: Classical/Instrumental     TRUE           1        0.00000
15:       Dance/Electronic     TRUE           2       26.00000
16: Classical/Instrumental    FALSE           3       19.66667
17:           Folk/Country     TRUE           2       90.50000
18:               R&B/Soul     TRUE           1        0.00000
# Group by genre for popularity analysis
genre_stats <- spotify_dt[, .(
  avg_popularity = mean(popularity, na.rm = TRUE),
  track_count = .N,
  avg_duration = mean(track_duration_ms)/1000
), by = main_genre][]

✅Now you try:

Grouping (i)

#comment and write your code below:

Think of it like a sentence:

i = “Which rows do you want?”

j = “What do you want to do with them?”

by = “How should they be grouped?”

5. Advanced Features

  1. Adding/Updating Columns (:=)

We use the empty brackets at the end to print but you could have also

# Add duration in seconds
spotify_dt[, duration_sec := track_duration_ms/1000][]
Index: <main_genre>
                                                      track_name
                                                          <char>
   1:                    Justified & Ancient - Stand by the Jams
   2:                            I Know You Want Me (Calle Ocho)
   3:                         From the Bottom of My Broken Heart
   4:                           Apeman - 2014 Remastered Version
   5:                         You Can't Always Get What You Want
  ---                                                           
9995:                               Kernkraft 400 (A Better Day)
9996:                               Never Say Never - Radio Edit
9997: Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
9998:                                                    Lay Low
9999:                                                Padam Padam
                     artist_name_s
                            <char>
   1:                      The KLF
   2:                      Pitbull
   3:               Britney Spears
   4:                    The Kinks
   5:           The Rolling Stones
  ---                             
9995:                   Topic, A7S
9996:                    Vandalism
9997: Spiller, Sophie Ellis-Bextor
9998:                       Tiësto
9999:                Kylie Minogue
                                                                  album_name
                                                                      <char>
   1:                                                       Songs Collection
   2:                                         Pitbull Starring In Rebelution
   3:                         ...Baby One More Time (Digital Deluxe Version)
   4: Lola vs. Powerman and the Moneygoround, Pt. One + Percy (Super Deluxe)
   5:                                                           Let It Bleed
  ---                                                                       
9995:                                           Kernkraft 400 (A Better Day)
9996:                                                        Never Say Never
9997:             Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
9998:                                                                Lay Low
9999:                                                            Padam Padam
      album_artist_name_s album_release_date disc_number track_number
                   <char>             <char>       <num>        <num>
   1:             The KLF         1992-08-03           1            3
   2:             Pitbull         2009-10-23           1            3
   3:      Britney Spears         1999-01-12           1            6
   4:           The Kinks         2014-10-20           1           11
   5:  The Rolling Stones         1969-12-05           1            9
  ---                                                                
9995:          Topic, A7S         2022-06-17           1            1
9996:           Vandalism         2005-10-24           1            1
9997:             Spiller         2000-08-14           1            1
9998:              Tiësto         2023-01-06           1            1
9999:       Kylie Minogue         2023-05-19           1            1
      track_duration_ms explicit popularity         isrc
                  <num>   <lgcl>      <num>       <char>
   1:            216270    FALSE          0 QMARG1760056
   2:            237120    FALSE         64 USJAY0900144
   3:            312533    FALSE         56 USJI19910455
   4:            233400    FALSE         42 GB5KW1499822
   5:            448720    FALSE          0 USA176910100
  ---                                                   
9995:            165800    FALSE         79 DECE72201091
9996:            176640    FALSE         17 AUVC00503711
9997:            227619    FALSE         62 GBCPZ0019728
9998:            153442    FALSE         87 NLZ542202348
9999:            166266    FALSE         69 GB5KW2301017
                      added_by            added_at
                        <char>              <POSc>
   1: spotify:user:bradnumber1 2020-03-05 09:20:39
   2: spotify:user:bradnumber1 2021-08-08 09:26:31
   3: spotify:user:bradnumber1 2021-08-08 09:26:31
   4: spotify:user:bradnumber1 2021-08-08 09:26:31
   5: spotify:user:bradnumber1 2021-08-08 09:26:31
  ---                                             
9995: spotify:user:bradnumber1 2023-07-11 10:57:25
9996: spotify:user:bradnumber1 2023-07-16 09:38:19
9997: spotify:user:bradnumber1 2023-07-16 09:39:17
9998: spotify:user:bradnumber1 2023-07-18 22:06:36
9999: spotify:user:bradnumber1 2023-07-25 11:57:02
                                                                                                                artist_genres
                                                                                                                       <char>
   1:                                                                             acid house,ambient house,big beat,hip house
   2:                                                                                             dance pop,miami hip hop,pop
   3:                                                                                                           dance pop,pop
   4: album rock,art rock,british invasion,classic rock,folk rock,glam rock,protopunk,psychedelic rock,rock,singer-songwriter
   5:                                                                           album rock,british invasion,classic rock,rock
  ---                                                                                                                        
9995:                                                    german dance,pop dance,pop edm,uk dance,pop dance,scandipop,uk dance
9996:                                                                                       australian dance,melbourne bounce
9997:                                                                  disco house,vocal house,dance pop,europop,new wave pop
9998:                                                        big room,brostep,dutch edm,edm,house,pop dance,slap house,trance
9999:                                                        australian dance,australian pop,dance pop,eurodance,new wave pop
      danceability energy   key loudness  mode speechiness acousticness
             <num>  <num> <num>    <num> <num>       <num>        <num>
   1:        0.617  0.872     8  -12.305     1      0.0480     0.015800
   2:        0.825  0.743     2   -5.995     1      0.1490     0.014200
   3:        0.677  0.665     7   -5.171     1      0.0305     0.560000
   4:        0.683  0.728     9   -8.920     1      0.2590     0.568000
   5:        0.319  0.627     0   -9.611     1      0.0687     0.675000
  ---                                                                  
9995:        0.623  0.727    11   -5.570     0      0.0562     0.184000
9996:        0.720  0.841     9   -6.373     1      0.0340     0.000354
9997:        0.719  0.806     9   -6.802     0      0.0389     0.000132
9998:        0.534  0.855     1   -4.923     0      0.1830     0.060700
9999:        0.744  0.620     5   -7.930     1      0.2460     0.214000
      instrumentalness liveness valence   tempo time_signature album_genres
                 <num>    <num>   <num>   <num>          <num>       <lgcl>
   1:         1.12e-01   0.4080   0.504 111.458              4           NA
   2:         2.12e-05   0.2370   0.800 127.045              4           NA
   3:         1.01e-06   0.3380   0.706  74.981              4           NA
   4:         5.08e-05   0.0384   0.833  75.311              4           NA
   5:         7.29e-05   0.2890   0.497  85.818              4           NA
  ---                                                                      
9995:         2.02e-05   0.3090   0.400 125.975              4           NA
9996:         1.12e-02   0.3380   0.767 130.978              4           NA
9997:         8.89e-02   0.3610   0.626 123.037              4           NA
9998:         2.63e-04   0.3460   0.420 122.060              4           NA
9999:         1.16e-03   0.1030   0.711 128.103              4           NA
                                    label
                                   <char>
   1:                 Jams Communications
   2: Mr.305/Polo Grounds Music/J Records
   3:                                Jive
   4:                   Sanctuary Records
   5:               Universal Music Group
  ---                                    
9995:                              Virgin
9996:                             Vicious
9997:                    Defected Records
9998:                     Musical Freedom
9999:                     Liberator Music
                                                                                                                                                                                    copyrights
                                                                                                                                                                                        <char>
   1:                                                                                                                                     C 1992 Copyright Control, P 1992 Jams Communications
   2:                                                                                                                      P (P) 2009 RCA/JIVE Label Group, a unit of Sony Music Entertainment
   3:                                                                                                                                                           P (P) 1999 Zomba Recording LLC
   4:                                                                               C © 2014 Sanctuary Records Group Ltd., a BMG Company, P ℗ 2014 Sanctuary Records Group Ltd., a BMG Company
   5:                                                                                                                 C © 2002 ABKCO Music & Records Inc., P ℗ 2002 ABKCO Music & Records Inc.
  ---                                                                                                                                                                                         
9995:                                                         C © 2022 Topic, under exclusive license to Universal Music GmbH, P ℗ 2022 Topic, under exclusive license to Universal Music GmbH
9996:                                                                       C 2005 Vicious, a division of Vicious Recordings Pty Ltd, P 2005 Vicious, a division of Vicious Recordings Pty Ltd
9997:                                                                                                                     C © 2021 Defected Records Limited, P ℗ 2021 Defected Records Limited
9998:                                                                                                                 C © 2023 Musical Freedom Label Ltd., P ℗ 2023 Musical Freedom Label Ltd.
9999: C 2023 Kylie Minogue/Darenote under exclusive license to BMG Rights Management (UK) Limited, P 2023 Kylie Minogue/Darenote under exclusive license to BMG Rights Management (UK) Limited
            main_genre duration_sec
                <char>        <num>
   1: Dance/Electronic      216.270
   2:              Pop      237.120
   3:              Pop      312.533
   4:             Rock      233.400
   5:             Rock      448.720
  ---                              
9995:              Pop      165.800
9996: Dance/Electronic      176.640
9997:              Pop      227.619
9998:              Pop      153.442
9999:              Pop      166.266
# Multiple column updates
spotify_dt[, ':='(
  popularity_scaled = popularity/100,
  is_popular = popularity > 75
)]

spotify_dt
Index: <main_genre>
                                                      track_name
                                                          <char>
   1:                    Justified & Ancient - Stand by the Jams
   2:                            I Know You Want Me (Calle Ocho)
   3:                         From the Bottom of My Broken Heart
   4:                           Apeman - 2014 Remastered Version
   5:                         You Can't Always Get What You Want
  ---                                                           
9995:                               Kernkraft 400 (A Better Day)
9996:                               Never Say Never - Radio Edit
9997: Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
9998:                                                    Lay Low
9999:                                                Padam Padam
                     artist_name_s
                            <char>
   1:                      The KLF
   2:                      Pitbull
   3:               Britney Spears
   4:                    The Kinks
   5:           The Rolling Stones
  ---                             
9995:                   Topic, A7S
9996:                    Vandalism
9997: Spiller, Sophie Ellis-Bextor
9998:                       Tiësto
9999:                Kylie Minogue
                                                                  album_name
                                                                      <char>
   1:                                                       Songs Collection
   2:                                         Pitbull Starring In Rebelution
   3:                         ...Baby One More Time (Digital Deluxe Version)
   4: Lola vs. Powerman and the Moneygoround, Pt. One + Percy (Super Deluxe)
   5:                                                           Let It Bleed
  ---                                                                       
9995:                                           Kernkraft 400 (A Better Day)
9996:                                                        Never Say Never
9997:             Groovejet (If This Ain't Love) [feat. Sophie Ellis-Bextor]
9998:                                                                Lay Low
9999:                                                            Padam Padam
      album_artist_name_s album_release_date disc_number track_number
                   <char>             <char>       <num>        <num>
   1:             The KLF         1992-08-03           1            3
   2:             Pitbull         2009-10-23           1            3
   3:      Britney Spears         1999-01-12           1            6
   4:           The Kinks         2014-10-20           1           11
   5:  The Rolling Stones         1969-12-05           1            9
  ---                                                                
9995:          Topic, A7S         2022-06-17           1            1
9996:           Vandalism         2005-10-24           1            1
9997:             Spiller         2000-08-14           1            1
9998:              Tiësto         2023-01-06           1            1
9999:       Kylie Minogue         2023-05-19           1            1
      track_duration_ms explicit popularity         isrc
                  <num>   <lgcl>      <num>       <char>
   1:            216270    FALSE          0 QMARG1760056
   2:            237120    FALSE         64 USJAY0900144
   3:            312533    FALSE         56 USJI19910455
   4:            233400    FALSE         42 GB5KW1499822
   5:            448720    FALSE          0 USA176910100
  ---                                                   
9995:            165800    FALSE         79 DECE72201091
9996:            176640    FALSE         17 AUVC00503711
9997:            227619    FALSE         62 GBCPZ0019728
9998:            153442    FALSE         87 NLZ542202348
9999:            166266    FALSE         69 GB5KW2301017
                      added_by            added_at
                        <char>              <POSc>
   1: spotify:user:bradnumber1 2020-03-05 09:20:39
   2: spotify:user:bradnumber1 2021-08-08 09:26:31
   3: spotify:user:bradnumber1 2021-08-08 09:26:31
   4: spotify:user:bradnumber1 2021-08-08 09:26:31
   5: spotify:user:bradnumber1 2021-08-08 09:26:31
  ---                                             
9995: spotify:user:bradnumber1 2023-07-11 10:57:25
9996: spotify:user:bradnumber1 2023-07-16 09:38:19
9997: spotify:user:bradnumber1 2023-07-16 09:39:17
9998: spotify:user:bradnumber1 2023-07-18 22:06:36
9999: spotify:user:bradnumber1 2023-07-25 11:57:02
                                                                                                                artist_genres
                                                                                                                       <char>
   1:                                                                             acid house,ambient house,big beat,hip house
   2:                                                                                             dance pop,miami hip hop,pop
   3:                                                                                                           dance pop,pop
   4: album rock,art rock,british invasion,classic rock,folk rock,glam rock,protopunk,psychedelic rock,rock,singer-songwriter
   5:                                                                           album rock,british invasion,classic rock,rock
  ---                                                                                                                        
9995:                                                    german dance,pop dance,pop edm,uk dance,pop dance,scandipop,uk dance
9996:                                                                                       australian dance,melbourne bounce
9997:                                                                  disco house,vocal house,dance pop,europop,new wave pop
9998:                                                        big room,brostep,dutch edm,edm,house,pop dance,slap house,trance
9999:                                                        australian dance,australian pop,dance pop,eurodance,new wave pop
      danceability energy   key loudness  mode speechiness acousticness
             <num>  <num> <num>    <num> <num>       <num>        <num>
   1:        0.617  0.872     8  -12.305     1      0.0480     0.015800
   2:        0.825  0.743     2   -5.995     1      0.1490     0.014200
   3:        0.677  0.665     7   -5.171     1      0.0305     0.560000
   4:        0.683  0.728     9   -8.920     1      0.2590     0.568000
   5:        0.319  0.627     0   -9.611     1      0.0687     0.675000
  ---                                                                  
9995:        0.623  0.727    11   -5.570     0      0.0562     0.184000
9996:        0.720  0.841     9   -6.373     1      0.0340     0.000354
9997:        0.719  0.806     9   -6.802     0      0.0389     0.000132
9998:        0.534  0.855     1   -4.923     0      0.1830     0.060700
9999:        0.744  0.620     5   -7.930     1      0.2460     0.214000
      instrumentalness liveness valence   tempo time_signature album_genres
                 <num>    <num>   <num>   <num>          <num>       <lgcl>
   1:         1.12e-01   0.4080   0.504 111.458              4           NA
   2:         2.12e-05   0.2370   0.800 127.045              4           NA
   3:         1.01e-06   0.3380   0.706  74.981              4           NA
   4:         5.08e-05   0.0384   0.833  75.311              4           NA
   5:         7.29e-05   0.2890   0.497  85.818              4           NA
  ---                                                                      
9995:         2.02e-05   0.3090   0.400 125.975              4           NA
9996:         1.12e-02   0.3380   0.767 130.978              4           NA
9997:         8.89e-02   0.3610   0.626 123.037              4           NA
9998:         2.63e-04   0.3460   0.420 122.060              4           NA
9999:         1.16e-03   0.1030   0.711 128.103              4           NA
                                    label
                                   <char>
   1:                 Jams Communications
   2: Mr.305/Polo Grounds Music/J Records
   3:                                Jive
   4:                   Sanctuary Records
   5:               Universal Music Group
  ---                                    
9995:                              Virgin
9996:                             Vicious
9997:                    Defected Records
9998:                     Musical Freedom
9999:                     Liberator Music
                                                                                                                                                                                    copyrights
                                                                                                                                                                                        <char>
   1:                                                                                                                                     C 1992 Copyright Control, P 1992 Jams Communications
   2:                                                                                                                      P (P) 2009 RCA/JIVE Label Group, a unit of Sony Music Entertainment
   3:                                                                                                                                                           P (P) 1999 Zomba Recording LLC
   4:                                                                               C © 2014 Sanctuary Records Group Ltd., a BMG Company, P ℗ 2014 Sanctuary Records Group Ltd., a BMG Company
   5:                                                                                                                 C © 2002 ABKCO Music & Records Inc., P ℗ 2002 ABKCO Music & Records Inc.
  ---                                                                                                                                                                                         
9995:                                                         C © 2022 Topic, under exclusive license to Universal Music GmbH, P ℗ 2022 Topic, under exclusive license to Universal Music GmbH
9996:                                                                       C 2005 Vicious, a division of Vicious Recordings Pty Ltd, P 2005 Vicious, a division of Vicious Recordings Pty Ltd
9997:                                                                                                                     C © 2021 Defected Records Limited, P ℗ 2021 Defected Records Limited
9998:                                                                                                                 C © 2023 Musical Freedom Label Ltd., P ℗ 2023 Musical Freedom Label Ltd.
9999: C 2023 Kylie Minogue/Darenote under exclusive license to BMG Rights Management (UK) Limited, P 2023 Kylie Minogue/Darenote under exclusive license to BMG Rights Management (UK) Limited
            main_genre duration_sec popularity_scaled is_popular
                <char>        <num>             <num>     <lgcl>
   1: Dance/Electronic      216.270              0.00      FALSE
   2:              Pop      237.120              0.64      FALSE
   3:              Pop      312.533              0.56      FALSE
   4:             Rock      233.400              0.42      FALSE
   5:             Rock      448.720              0.00      FALSE
  ---                                                           
9995:              Pop      165.800              0.79       TRUE
9996: Dance/Electronic      176.640              0.17      FALSE
9997:              Pop      227.619              0.62      FALSE
9998:              Pop      153.442              0.87       TRUE
9999:              Pop      166.266              0.69      FALSE

✅Now you try:

#comment and write your code below:

# Create new columns
spotify_dt[, `:=`(
  is_popular = popularity > 75,
  loudness_energy_ratio = loudness / energy
)]

# Verify new columns
names(spotify_dt) 
 [1] "track_name"            "artist_name_s"         "album_name"           
 [4] "album_artist_name_s"   "album_release_date"    "disc_number"          
 [7] "track_number"          "track_duration_ms"     "explicit"             
[10] "popularity"            "isrc"                  "added_by"             
[13] "added_at"              "artist_genres"         "danceability"         
[16] "energy"                "key"                   "loudness"             
[19] "mode"                  "speechiness"           "acousticness"         
[22] "instrumentalness"      "liveness"              "valence"              
[25] "tempo"                 "time_signature"        "album_genres"         
[28] "label"                 "copyrights"            "main_genre"           
[31] "duration_sec"          "popularity_scaled"     "is_popular"           
[34] "loudness_energy_ratio"
#from data.table
spotify_dt[, names(.SD)]
 [1] "track_name"            "artist_name_s"         "album_name"           
 [4] "album_artist_name_s"   "album_release_date"    "disc_number"          
 [7] "track_number"          "track_duration_ms"     "explicit"             
[10] "popularity"            "isrc"                  "added_by"             
[13] "added_at"              "artist_genres"         "danceability"         
[16] "energy"                "key"                   "loudness"             
[19] "mode"                  "speechiness"           "acousticness"         
[22] "instrumentalness"      "liveness"              "valence"              
[25] "tempo"                 "time_signature"        "album_genres"         
[28] "label"                 "copyrights"            "main_genre"           
[31] "duration_sec"          "popularity_scaled"     "is_popular"           
[34] "loudness_energy_ratio"

Working with Dates

Here’s how to handle dates in data.table:

# Load lubridate
library(lubridate)

# Convert and extract date components
spotify_dt <- spotify_dt[, `:=`(
  release_date = as.Date(album_release_date),
  release_year = year(as.Date(album_release_date)),
  release_month = month(as.Date(album_release_date)),
  release_day = day(as.Date(album_release_date))
)]

# View the new date columns
spotify_dt[, .(release_date, release_year, release_month, release_day)][1:5]
   release_date release_year release_month release_day
         <Date>        <num>         <num>       <int>
1:   1992-08-03         1992             8           3
2:   2009-10-23         2009            10          23
3:   1999-01-12         1999             1          12
4:   2014-10-20         2014            10          20
5:   1969-12-05         1969            12           5
class(spotify_dt$album_release_date)
[1] "character"

Ordering Results

Data.table provides efficient ways to sort:

# Sort genres by popularity (descending)
genre_stats[order(-avg_popularity)]
                main_genre avg_popularity track_count avg_duration
                    <char>          <num>       <int>        <num>
 1:       Reggae/Caribbean       48.22222          27     210.8963
 2:                    Pop       40.27116        5694     223.3444
 3:                   Rock       37.57563        2413     233.6300
 4:               R&B/Soul       34.93709         302     227.6889
 5:           Folk/Country       34.65934          91     205.5994
 6:       Dance/Electronic       32.76378         254     246.2988
 7:                Hip Hop       32.50935         214     232.4100
 8:                  Other       26.09831         946     208.5390
 9:             Jazz/Blues       22.24074          54     168.0612
10: Classical/Instrumental       14.75000           4     159.5265
# Get top 5 most popular genres
genre_stats[order(-avg_popularity)][1:5]
         main_genre avg_popularity track_count avg_duration
             <char>          <num>       <int>        <num>
1: Reggae/Caribbean       48.22222          27     210.8963
2:              Pop       40.27116        5694     223.3444
3:             Rock       37.57563        2413     233.6300
4:         R&B/Soul       34.93709         302     227.6889
5:     Folk/Country       34.65934          91     205.5994
year_genre_stats <- spotify_dt[, .(
    avg_popularity = mean(popularity),
    track_count = .N,
    avg_energy = mean(energy)
), by = .(release_year)]

# Sort by multiple columns
year_genre_stats[order(release_year, -avg_popularity)]
    release_year avg_popularity track_count avg_energy
           <num>          <num>       <int>      <num>
 1:         1956       68.00000           1  0.6600000
 2:         1958       55.33333           6  0.5676667
 3:         1959       35.00000           2  0.2385000
 4:         1960       61.00000           2  0.8260000
 5:         1961       24.16667           6  0.6656667
 6:         1962       33.57895          19  0.5368947
 7:         1963       32.13333          45  0.5427844
 8:         1964       40.00000          46  0.6658696
 9:         1965       52.51613          31  0.5777097
10:         1966       33.30233          43  0.6424419
11:         1967       44.56250          48  0.5366667
12:         1968       43.87805          41  0.5942927
13:         1969       48.48889          45  0.5497111
14:         1970       50.62500          56  0.5608393
15:         1971       42.90909          44  0.6505000
16:         1972       42.32500          40  0.5486250
17:         1973       49.07273          55  0.5865455
18:         1974       38.13333          30  0.6115667
19:         1975       43.42500          40  0.6191250
20:         1976       50.14634          41  0.6497317
21:         1977       50.55172          58  0.6022759
22:         1978       48.56604          53  0.6504528
23:         1979       44.45283          53  0.6116604
24:         1980       51.33333          45  0.6395556
25:         1981       51.20000          45  0.6677111
26:         1982       47.80851          47  0.6705745
27:         1983       42.55357          56  0.7012679
28:         1984       48.64815          54  0.6755185
29:         1985       40.46154          65  0.7196308
30:         1986       50.05556          72  0.6955417
31:         1987       36.12844         109  0.6350550
32:         1988       40.35294          85  0.6454235
33:         1989       44.08974          78  0.6550231
34:         1990       42.80247          81  0.6516914
35:         1991       38.70787          89  0.6743933
36:         1992       37.28409          88  0.6474659
37:         1993       38.81609          87  0.6679770
38:         1994       39.17073          82  0.6669024
39:         1995       37.00781         128  0.6582813
40:         1996       37.84211         114  0.6303070
41:         1997       39.68217         129  0.7040884
42:         1998       36.08088         136  0.7423088
43:         1999       43.38400         125  0.7390288
44:         2000       36.28859         149  0.7102148
45:         2001       37.68280         186  0.7078441
46:         2002       37.61765         170  0.6864525
47:         2003       36.37500         176  0.7010170
48:         2004       38.66857         175  0.7082629
49:         2005       30.91250         240  0.7182292
50:         2006       33.20321         187  0.6895882
51:         2007       32.03167         221  0.7046335
52:         2008       32.25574         305  0.7465705
53:         2009       31.95455         374  0.7561578
54:         2010       32.20896         335  0.7757433
55:         2011       33.42559         383  0.7296240
56:         2012       35.45299         351  0.7361425
57:         2013       31.26780         295  0.7019861
58:         2014       32.59850         401  0.6773738
59:         2015       32.12250         400  0.6740375
60:         2016       33.14550         378  0.6490534
61:         2017       36.43391         348  0.6564569
62:         2018       43.81609         261  0.6452682
63:         2019       48.44550         211  0.6600806
64:         2020       49.54688         256  0.6631406
65:         2021       41.00966         207  0.6741739
66:         2022       68.91667          60  0.7245000
67:         2023       76.78788          33  0.6893333
68:           NA       36.23457        1377         NA
    release_year avg_popularity track_count avg_energy

🔍 Practice Exercise:

Using the Spotify dataset, create an analysis that:

  1. Filters for songs after 2010

  2. Calculates average audio features by genre

  3. Identifies the top 3 genres by popularity

# solution here
spotify_dt[year(as.Date(album_release_date)) >= 2010,
          .(avg_popularity = mean(popularity),
            avg_danceability = mean(danceability),
            avg_energy = mean(energy),
            track_count = .N),
          by = main_genre
          ][order(-avg_popularity)
          ][1:3]
         main_genre avg_popularity avg_danceability avg_energy track_count
             <char>          <num>            <num>      <num>       <int>
1: Reggae/Caribbean       43.57143        0.7291429  0.7142857           7
2:              Pop       42.13373        0.6329859  0.6930078        2834
3:          Hip Hop       37.21154        0.7300096  0.7036635         104

Ordering Results

Data.table provides efficient ways to sort:

# Sort genres by popularity (descending)
genre_stats[order(-avg_popularity)]
                main_genre avg_popularity track_count avg_duration
                    <char>          <num>       <int>        <num>
 1:       Reggae/Caribbean       48.22222          27     210.8963
 2:                    Pop       40.27116        5694     223.3444
 3:                   Rock       37.57563        2413     233.6300
 4:               R&B/Soul       34.93709         302     227.6889
 5:           Folk/Country       34.65934          91     205.5994
 6:       Dance/Electronic       32.76378         254     246.2988
 7:                Hip Hop       32.50935         214     232.4100
 8:                  Other       26.09831         946     208.5390
 9:             Jazz/Blues       22.24074          54     168.0612
10: Classical/Instrumental       14.75000           4     159.5265
# Sort by multiple columns
year_genre_stats[order(release_year, -avg_popularity)]
    release_year avg_popularity track_count avg_energy
           <num>          <num>       <int>      <num>
 1:         1956       68.00000           1  0.6600000
 2:         1958       55.33333           6  0.5676667
 3:         1959       35.00000           2  0.2385000
 4:         1960       61.00000           2  0.8260000
 5:         1961       24.16667           6  0.6656667
 6:         1962       33.57895          19  0.5368947
 7:         1963       32.13333          45  0.5427844
 8:         1964       40.00000          46  0.6658696
 9:         1965       52.51613          31  0.5777097
10:         1966       33.30233          43  0.6424419
11:         1967       44.56250          48  0.5366667
12:         1968       43.87805          41  0.5942927
13:         1969       48.48889          45  0.5497111
14:         1970       50.62500          56  0.5608393
15:         1971       42.90909          44  0.6505000
16:         1972       42.32500          40  0.5486250
17:         1973       49.07273          55  0.5865455
18:         1974       38.13333          30  0.6115667
19:         1975       43.42500          40  0.6191250
20:         1976       50.14634          41  0.6497317
21:         1977       50.55172          58  0.6022759
22:         1978       48.56604          53  0.6504528
23:         1979       44.45283          53  0.6116604
24:         1980       51.33333          45  0.6395556
25:         1981       51.20000          45  0.6677111
26:         1982       47.80851          47  0.6705745
27:         1983       42.55357          56  0.7012679
28:         1984       48.64815          54  0.6755185
29:         1985       40.46154          65  0.7196308
30:         1986       50.05556          72  0.6955417
31:         1987       36.12844         109  0.6350550
32:         1988       40.35294          85  0.6454235
33:         1989       44.08974          78  0.6550231
34:         1990       42.80247          81  0.6516914
35:         1991       38.70787          89  0.6743933
36:         1992       37.28409          88  0.6474659
37:         1993       38.81609          87  0.6679770
38:         1994       39.17073          82  0.6669024
39:         1995       37.00781         128  0.6582813
40:         1996       37.84211         114  0.6303070
41:         1997       39.68217         129  0.7040884
42:         1998       36.08088         136  0.7423088
43:         1999       43.38400         125  0.7390288
44:         2000       36.28859         149  0.7102148
45:         2001       37.68280         186  0.7078441
46:         2002       37.61765         170  0.6864525
47:         2003       36.37500         176  0.7010170
48:         2004       38.66857         175  0.7082629
49:         2005       30.91250         240  0.7182292
50:         2006       33.20321         187  0.6895882
51:         2007       32.03167         221  0.7046335
52:         2008       32.25574         305  0.7465705
53:         2009       31.95455         374  0.7561578
54:         2010       32.20896         335  0.7757433
55:         2011       33.42559         383  0.7296240
56:         2012       35.45299         351  0.7361425
57:         2013       31.26780         295  0.7019861
58:         2014       32.59850         401  0.6773738
59:         2015       32.12250         400  0.6740375
60:         2016       33.14550         378  0.6490534
61:         2017       36.43391         348  0.6564569
62:         2018       43.81609         261  0.6452682
63:         2019       48.44550         211  0.6600806
64:         2020       49.54688         256  0.6631406
65:         2021       41.00966         207  0.6741739
66:         2022       68.91667          60  0.7245000
67:         2023       76.78788          33  0.6893333
68:           NA       36.23457        1377         NA
    release_year avg_popularity track_count avg_energy
# Get top 5 most popular genres
genre_stats[order(-avg_popularity)][1:5]
         main_genre avg_popularity track_count avg_duration
             <char>          <num>       <int>        <num>
1: Reggae/Caribbean       48.22222          27     210.8963
2:              Pop       40.27116        5694     223.3444
3:             Rock       37.57563        2413     233.6300
4:         R&B/Soul       34.93709         302     227.6889
5:     Folk/Country       34.65934          91     205.5994

Chaining Operations

Data.table allows you to chain operations efficiently:

# Complex analysis chain
spotify_dt[popularity > 70
          ][, .(avg_energy = mean(energy),
                track_count = .N),
            by = main_genre
          ][order(-avg_energy)]
         main_genre avg_energy track_count
             <char>      <num>       <int>
1: Dance/Electronic  0.7640625          16
2:              Pop  0.6937229        1065
3:             Rock  0.6842514         395
4: Reggae/Caribbean  0.6568333           6
5:          Hip Hop  0.6486471          34
6:         R&B/Soul  0.6468684          38
7:            Other  0.6176429          28
8:     Folk/Country  0.5845833          12
  1. Special Symbols
  • .N: Counts rows (like n() in tidyverse)
  • .SD: Subset of data (useful for applying functions to multiple columns)
  • .(): Create a list (like select() or for multiple aggregations)
# Use .SD for multiple column operations
spotify_dt[, lapply(.SD, mean), 
          .SDcols = c("danceability", "energy", "valence"),
          by = main_genre]
                main_genre danceability    energy   valence
                    <char>        <num>     <num>     <num>
 1:       Dance/Electronic    0.6779331 0.8231969 0.5845252
 2:                    Pop    0.6243319 0.6954419 0.5665183
 3:                   Rock    0.5480435 0.6759753 0.5957726
 4:                Hip Hop    0.7607477 0.7146449 0.6088201
 5:                  Other           NA        NA        NA
 6:               R&B/Soul    0.6557980 0.6104570 0.6947848
 7:             Jazz/Blues    0.5241852 0.5895185 0.7212222
 8:           Folk/Country    0.5683407 0.5402066 0.5439121
 9:       Reggae/Caribbean    0.7564815 0.6411481 0.7690000
10: Classical/Instrumental    0.5580000 0.5570000 0.6500000

Performance Tips (Extra)

  1. set key columns for faster subsetting:

# Set key for faster lookups setkey(spotify_dt, main_genre) # Fast subset using key spotify_dt["Rock"]

# Set key for faster lookups 

setkey(spotify_dt, main_genre)  

# Fast subset using key 

spotify_dt["Rock"]
Key: <main_genre>
                                                   track_name
                                                       <char>
   1:                        Apeman - 2014 Remastered Version
   2:                      You Can't Always Get What You Want
   3:                              Don't Stop - 2004 Remaster
   4: Something About The Way You Look Tonight - Edit Version
   5:                                           Juke Box Hero
  ---                                                        
2409:                                     Blowin' in the Wind
2410:                                           She's My Baby
2411:                      Yellow Submarine - Remastered 2009
2412:                                     When You Were Young
2413:                                     Prisoner of Society
           artist_name_s
                  <char>
   1:          The Kinks
   2: The Rolling Stones
   3:      Fleetwood Mac
   4:         Elton John
   5:          Foreigner
  ---                   
2409:          Bob Dylan
2410:     Johnny O'Keefe
2411:        The Beatles
2412:        The Killers
2413:     The Living End
                                                                  album_name
                                                                      <char>
   1: Lola vs. Powerman and the Moneygoround, Pt. One + Percy (Super Deluxe)
   2:                                                           Let It Bleed
   3:                                                                Rumours
   4:                          Candle In The Wind 1997 / Something About ...
   5:                                                           4 (Expanded)
  ---                                                                       
2409:                                             The Freewheelin' Bob Dylan
2410:                                            The Very Best Of (Standard)
2411:                                                  Revolver (Remastered)
2412:                                                             Sam's Town
2413:                                                         The Living End
      album_artist_name_s album_release_date disc_number track_number
                   <char>             <char>       <num>        <num>
   1:           The Kinks         2014-10-20           1           11
   2:  The Rolling Stones         1969-12-05           1            9
   3:       Fleetwood Mac         1977-02-04           1            4
   4:          Elton John         1997-01-01           1            1
   5:           Foreigner               1981           1            2
  ---                                                                
2409:           Bob Dylan         1963-05-27           1            1
2410:      Johnny O'Keefe         2008-08-23           1            4
2411:         The Beatles         1966-08-05           1            6
2412:         The Killers         2006-10-03           1            3
2413:      The Living End         1999-01-15           1            1
      track_duration_ms explicit popularity         isrc
                  <num>   <lgcl>      <num>       <char>
   1:            233400    FALSE         42 GB5KW1499822
   2:            448720    FALSE          0 USA176910100
   3:            193346    FALSE         79 USWB10400049
   4:            240546    FALSE         61 GBAMS9700013
   5:            259800    FALSE         74 USAT20803006
  ---                                                   
2409:            165426    FALSE         71 USSM19900534
2410:            125226    FALSE         16 AUWA00800460
2411:            158880    FALSE         61 GBAYE0601498
2412:            220426    FALSE          1 USUM70605164
2413:            229066    FALSE         47 AUMPO9800004
                      added_by            added_at
                        <char>              <POSc>
   1: spotify:user:bradnumber1 2021-08-08 09:26:31
   2: spotify:user:bradnumber1 2021-08-08 09:26:31
   3: spotify:user:bradnumber1 2022-08-31 00:08:18
   4: spotify:user:bradnumber1 2021-08-08 09:26:31
   5: spotify:user:bradnumber1 2021-08-08 09:26:31
  ---                                             
2409: spotify:user:bradnumber1 2021-08-08 09:26:31
2410: spotify:user:bradnumber1 2021-08-08 09:26:31
2411: spotify:user:bradnumber1 2021-08-08 09:26:31
2412: spotify:user:bradnumber1 2021-08-08 09:26:31
2413: spotify:user:bradnumber1 2021-08-08 09:26:31
                                                                                                                artist_genres
                                                                                                                       <char>
   1: album rock,art rock,british invasion,classic rock,folk rock,glam rock,protopunk,psychedelic rock,rock,singer-songwriter
   2:                                                                           album rock,british invasion,classic rock,rock
   3:                                                                       album rock,classic rock,rock,soft rock,yacht rock
   4:                                                                                   glam rock,mellow gold,piano rock,rock
   5:                                  album rock,classic rock,glam metal,hard rock,heartland rock,mellow gold,rock,soft rock
  ---                                                                                                                        
2409:                                              classic rock,country rock,folk,folk rock,rock,roots rock,singer-songwriter
2410:                                                                                                         australian rock
2411:                                               beatlesque,british invasion,classic rock,merseybeat,psychedelic rock,rock
2412:                                                             alternative rock,dance rock,modern rock,permanent wave,rock
2413:                                                              australian alternative rock,australian rock,australian ska
      danceability energy   key loudness  mode speechiness acousticness
             <num>  <num> <num>    <num> <num>       <num>        <num>
   1:        0.683 0.7280     9   -8.920     1      0.2590     0.568000
   2:        0.319 0.6270     0   -9.611     1      0.0687     0.675000
   3:        0.671 0.7100     9   -7.724     1      0.0356     0.039300
   4:        0.480 0.6280     6   -7.643     1      0.0262     0.174000
   5:        0.357 0.6530     9   -5.554     1      0.0654     0.082800
  ---                                                                  
2409:        0.380 0.0993     2  -20.567     1      0.0509     0.914000
2410:        0.582 0.7720     2   -5.028     1      0.0332     0.691000
2411:        0.604 0.5490     1   -9.873     1      0.0389     0.531000
2412:        0.467 0.9880    11   -3.313     1      0.1120     0.000152
2413:        0.275 0.9930     9   -5.474     0      0.1380     0.000186
      instrumentalness liveness valence   tempo time_signature album_genres
                 <num>    <num>   <num>   <num>          <num>       <lgcl>
   1:         5.08e-05   0.0384   0.833  75.311              4           NA
   2:         7.29e-05   0.2890   0.497  85.818              4           NA
   3:         1.12e-05   0.0387   0.834 118.745              4           NA
   4:         3.28e-05   0.0753   0.541 143.412              4           NA
   5:         0.00e+00   0.0844   0.522 176.647              4           NA
  ---                                                                      
2409:         2.32e-05   0.0605   0.440 174.874              4           NA
2410:         1.75e-04   0.1240   0.731 131.613              4           NA
2411:         0.00e+00   0.4380   0.696 111.398              4           NA
2412:         4.84e-02   0.2800   0.321 130.435              4           NA
2413:         0.00e+00   0.3020   0.190 148.725              4           NA
                      label
                     <char>
   1:     Sanctuary Records
   2: Universal Music Group
   3:  Rhino/Warner Records
   4:                   EMI
   5:        Rhino Atlantic
  ---                      
2409:              Columbia
2410:          WM Australia
2411:         EMI Catalogue
2412: Universal Music Group
2413:               Reprise
                                                                                                                                       copyrights
                                                                                                                                           <char>
   1:                                  C © 2014 Sanctuary Records Group Ltd., a BMG Company, P ℗ 2014 Sanctuary Records Group Ltd., a BMG Company
   2:                                                                    C © 2002 ABKCO Music & Records Inc., P ℗ 2002 ABKCO Music & Records Inc.
   3:                                                                                  C © 2004 Warner Records Inc., P ℗ 2004 Warner Records Inc.
   4:                                                         C © 1997 Mercury Records Limited, P This Compilation ℗ 1997 Mercury Records Limited
   5: C © 2002 Atlantic Recording Corp., marketed by Rhino Entertainment Company, a Warner Music Group company, P ℗ 2002 Atlantic Recording Corp.
  ---                                                                                                                                            
2409:                                                                                    P Originally Released 1963 Sony Music Entertainment Inc.
2410:                                                    C © 2008 Warner Music Australia Pty Limited, P ℗ 2008 Warner Music Australia Pty Limited
2411:                                   C © 2015 Apple Corps Ltd., P ℗ 2015 Calderstone Productions Limited (a division of Universal Music Group)
2412:                                                            C © 2006 The Island Def Jam Music Group, P ℗ 2006 The Island Def Jam Music Group
2413:                                                                                    C © 1999 Reprise Records, P ℗ 1998, 1999 Reprise Records
      main_genre duration_sec popularity_scaled is_popular
          <char>        <num>             <num>     <lgcl>
   1:       Rock      233.400              0.42      FALSE
   2:       Rock      448.720              0.00      FALSE
   3:       Rock      193.346              0.79       TRUE
   4:       Rock      240.546              0.61      FALSE
   5:       Rock      259.800              0.74      FALSE
  ---                                                     
2409:       Rock      165.426              0.71      FALSE
2410:       Rock      125.226              0.16      FALSE
2411:       Rock      158.880              0.61      FALSE
2412:       Rock      220.426              0.01      FALSE
2413:       Rock      229.066              0.47      FALSE
      loudness_energy_ratio release_date release_year release_month release_day
                      <num>       <Date>        <num>         <num>       <int>
   1:            -12.252747   2014-10-20         2014            10          20
   2:            -15.328549   1969-12-05         1969            12           5
   3:            -10.878873   1977-02-04         1977             2           4
   4:            -12.170382   1997-01-01         1997             1           1
   5:             -8.505360         <NA>           NA            NA          NA
  ---                                                                          
2409:           -207.119839   1963-05-27         1963             5          27
2410:             -6.512953   2008-08-23         2008             8          23
2411:            -17.983607   1966-08-05         1966             8           5
2412:             -3.353239   2006-10-03         2006            10           3
2413:             -5.512588   1999-01-15         1999             1          15
  1. Avoid copies with :=:
# More efficient than creating new objects 
spotify_dt[, efficiency_ratio := energy/danceability]
  1. Use .SDcols for column operations:
# Efficient multiple column operations 
audio_features <- c("danceability", "energy", "valence") 

spotify_dt[, lapply(.SD, mean), .SDcols = audio_features, by = main_genre]
Key: <main_genre>
                main_genre danceability    energy   valence
                    <char>        <num>     <num>     <num>
 1: Classical/Instrumental    0.5580000 0.5570000 0.6500000
 2:       Dance/Electronic    0.6779331 0.8231969 0.5845252
 3:           Folk/Country    0.5683407 0.5402066 0.5439121
 4:                Hip Hop    0.7607477 0.7146449 0.6088201
 5:             Jazz/Blues    0.5241852 0.5895185 0.7212222
 6:                  Other           NA        NA        NA
 7:                    Pop    0.6243319 0.6954419 0.5665183
 8:               R&B/Soul    0.6557980 0.6104570 0.6947848
 9:       Reggae/Caribbean    0.7564815 0.6411481 0.7690000
10:                   Rock    0.5480435 0.6759753 0.5957726

Watch out for these common mistakes:

  • Forgetting the .() when selecting multiple columns
  • Using $ instead of := for assignments
  • Forgetting to add [] at the end when you want to see results

Remember:

  • tidyverse is great for readability and smaller datasets
  • data.table shines with big data and performance needs
  • Both have their place in your R toolkit

🎊🎉 Congratulations you finished week 4 and now on to a mini project analysis using data.table functions.

💡 Mini-Project Guidelines:

Create a comprehensive analysis using data.table that:

Demonstrates at least three different data.table operations Includes performance comparisons with equivalent dplyr operations Creates insightful visualizations from the results

Remember to:

  • Document your code

  • Explain your analysis choices

  • Consider performance implications

  • Think about scalability

Mini Project - Popularity by Genres/Tempo-Energy Ratio

I wanted to analyze the average popularity between the genres to identify trends and figure out which genre is the most popular and least popular.

# Average Popularity by Genre
spotify_dt_summary <- spotify_dt[, .(avg_popularity = mean(popularity, na.rm = TRUE)), by = main_genre]
spotify_dt_summary
Key: <main_genre>
                main_genre avg_popularity
                    <char>          <num>
 1: Classical/Instrumental       14.75000
 2:       Dance/Electronic       32.76378
 3:           Folk/Country       34.65934
 4:                Hip Hop       32.50935
 5:             Jazz/Blues       22.24074
 6:                  Other       26.09831
 7:                    Pop       40.27116
 8:               R&B/Soul       34.93709
 9:       Reggae/Caribbean       48.22222
10:                   Rock       37.57563
# dplyr operation of Average Popularity by Genre
spotify_dt_summarydplyr <- spotify_dt %>% group_by(main_genre) %>% summarise(avg_popularity = mean(popularity, na.rm = TRUE))
spotify_dt_summarydplyr
# A tibble: 10 × 2
   main_genre             avg_popularity
   <chr>                           <dbl>
 1 Classical/Instrumental           14.8
 2 Dance/Electronic                 32.8
 3 Folk/Country                     34.7
 4 Hip Hop                          32.5
 5 Jazz/Blues                       22.2
 6 Other                            26.1
 7 Pop                              40.3
 8 R&B/Soul                         34.9
 9 Reggae/Caribbean                 48.2
10 Rock                             37.6

Here I have filtered out the top tracks by popularity and specifically in the Hip-Hop genre. By filtering out top tracks based on certain criteria it allows us to focus on a subset of the data for deeper analysis. Could be useful for exploring highly rated songs within the “Hip-Hop” genre.

# Filtering top tracks by popularity and Hip Hop genre
dt_top_tracks <- spotify_dt[popularity > 75 & main_genre == "Hip Hop"]
dt_top_tracks
Key: <main_genre>
                                        track_name
                                            <char>
 1:                              THATS WHAT I WANT
 2:                            HIGHEST IN THE ROOM
 3:                                        The Box
 4:                     Gangnam Style (강남스타일)
 5:                                     Without Me
 6:                                        Hey Ya!
 7:                             U Can't Touch This
 8:                                        HUMBLE.
 9:                                          Crazy
10:                                   Ready or Not
11: STAR WALKIN' (League of Legends Worlds Anthem)
12:                                       Regulate
13:                Killing Me Softly With His Song
14:                                           Trip
15:                                  Lose Yourself
16:                                         DÁKITI
17:                   ROCKSTAR (feat. Roddy Ricch)
18:                                      Heartless
19:        For The Night (feat. Lil Baby & DaBaby)
20:                                        Players
21:                                       Go Crazy
22:                    Godzilla (feat. Juice WRLD)
23:      Praise The Lord (Da Shine) (feat. Skepta)
24:                     edamame (feat. Rich Brian)
25:                                       Mask Off
26:                                   Lucid Dreams
27:                                     Not Afraid
28:                                           SAD!
29:                                    Ms. Jackson
30:                                     SICKO MODE
                                        track_name
                                 artist_name_s
                                        <char>
 1:                                  Lil Nas X
 2:                               Travis Scott
 3:                                Roddy Ricch
 4:                                        PSY
 5:                                     Eminem
 6:                                    Outkast
 7:                                  MC Hammer
 8:                             Kendrick Lamar
 9:                             Gnarls Barkley
10: Fugees, Ms. Lauryn Hill, Wyclef Jean, Pras
11:                                  Lil Nas X
12:                        Warren G, Nate Dogg
13:                    Fugees, Ms. Lauryn Hill
14:                                   Ella Mai
15:                                     Eminem
16:                          Bad Bunny, Jhayco
17:                        DaBaby, Roddy Ricch
18:                                 Kanye West
19:                Pop Smoke, Lil Baby, DaBaby
20:                                  Coi Leray
21:                    Chris Brown, Young Thug
22:                         Eminem, Juice WRLD
23:                         A$AP Rocky, Skepta
24:                          bbno$, Rich Brian
25:                                     Future
26:                                 Juice WRLD
27:                                     Eminem
28:                               XXXTENTACION
29:                                    Outkast
30:                               Travis Scott
                                 artist_name_s
                                        album_name
                                            <char>
 1:                                        MONTERO
 2:                            HIGHEST IN THE ROOM
 3:          Please Excuse Me for Being Antisocial
 4:                     Gangnam Style (강남스타일)
 5:                                The Eminem Show
 6:                    Speakerboxxx/The Love Below
 7:                   Please Hammer Don't Hurt 'Em
 8:                                          DAMN.
 9:                                  St. Elsewhere
10:                   The Score (Expanded Edition)
11: STAR WALKIN' (League of Legends Worlds Anthem)
12:                           Regulate… G Funk Era
13:                   The Score (Expanded Edition)
14:                                       Ella Mai
15:                                        SHADYXV
16:                       EL ÚLTIMO TOUR DEL MUNDO
17:                               BLAME IT ON BABY
18:                              808s & Heartbreak
19:           Shoot For The Stars Aim For The Moon
20:                                        Players
21:                                      Slime & B
22:                        Music To Be Murdered By
23:                                        TESTING
24:                     edamame (feat. Rich Brian)
25:                                         FUTURE
26:                        Goodbye & Good Riddance
27:                                       Recovery
28:                                              ?
29:                                      Stankonia
30:                                     ASTROWORLD
                                        album_name
                     album_artist_name_s album_release_date disc_number
                                  <char>             <char>       <num>
 1:                            Lil Nas X         2021-09-17           1
 2:                         Travis Scott         2019-10-04           1
 3:                          Roddy Ricch         2019-12-06           1
 4:                                  PSY         2012-01-01           1
 5:                               Eminem         2002-05-26           1
 6:                              Outkast               2003           2
 7:                            MC Hammer         1990-02-20           1
 8:                       Kendrick Lamar         2017-04-14           1
 9:                       Gnarls Barkley         2006-04-24           1
10: Fugees, Ms. Lauryn Hill, Wyclef Jean         1996-02-13           1
11:                            Lil Nas X         2022-09-22           1
12:                             Warren G         1994-06-07           1
13: Fugees, Ms. Lauryn Hill, Wyclef Jean         1996-02-13           1
14:                             Ella Mai         2018-10-12           1
15:                      Various Artists         2014-11-24           2
16:                            Bad Bunny         2020-11-27           1
17:                               DaBaby         2020-04-17           1
18:                           Kanye West         2008-11-24           1
19:                            Pop Smoke         2020-07-03           1
20:                            Coi Leray         2022-11-30           1
21:              Chris Brown, Young Thug         2020-05-08           1
22:                               Eminem         2020-01-17           1
23:                           A$AP Rocky         2018-05-25           1
24:                    bbno$, Rich Brian         2021-07-24           1
25:                               Future         2017-06-30           1
26:                           Juice WRLD         2018-12-10           1
27:                               Eminem         2010-06-18           1
28:                         XXXTENTACION         2018-03-16           1
29:                              Outkast         2000-10-31           1
30:                         Travis Scott         2018-08-03           1
                     album_artist_name_s album_release_date disc_number
    track_number track_duration_ms explicit popularity         isrc
           <num>             <num>   <lgcl>      <num>       <char>
 1:            4            143901     TRUE         85 USSM12105732
 2:            1            175720     TRUE         86 USSM11904276
 3:            2            196652     TRUE         85 USAT21906978
 4:            1            219493    FALSE         77 USUM71210283
 5:           10            290320     TRUE         91 USIR10211038
 6:            9            235213    FALSE         86 USAR10300924
 7:            2            257359    FALSE         77 USCA29000294
 8:            8            177000     TRUE         88 USUM71703085
 9:            2            177933    FALSE         81 GBAHT0600202
10:            3            226986     TRUE         76 USSM19600051
11:            1            210575     TRUE         84 USSM12208809
12:            1            248866     TRUE         78 USPR39401366
13:            8            298773     TRUE         81 USSM19600055
14:           13            213993    FALSE         77 USUM71810720
15:            3            320626     TRUE         76 USIR10211559
16:           11            205090     TRUE         82 QMFME2004132
17:            7            181733     TRUE         81 USUM72007941
18:            3            211000    FALSE         88 USUM70840511
19:            3            190476     TRUE         82 USUM72013355
20:            1            139560     TRUE         88 USUM72222387
21:            2            176960     TRUE         79 USRC12001468
22:            7            210800     TRUE         85 USUM72000788
23:            5            205040     TRUE         86 USRC11800930
24:            1            133706     TRUE         76 QMUY42100151
25:            7            204600     TRUE         85 USSM11701444
26:            3            239835     TRUE         87 USUG11800685
27:            7            248133     TRUE         83 USUM71011769
28:            4            166605     TRUE         84 USUG11800208
29:            5            270506     TRUE         85 USLF20000357
30:            3            312820     TRUE         87 USSM11806660
    track_number track_duration_ms explicit popularity         isrc
                    added_by            added_at
                      <char>              <POSc>
 1: spotify:user:bradnumber1 2022-04-27 22:11:04
 2: spotify:user:bradnumber1 2021-08-08 09:26:31
 3: spotify:user:bradnumber1 2021-08-08 09:26:31
 4: spotify:user:bradnumber1 2021-08-08 09:26:31
 5: spotify:user:bradnumber1 2021-08-08 09:26:31
 6: spotify:user:bradnumber1 2021-08-08 09:26:31
 7: spotify:user:bradnumber1 2021-08-08 09:26:31
 8: spotify:user:bradnumber1 2021-08-08 09:26:31
 9: spotify:user:bradnumber1 2021-08-08 09:26:31
10: spotify:user:bradnumber1 2021-08-08 09:26:31
11: spotify:user:bradnumber1 2022-12-11 23:08:28
12: spotify:user:bradnumber1 2021-08-08 09:26:31
13: spotify:user:bradnumber1 2021-08-08 09:26:31
14: spotify:user:bradnumber1 2021-08-08 09:26:31
15: spotify:user:bradnumber1 2021-08-08 09:26:31
16: spotify:user:bradnumber1 2021-08-08 09:26:31
17: spotify:user:bradnumber1 2021-08-08 09:26:31
18: spotify:user:bradnumber1 2021-08-08 09:26:31
19: spotify:user:bradnumber1 2021-08-08 09:26:31
20: spotify:user:bradnumber1 2023-05-11 07:58:24
21: spotify:user:bradnumber1 2021-08-08 09:26:31
22: spotify:user:bradnumber1 2021-08-08 09:26:31
23: spotify:user:bradnumber1 2021-08-08 09:26:31
24: spotify:user:bradnumber1 2022-01-04 09:42:24
25: spotify:user:bradnumber1 2021-08-08 09:26:31
26: spotify:user:bradnumber1 2021-08-08 09:26:31
27: spotify:user:bradnumber1 2021-08-08 09:26:31
28: spotify:user:bradnumber1 2021-08-08 09:26:31
29: spotify:user:bradnumber1 2021-08-08 09:26:31
30: spotify:user:bradnumber1 2021-08-08 09:26:31
                    added_by            added_at
                                                                                                                                              artist_genres
                                                                                                                                                     <char>
 1:                                                                                                                                          lgbtq+ hip hop
 2:                                                                                                                                  hip hop,rap,slap house
 3:                                                                                                                                    melodic rap,rap,trap
 4:                                                                                                                         k-rap,korean old school hip hop
 5:                                                                                                                             detroit hip hop,hip hop,rap
 6:                                                                     atl hip hop,dirty south rap,hip hop,old school atlanta hip hop,rap,southern hip hop
 7:                                                                                                                                                 hip hop
 8:                                                                                                            conscious hip hop,hip hop,rap,west coast rap
 9:                                                                                                                                    atl hip hop,neo soul
10: east coast hip hop,hip hop,neo soul,new jersey rap,alternative r&b,conscious hip hop,hip hop,neo soul,new jersey rap,r&b,soul,rap kreyol,new jersey rap
11:                                                                                                                                          lgbtq+ hip hop
12:                                 g funk,gangster rap,hardcore hip hop,hip hop,west coast rap,g funk,gangster rap,hardcore hip hop,hip hop,west coast rap
13:                           east coast hip hop,hip hop,neo soul,new jersey rap,alternative r&b,conscious hip hop,hip hop,neo soul,new jersey rap,r&b,soul
14:                                                                                                                             r&b,rap,uk contemporary r&b
15:                                                                                                                             detroit hip hop,hip hop,rap
16:                                                                                 reggaeton,trap latino,urbano latino,reggaeton,trap latino,urbano latino
17:                                                                                            hip hop,north carolina hip hop,rap,trap,melodic rap,rap,trap
18:                                                                                                                                 chicago rap,hip hop,rap
19:                                                                brooklyn drill,rap,atl hip hop,atl trap,rap,trap,hip hop,north carolina hip hop,rap,trap
20:                                                                                                                   new jersey underground rap,trap queen
21:                                                                                          r&b,rap,atl hip hop,atl trap,gangster rap,melodic rap,rap,trap
22:                                                                                                 detroit hip hop,hip hop,rap,chicago rap,melodic rap,rap
23:                                                                east coast hip hop,hip hop,rap,trap,birmingham grime,grime,instrumental grime,uk hip hop
24:                                                                                                  canadian hip hop,dark trap,meme rap,indonesian hip hop
25:                                                                                                           atl hip hop,hip hop,rap,southern hip hop,trap
26:                                                                                                                             chicago rap,melodic rap,rap
27:                                                                                                                             detroit hip hop,hip hop,rap
28:                                                                                                                                   emo rap,miami hip hop
29:                                                                     atl hip hop,dirty south rap,hip hop,old school atlanta hip hop,rap,southern hip hop
30:                                                                                                                                  hip hop,rap,slap house
                                                                                                                                              artist_genres
    danceability energy   key loudness  mode speechiness acousticness
           <num>  <num> <num>    <num> <num>       <num>        <num>
 1:        0.737  0.846     1   -4.510     0      0.2200     0.006140
 2:        0.598  0.427     7   -8.764     0      0.0317     0.054600
 3:        0.896  0.586    10   -6.687     0      0.0559     0.104000
 4:        0.727  0.937    11   -2.871     0      0.2860     0.004170
 5:        0.908  0.669     7   -2.827     1      0.0738     0.002860
 6:        0.727  0.974     4   -2.261     0      0.0664     0.103000
 7:        0.867  0.517    11  -13.222     0      0.0875     0.004560
 8:        0.908  0.621     1   -6.638     0      0.1020     0.000282
 9:        0.855  0.587     8   -4.589     1      0.0336     0.050500
10:        0.607  0.422    10  -13.364     0      0.4990     0.209000
11:        0.637  0.715     2   -4.971     0      0.0455     0.148000
12:        0.849  0.521    11  -12.999     0      0.2160     0.304000
13:        0.765  0.290     4  -17.117     0      0.1390     0.027900
14:        0.477  0.610    11   -5.628     0      0.1440     0.225000
15:        0.689  0.735     2   -4.545     1      0.2670     0.009220
16:        0.731  0.573     4  -10.059     0      0.0544     0.401000
17:        0.746  0.690    11   -7.956     1      0.1640     0.247000
18:        0.790  0.647    10   -5.983     0      0.1360     0.051500
19:        0.823  0.586     6   -6.606     0      0.2000     0.114000
20:        0.954  0.516     6   -5.817     1      0.1600     0.030000
21:        0.755  0.578     0   -8.932     0      0.1450     0.316000
22:        0.808  0.745    10   -5.260     0      0.3420     0.145000
23:        0.854  0.569     5   -8.151     0      0.1060     0.060900
24:        0.815  0.848     4   -4.704     0      0.1150     0.023000
25:        0.833  0.434     2   -8.795     1      0.4310     0.010200
26:        0.511  0.566     6   -7.230     0      0.2000     0.349000
27:        0.855  0.954     0   -1.190     0      0.2640     0.529000
28:        0.740  0.613     8   -4.880     1      0.1450     0.258000
29:        0.843  0.806     4   -5.946     0      0.2690     0.143000
30:        0.834  0.730     8   -3.714     1      0.2220     0.005130
    danceability energy   key loudness  mode speechiness acousticness
    instrumentalness liveness valence   tempo time_signature album_genres
               <num>    <num>   <num>   <num>          <num>       <lgcl>
 1:         0.00e+00   0.0486  0.5460  87.981              4           NA
 2:         5.83e-06   0.2100  0.0605  76.469              4           NA
 3:         0.00e+00   0.7900  0.6420 116.971              4           NA
 4:         0.00e+00   0.0910  0.7490 132.067              4           NA
 5:         0.00e+00   0.2370  0.6620 112.238              4           NA
 6:         5.32e-04   0.1740  0.9650  79.526              4           NA
 7:         3.39e-04   0.0864  0.8660 133.148              4           NA
 8:         5.39e-05   0.0958  0.4210 150.011              4           NA
 9:         7.14e-03   0.1050  0.6400 111.970              4           NA
10:         0.00e+00   0.0839  0.5630 130.470              3           NA
11:         0.00e+00   0.0892  0.3080 141.872              4           NA
12:         1.65e-06   0.1150  0.7630  95.265              4           NA
13:         0.00e+00   0.5710  0.5060  92.409              4           NA
14:         0.00e+00   0.1070  0.3580  79.882              4           NA
15:         7.20e-04   0.3650  0.0590 171.403              4           NA
16:         5.22e-05   0.1130  0.1450 109.928              4           NA
17:         0.00e+00   0.1010  0.4970  89.977              4           NA
18:         0.00e+00   0.2480  0.6540  87.999              4           NA
19:         0.00e+00   0.1930  0.3470 125.971              4           NA
20:         7.54e-06   0.0504  0.6240 105.001              4           NA
21:         0.00e+00   0.2500  0.5810  94.148              4           NA
22:         0.00e+00   0.2920  0.8290 165.995              4           NA
23:         8.16e-02   0.1000  0.2940  80.015              4           NA
24:         1.62e-04   0.0265  0.6840 106.032              4           NA
25:         2.19e-02   0.1650  0.2810 150.062              4           NA
26:         0.00e+00   0.3400  0.2180  83.903              4           NA
27:         0.00e+00   0.2050  0.6680 114.635              5           NA
28:         3.72e-03   0.1230  0.4730  75.023              4           NA
29:         0.00e+00   0.0771  0.6130  94.948              4           NA
30:         0.00e+00   0.1240  0.4460 155.008              4           NA
    instrumentalness liveness valence   tempo time_signature album_genres
                                                      label
                                                     <char>
 1:                                                Columbia
 2:                                      Cactus Jack / Epic
 3:                                        Atlantic Records
 4:               Silent Records/Universal Republic Records
 5:                                               Aftermath
 6:                                                  Arista
 7:                                   EMI/EMI Records (USA)
 8:                                               Aftermath
 9:                                                   WM UK
10:                                                Columbia
11:                                                Columbia
12:                                          Island Records
13:                                                Columbia
14:                              10 Summers / Interscope PS
15:                                                      3H
16:                                 Rimas Entertainment LLC
17:              South Coast Music Group/Interscope Records
18:                                             Roc-A-Fella
19:                                 Victor Victor Worldwide
20:                               Uptown / Republic Records
21: Chris Brown Entertainment/300 Entertainment/RCA Records
22:                      Shady/Aftermath/Interscope Records
23:           A$AP Worldwide/Polo Grounds Music/RCA Records
24:          bbno$, Rich Brian appears courtesy of 88Rising
25:                                       Epic/Freebandz/A1
26:                            Juice WRLD Mixtape / ISR P&D
27:                                               Aftermath
28:                                  Bad Vibes Forever, LLC
29:                                   Arista/LaFace Records
30:                       Cactus Jack / Epic / Grand Hustle
                                                      label
                                                                                                                                                                                                                                                                                                                              copyrights
                                                                                                                                                                                                                                                                                                                                  <char>
 1:                                                                                                                                                                                                                                                                  P (P) 2021 Columbia Records, a Division of Sony Music Entertainment
 2:                                                                                                                                                                                                                                                                                           P (P) 2019 Epic Records. With Cactus Jack.
 3: C © 2019 Atlantic Recording Corporation for the United States and WEA International Inc. for the world outside of the United States. A Warner Music Group Company, P ℗ 2019 Atlantic Recording Corporation for the United States and WEA International Inc. for the world outside of the United States. A Warner Music Group Company
 4:                                                                                                                                                                 C © 2012 Schoolboy/Universal Republic Records, a division of UMG Recordings, Inc., P ℗ 2012 Schoolboy/Universal Republic Records, a division of UMG Recordings, Inc.
 5:                                                                                                                                                                                                                                                                               C © 2002 Aftermath Records, P ℗ 2002 Aftermath Records
 6:                                                                                                                                                                                                                                                                                                        P (P) 2003 Arista Records LLC
 7:                                                                                                                                                                                                                                                                         C © 1990 Capitol Records, LLC, P ℗ 1990 Capitol Records, LLC
 8:                                                                                                                                                                                                                       C © 2017 Aftermath/Interscope (Top Dawg Entertainment), P ℗ 2017 Aftermath/Interscope (Top Dawg Entertainment)
 9:                                                                                                                                                                                      C © 2006 Gnarls Barkley under exclusive license to Warner Music UK Ltd, P ℗ 2006 Gnarls Barkley under exclusive license to Warner Music UK Ltd.
10:                                                                                                                                                                                                                                                                  P (P) 1996 Columbia Records, a division of Sony Music Entertainment
11:                                                                                                                                                                                                                                                                  P (P) 2022 Columbia Records, a Division of Sony Music Entertainment
12:                                                                                                                                                                                                                                    C © 2007 The Island Def Jam Music Group, P This Compilation ℗ 2007 The Island Def Jam Music Group
13:                                                                                                                                                                                                                                                                  P (P) 1996 Columbia Records, a division of Sony Music Entertainment
14:                                                                                                                                                                                                                                                                   C © 2018 10 Summers Records, LLC, P ℗ 2018 10 Summers Records, LLC
15:                                                                                                                                                                                                                                C © 2014 Shady Records/Interscope Records, P This Compilation ℗ 2014 Shady Records/Interscope Records
16:                                                                                                                                                                                                                                                               C (C) 2020 Rimas Entertainment LLC, P (P) 2020 Rimas Entertainment LLC
17:                                                                                                                                                                                                                                                                             C © 2020 Interscope Records, P ℗ 2020 Interscope Records
18:                                                                                                                                                                                                                                                                         C © 2008 UMG Recordings, Inc., P ℗ 2008 UMG Recordings, Inc.
19:                                                                                                                                                     C © 2020 Republic Records, a division of UMG Recordings, Inc. & Victor Victor Worldwide, P ℗ 2020 Republic Records, a division of UMG Recordings, Inc. & Victor Victor Worldwide
20:                                                                                                                                                                   C © 2022 Uptown Records, a division of Republic Records and UMG Recordings, Inc., P ℗ 2022 Uptown Records, a division of Republic Records and UMG Recordings, Inc.
21:                                                                                                                                                                                                                                                    P (P) 2020 Chris Brown Entertainment, LLC, under exclusive license to RCA Records
22:                                                                                                                                                                                                                                                                   C © 2020 Marshall B. Mathers III, P ℗ 2020 Marshall B. Mathers III
23:                                                                                                                                                                                                                          P (P) 2018 A$AP Rocky Recordings LLC/RCA Records. Marketed & Distributed by Polo Grounds Music/RCA Records.
24:                                                                                                                                                                                                                         C 2021 bbno$, Rich Brian appears courtesy of 88Rising, P 2021 bbno$, Rich Brian appears courtesy of 88Rising
25:                                                                                                                                                                                                                                                  P (P) 2017 Epic Records, a division of Sony Music Entertainment with A1 / Freebandz
26:                                                                                                                                                                                           C © 2018 Grade A Productions, LLC, distributed by Interscope Records, P ℗ 2018 Grade A Productions, LLC, distributed by Interscope Records
27:                                                                                                                                                                                                                                                                               C © 2010 Aftermath Records, P ℗ 2010 Aftermath Records
28:                                                                                                                                                                                                                                                                               C © 2018 Bad Vibes Forever, P ℗ 2018 Bad Vibes Forever
29:                                                                                                                                                                                                                                                                                                        P (P) 2000 Arista Records LLC
30:                                                                                                                                                                                                                                                                     P (P) 2018 Epic Records/Sony. With Cactus Jack and Grand Hustle.
                                                                                                                                                                                                                                                                                                                              copyrights
    main_genre duration_sec popularity_scaled is_popular loudness_energy_ratio
        <char>        <num>             <num>     <lgcl>                 <num>
 1:    Hip Hop      143.901              0.85       TRUE             -5.330969
 2:    Hip Hop      175.720              0.86       TRUE            -20.524590
 3:    Hip Hop      196.652              0.85       TRUE            -11.411263
 4:    Hip Hop      219.493              0.77       TRUE             -3.064034
 5:    Hip Hop      290.320              0.91       TRUE             -4.225710
 6:    Hip Hop      235.213              0.86       TRUE             -2.321355
 7:    Hip Hop      257.359              0.77       TRUE            -25.574468
 8:    Hip Hop      177.000              0.88       TRUE            -10.689211
 9:    Hip Hop      177.933              0.81       TRUE             -7.817717
10:    Hip Hop      226.986              0.76       TRUE            -31.668246
11:    Hip Hop      210.575              0.84       TRUE             -6.952448
12:    Hip Hop      248.866              0.78       TRUE            -24.950096
13:    Hip Hop      298.773              0.81       TRUE            -59.024138
14:    Hip Hop      213.993              0.77       TRUE             -9.226230
15:    Hip Hop      320.626              0.76       TRUE             -6.183673
16:    Hip Hop      205.090              0.82       TRUE            -17.554974
17:    Hip Hop      181.733              0.81       TRUE            -11.530435
18:    Hip Hop      211.000              0.88       TRUE             -9.247295
19:    Hip Hop      190.476              0.82       TRUE            -11.273038
20:    Hip Hop      139.560              0.88       TRUE            -11.273256
21:    Hip Hop      176.960              0.79       TRUE            -15.453287
22:    Hip Hop      210.800              0.85       TRUE             -7.060403
23:    Hip Hop      205.040              0.86       TRUE            -14.325132
24:    Hip Hop      133.706              0.76       TRUE             -5.547170
25:    Hip Hop      204.600              0.85       TRUE            -20.264977
26:    Hip Hop      239.835              0.87       TRUE            -12.773852
27:    Hip Hop      248.133              0.83       TRUE             -1.247379
28:    Hip Hop      166.605              0.84       TRUE             -7.960848
29:    Hip Hop      270.506              0.85       TRUE             -7.377171
30:    Hip Hop      312.820              0.87       TRUE             -5.087671
    main_genre duration_sec popularity_scaled is_popular loudness_energy_ratio
    release_date release_year release_month release_day efficiency_ratio
          <Date>        <num>         <num>       <int>            <num>
 1:   2021-09-17         2021             9          17        1.1478969
 2:   2019-10-04         2019            10           4        0.7140468
 3:   2019-12-06         2019            12           6        0.6540179
 4:   2012-01-01         2012             1           1        1.2888583
 5:   2002-05-26         2002             5          26        0.7367841
 6:         <NA>           NA            NA          NA        1.3397524
 7:   1990-02-20         1990             2          20        0.5963091
 8:   2017-04-14         2017             4          14        0.6839207
 9:   2006-04-24         2006             4          24        0.6865497
10:   1996-02-13         1996             2          13        0.6952224
11:   2022-09-22         2022             9          22        1.1224490
12:   1994-06-07         1994             6           7        0.6136631
13:   1996-02-13         1996             2          13        0.3790850
14:   2018-10-12         2018            10          12        1.2788260
15:   2014-11-24         2014            11          24        1.0667634
16:   2020-11-27         2020            11          27        0.7838577
17:   2020-04-17         2020             4          17        0.9249330
18:   2008-11-24         2008            11          24        0.8189873
19:   2020-07-03         2020             7           3        0.7120292
20:   2022-11-30         2022            11          30        0.5408805
21:   2020-05-08         2020             5           8        0.7655629
22:   2020-01-17         2020             1          17        0.9220297
23:   2018-05-25         2018             5          25        0.6662763
24:   2021-07-24         2021             7          24        1.0404908
25:   2017-06-30         2017             6          30        0.5210084
26:   2018-12-10         2018            12          10        1.1076321
27:   2010-06-18         2010             6          18        1.1157895
28:   2018-03-16         2018             3          16        0.8283784
29:   2000-10-31         2000            10          31        0.9561091
30:   2018-08-03         2018             8           3        0.8752998
    release_date release_year release_month release_day efficiency_ratio
# dplyr operation to filter top tracks
dt_top_tracks_dplyr <- spotify_dt %>% filter(popularity > 75 & main_genre == "Hip Hop")
dt_top_tracks_dplyr
Key: <main_genre>
                                        track_name
                                            <char>
 1:                              THATS WHAT I WANT
 2:                            HIGHEST IN THE ROOM
 3:                                        The Box
 4:                     Gangnam Style (강남스타일)
 5:                                     Without Me
 6:                                        Hey Ya!
 7:                             U Can't Touch This
 8:                                        HUMBLE.
 9:                                          Crazy
10:                                   Ready or Not
11: STAR WALKIN' (League of Legends Worlds Anthem)
12:                                       Regulate
13:                Killing Me Softly With His Song
14:                                           Trip
15:                                  Lose Yourself
16:                                         DÁKITI
17:                   ROCKSTAR (feat. Roddy Ricch)
18:                                      Heartless
19:        For The Night (feat. Lil Baby & DaBaby)
20:                                        Players
21:                                       Go Crazy
22:                    Godzilla (feat. Juice WRLD)
23:      Praise The Lord (Da Shine) (feat. Skepta)
24:                     edamame (feat. Rich Brian)
25:                                       Mask Off
26:                                   Lucid Dreams
27:                                     Not Afraid
28:                                           SAD!
29:                                    Ms. Jackson
30:                                     SICKO MODE
                                        track_name
                                 artist_name_s
                                        <char>
 1:                                  Lil Nas X
 2:                               Travis Scott
 3:                                Roddy Ricch
 4:                                        PSY
 5:                                     Eminem
 6:                                    Outkast
 7:                                  MC Hammer
 8:                             Kendrick Lamar
 9:                             Gnarls Barkley
10: Fugees, Ms. Lauryn Hill, Wyclef Jean, Pras
11:                                  Lil Nas X
12:                        Warren G, Nate Dogg
13:                    Fugees, Ms. Lauryn Hill
14:                                   Ella Mai
15:                                     Eminem
16:                          Bad Bunny, Jhayco
17:                        DaBaby, Roddy Ricch
18:                                 Kanye West
19:                Pop Smoke, Lil Baby, DaBaby
20:                                  Coi Leray
21:                    Chris Brown, Young Thug
22:                         Eminem, Juice WRLD
23:                         A$AP Rocky, Skepta
24:                          bbno$, Rich Brian
25:                                     Future
26:                                 Juice WRLD
27:                                     Eminem
28:                               XXXTENTACION
29:                                    Outkast
30:                               Travis Scott
                                 artist_name_s
                                        album_name
                                            <char>
 1:                                        MONTERO
 2:                            HIGHEST IN THE ROOM
 3:          Please Excuse Me for Being Antisocial
 4:                     Gangnam Style (강남스타일)
 5:                                The Eminem Show
 6:                    Speakerboxxx/The Love Below
 7:                   Please Hammer Don't Hurt 'Em
 8:                                          DAMN.
 9:                                  St. Elsewhere
10:                   The Score (Expanded Edition)
11: STAR WALKIN' (League of Legends Worlds Anthem)
12:                           Regulate… G Funk Era
13:                   The Score (Expanded Edition)
14:                                       Ella Mai
15:                                        SHADYXV
16:                       EL ÚLTIMO TOUR DEL MUNDO
17:                               BLAME IT ON BABY
18:                              808s & Heartbreak
19:           Shoot For The Stars Aim For The Moon
20:                                        Players
21:                                      Slime & B
22:                        Music To Be Murdered By
23:                                        TESTING
24:                     edamame (feat. Rich Brian)
25:                                         FUTURE
26:                        Goodbye & Good Riddance
27:                                       Recovery
28:                                              ?
29:                                      Stankonia
30:                                     ASTROWORLD
                                        album_name
                     album_artist_name_s album_release_date disc_number
                                  <char>             <char>       <num>
 1:                            Lil Nas X         2021-09-17           1
 2:                         Travis Scott         2019-10-04           1
 3:                          Roddy Ricch         2019-12-06           1
 4:                                  PSY         2012-01-01           1
 5:                               Eminem         2002-05-26           1
 6:                              Outkast               2003           2
 7:                            MC Hammer         1990-02-20           1
 8:                       Kendrick Lamar         2017-04-14           1
 9:                       Gnarls Barkley         2006-04-24           1
10: Fugees, Ms. Lauryn Hill, Wyclef Jean         1996-02-13           1
11:                            Lil Nas X         2022-09-22           1
12:                             Warren G         1994-06-07           1
13: Fugees, Ms. Lauryn Hill, Wyclef Jean         1996-02-13           1
14:                             Ella Mai         2018-10-12           1
15:                      Various Artists         2014-11-24           2
16:                            Bad Bunny         2020-11-27           1
17:                               DaBaby         2020-04-17           1
18:                           Kanye West         2008-11-24           1
19:                            Pop Smoke         2020-07-03           1
20:                            Coi Leray         2022-11-30           1
21:              Chris Brown, Young Thug         2020-05-08           1
22:                               Eminem         2020-01-17           1
23:                           A$AP Rocky         2018-05-25           1
24:                    bbno$, Rich Brian         2021-07-24           1
25:                               Future         2017-06-30           1
26:                           Juice WRLD         2018-12-10           1
27:                               Eminem         2010-06-18           1
28:                         XXXTENTACION         2018-03-16           1
29:                              Outkast         2000-10-31           1
30:                         Travis Scott         2018-08-03           1
                     album_artist_name_s album_release_date disc_number
    track_number track_duration_ms explicit popularity         isrc
           <num>             <num>   <lgcl>      <num>       <char>
 1:            4            143901     TRUE         85 USSM12105732
 2:            1            175720     TRUE         86 USSM11904276
 3:            2            196652     TRUE         85 USAT21906978
 4:            1            219493    FALSE         77 USUM71210283
 5:           10            290320     TRUE         91 USIR10211038
 6:            9            235213    FALSE         86 USAR10300924
 7:            2            257359    FALSE         77 USCA29000294
 8:            8            177000     TRUE         88 USUM71703085
 9:            2            177933    FALSE         81 GBAHT0600202
10:            3            226986     TRUE         76 USSM19600051
11:            1            210575     TRUE         84 USSM12208809
12:            1            248866     TRUE         78 USPR39401366
13:            8            298773     TRUE         81 USSM19600055
14:           13            213993    FALSE         77 USUM71810720
15:            3            320626     TRUE         76 USIR10211559
16:           11            205090     TRUE         82 QMFME2004132
17:            7            181733     TRUE         81 USUM72007941
18:            3            211000    FALSE         88 USUM70840511
19:            3            190476     TRUE         82 USUM72013355
20:            1            139560     TRUE         88 USUM72222387
21:            2            176960     TRUE         79 USRC12001468
22:            7            210800     TRUE         85 USUM72000788
23:            5            205040     TRUE         86 USRC11800930
24:            1            133706     TRUE         76 QMUY42100151
25:            7            204600     TRUE         85 USSM11701444
26:            3            239835     TRUE         87 USUG11800685
27:            7            248133     TRUE         83 USUM71011769
28:            4            166605     TRUE         84 USUG11800208
29:            5            270506     TRUE         85 USLF20000357
30:            3            312820     TRUE         87 USSM11806660
    track_number track_duration_ms explicit popularity         isrc
                    added_by            added_at
                      <char>              <POSc>
 1: spotify:user:bradnumber1 2022-04-27 22:11:04
 2: spotify:user:bradnumber1 2021-08-08 09:26:31
 3: spotify:user:bradnumber1 2021-08-08 09:26:31
 4: spotify:user:bradnumber1 2021-08-08 09:26:31
 5: spotify:user:bradnumber1 2021-08-08 09:26:31
 6: spotify:user:bradnumber1 2021-08-08 09:26:31
 7: spotify:user:bradnumber1 2021-08-08 09:26:31
 8: spotify:user:bradnumber1 2021-08-08 09:26:31
 9: spotify:user:bradnumber1 2021-08-08 09:26:31
10: spotify:user:bradnumber1 2021-08-08 09:26:31
11: spotify:user:bradnumber1 2022-12-11 23:08:28
12: spotify:user:bradnumber1 2021-08-08 09:26:31
13: spotify:user:bradnumber1 2021-08-08 09:26:31
14: spotify:user:bradnumber1 2021-08-08 09:26:31
15: spotify:user:bradnumber1 2021-08-08 09:26:31
16: spotify:user:bradnumber1 2021-08-08 09:26:31
17: spotify:user:bradnumber1 2021-08-08 09:26:31
18: spotify:user:bradnumber1 2021-08-08 09:26:31
19: spotify:user:bradnumber1 2021-08-08 09:26:31
20: spotify:user:bradnumber1 2023-05-11 07:58:24
21: spotify:user:bradnumber1 2021-08-08 09:26:31
22: spotify:user:bradnumber1 2021-08-08 09:26:31
23: spotify:user:bradnumber1 2021-08-08 09:26:31
24: spotify:user:bradnumber1 2022-01-04 09:42:24
25: spotify:user:bradnumber1 2021-08-08 09:26:31
26: spotify:user:bradnumber1 2021-08-08 09:26:31
27: spotify:user:bradnumber1 2021-08-08 09:26:31
28: spotify:user:bradnumber1 2021-08-08 09:26:31
29: spotify:user:bradnumber1 2021-08-08 09:26:31
30: spotify:user:bradnumber1 2021-08-08 09:26:31
                    added_by            added_at
                                                                                                                                              artist_genres
                                                                                                                                                     <char>
 1:                                                                                                                                          lgbtq+ hip hop
 2:                                                                                                                                  hip hop,rap,slap house
 3:                                                                                                                                    melodic rap,rap,trap
 4:                                                                                                                         k-rap,korean old school hip hop
 5:                                                                                                                             detroit hip hop,hip hop,rap
 6:                                                                     atl hip hop,dirty south rap,hip hop,old school atlanta hip hop,rap,southern hip hop
 7:                                                                                                                                                 hip hop
 8:                                                                                                            conscious hip hop,hip hop,rap,west coast rap
 9:                                                                                                                                    atl hip hop,neo soul
10: east coast hip hop,hip hop,neo soul,new jersey rap,alternative r&b,conscious hip hop,hip hop,neo soul,new jersey rap,r&b,soul,rap kreyol,new jersey rap
11:                                                                                                                                          lgbtq+ hip hop
12:                                 g funk,gangster rap,hardcore hip hop,hip hop,west coast rap,g funk,gangster rap,hardcore hip hop,hip hop,west coast rap
13:                           east coast hip hop,hip hop,neo soul,new jersey rap,alternative r&b,conscious hip hop,hip hop,neo soul,new jersey rap,r&b,soul
14:                                                                                                                             r&b,rap,uk contemporary r&b
15:                                                                                                                             detroit hip hop,hip hop,rap
16:                                                                                 reggaeton,trap latino,urbano latino,reggaeton,trap latino,urbano latino
17:                                                                                            hip hop,north carolina hip hop,rap,trap,melodic rap,rap,trap
18:                                                                                                                                 chicago rap,hip hop,rap
19:                                                                brooklyn drill,rap,atl hip hop,atl trap,rap,trap,hip hop,north carolina hip hop,rap,trap
20:                                                                                                                   new jersey underground rap,trap queen
21:                                                                                          r&b,rap,atl hip hop,atl trap,gangster rap,melodic rap,rap,trap
22:                                                                                                 detroit hip hop,hip hop,rap,chicago rap,melodic rap,rap
23:                                                                east coast hip hop,hip hop,rap,trap,birmingham grime,grime,instrumental grime,uk hip hop
24:                                                                                                  canadian hip hop,dark trap,meme rap,indonesian hip hop
25:                                                                                                           atl hip hop,hip hop,rap,southern hip hop,trap
26:                                                                                                                             chicago rap,melodic rap,rap
27:                                                                                                                             detroit hip hop,hip hop,rap
28:                                                                                                                                   emo rap,miami hip hop
29:                                                                     atl hip hop,dirty south rap,hip hop,old school atlanta hip hop,rap,southern hip hop
30:                                                                                                                                  hip hop,rap,slap house
                                                                                                                                              artist_genres
    danceability energy   key loudness  mode speechiness acousticness
           <num>  <num> <num>    <num> <num>       <num>        <num>
 1:        0.737  0.846     1   -4.510     0      0.2200     0.006140
 2:        0.598  0.427     7   -8.764     0      0.0317     0.054600
 3:        0.896  0.586    10   -6.687     0      0.0559     0.104000
 4:        0.727  0.937    11   -2.871     0      0.2860     0.004170
 5:        0.908  0.669     7   -2.827     1      0.0738     0.002860
 6:        0.727  0.974     4   -2.261     0      0.0664     0.103000
 7:        0.867  0.517    11  -13.222     0      0.0875     0.004560
 8:        0.908  0.621     1   -6.638     0      0.1020     0.000282
 9:        0.855  0.587     8   -4.589     1      0.0336     0.050500
10:        0.607  0.422    10  -13.364     0      0.4990     0.209000
11:        0.637  0.715     2   -4.971     0      0.0455     0.148000
12:        0.849  0.521    11  -12.999     0      0.2160     0.304000
13:        0.765  0.290     4  -17.117     0      0.1390     0.027900
14:        0.477  0.610    11   -5.628     0      0.1440     0.225000
15:        0.689  0.735     2   -4.545     1      0.2670     0.009220
16:        0.731  0.573     4  -10.059     0      0.0544     0.401000
17:        0.746  0.690    11   -7.956     1      0.1640     0.247000
18:        0.790  0.647    10   -5.983     0      0.1360     0.051500
19:        0.823  0.586     6   -6.606     0      0.2000     0.114000
20:        0.954  0.516     6   -5.817     1      0.1600     0.030000
21:        0.755  0.578     0   -8.932     0      0.1450     0.316000
22:        0.808  0.745    10   -5.260     0      0.3420     0.145000
23:        0.854  0.569     5   -8.151     0      0.1060     0.060900
24:        0.815  0.848     4   -4.704     0      0.1150     0.023000
25:        0.833  0.434     2   -8.795     1      0.4310     0.010200
26:        0.511  0.566     6   -7.230     0      0.2000     0.349000
27:        0.855  0.954     0   -1.190     0      0.2640     0.529000
28:        0.740  0.613     8   -4.880     1      0.1450     0.258000
29:        0.843  0.806     4   -5.946     0      0.2690     0.143000
30:        0.834  0.730     8   -3.714     1      0.2220     0.005130
    danceability energy   key loudness  mode speechiness acousticness
    instrumentalness liveness valence   tempo time_signature album_genres
               <num>    <num>   <num>   <num>          <num>       <lgcl>
 1:         0.00e+00   0.0486  0.5460  87.981              4           NA
 2:         5.83e-06   0.2100  0.0605  76.469              4           NA
 3:         0.00e+00   0.7900  0.6420 116.971              4           NA
 4:         0.00e+00   0.0910  0.7490 132.067              4           NA
 5:         0.00e+00   0.2370  0.6620 112.238              4           NA
 6:         5.32e-04   0.1740  0.9650  79.526              4           NA
 7:         3.39e-04   0.0864  0.8660 133.148              4           NA
 8:         5.39e-05   0.0958  0.4210 150.011              4           NA
 9:         7.14e-03   0.1050  0.6400 111.970              4           NA
10:         0.00e+00   0.0839  0.5630 130.470              3           NA
11:         0.00e+00   0.0892  0.3080 141.872              4           NA
12:         1.65e-06   0.1150  0.7630  95.265              4           NA
13:         0.00e+00   0.5710  0.5060  92.409              4           NA
14:         0.00e+00   0.1070  0.3580  79.882              4           NA
15:         7.20e-04   0.3650  0.0590 171.403              4           NA
16:         5.22e-05   0.1130  0.1450 109.928              4           NA
17:         0.00e+00   0.1010  0.4970  89.977              4           NA
18:         0.00e+00   0.2480  0.6540  87.999              4           NA
19:         0.00e+00   0.1930  0.3470 125.971              4           NA
20:         7.54e-06   0.0504  0.6240 105.001              4           NA
21:         0.00e+00   0.2500  0.5810  94.148              4           NA
22:         0.00e+00   0.2920  0.8290 165.995              4           NA
23:         8.16e-02   0.1000  0.2940  80.015              4           NA
24:         1.62e-04   0.0265  0.6840 106.032              4           NA
25:         2.19e-02   0.1650  0.2810 150.062              4           NA
26:         0.00e+00   0.3400  0.2180  83.903              4           NA
27:         0.00e+00   0.2050  0.6680 114.635              5           NA
28:         3.72e-03   0.1230  0.4730  75.023              4           NA
29:         0.00e+00   0.0771  0.6130  94.948              4           NA
30:         0.00e+00   0.1240  0.4460 155.008              4           NA
    instrumentalness liveness valence   tempo time_signature album_genres
                                                      label
                                                     <char>
 1:                                                Columbia
 2:                                      Cactus Jack / Epic
 3:                                        Atlantic Records
 4:               Silent Records/Universal Republic Records
 5:                                               Aftermath
 6:                                                  Arista
 7:                                   EMI/EMI Records (USA)
 8:                                               Aftermath
 9:                                                   WM UK
10:                                                Columbia
11:                                                Columbia
12:                                          Island Records
13:                                                Columbia
14:                              10 Summers / Interscope PS
15:                                                      3H
16:                                 Rimas Entertainment LLC
17:              South Coast Music Group/Interscope Records
18:                                             Roc-A-Fella
19:                                 Victor Victor Worldwide
20:                               Uptown / Republic Records
21: Chris Brown Entertainment/300 Entertainment/RCA Records
22:                      Shady/Aftermath/Interscope Records
23:           A$AP Worldwide/Polo Grounds Music/RCA Records
24:          bbno$, Rich Brian appears courtesy of 88Rising
25:                                       Epic/Freebandz/A1
26:                            Juice WRLD Mixtape / ISR P&D
27:                                               Aftermath
28:                                  Bad Vibes Forever, LLC
29:                                   Arista/LaFace Records
30:                       Cactus Jack / Epic / Grand Hustle
                                                      label
                                                                                                                                                                                                                                                                                                                              copyrights
                                                                                                                                                                                                                                                                                                                                  <char>
 1:                                                                                                                                                                                                                                                                  P (P) 2021 Columbia Records, a Division of Sony Music Entertainment
 2:                                                                                                                                                                                                                                                                                           P (P) 2019 Epic Records. With Cactus Jack.
 3: C © 2019 Atlantic Recording Corporation for the United States and WEA International Inc. for the world outside of the United States. A Warner Music Group Company, P ℗ 2019 Atlantic Recording Corporation for the United States and WEA International Inc. for the world outside of the United States. A Warner Music Group Company
 4:                                                                                                                                                                 C © 2012 Schoolboy/Universal Republic Records, a division of UMG Recordings, Inc., P ℗ 2012 Schoolboy/Universal Republic Records, a division of UMG Recordings, Inc.
 5:                                                                                                                                                                                                                                                                               C © 2002 Aftermath Records, P ℗ 2002 Aftermath Records
 6:                                                                                                                                                                                                                                                                                                        P (P) 2003 Arista Records LLC
 7:                                                                                                                                                                                                                                                                         C © 1990 Capitol Records, LLC, P ℗ 1990 Capitol Records, LLC
 8:                                                                                                                                                                                                                       C © 2017 Aftermath/Interscope (Top Dawg Entertainment), P ℗ 2017 Aftermath/Interscope (Top Dawg Entertainment)
 9:                                                                                                                                                                                      C © 2006 Gnarls Barkley under exclusive license to Warner Music UK Ltd, P ℗ 2006 Gnarls Barkley under exclusive license to Warner Music UK Ltd.
10:                                                                                                                                                                                                                                                                  P (P) 1996 Columbia Records, a division of Sony Music Entertainment
11:                                                                                                                                                                                                                                                                  P (P) 2022 Columbia Records, a Division of Sony Music Entertainment
12:                                                                                                                                                                                                                                    C © 2007 The Island Def Jam Music Group, P This Compilation ℗ 2007 The Island Def Jam Music Group
13:                                                                                                                                                                                                                                                                  P (P) 1996 Columbia Records, a division of Sony Music Entertainment
14:                                                                                                                                                                                                                                                                   C © 2018 10 Summers Records, LLC, P ℗ 2018 10 Summers Records, LLC
15:                                                                                                                                                                                                                                C © 2014 Shady Records/Interscope Records, P This Compilation ℗ 2014 Shady Records/Interscope Records
16:                                                                                                                                                                                                                                                               C (C) 2020 Rimas Entertainment LLC, P (P) 2020 Rimas Entertainment LLC
17:                                                                                                                                                                                                                                                                             C © 2020 Interscope Records, P ℗ 2020 Interscope Records
18:                                                                                                                                                                                                                                                                         C © 2008 UMG Recordings, Inc., P ℗ 2008 UMG Recordings, Inc.
19:                                                                                                                                                     C © 2020 Republic Records, a division of UMG Recordings, Inc. & Victor Victor Worldwide, P ℗ 2020 Republic Records, a division of UMG Recordings, Inc. & Victor Victor Worldwide
20:                                                                                                                                                                   C © 2022 Uptown Records, a division of Republic Records and UMG Recordings, Inc., P ℗ 2022 Uptown Records, a division of Republic Records and UMG Recordings, Inc.
21:                                                                                                                                                                                                                                                    P (P) 2020 Chris Brown Entertainment, LLC, under exclusive license to RCA Records
22:                                                                                                                                                                                                                                                                   C © 2020 Marshall B. Mathers III, P ℗ 2020 Marshall B. Mathers III
23:                                                                                                                                                                                                                          P (P) 2018 A$AP Rocky Recordings LLC/RCA Records. Marketed & Distributed by Polo Grounds Music/RCA Records.
24:                                                                                                                                                                                                                         C 2021 bbno$, Rich Brian appears courtesy of 88Rising, P 2021 bbno$, Rich Brian appears courtesy of 88Rising
25:                                                                                                                                                                                                                                                  P (P) 2017 Epic Records, a division of Sony Music Entertainment with A1 / Freebandz
26:                                                                                                                                                                                           C © 2018 Grade A Productions, LLC, distributed by Interscope Records, P ℗ 2018 Grade A Productions, LLC, distributed by Interscope Records
27:                                                                                                                                                                                                                                                                               C © 2010 Aftermath Records, P ℗ 2010 Aftermath Records
28:                                                                                                                                                                                                                                                                               C © 2018 Bad Vibes Forever, P ℗ 2018 Bad Vibes Forever
29:                                                                                                                                                                                                                                                                                                        P (P) 2000 Arista Records LLC
30:                                                                                                                                                                                                                                                                     P (P) 2018 Epic Records/Sony. With Cactus Jack and Grand Hustle.
                                                                                                                                                                                                                                                                                                                              copyrights
    main_genre duration_sec popularity_scaled is_popular loudness_energy_ratio
        <char>        <num>             <num>     <lgcl>                 <num>
 1:    Hip Hop      143.901              0.85       TRUE             -5.330969
 2:    Hip Hop      175.720              0.86       TRUE            -20.524590
 3:    Hip Hop      196.652              0.85       TRUE            -11.411263
 4:    Hip Hop      219.493              0.77       TRUE             -3.064034
 5:    Hip Hop      290.320              0.91       TRUE             -4.225710
 6:    Hip Hop      235.213              0.86       TRUE             -2.321355
 7:    Hip Hop      257.359              0.77       TRUE            -25.574468
 8:    Hip Hop      177.000              0.88       TRUE            -10.689211
 9:    Hip Hop      177.933              0.81       TRUE             -7.817717
10:    Hip Hop      226.986              0.76       TRUE            -31.668246
11:    Hip Hop      210.575              0.84       TRUE             -6.952448
12:    Hip Hop      248.866              0.78       TRUE            -24.950096
13:    Hip Hop      298.773              0.81       TRUE            -59.024138
14:    Hip Hop      213.993              0.77       TRUE             -9.226230
15:    Hip Hop      320.626              0.76       TRUE             -6.183673
16:    Hip Hop      205.090              0.82       TRUE            -17.554974
17:    Hip Hop      181.733              0.81       TRUE            -11.530435
18:    Hip Hop      211.000              0.88       TRUE             -9.247295
19:    Hip Hop      190.476              0.82       TRUE            -11.273038
20:    Hip Hop      139.560              0.88       TRUE            -11.273256
21:    Hip Hop      176.960              0.79       TRUE            -15.453287
22:    Hip Hop      210.800              0.85       TRUE             -7.060403
23:    Hip Hop      205.040              0.86       TRUE            -14.325132
24:    Hip Hop      133.706              0.76       TRUE             -5.547170
25:    Hip Hop      204.600              0.85       TRUE            -20.264977
26:    Hip Hop      239.835              0.87       TRUE            -12.773852
27:    Hip Hop      248.133              0.83       TRUE             -1.247379
28:    Hip Hop      166.605              0.84       TRUE             -7.960848
29:    Hip Hop      270.506              0.85       TRUE             -7.377171
30:    Hip Hop      312.820              0.87       TRUE             -5.087671
    main_genre duration_sec popularity_scaled is_popular loudness_energy_ratio
    release_date release_year release_month release_day efficiency_ratio
          <Date>        <num>         <num>       <int>            <num>
 1:   2021-09-17         2021             9          17        1.1478969
 2:   2019-10-04         2019            10           4        0.7140468
 3:   2019-12-06         2019            12           6        0.6540179
 4:   2012-01-01         2012             1           1        1.2888583
 5:   2002-05-26         2002             5          26        0.7367841
 6:         <NA>           NA            NA          NA        1.3397524
 7:   1990-02-20         1990             2          20        0.5963091
 8:   2017-04-14         2017             4          14        0.6839207
 9:   2006-04-24         2006             4          24        0.6865497
10:   1996-02-13         1996             2          13        0.6952224
11:   2022-09-22         2022             9          22        1.1224490
12:   1994-06-07         1994             6           7        0.6136631
13:   1996-02-13         1996             2          13        0.3790850
14:   2018-10-12         2018            10          12        1.2788260
15:   2014-11-24         2014            11          24        1.0667634
16:   2020-11-27         2020            11          27        0.7838577
17:   2020-04-17         2020             4          17        0.9249330
18:   2008-11-24         2008            11          24        0.8189873
19:   2020-07-03         2020             7           3        0.7120292
20:   2022-11-30         2022            11          30        0.5408805
21:   2020-05-08         2020             5           8        0.7655629
22:   2020-01-17         2020             1          17        0.9220297
23:   2018-05-25         2018             5          25        0.6662763
24:   2021-07-24         2021             7          24        1.0404908
25:   2017-06-30         2017             6          30        0.5210084
26:   2018-12-10         2018            12          10        1.1076321
27:   2010-06-18         2010             6          18        1.1157895
28:   2018-03-16         2018             3          16        0.8283784
29:   2000-10-31         2000            10          31        0.9561091
30:   2018-08-03         2018             8           3        0.8752998
    release_date release_year release_month release_day efficiency_ratio

Here I wanted to analyze the ratio between the tempo and the energy to understand the relationship between those two musical characteristics which could help figure out which songs have higher ratios and whether there was a correlation between the two.

# Creating a new column measuring the tempo to energy ratio
spotify_dt[, tempo_energy_ratio := tempo / energy]
spotify_dt
Key: <main_genre>
                                                                  track_name
                                                                      <char>
   1:                                                           Dance Wiv Me
   2: The Good, the Bad and the Ugly - From "The Good, the Bad and the Ugly"
   3:                                                                  Sandy
   4:                                                           The Stripper
   5:                                Justified & Ancient - Stand by the Jams
  ---                                                                       
9995:                                                    Blowin' in the Wind
9996:                                                          She's My Baby
9997:                                     Yellow Submarine - Remastered 2009
9998:                                                    When You Were Young
9999:                                                    Prisoner of Society
                                         artist_name_s
                                                <char>
   1:                                    Dizzee Rascal
   2: Hugo Montenegro & His Orchestra, Hugo Montenegro
   3:                                       Larry Hall
   4:                       David Rose & His Orchestra
   5:                                          The KLF
  ---                                                 
9995:                                        Bob Dylan
9996:                                   Johnny O'Keefe
9997:                                      The Beatles
9998:                                      The Killers
9999:                                   The Living End
                                    album_name             album_artist_name_s
                                        <char>                          <char>
   1:                          Tongue N' Cheek                   Dizzee Rascal
   2: All-Time Greatest Movie Themes & Schemes Hugo Montenegro & His Orchestra
   3:    Totally Cool '60s: Lost Hits & Misses                 Various Artists
   4:                    Music Of The Stripper      David Rose & His Orchestra
   5:                         Songs Collection                         The KLF
  ---                                                                         
9995:               The Freewheelin' Bob Dylan                       Bob Dylan
9996:              The Very Best Of (Standard)                  Johnny O'Keefe
9997:                    Revolver (Remastered)                     The Beatles
9998:                               Sam's Town                     The Killers
9999:                           The Living End                  The Living End
      album_release_date disc_number track_number track_duration_ms explicit
                  <char>       <num>        <num>             <num>   <lgcl>
   1:         2009-09-25           1            3            204093     TRUE
   2:         1999-08-24           1           15            165693    FALSE
   3:         2016-12-02           1            9            150680    FALSE
   4:         2006-01-01           1            1            117640    FALSE
   5:         1992-08-03           1            3            216270    FALSE
  ---                                                                       
9995:         1963-05-27           1            1            165426    FALSE
9996:         2008-08-23           1            4            125226    FALSE
9997:         1966-08-05           1            6            158880    FALSE
9998:         2006-10-03           1            3            220426    FALSE
9999:         1999-01-15           1            1            229066    FALSE
      popularity         isrc                 added_by            added_at
           <num>       <char>                   <char>              <POSc>
   1:          0 GBWHS0800003 spotify:user:bradnumber1 2021-08-08 09:26:31
   2:         37 USRC16707524 spotify:user:bradnumber1 2021-08-08 09:26:31
   3:          0 USBT21603678 spotify:user:bradnumber1 2021-08-08 09:26:31
   4:         22 ADA010903901 spotify:user:bradnumber1 2021-08-08 09:26:31
   5:          0 QMARG1760056 spotify:user:bradnumber1 2020-03-05 09:20:39
  ---                                                                     
9995:         71 USSM19900534 spotify:user:bradnumber1 2021-08-08 09:26:31
9996:         16 AUWA00800460 spotify:user:bradnumber1 2021-08-08 09:26:31
9997:         61 GBAYE0601498 spotify:user:bradnumber1 2021-08-08 09:26:31
9998:          1 USUM70605164 spotify:user:bradnumber1 2021-08-08 09:26:31
9999:         47 AUMPO9800004 spotify:user:bradnumber1 2021-08-08 09:26:31
                                                                   artist_genres
                                                                          <char>
   1:                                                   grime,instrumental grime
   2:                              easy listening,man's orchestra,easy listening
   3:                                                       instrumental worship
   4:                                                light music,man's orchestra
   5:                                acid house,ambient house,big beat,hip house
  ---                                                                           
9995: classic rock,country rock,folk,folk rock,rock,roots rock,singer-songwriter
9996:                                                            australian rock
9997:  beatlesque,british invasion,classic rock,merseybeat,psychedelic rock,rock
9998:                alternative rock,dance rock,modern rock,permanent wave,rock
9999:                 australian alternative rock,australian rock,australian ska
      danceability energy   key loudness  mode speechiness acousticness
             <num>  <num> <num>    <num> <num>       <num>        <num>
   1:        0.878 0.7460    11   -4.281     1      0.0451     0.047600
   2:        0.531 0.6480     0  -10.648     1      0.0412     0.649000
   3:        0.359 0.4140     0  -11.949     1      0.0339     0.636000
   4:        0.464 0.4200     5  -12.649     1      0.0280     0.843000
   5:        0.617 0.8720     8  -12.305     1      0.0480     0.015800
  ---                                                                  
9995:        0.380 0.0993     2  -20.567     1      0.0509     0.914000
9996:        0.582 0.7720     2   -5.028     1      0.0332     0.691000
9997:        0.604 0.5490     1   -9.873     1      0.0389     0.531000
9998:        0.467 0.9880    11   -3.313     1      0.1120     0.000152
9999:        0.275 0.9930     9   -5.474     0      0.1380     0.000186
      instrumentalness liveness valence   tempo time_signature album_genres
                 <num>    <num>   <num>   <num>          <num>       <lgcl>
   1:         0.00e+00   0.1540   0.792 111.996              4           NA
   2:         5.01e-01   0.4420   0.589 116.572              4           NA
   3:         1.08e-03   0.3670   0.628 158.741              4           NA
   4:         9.73e-01   0.2620   0.591  94.570              4           NA
   5:         1.12e-01   0.4080   0.504 111.458              4           NA
  ---                                                                      
9995:         2.32e-05   0.0605   0.440 174.874              4           NA
9996:         1.75e-04   0.1240   0.731 131.613              4           NA
9997:         0.00e+00   0.4380   0.696 111.398              4           NA
9998:         4.84e-02   0.2800   0.321 130.435              4           NA
9999:         0.00e+00   0.3020   0.190 148.725              4           NA
                      label
                     <char>
   1:       Liberator Music
   2:     RCA Records Label
   3:           Record Town
   4:        Gambit Records
   5:   Jams Communications
  ---                      
9995:              Columbia
9996:          WM Australia
9997:         EMI Catalogue
9998: Universal Music Group
9999:               Reprise
                                                                                                     copyrights
                                                                                                         <char>
   1:                                                                  C 2009 Dirtee Stank, P 2009 Dirtee Stank
   2:                                                                              P (P) 1999 BMG Entertainment
   3:                                                             C 2016 Sunset Blvd., P 2016 Copyright Control
   4:                                                                                     C 2006 Gambit Records
   5:                                                      C 1992 Copyright Control, P 1992 Jams Communications
  ---                                                                                                          
9995:                                                  P Originally Released 1963 Sony Music Entertainment Inc.
9996:                  C © 2008 Warner Music Australia Pty Limited, P ℗ 2008 Warner Music Australia Pty Limited
9997: C © 2015 Apple Corps Ltd., P ℗ 2015 Calderstone Productions Limited (a division of Universal Music Group)
9998:                          C © 2006 The Island Def Jam Music Group, P ℗ 2006 The Island Def Jam Music Group
9999:                                                  C © 1999 Reprise Records, P ℗ 1998, 1999 Reprise Records
                  main_genre duration_sec popularity_scaled is_popular
                      <char>        <num>             <num>     <lgcl>
   1: Classical/Instrumental      204.093              0.00      FALSE
   2: Classical/Instrumental      165.693              0.37      FALSE
   3: Classical/Instrumental      150.680              0.00      FALSE
   4: Classical/Instrumental      117.640              0.22      FALSE
   5:       Dance/Electronic      216.270              0.00      FALSE
  ---                                                                 
9995:                   Rock      165.426              0.71      FALSE
9996:                   Rock      125.226              0.16      FALSE
9997:                   Rock      158.880              0.61      FALSE
9998:                   Rock      220.426              0.01      FALSE
9999:                   Rock      229.066              0.47      FALSE
      loudness_energy_ratio release_date release_year release_month release_day
                      <num>       <Date>        <num>         <num>       <int>
   1:             -5.738606   2009-09-25         2009             9          25
   2:            -16.432099   1999-08-24         1999             8          24
   3:            -28.862319   2016-12-02         2016            12           2
   4:            -30.116667   2006-01-01         2006             1           1
   5:            -14.111239   1992-08-03         1992             8           3
  ---                                                                          
9995:           -207.119839   1963-05-27         1963             5          27
9996:             -6.512953   2008-08-23         2008             8          23
9997:            -17.983607   1966-08-05         1966             8           5
9998:             -3.353239   2006-10-03         2006            10           3
9999:             -5.512588   1999-01-15         1999             1          15
      efficiency_ratio tempo_energy_ratio
                 <num>              <num>
   1:        0.8496583           150.1287
   2:        1.2203390           179.8951
   3:        1.1532033           383.4324
   4:        0.9051724           225.1667
   5:        1.4132901           127.8188
  ---                                    
9995:        0.2613158          1761.0675
9996:        1.3264605           170.4832
9997:        0.9089404           202.9107
9998:        2.1156317           132.0192
9999:        3.6109091           149.7734
# dplyr operation for tempo to energy ratio
spotify_dt_newcol <- spotify_dt %>% mutate(tempo_energy_ratio = tempo / energy)
spotify_dt_newcol
Key: <main_genre>
                                                                  track_name
                                                                      <char>
   1:                                                           Dance Wiv Me
   2: The Good, the Bad and the Ugly - From "The Good, the Bad and the Ugly"
   3:                                                                  Sandy
   4:                                                           The Stripper
   5:                                Justified & Ancient - Stand by the Jams
  ---                                                                       
9995:                                                    Blowin' in the Wind
9996:                                                          She's My Baby
9997:                                     Yellow Submarine - Remastered 2009
9998:                                                    When You Were Young
9999:                                                    Prisoner of Society
                                         artist_name_s
                                                <char>
   1:                                    Dizzee Rascal
   2: Hugo Montenegro & His Orchestra, Hugo Montenegro
   3:                                       Larry Hall
   4:                       David Rose & His Orchestra
   5:                                          The KLF
  ---                                                 
9995:                                        Bob Dylan
9996:                                   Johnny O'Keefe
9997:                                      The Beatles
9998:                                      The Killers
9999:                                   The Living End
                                    album_name             album_artist_name_s
                                        <char>                          <char>
   1:                          Tongue N' Cheek                   Dizzee Rascal
   2: All-Time Greatest Movie Themes & Schemes Hugo Montenegro & His Orchestra
   3:    Totally Cool '60s: Lost Hits & Misses                 Various Artists
   4:                    Music Of The Stripper      David Rose & His Orchestra
   5:                         Songs Collection                         The KLF
  ---                                                                         
9995:               The Freewheelin' Bob Dylan                       Bob Dylan
9996:              The Very Best Of (Standard)                  Johnny O'Keefe
9997:                    Revolver (Remastered)                     The Beatles
9998:                               Sam's Town                     The Killers
9999:                           The Living End                  The Living End
      album_release_date disc_number track_number track_duration_ms explicit
                  <char>       <num>        <num>             <num>   <lgcl>
   1:         2009-09-25           1            3            204093     TRUE
   2:         1999-08-24           1           15            165693    FALSE
   3:         2016-12-02           1            9            150680    FALSE
   4:         2006-01-01           1            1            117640    FALSE
   5:         1992-08-03           1            3            216270    FALSE
  ---                                                                       
9995:         1963-05-27           1            1            165426    FALSE
9996:         2008-08-23           1            4            125226    FALSE
9997:         1966-08-05           1            6            158880    FALSE
9998:         2006-10-03           1            3            220426    FALSE
9999:         1999-01-15           1            1            229066    FALSE
      popularity         isrc                 added_by            added_at
           <num>       <char>                   <char>              <POSc>
   1:          0 GBWHS0800003 spotify:user:bradnumber1 2021-08-08 09:26:31
   2:         37 USRC16707524 spotify:user:bradnumber1 2021-08-08 09:26:31
   3:          0 USBT21603678 spotify:user:bradnumber1 2021-08-08 09:26:31
   4:         22 ADA010903901 spotify:user:bradnumber1 2021-08-08 09:26:31
   5:          0 QMARG1760056 spotify:user:bradnumber1 2020-03-05 09:20:39
  ---                                                                     
9995:         71 USSM19900534 spotify:user:bradnumber1 2021-08-08 09:26:31
9996:         16 AUWA00800460 spotify:user:bradnumber1 2021-08-08 09:26:31
9997:         61 GBAYE0601498 spotify:user:bradnumber1 2021-08-08 09:26:31
9998:          1 USUM70605164 spotify:user:bradnumber1 2021-08-08 09:26:31
9999:         47 AUMPO9800004 spotify:user:bradnumber1 2021-08-08 09:26:31
                                                                   artist_genres
                                                                          <char>
   1:                                                   grime,instrumental grime
   2:                              easy listening,man's orchestra,easy listening
   3:                                                       instrumental worship
   4:                                                light music,man's orchestra
   5:                                acid house,ambient house,big beat,hip house
  ---                                                                           
9995: classic rock,country rock,folk,folk rock,rock,roots rock,singer-songwriter
9996:                                                            australian rock
9997:  beatlesque,british invasion,classic rock,merseybeat,psychedelic rock,rock
9998:                alternative rock,dance rock,modern rock,permanent wave,rock
9999:                 australian alternative rock,australian rock,australian ska
      danceability energy   key loudness  mode speechiness acousticness
             <num>  <num> <num>    <num> <num>       <num>        <num>
   1:        0.878 0.7460    11   -4.281     1      0.0451     0.047600
   2:        0.531 0.6480     0  -10.648     1      0.0412     0.649000
   3:        0.359 0.4140     0  -11.949     1      0.0339     0.636000
   4:        0.464 0.4200     5  -12.649     1      0.0280     0.843000
   5:        0.617 0.8720     8  -12.305     1      0.0480     0.015800
  ---                                                                  
9995:        0.380 0.0993     2  -20.567     1      0.0509     0.914000
9996:        0.582 0.7720     2   -5.028     1      0.0332     0.691000
9997:        0.604 0.5490     1   -9.873     1      0.0389     0.531000
9998:        0.467 0.9880    11   -3.313     1      0.1120     0.000152
9999:        0.275 0.9930     9   -5.474     0      0.1380     0.000186
      instrumentalness liveness valence   tempo time_signature album_genres
                 <num>    <num>   <num>   <num>          <num>       <lgcl>
   1:         0.00e+00   0.1540   0.792 111.996              4           NA
   2:         5.01e-01   0.4420   0.589 116.572              4           NA
   3:         1.08e-03   0.3670   0.628 158.741              4           NA
   4:         9.73e-01   0.2620   0.591  94.570              4           NA
   5:         1.12e-01   0.4080   0.504 111.458              4           NA
  ---                                                                      
9995:         2.32e-05   0.0605   0.440 174.874              4           NA
9996:         1.75e-04   0.1240   0.731 131.613              4           NA
9997:         0.00e+00   0.4380   0.696 111.398              4           NA
9998:         4.84e-02   0.2800   0.321 130.435              4           NA
9999:         0.00e+00   0.3020   0.190 148.725              4           NA
                      label
                     <char>
   1:       Liberator Music
   2:     RCA Records Label
   3:           Record Town
   4:        Gambit Records
   5:   Jams Communications
  ---                      
9995:              Columbia
9996:          WM Australia
9997:         EMI Catalogue
9998: Universal Music Group
9999:               Reprise
                                                                                                     copyrights
                                                                                                         <char>
   1:                                                                  C 2009 Dirtee Stank, P 2009 Dirtee Stank
   2:                                                                              P (P) 1999 BMG Entertainment
   3:                                                             C 2016 Sunset Blvd., P 2016 Copyright Control
   4:                                                                                     C 2006 Gambit Records
   5:                                                      C 1992 Copyright Control, P 1992 Jams Communications
  ---                                                                                                          
9995:                                                  P Originally Released 1963 Sony Music Entertainment Inc.
9996:                  C © 2008 Warner Music Australia Pty Limited, P ℗ 2008 Warner Music Australia Pty Limited
9997: C © 2015 Apple Corps Ltd., P ℗ 2015 Calderstone Productions Limited (a division of Universal Music Group)
9998:                          C © 2006 The Island Def Jam Music Group, P ℗ 2006 The Island Def Jam Music Group
9999:                                                  C © 1999 Reprise Records, P ℗ 1998, 1999 Reprise Records
                  main_genre duration_sec popularity_scaled is_popular
                      <char>        <num>             <num>     <lgcl>
   1: Classical/Instrumental      204.093              0.00      FALSE
   2: Classical/Instrumental      165.693              0.37      FALSE
   3: Classical/Instrumental      150.680              0.00      FALSE
   4: Classical/Instrumental      117.640              0.22      FALSE
   5:       Dance/Electronic      216.270              0.00      FALSE
  ---                                                                 
9995:                   Rock      165.426              0.71      FALSE
9996:                   Rock      125.226              0.16      FALSE
9997:                   Rock      158.880              0.61      FALSE
9998:                   Rock      220.426              0.01      FALSE
9999:                   Rock      229.066              0.47      FALSE
      loudness_energy_ratio release_date release_year release_month release_day
                      <num>       <Date>        <num>         <num>       <int>
   1:             -5.738606   2009-09-25         2009             9          25
   2:            -16.432099   1999-08-24         1999             8          24
   3:            -28.862319   2016-12-02         2016            12           2
   4:            -30.116667   2006-01-01         2006             1           1
   5:            -14.111239   1992-08-03         1992             8           3
  ---                                                                          
9995:           -207.119839   1963-05-27         1963             5          27
9996:             -6.512953   2008-08-23         2008             8          23
9997:            -17.983607   1966-08-05         1966             8           5
9998:             -3.353239   2006-10-03         2006            10           3
9999:             -5.512588   1999-01-15         1999             1          15
      efficiency_ratio tempo_energy_ratio
                 <num>              <num>
   1:        0.8496583           150.1287
   2:        1.2203390           179.8951
   3:        1.1532033           383.4324
   4:        0.9051724           225.1667
   5:        1.4132901           127.8188
  ---                                    
9995:        0.2613158          1761.0675
9996:        1.3264605           170.4832
9997:        0.9089404           202.9107
9998:        2.1156317           132.0192
9999:        3.6109091           149.7734

I chose to incorporate a bar graph of the average popularity by genre to get a good idea of which genres are most popular and least.

# ggplot (bar graph) of the average popularity by genres
ggplot(spotify_dt_summary, aes(x = reorder(main_genre, -avg_popularity), y = avg_popularity, fill = main_genre)) +
  geom_bar(stat = "identity") +
  coord_flip() +
  theme_minimal() +
  labs(title = "Average Popularity by Genre", x = "Genre", y = "Average Popularity")

ChatGPT Reliability

In order to compare operation differences with AI, I used ChatGPT and entered this prompt:


“You are a data scientist using exploring a large data set using exploratory data analysis principles. Your task is to create code in R using data.table functions to find the highest popularity track within all the genres, and find the lowest popular track.”

This is the result it gave me with tweaks of my own for the naming conventions in order to actually run the code:

# Highest Popularity Track by Genre
highest_popularity_tracks <- spotify_dt[, .SD[popularity == max(popularity, na.rm = TRUE)], by = main_genre]

# Display result
highest_popularity_tracks
Key: <main_genre>
                main_genre
                    <char>
 1: Classical/Instrumental
 2:       Dance/Electronic
 3:       Dance/Electronic
 4:           Folk/Country
 5:                Hip Hop
 6:             Jazz/Blues
 7:             Jazz/Blues
 8:                  Other
 9:                    Pop
10:               R&B/Soul
11:       Reggae/Caribbean
12:                   Rock
13:                   Rock
                                                                track_name
                                                                    <char>
 1: The Good, the Bad and the Ugly - From "The Good, the Bad and the Ugly"
 2:                                                         Afraid To Feel
 3:                                                                  Faded
 4:                                                               Fast Car
 5:                                                             Without Me
 6:                                                              Take Five
 7:                                               Everybody Loves Somebody
 8:               Until I Found You (with Em Beihold) - Em Beihold Version
 9:                                                                vampire
10:                                                            Billie Jean
11:                                                           Is This Love
12:                                                     Livin' On A Prayer
13:                                                             In the End
                                       artist_name_s
                                              <char>
 1: Hugo Montenegro & His Orchestra, Hugo Montenegro
 2:                                        LF SYSTEM
 3:                                      Alan Walker
 4:                                       Luke Combs
 5:                                           Eminem
 6:                         The Dave Brubeck Quartet
 7:                                      Dean Martin
 8:                      Stephen Sanchez, Em Beihold
 9:                                   Olivia Rodrigo
10:                                  Michael Jackson
11:                         Bob Marley & The Wailers
12:                                         Bon Jovi
13:                                      Linkin Park
                                  album_name             album_artist_name_s
                                      <char>                          <char>
 1: All-Time Greatest Movie Themes & Schemes Hugo Montenegro & His Orchestra
 2:                           Afraid To Feel                       LF SYSTEM
 3:                          Different World                     Alan Walker
 4:                              Gettin' Old                      Luke Combs
 5:                          The Eminem Show                          Eminem
 6:                                 Time Out        The Dave Brubeck Quartet
 7:                 Everybody Loves Somebody                     Dean Martin
 8:   Until I Found You (Em Beihold Version)     Stephen Sanchez, Em Beihold
 9:                                  vampire                  Olivia Rodrigo
10:         Thriller 25 Super Deluxe Edition                 Michael Jackson
11:                                     Kaya        Bob Marley & The Wailers
12:                        Slippery When Wet                        Bon Jovi
13:            Hybrid Theory (Bonus Edition)                     Linkin Park
    album_release_date disc_number track_number track_duration_ms explicit
                <char>       <num>        <num>             <num>   <lgcl>
 1:         1999-08-24           1           15            165693    FALSE
 2:         2022-05-02           1            1            177524    FALSE
 3:         2018-12-14           1           15            212106    FALSE
 4:         2023-03-24           1           14            265493    FALSE
 5:         2002-05-26           1           10            290320     TRUE
 6:         1959-12-14           1            3            324133    FALSE
 7:               1964           1            1            164546    FALSE
 8:         2022-04-22           1            1            176440    FALSE
 9:         2023-06-30           1            1            219724     TRUE
10:         2008-02-08           1            6            293826    FALSE
11:               1978           1            3            232200    FALSE
12:         1986-08-16           1            3            249293    FALSE
13:         2000-10-24           1            8            216880    FALSE
    popularity         isrc                 added_by            added_at
         <num>       <char>                   <char>              <POSc>
 1:         37 USRC16707524 spotify:user:bradnumber1 2021-08-08 09:26:31
 2:         83 GBAHT2200492 spotify:user:bradnumber1 2022-08-28 08:07:11
 3:         83 NOG841549010 spotify:user:bradnumber1 2021-08-08 09:26:31
 4:         92 US6XF2200436 spotify:user:bradnumber1 2023-05-07 11:59:41
 5:         91 USIR10211038 spotify:user:bradnumber1 2021-08-08 09:26:31
 6:         70 USSM15900108 spotify:user:bradnumber1 2021-08-08 09:26:31
 7:         70 USQX91301936 spotify:user:bradnumber1 2021-08-08 09:26:31
 8:         93 USUG12202266 spotify:user:bradnumber1 2023-02-14 22:36:02
 9:         98 USUG12304091 spotify:user:bradnumber1 2023-07-05 22:55:39
10:         84 USSM19902991 spotify:user:bradnumber1 2021-08-08 09:26:31
11:         81 USIR27800259 spotify:user:bradnumber1 2021-08-08 09:26:31
12:         89 USPR38619998 spotify:user:bradnumber1 2021-08-08 09:26:31
13:         89 USWB10002407 spotify:user:bradnumber1 2021-08-08 09:26:31
                                            artist_genres danceability energy
                                                   <char>        <num>  <num>
 1:         easy listening,man's orchestra,easy listening        0.531  0.648
 2:                                              uk dance        0.578  0.912
 3:                                         electro house        0.468  0.627
 4:                          contemporary country,country        0.712  0.603
 5:                           detroit hip hop,hip hop,rap        0.908  0.669
 6:                               bebop,jazz,jazz quartet        0.454  0.260
 7:      adult standards,easy listening,lounge,vocal jazz        0.366  0.391
 8: gen z singer-songwriter,alt z,gen z singer-songwriter        0.551  0.550
 9:                                                   pop        0.511  0.532
10:                                              r&b,soul        0.920  0.654
11:                                   reggae,roots reggae        0.776  0.559
12:                                       glam metal,rock        0.532  0.887
13: alternative metal,nu metal,post-grunge,rap metal,rock        0.556  0.864
      key loudness  mode speechiness acousticness instrumentalness liveness
    <num>    <num> <num>       <num>        <num>            <num>    <num>
 1:     0  -10.648     1      0.0412      0.64900         5.01e-01   0.4420
 2:     1   -3.929     1      0.1140      0.01660         3.62e-03   0.2730
 3:     6   -5.085     1      0.0476      0.02810         7.97e-06   0.1100
 4:     8   -5.520     1      0.0262      0.18600         0.00e+00   0.1150
 5:     7   -2.827     1      0.0738      0.00286         0.00e+00   0.2370
 6:     8  -13.193     0      0.0401      0.53900         7.80e-04   0.0675
 7:     3  -12.322     1      0.0425      0.85400         0.00e+00   0.3520
 8:    10   -5.339     1      0.0286      0.77700         0.00e+00   0.2020
 9:     5   -5.745     1      0.0560      0.16900         0.00e+00   0.3110
10:    11   -3.051     0      0.0401      0.02360         1.53e-02   0.0360
11:     6   -8.375     0      0.1030      0.11000         0.00e+00   0.0671
12:     0   -3.757     1      0.0335      0.07780         2.14e-04   0.2940
13:     3   -5.870     0      0.0584      0.00958         0.00e+00   0.2090
    valence   tempo time_signature album_genres
      <num>   <num>          <num>       <lgcl>
 1:   0.589 116.572              4           NA
 2:   0.680 127.870              4           NA
 3:   0.159 179.642              4           NA
 4:   0.670  97.994              4           NA
 5:   0.662 112.238              4           NA
 6:   0.598 174.322              5           NA
 7:   0.461 119.182              3           NA
 8:   0.342 101.190              3           NA
 9:   0.322 137.827              4           NA
10:   0.847 117.046              4           NA
11:   0.758 122.242              4           NA
12:   0.795 122.511              4           NA
13:   0.400 105.143              4           NA
                                     label
                                    <char>
 1:                      RCA Records Label
 2:                         Warner Records
 3:                         MER Recordings
 4: River House Artists/Columbia Nashville
 5:                              Aftermath
 6:                        Columbia/Legacy
 7:                      Legacy Recordings
 8:                       Republic Records
 9:                      Olivia Rodrigo PS
10:                            Epic/Legacy
11:                         Island Records
12:                         Island Records
13:                         Warner Records
                                                                                                                                                                                                                                              copyrights
                                                                                                                                                                                                                                                  <char>
 1:                                                                                                                                                                                                                         P (P) 1999 BMG Entertainment
 2:                                                                                                                                                                                   C © 2022 Warner Music UK Limited, P ℗ 2022 Warner Music UK Limited
 3:                                                                                                                                                                         P (P) 2018 MER under exclusive license to Sony Music Entertainment Sweden AB
 4:                                                                                                                                        P (P) 2023 River House Artists LLC, under exclusive license to Sony Music Entertainment. All rights reserved.
 5:                                                                                                                                                                                               C © 2002 Aftermath Records, P ℗ 2002 Aftermath Records
 6:                                                                                                                                                                      P Originally released 1959. All rights reserved by SONY BMG MUSIC ENTERTAINMENT
 7:                                                                                                                                                                                 P Originally released 1964. All rights reserved by Dean Family Trust
 8: C © 2022 Stephen Sanchez, under exclusive license to Mercury Records/Republic Records, a division of UMG Recordings, Inc., P ℗ 2022 Stephen Sanchez, under exclusive license to Mercury Records/Republic Records, a division of UMG Recordings, Inc.
 9:                                                                                                               C © 2023 Olivia Rodrigo, under exclusive license to Geffen Records, P ℗ 2023 Olivia Rodrigo, under exclusive license to Geffen Records
10:                                                                                                                                                                                                          P (P) 1982, 2001, 2008 MJJ Productions Inc.
11:                                                                                                                                                                     C © 1978 The Island Def Jam Music Group, P ℗ 2001 The Island Def Jam Music Group
12:                                                                                                                                                                                         C © 1986 UMG Recordings, Inc., P ℗ 1986 UMG Recordings, Inc.
13:                                                                                                                                                                                           C © 2000 Warner Records Inc., P ℗ 2000 Warner Records Inc.
    duration_sec popularity_scaled is_popular loudness_energy_ratio
           <num>             <num>     <lgcl>                 <num>
 1:      165.693              0.37      FALSE            -16.432099
 2:      177.524              0.83       TRUE             -4.308114
 3:      212.106              0.83       TRUE             -8.110048
 4:      265.493              0.92       TRUE             -9.154229
 5:      290.320              0.91       TRUE             -4.225710
 6:      324.133              0.70      FALSE            -50.742308
 7:      164.546              0.70      FALSE            -31.514066
 8:      176.440              0.93       TRUE             -9.707273
 9:      219.724              0.98       TRUE            -10.798872
10:      293.826              0.84       TRUE             -4.665138
11:      232.200              0.81       TRUE            -14.982111
12:      249.293              0.89       TRUE             -4.235626
13:      216.880              0.89       TRUE             -6.793981
    release_date release_year release_month release_day efficiency_ratio
          <Date>        <num>         <num>       <int>            <num>
 1:   1999-08-24         1999             8          24        1.2203390
 2:   2022-05-02         2022             5           2        1.5778547
 3:   2018-12-14         2018            12          14        1.3397436
 4:   2023-03-24         2023             3          24        0.8469101
 5:   2002-05-26         2002             5          26        0.7367841
 6:   1959-12-14         1959            12          14        0.5726872
 7:         <NA>           NA            NA          NA        1.0683060
 8:   2022-04-22         2022             4          22        0.9981851
 9:   2023-06-30         2023             6          30        1.0410959
10:   2008-02-08         2008             2           8        0.7108696
11:         <NA>           NA            NA          NA        0.7203608
12:   1986-08-16         1986             8          16        1.6672932
13:   2000-10-24         2000            10          24        1.5539568
    tempo_energy_ratio
                 <num>
 1:           179.8951
 2:           140.2083
 3:           286.5104
 4:           162.5108
 5:           167.7698
 6:           670.4692
 7:           304.8133
 8:           183.9818
 9:           259.0733
10:           178.9694
11:           218.6798
12:           138.1184
13:           121.6933
# Lowest Popularity Track by Genre
lowest_popularity_tracks <- spotify_dt[, .SD[popularity == min(popularity, na.rm = TRUE)], by = main_genre]

# Display result
lowest_popularity_tracks
Key: <main_genre>
                  main_genre                              track_name
                      <char>                                  <char>
   1: Classical/Instrumental                            Dance Wiv Me
   2: Classical/Instrumental                                   Sandy
   3:       Dance/Electronic Justified & Ancient - Stand by the Jams
   4:       Dance/Electronic                         Twist And Shout
   5:       Dance/Electronic                          Talk Like That
  ---                                                               
2665:                   Rock                        Bring Me To Life
2666:                   Rock                                 Cocaine
2667:                   Rock                            Endless Love
2668:                   Rock                           Love Your Way
2669:                   Rock                   How Deep Is Your Love
                                         artist_name_s
                                                <char>
   1:                                    Dizzee Rascal
   2:                                       Larry Hall
   3:                                          The KLF
   4: Chaka Demus & Pliers, Jack Radics, The Taxi Gang
   5:                                      The Presets
  ---                                                 
2665:                                      Evanescence
2666:                                        J.J. Cale
2667:                        Lionel Richie, Diana Ross
2668:                                     Powderfinger
2669:                                         Bee Gees
                                 album_name  album_artist_name_s
                                     <char>               <char>
   1:                       Tongue N' Cheek        Dizzee Rascal
   2: Totally Cool '60s: Lost Hits & Misses      Various Artists
   3:                      Songs Collection              The KLF
   4:                              Tease Me Chaka Demus & Pliers
   5:                            Apocalypso          The Presets
  ---                                                           
2665:                                Fallen          Evanescence
2666:          Gold (International Version)            J.J. Cale
2667:                  Love's Greatest Hits      Various Artists
2668:                        Vulture Street         Powderfinger
2669:                              Greatest             Bee Gees
      album_release_date disc_number track_number track_duration_ms explicit
                  <char>       <num>        <num>             <num>   <lgcl>
   1:         2009-09-25           1            3            204093     TRUE
   2:         2016-12-02           1            9            150680    FALSE
   3:         1992-08-03           1            3            216270    FALSE
   4:         1993-01-01           1           13            237506    FALSE
   5:         2009-01-01           1            6            221066    FALSE
  ---                                                                       
2665:               2003           1            2            237293    FALSE
2666:         2007-01-01           1           17            171760    FALSE
2667:         2006-01-01           1            2            265613    FALSE
2668:         2003-01-01           1            4            271162    FALSE
2669:         1979-01-01           1            6            245200    FALSE
      popularity         isrc                 added_by            added_at
           <num>       <char>                   <char>              <POSc>
   1:          0 GBWHS0800003 spotify:user:bradnumber1 2021-08-08 09:26:31
   2:          0 USBT21603678 spotify:user:bradnumber1 2021-08-08 09:26:31
   3:          0 QMARG1760056 spotify:user:bradnumber1 2020-03-05 09:20:39
   4:          0 USIR19390126 spotify:user:bradnumber1 2021-08-08 09:26:31
   5:          0 AUUM70800157 spotify:user:bradnumber1 2021-08-08 09:26:31
  ---                                                                     
2665:          0 USWU30200093 spotify:user:bradnumber1 2021-08-08 09:26:31
2666:          0 NLF057690004 spotify:user:bradnumber1 2021-08-08 09:26:31
2667:          0 USMO18190008 spotify:user:bradnumber1 2021-08-08 09:26:31
2668:          0 AUUM00330025 spotify:user:bradnumber1 2021-08-08 09:26:31
2669:          0 GBAKW7701005 spotify:user:bradnumber1 2021-08-08 09:27:32
                                                          artist_genres
                                                                 <char>
   1:                                          grime,instrumental grime
   2:                                              instrumental worship
   3:                       acid house,ambient house,big beat,hip house
   4:                            funana,deep ragga,old school dancehall
   5:               alternative dance,australian dance,australian indie
  ---                                                                  
2665:                                                 alternative metal
2666:           blues,blues rock,folk rock,roots rock,singer-songwriter
2667: soft rock,adult standards,disco,motown,quiet storm,soft rock,soul
2668:                       australian alternative rock,australian rock
2669:                                       disco,mellow gold,soft rock
      danceability energy   key loudness  mode speechiness acousticness
             <num>  <num> <num>    <num> <num>       <num>        <num>
   1:        0.878  0.746    11   -4.281     1      0.0451      0.04760
   2:        0.359  0.414     0  -11.949     1      0.0339      0.63600
   3:        0.617  0.872     8  -12.305     1      0.0480      0.01580
   4:        0.802  0.820     2   -9.196     1      0.0334      0.22400
   5:        0.573  0.941     6   -3.460     0      0.0315      0.00130
  ---                                                                  
2665:        0.322  0.947     4   -3.173     0      0.1200      0.00846
2666:        0.802  0.439     6  -12.521     1      0.0428      0.37700
2667:        0.439  0.338    10  -10.108     1      0.0252      0.48400
2668:        0.303  0.774     6   -4.773     0      0.0446      0.00455
2669:        0.633  0.357     5   -9.366     0      0.0264      0.10500
      instrumentalness liveness valence   tempo time_signature album_genres
                 <num>    <num>   <num>   <num>          <num>       <lgcl>
   1:         0.00e+00    0.154   0.792 111.996              4           NA
   2:         1.08e-03    0.367   0.628 158.741              4           NA
   3:         1.12e-01    0.408   0.504 111.458              4           NA
   4:         0.00e+00    0.298   0.977 128.954              4           NA
   5:         4.06e-05    0.180   0.864 135.017              4           NA
  ---                                                                      
2665:         3.95e-06    0.238   0.269 189.994              4           NA
2666:         1.89e-01    0.120   0.596 103.681              4           NA
2667:         4.20e-04    0.188   0.216  93.573              4           NA
2668:         1.28e-06    0.140   0.457 170.067              4           NA
2669:         0.00e+00    0.133   0.674 104.938              4           NA
                      label
                     <char>
   1:       Liberator Music
   2:           Record Town
   3:   Jams Communications
   4:        Island Records
   5:               Modular
  ---                      
2665:               Wind Up
2666:               Mercury
2667:     Manhattan Records
2668: Universal Music Group
2669: Universal Music Group
                                                                                                                                                                                                                                                                                                    copyrights
                                                                                                                                                                                                                                                                                                        <char>
   1:                                                                                                                                                                                                                                                                 C 2009 Dirtee Stank, P 2009 Dirtee Stank
   2:                                                                                                                                                                                                                                                            C 2016 Sunset Blvd., P 2016 Copyright Control
   3:                                                                                                                                                                                                                                                     C 1992 Copyright Control, P 1992 Jams Communications
   4:                                                                                                                                                     C © 1993 Island Records, a division of Universal Music Operations Limited, P ℗ 1993 Island Records, a division of Universal Music Operations Limited
   5:                                                                                                                                                                                                                                                 C © 2009 Modular Recordings, P ℗ 2009 Modular Recordings
  ---                                                                                                                                                                                                                                                                                                         
2665: C (C) 2003 Wind-Up Records, LLCThis label copy information is the subject of copyright protection. All rights reserved.(C) EMI Music Germany GmbH & Co. KG, P (P) 2003 The copyright in this sound recording is owned by Wind-Up Records, LLC under exclusive licence to EMI Music Germany GmbH & Co. KG
2666:                                                                                                                                                                                                                 C © 2007 Universal International Music B.V., P ℗ 2007 Universal International Music B.V.
2667:                                                                                                                                                                                                                                  C © 2006 Manhattan Records, P This Compilation ℗ 2006 Manhattan Records
2668:                                                                                                                                                                                                                 C © 2003 Universal Music Australia Pty Ltd., P ℗ 2003 Universal Music Australia Pty Ltd.
2669:                                                                             C © 2007 Barry Gibb, The Estate of Robin Gibb and Yvonne Gibb, under exclusive license to Capitol Music Group, P ℗ 2007 Barry Gibb, The Estate of Robin Gibb and Yvonne Gibb, under exclusive license to Capitol Music Group
      duration_sec popularity_scaled is_popular loudness_energy_ratio
             <num>             <num>     <lgcl>                 <num>
   1:      204.093                 0      FALSE             -5.738606
   2:      150.680                 0      FALSE            -28.862319
   3:      216.270                 0      FALSE            -14.111239
   4:      237.506                 0      FALSE            -11.214634
   5:      221.066                 0      FALSE             -3.676939
  ---                                                                
2665:      237.293                 0      FALSE             -3.350581
2666:      171.760                 0      FALSE            -28.521640
2667:      265.613                 0      FALSE            -29.905325
2668:      271.162                 0      FALSE             -6.166667
2669:      245.200                 0      FALSE            -26.235294
      release_date release_year release_month release_day efficiency_ratio
            <Date>        <num>         <num>       <int>            <num>
   1:   2009-09-25         2009             9          25        0.8496583
   2:   2016-12-02         2016            12           2        1.1532033
   3:   1992-08-03         1992             8           3        1.4132901
   4:   1993-01-01         1993             1           1        1.0224439
   5:   2009-01-01         2009             1           1        1.6422339
  ---                                                                     
2665:         <NA>           NA            NA          NA        2.9409938
2666:   2007-01-01         2007             1           1        0.5473815
2667:   2006-01-01         2006             1           1        0.7699317
2668:   2003-01-01         2003             1           1        2.5544554
2669:   1979-01-01         1979             1           1        0.5639810
      tempo_energy_ratio
                   <num>
   1:           150.1287
   2:           383.4324
   3:           127.8188
   4:           157.2610
   5:           143.4825
  ---                   
2665:           200.6272
2666:           236.1754
2667:           276.8432
2668:           219.7248
2669:           293.9440

The results are not that different from the lesson and ChatGPT, we can conclude it is a reliable source to use when stuck in a situation where we can not figure out what the issue is however I had to adjust a few variables.