R Markdown

This is a random data set that I pull from Kaggle. It contained a list of the top trending videos on YouTube from the US in a random day back in 2017. I want to build a linear regression model to see if number of likes can be predicted by views.

library(tidyverse)
## Warning: package 'tidyverse' was built under R version 3.6.3
## -- Attaching packages --------------------------------------- tidyverse 1.3.0 --
## v ggplot2 3.3.3     v purrr   0.3.3
## v tibble  3.1.0     v dplyr   1.0.5
## v tidyr   1.1.3     v stringr 1.4.0
## v readr   1.3.1     v forcats 0.4.0
## Warning: package 'ggplot2' was built under R version 3.6.3
## Warning: package 'tibble' was built under R version 3.6.3
## Warning: package 'tidyr' was built under R version 3.6.3
## Warning: package 'readr' was built under R version 3.6.2
## Warning: package 'purrr' was built under R version 3.6.2
## Warning: package 'dplyr' was built under R version 3.6.3
## Warning: package 'stringr' was built under R version 3.6.2
## Warning: package 'forcats' was built under R version 3.6.2
## -- Conflicts ------------------------------------------ tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()
df = read.csv('USvideos.csv', stringsAsFactors = FALSE)
head(df)
##      video_id                                                          title
## 1 2kyS6SvSYSE                             WE WANT TO TALK ABOUT OUR MARRIAGE
## 2 1ZAPwfrtAFY The Trump Presidency: Last Week Tonight with John Oliver (HBO)
## 3 5qpjK5DgCt4          Racist Superman | Rudy Mancuso, King Bach & Lele Pons
## 4 puqaWrEC7tY                               Nickelback Lyrics: Real or Fake?
## 5 d380meD0W0M                                       I Dare You: GOING BALD!?
## 6 gHZ1Qz0KiKM                                          2 Weeks with iPhone X
##           channel_title category_id   views  likes dislikes comment_count
## 1          CaseyNeistat          22  748374  57527     2966         15954
## 2       LastWeekTonight          24 2418783  97185     6146         12703
## 3          Rudy Mancuso          23 3191434 146033     5339          8181
## 4 Good Mythical Morning          24  343168  10172      666          2146
## 5              nigahiga          24 2095731 132235     1989         17518
## 6              iJustine          28  119180   9763      511          1434
dim(df)
## [1] 40949     8
cor(df %>% dplyr::select(views, likes, dislikes, comment_count)) %>% round(., 3)
##               views likes dislikes comment_count
## views         1.000 0.849    0.472         0.618
## likes         0.849 1.000    0.447         0.803
## dislikes      0.472 0.447    1.000         0.700
## comment_count 0.618 0.803    0.700         1.000
chart1 = with(df, 
              plot(views, likes,
                   xlab = 'views', 
                   ylab = 'likes', 
                   main = 'Views x Likes'))

chart1
## NULL
mod = lm(likes ~ views, data = df)
summary(mod)
## 
## Call:
## lm(formula = likes ~ views, data = df)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -2249306   -17006   -11581      675  3019010 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 1.221e+04  6.271e+02   19.47   <2e-16 ***
## views       2.629e-02  8.079e-05  325.38   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 120900 on 40947 degrees of freedom
## Multiple R-squared:  0.7211, Adjusted R-squared:  0.7211 
## F-statistic: 1.059e+05 on 1 and 40947 DF,  p-value: < 2.2e-16
par(mfrow = c(2, 2))
plot(mod)

The residual plot and the q-q plot definitely suggests that we should not build a linear regression model using the variables without any sort of transformation. Perhaps we should first bucket the variables, e.g. cut views into deciles and then build a model on each. Or, remove any outlier and then apply Box-Cox to transform the data to make sure it becomes constant variance and suitable for linear regression.