DsLabs

Loading libraries and data

library(dslabs)

Warning: package 'dslabs' was built under R version 4.6.1

library(dplyr)

Warning: package 'dplyr' was built under R version 4.6.1

Attaching package: 'dplyr'

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

library(ggplot2)
# data(package = "dslabs")
# list.files(system.file("script", package = "dslabs"))
data("movielens")
head(movielens)

  movieId                                   title year
1      31                         Dangerous Minds 1995
2    1029                                   Dumbo 1941
3    1061                                Sleepers 1996
4    1129                    Escape from New York 1981
5    1172 Cinema Paradiso (Nuovo cinema Paradiso) 1989
6    1263                        Deer Hunter, The 1978
                            genres userId rating  timestamp
1                            Drama      1    2.5 1260759144
2 Animation|Children|Drama|Musical      1    3.0 1260759179
3                         Thriller      1    3.0 1260759182
4 Action|Adventure|Sci-Fi|Thriller      1    2.0 1260759185
5                            Drama      1    4.0 1260759205
6                        Drama|War      1    2.0 1260759151

# random check on how many years is in the data-set
count(movielens, year)

    year    n
1   1902    6
2   1915    2
3   1916    1
4   1917    2
5   1918    2
6   1919    1
7   1920   15
8   1921   12
9   1922   28
10  1923    3
11  1924   10
12  1925   40
13  1926   20
14  1927   38
15  1928   21
16  1929   18
17  1930   31
18  1931   98
19  1932   44
20  1933   74
21  1934   75
22  1935   73
23  1936   65
24  1937  125
25  1938  100
26  1939  288
27  1940  317
28  1941  254
29  1942  221
30  1943   42
31  1944  162
32  1945   75
33  1946  193
34  1947   87
35  1948  119
36  1949   99
37  1950  236
38  1951  243
39  1952  152
40  1953  218
41  1954  346
42  1955  290
43  1956  170
44  1957  316
45  1958  246
46  1959  368
47  1960  304
48  1961  296
49  1962  368
50  1963  352
51  1964  429
52  1965  249
53  1966  226
54  1967  434
55  1968  473
56  1969  282
57  1970  295
58  1971  641
59  1972  449
60  1973  525
61  1974  540
62  1975  712
63  1976  471
64  1977  674
65  1978  602
66  1979  885
67  1980 1123
68  1981  970
69  1982 1240
70  1983 1014
71  1984 1581
72  1985 1427
73  1986 1769
74  1987 1646
75  1988 1689
76  1989 2198
77  1990 2105
78  1991 1914
79  1992 2301
80  1993 4214
81  1994 5861
82  1995 6635
83  1996 5109
84  1997 4060
85  1998 4019
86  1999 5034
87  2000 4054
88  2001 3442
89  2002 3090
90  2003 2508
91  2004 2879
92  2005 1852
93  2006 1786
94  2007 1606
95  2008 1544
96  2009 1375
97  2010 1238
98  2011  926
99  2012  939
100 2013  839
101 2014  798
102 2015  535
103 2016  124
104   NA    7

Summarizing and cleaning the data

# Summarizing each movie as number of ratings, average rating, and their release year
movie <- movielens |>
  group_by(movieId, year) |>
  summarize(num_r = n(),
            # taking average
            avg_rating = mean(rating),
            .groups = "drop") |>
  # dropping movies having fewer ratings
  filter(num_r >= 10)

Creating my visualization

# Creating scatterplot with the new (cleaned) data-set
ggplot(movie, aes(x = num_r, y = avg_rating, color = year)) +
  geom_point(alpha = 0.5) + 
  # using log as the numbers are very skewed
  scale_x_log10() +
  # using orange and dark-blue color as it is easy to view
  scale_color_gradient(low = "orange", high = "darkblue",
                       name = "Year") +
  # adding title, subtitle,source, x axis, and y axis name
  labs(title = "Popular Movies with Higher Rating?",
       subtitle = "Each point is a movie and color shows its release year",
       x = "Ratings in number (log scle)",
       y = "Average rating (0.5 to 5)",
       caption = "Source: DS Labs (movielens dataset)") +
  theme_minimal() 

My Work

Above is the visualization and cleaning I did as per the requirement of the assignment. I chose “movielens” dataset from “dslabs” which is interesting as it covers the movie, its release date, genre, timestamp, and the ratings. This dataset has 100,004 observations and 7 variables however, I used only 3 variables; year, movieId, and rating. Then, I cleaned and summarized the data dropping movies having fewer ratings (less than 10). I calculated the number of ratings and the average rating per movie, and started doing some visualization. I used scatterplot as my preference, and using some functions to make the plot clean like log, different color, changing the alpha. And finally, giving titles and subtitles. As per the result, popular movies are not dramatically better rated, but they are more consistently rated. Movies with more ratings have averages that cluster in a narrow, fairly high band (roughly 3 to 4.5 stars), while movies with few ratings range from very low to very high.