Objective

From an existing dataset of user-item ratings, such as our toy books dataset, MovieLens, Jester [http://eigentaste.berkeley.edu/dataset/] or another dataset of your choosing. Implement at least two of these recommendation algorithms:

• Content-Based Filtering

• User-User Collaborative Filtering

• Item-Item Collaborative Filtering

You should evaluate and compare different approaches, using different algorithms, normalization techniques, similarity methods, neighborhood sizes, etc. You don’t need to be exhaustive—these are just some suggested possibilities.

Data Loading and Data Preparation

library(recommenderlab)
library(reshape2)
library(RCurl)
library(pander)
library(ggplot2)
library(knitr)
library(dplyr)
library(tidyr)
library(ggplot2)

ratings <- read.csv("https://raw.githubusercontent.com/PriyaShaji/Data612/master/Project_2/ratings.csv")

movies <- read.csv("https://raw.githubusercontent.com/PriyaShaji/Data612/master/Project_2/movies.csv")

knitr::kable(head(movies))

knitr::kable(head(ratings))

Movie_Matrix <- ratings %>% select(-timestamp) %>% spread(movieId, rating)

row.names(Movie_Matrix) <- Movie_Matrix[, 1]

Movie_Matrix <- Movie_Matrix[-c(1)]
Movie_Matrix <- as(as.matrix(Movie_Matrix), "realRatingMatrix")

Movie_Matrix

610 x 9724 rating matrix of class 'realRatingMatrix' with 100836 ratings.

Ratings_distribution <- as.vector(Movie_Matrix@data)
Ratings_distribution <- Ratings_distribution[Ratings_distribution != 0]
ggplot() + aes(Ratings_distribution) + geom_histogram(binwidth = 0.5) + xlab("Ratings") + 
    ylab("No. of Ratings")

(movies <- Movie_Matrix[rowCounts(Movie_Matrix) > 50, colCounts(Movie_Matrix) > 
    50])

378 x 436 rating matrix of class 'realRatingMatrix' with 36214 ratings.

avg <- rowMeans(movies)
ggplot() + aes(avg) + geom_histogram(binwidth = 0.1) + xlab("Average Rating") + 
    ylab(" No. of Ratings")

Data Normalization

recommender lab normalizes the data when building a model. Let us normalize the ratings and confirm that all averages are 0 now to see what kind of effect it has.

movie_Normalization <- normalize(movies)
avg <- round(rowMeans(movie_Normalization), 5)
table(avg)

avg
  0 
378 

min_Items <- quantile(rowCounts(movies), 0.95)
min_Users <- quantile(colCounts(movies), 0.95)

image(movies[rowCounts(movies) > min_Items, colCounts(movies) > min_Users], 
    main = "Heatmap of the Top Users and Movies (Non-Normalized")

image(movie_Normalization[rowCounts(movie_Normalization) > min_Items, colCounts(movie_Normalization) > 
    min_Users], main = "Heatmap of the Top Users and Movies (Normalized)")

Reviewing rows in two heatmaps above, we can see that after normalization, the average rating is more uniform. Visually it does appear that bias is reduced.

Algorithms Used

set.seed(80)
train_set <- sample(x = c(TRUE, FALSE), size = nrow(movies), replace = TRUE, 
    prob = c(0.8, 0.2))

movie_Train <- movies[train_set, ]
movie_Test <- movies[!train_set, ]

(model <- Recommender(movie_Train, method = "IBCF"))

Recommender of type 'IBCF' for 'realRatingMatrix' 
learned using 302 users.

similarityMatrix <- getModel(model)$sim
which_max <- order(colSums(similarityMatrix > 0), decreasing = TRUE)[1:10]
topMovies <- as.data.frame(as.integer(rownames(similarityMatrix)[which_max]))
colnames(topMovies) <- c("movieId")
data <- topMovies %>% inner_join(movies, by = "movieId") %>% select(Movie = "title")
knitr::kable(data, format = "html") %>% kableExtra::kable_styling(bootstrap_options = c("striped", 
    "hover"))

(pred <- predict(model, newdata = movie_Test, n = 6))

Recommendations as 'topNList' with n = 6 for 76 users. 

user1 <- as.data.frame(movie_Test@data[1, movie_Test@data[1, ] > 0])
colnames(user1) <- c("Rating")
user1[c("movieId")] <- as.integer(rownames(user1))
data <- movies %>% inner_join(user1, by = "movieId") %>% select(Movie = "title", 
    Rating) %>% arrange(desc(Rating))
knitr::kable(data, format = "html") %>% kableExtra::kable_styling(bootstrap_options = c("striped", 
    "hover"))

recommended <- pred@itemLabels[pred@items[[1]]]
recommended <- as.data.frame(as.integer(recommended))
colnames(recommended) <- c("movieId")
data <- recommended %>% inner_join(movies, by = "movieId") %>% select(Movie = "title")
knitr::kable(data, format = "html") %>% kableExtra::kable_styling(bootstrap_options = c("striped", 
    "hover"))

(model <- Recommender(movie_Train, method = "UBCF"))

Recommender of type 'UBCF' for 'realRatingMatrix' 
learned using 302 users.

(pred <- predict(model, newdata = movie_Test, n = 6))

Recommendations as 'topNList' with n = 6 for 76 users. 

# Recommendations for the first user
recommended <- pred@itemLabels[pred@items[[1]]]
recommended <- as.data.frame(as.integer(recommended))
colnames(recommended) <- c("movieId")
data <- recommended %>% inner_join(movies, by = "movieId") %>% select(Movie = "title")
knitr::kable(data, format = "html") %>% kableExtra::kable_styling(bootstrap_options = c("striped", 
    "hover"))

model <- Recommender(movie_Train, method = "UBCF", parameter = list(normalize = NULL))
pred <- predict(model, newdata = movie_Test, n = 6)
recommended <- pred@itemLabels[pred@items[[1]]]
recommended <- as.data.frame(as.integer(recommended))
colnames(recommended) <- c("movieId")
data <- recommended %>% inner_join(movies, by = "movieId") %>% select(Movie = "title")
knitr::kable(data, format = "html") %>% kableExtra::kable_styling(bootstrap_options = c("striped", 
    "hover"))

Summary

For both Item-Item and User-User collaborative filtering, the recommendations which the first user got was almost similar. Movie ratings extracted for the first user consisted of drama genre movies. Therefore it was interesting to see that the movies recommended to the same user also consisted of drama genre movies.

The algorithms are carried out using recommenderlab package in R. It is a basic recommender system. More development and testing would be needed for a usable recommender system.

Comparing the results of UBCF with IBCF helps in understanding the algorithm better. UBCF needs to access the initial data, so it is a lazy-learning model. Since it needs to keep the entire database in memory, it doesn’t work well in the presence of a big rating matrix.

However, UBCF’s accuracy is proven to be slightly more accurate than IBCF, so it’s a good option if the dataset is not too big.