Web Scraping in R (using rvest)
Jennifer Johnson
16/06/2020
Scraping a webpage (IMDb) using R
Exercise found at: https://www.analyticsvidhya.com/blog/2017/03/beginners-guide-on-web-scraping-in-r-using-rvest-with-hands-on-knowledge/
# install.packages('rvest')library(tidyverse)## -- Attaching packages ---------------------------------------------------------------------------------------------------------- tidyverse 1.3.0 --## v ggplot2 3.3.1 v purrr 0.3.4
## v tibble 3.0.1 v dplyr 1.0.0
## v tidyr 1.1.0 v stringr 1.4.0
## v readr 1.3.1 v forcats 0.5.0## -- Conflicts ------------------------------------------------------------------------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()#Loading the rvest package
library('rvest')## Loading required package: xml2##
## Attaching package: 'rvest'## The following object is masked from 'package:purrr':
##
## pluck## The following object is masked from 'package:readr':
##
## guess_encoding#Specifying the url for desired website to be scraped
url <- 'http://www.imdb.com/search/title?count=100&release_date=2016,2016&title_type=feature'#Reading the HTML code from the website
webpage <- read_html(url)Now, we’ll be scraping the following data from this website.
Rank: The rank of the film from 1 to 100 on the list of 100 most popular feature films released in 2016. Title: The title of the feature film. Description: The description of the feature film. Runtime: The duration of the feature film. Genre: The genre of the feature film, Rating: The IMDb rating of the feature film. Metascore: The metascore on IMDb website for the feature film. Votes: Votes cast in favor of the feature film. Gross_Earning_in_Mil: The gross earnings of the feature film in millions. Director: The main director of the feature film. Note, in case of multiple directors, I’ll take only the first. Actor: The main actor in the feature film. Note, in case of multiple actors, I’ll take only the first.
Screenshot of IMDb Scraping Fields
Step 1
Now, we will start by scraping the Rank field. For that, we’ll use the selector gadget to get the specific CSS selectors that encloses the rankings. You can click on the extension in your browser and select the rankings field with the cursor.
Screenshot Rank Field CSS Selector
Make sure that all the rankings are selected. You can select some more ranking sections in case you are not able to get all of them and you can also de-select them by clicking on the selected section to make sure that you only have those sections highlighted that you want to scrape for that go.
Step 2 Once you are sure that you have made the right selections, you need to copy the corresponding CSS selector that you can view in the bottom center.
Step 3 Once you know the CSS selector that contains the rankings, you can use this simple R code to get all the rankings:
#Using CSS selectors to scrape the rankings section
rank_data_html <- html_nodes(webpage,'.text-primary')#Converting the ranking data to text
rank_data <- html_text(rank_data_html)#Let's have a look at the rankings
head(rank_data)## [1] "1." "2." "3." "4." "5." "6."Step 4 Once you have the data, make sure that it looks in the desired format. I am preprocessing my data to convert it to numerical format.
#Data-Preprocessing: Converting rankings to numerical
rank_data<-as.numeric(rank_data)#Let's have another look at the rankings
head(rank_data)## [1] 1 2 3 4 5 6Step 5 Now you can clear the selector section and select all the titles. You can visually inspect that all the titles are selected. Make any required additions and deletions with the help of your curser. I have done the same here.
Screenshot Title Field CSS Selector
Step 6 Again, I have the corresponding CSS selector for the titles – .lister-item-header a. I will use this selector to scrape all the titles using the following code.
#Using CSS selectors to scrape the title section
title_data_html <- html_nodes(webpage,'.lister-item-header a')#Converting the title data to text
title_data <- html_text(title_data_html)#Let's have a look at the title
head(title_data)## [1] "Moana"
## [2] "Moonlight"
## [3] "Suicide Squad"
## [4] "Rogue One: A Star Wars Story"
## [5] "Miss Peregrine's Home for Peculiar Children"
## [6] "La La Land"Step 7 In the following code, I have done the same thing for scraping – Description, Runtime, Genre, Rating, Metascore, Votes, Gross_Earning_in_Mil , Director and Actor data.
#Using CSS selectors to scrape the description section
description_data_html <- html_nodes(webpage,'.ratings-bar+ .text-muted')#Converting the description data to text
description_data <- html_text(description_data_html)#Let's have a look at the description data
head(description_data)## [1] "\n In Ancient Polynesia, when a terrible curse incurred by the Demigod Maui reaches Moana's island, she answers the Ocean's call to seek out the Demigod to set things right."
## [2] "\n A young African-American man grapples with his identity and sexuality while experiencing the everyday struggles of childhood, adolescence, and burgeoning adulthood."
## [3] "\n A secret government agency recruits some of the most dangerous incarcerated super-villains to form a defensive task force. Their first mission: save the world from the apocalypse."
## [4] "\n The daughter of an Imperial scientist joins the Rebel Alliance in a risky move to steal the Death Star plans."
## [5] "\n When Jacob (Asa Butterfield) discovers clues to a mystery that stretches across time, he finds Miss Peregrine's Home for Peculiar Children. But the danger deepens after he gets to know the residents and learns about their special powers."
## [6] "\n While navigating their careers in Los Angeles, a pianist and an actress fall in love while attempting to reconcile their aspirations for the future."#Data-Preprocessing: removing '\n'
description_data<-gsub("\n","",description_data)#Let's have another look at the description data
head(description_data)## [1] " In Ancient Polynesia, when a terrible curse incurred by the Demigod Maui reaches Moana's island, she answers the Ocean's call to seek out the Demigod to set things right."
## [2] " A young African-American man grapples with his identity and sexuality while experiencing the everyday struggles of childhood, adolescence, and burgeoning adulthood."
## [3] " A secret government agency recruits some of the most dangerous incarcerated super-villains to form a defensive task force. Their first mission: save the world from the apocalypse."
## [4] " The daughter of an Imperial scientist joins the Rebel Alliance in a risky move to steal the Death Star plans."
## [5] " When Jacob (Asa Butterfield) discovers clues to a mystery that stretches across time, he finds Miss Peregrine's Home for Peculiar Children. But the danger deepens after he gets to know the residents and learns about their special powers."
## [6] " While navigating their careers in Los Angeles, a pianist and an actress fall in love while attempting to reconcile their aspirations for the future."# unsuccessful attempt 1 to get rid of spacing issue with "
description_data<-gsub("\n ","",description_data)#Let's have another look at the description data
head(description_data)## [1] " In Ancient Polynesia, when a terrible curse incurred by the Demigod Maui reaches Moana's island, she answers the Ocean's call to seek out the Demigod to set things right."
## [2] " A young African-American man grapples with his identity and sexuality while experiencing the everyday struggles of childhood, adolescence, and burgeoning adulthood."
## [3] " A secret government agency recruits some of the most dangerous incarcerated super-villains to form a defensive task force. Their first mission: save the world from the apocalypse."
## [4] " The daughter of an Imperial scientist joins the Rebel Alliance in a risky move to steal the Death Star plans."
## [5] " When Jacob (Asa Butterfield) discovers clues to a mystery that stretches across time, he finds Miss Peregrine's Home for Peculiar Children. But the danger deepens after he gets to know the residents and learns about their special powers."
## [6] " While navigating their careers in Los Angeles, a pianist and an actress fall in love while attempting to reconcile their aspirations for the future."# unsuccessful attempt 2 to get rid of spacing issue with "
description_data<-gsub("\n####","",description_data)#Let's have another look at the description data
head(description_data)## [1] " In Ancient Polynesia, when a terrible curse incurred by the Demigod Maui reaches Moana's island, she answers the Ocean's call to seek out the Demigod to set things right."
## [2] " A young African-American man grapples with his identity and sexuality while experiencing the everyday struggles of childhood, adolescence, and burgeoning adulthood."
## [3] " A secret government agency recruits some of the most dangerous incarcerated super-villains to form a defensive task force. Their first mission: save the world from the apocalypse."
## [4] " The daughter of an Imperial scientist joins the Rebel Alliance in a risky move to steal the Death Star plans."
## [5] " When Jacob (Asa Butterfield) discovers clues to a mystery that stretches across time, he finds Miss Peregrine's Home for Peculiar Children. But the danger deepens after he gets to know the residents and learns about their special powers."
## [6] " While navigating their careers in Los Angeles, a pianist and an actress fall in love while attempting to reconcile their aspirations for the future."# successful attempt 3 to get rid of spacing issue with "
description_data<-gsub(" ","",description_data)#Let's have another look at the description data
head(description_data)## [1] "In Ancient Polynesia, when a terrible curse incurred by the Demigod Maui reaches Moana's island, she answers the Ocean's call to seek out the Demigod to set things right."
## [2] "A young African-American man grapples with his identity and sexuality while experiencing the everyday struggles of childhood, adolescence, and burgeoning adulthood."
## [3] "A secret government agency recruits some of the most dangerous incarcerated super-villains to form a defensive task force. Their first mission: save the world from the apocalypse."
## [4] "The daughter of an Imperial scientist joins the Rebel Alliance in a risky move to steal the Death Star plans."
## [5] "When Jacob (Asa Butterfield) discovers clues to a mystery that stretches across time, he finds Miss Peregrine's Home for Peculiar Children. But the danger deepens after he gets to know the residents and learns about their special powers."
## [6] "While navigating their careers in Los Angeles, a pianist and an actress fall in love while attempting to reconcile their aspirations for the future."#Using CSS selectors to scrape the Movie runtime section
runtime_data_html <- html_nodes(webpage,'.text-muted .runtime')#Converting the runtime data to text
runtime_data <- html_text(runtime_data_html)#Let's have a look at the runtime
head(runtime_data)## [1] "107 min" "111 min" "123 min" "133 min" "127 min" "128 min"#Data-Preprocessing: removing mins and converting it to numerical
runtime_data<-gsub(" min","",runtime_data)
runtime_data<-as.numeric(runtime_data)#Let's have another look at the runtime data
head(runtime_data)## [1] 107 111 123 133 127 128#Using CSS selectors to scrape the Movie genre section
genre_data_html <- html_nodes(webpage,'.genre')#Converting the genre data to text
genre_data <- html_text(genre_data_html)#Let's have a look at the runtime
head(genre_data)## [1] "\nAnimation, Adventure, Comedy "
## [2] "\nDrama "
## [3] "\nAction, Adventure, Fantasy "
## [4] "\nAction, Adventure, Sci-Fi "
## [5] "\nAdventure, Drama, Family "
## [6] "\nComedy, Drama, Music "#Data-Preprocessing: removing \n
genre_data<-gsub("\n","",genre_data)#Data-Preprocessing: removing excess spaces
genre_data<-gsub(" ","",genre_data)#taking only the first genre of each movie
genre_data<-gsub(",.*","",genre_data)#Convering each genre from text to factor
genre_data<-as.factor(genre_data)#Let's have another look at the genre data
head(genre_data)## [1] Animation Drama Action Action Adventure Comedy
## Levels: Action Adventure Animation Biography Comedy Crime Drama Horror#Using CSS selectors to scrape the IMDB rating section
rating_data_html <- html_nodes(webpage,'.ratings-imdb-rating strong')#Converting the ratings data to text
rating_data <- html_text(rating_data_html)#Let's have a look at the ratings
head(rating_data)## [1] "7.6" "7.4" "6.0" "7.8" "6.7" "8.0"#Data-Preprocessing: converting ratings to numerical
rating_data<-as.numeric(rating_data)#Let's have another look at the ratings data
head(rating_data)## [1] 7.6 7.4 6.0 7.8 6.7 8.0#Using CSS selectors to scrape the votes section
votes_data_html <- html_nodes(webpage,'.sort-num_votes-visible span:nth-child(2)')#Converting the votes data to text
votes_data <- html_text(votes_data_html)#Let's have a look at the votes data
head(votes_data)## [1] "255,125" "258,773" "580,892" "533,067" "150,584" "480,918"#Data-Preprocessing: removing commas
votes_data<-gsub(",","",votes_data)#Data-Preprocessing: converting votes to numerical
votes_data<-as.numeric(votes_data)#Let's have another look at the votes data
head(votes_data)## [1] 255125 258773 580892 533067 150584 480918#Using CSS selectors to scrape the directors section
directors_data_html <- html_nodes(webpage,'.text-muted+ p a:nth-child(1)')#Converting the directors data to text
directors_data <- html_text(directors_data_html)#Let's have a look at the directors data
head(directors_data)## [1] "Ron Clements" "Barry Jenkins" "David Ayer" "Gareth Edwards"
## [5] "Tim Burton" "Damien Chazelle"#Data-Preprocessing: converting directors data into factors
directors_data<-as.factor(directors_data)#Using CSS selectors to scrape the actors section
actors_data_html <- html_nodes(webpage,'.lister-item-content .ghost+ a')#Converting the gross actors data to text
actors_data <- html_text(actors_data_html)#Let's have a look at the actors data
head(actors_data)## [1] "Auli'i Cravalho" "Mahershala Ali" "Will Smith" "Felicity Jones"
## [5] "Eva Green" "Ryan Gosling"#Data-Preprocessing: converting actors data into factors
#**unsure what this is doing exactly**
actors_data<-as.factor(actors_data)#Let's have a look at the actors data
head(actors_data)## [1] Auli'i Cravalho Mahershala Ali Will Smith Felicity Jones
## [5] Eva Green Ryan Gosling
## 92 Levels: Aamir Khan Adam Driver Adam Sandler ... Zoey DeutchBut, I want you to closely follow what happens when I do the same thing for Metascore data.
#Using CSS selectors to scrape the metascore section
metascore_data_html <- html_nodes(webpage,'.metascore')#Converting the runtime data to text
metascore_data <- html_text(metascore_data_html)#Let's have a look at the metascore
head(metascore_data)## [1] "81 " "99 " "40 " "65 " "57 "
## [6] "94 "#Data-Preprocessing: removing extra space in metascore
metascore_data<-gsub(" ","",metascore_data)#Let's have a look at the metascore
head(metascore_data)## [1] "81" "99" "40" "65" "57" "94"#Lets check the length of metascore data
length(metascore_data)## [1] 98Step 8 The length of the metascore data is 98 while we are scraping the data for 100 movies. The reason this happened is that there are 2 movies that don’t have the corresponding Metascore fields.
Missing Metascore
Step 9 It is a practical situation which can arise while scraping any website. Unfortunately, if we simply add NA’s to last 4 entries, it will map NA as Metascore for movies 96 to 100 while in reality, the data is missing for some other movies. After a visual inspection, I found that the Metascore is missing for movies 22 and 80. I have written the following function to get around this problem.
for (i in c(22,80)){
a<-metascore_data[1:(i-1)]
b<-metascore_data[i:length(metascore_data)]
metascore_data<-append(a,list("NA"))
metascore_data<-append(metascore_data,b)
}#Data-Preprocessing: converting metascore to numerical
metascore_data<-as.numeric(metascore_data)## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion#Let's have another look at length of the metascore data
length(metascore_data)## [1] 100#Let's look at summary statistics
summary(metascore_data)## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 21.00 47.25 62.00 60.19 73.50 99.00 2Step 10 The same thing happens with the Gross variable which represents gross earnings of that movie in millions. I have use the same solution to work my way around:
#Using CSS selectors to scrap the gross revenue section
gross_data_html <- html_nodes(webpage,'.ghost~ .text-muted+ span')#Converting the gross revenue data to text
gross_data <- html_text(gross_data_html)#Let's have a look at the gross data
head(gross_data)## [1] "$248.76M" "$27.85M" "$325.10M" "$532.18M" "$87.24M" "$151.10M"#Data-Preprocessing: removing '$' and 'M' signs
gross_data<-gsub("[^0-9]*","",gross_data)head(gross_data)## [1] "24876" "2785" "32510" "53218" "8724" "15110"#Let's check the length of gross data
length(gross_data)## [1] 90#Filling missing entries with NA
for (i in c(29,33,40,41,43,72,74,75,76,100)){
a<-gross_data[1:(i-1)]
b<-gross_data[i:length(gross_data)]
gross_data<-append(a,list("NA"))
gross_data<-append(gross_data,b)
}#Data-Preprocessing: converting gross to numerical
unlist(gross_data)## [1] "24876" "2785" "32510" "53218" "8724" "15110" "33036" "34127" "10055"
## [10] "3626" "6721" "23264" "40808" "36307" "502" "5870" "23404" "588"
## [19] "201" "16961" "213" "9343" "13829" "5625" "5465" "1064" "12664"
## [28] "3434" "NA" "15885" "5285" "15544" "NA" "4770" "133" "10001"
## [37] "27040" "8626" "10314" "NA" "NA" "8922" "NA" "3582" "9769"
## [46] "5174" "1443" "7540" "2686" "770" "6143" "16243" "15371" "12744"
## [55] "3115" "6508" "3008" "4737" "421" "3559" "858" "5512" "7208"
## [64] "10247" "520" "710" "36400" "12834" "4303" "4684" "6727" "NA"
## [73] "12507" "NA" "NA" "NA" "3035" "6032" "6618" "2641" "4010"
## [82] "48630" "11326" "1239" "1279" "2683" "8205" "018" "6268" "3492"
## [91] "066" "811" "36838" "2159" "3189" "4601" "1091" "214" "5764"
## [100] "NA" "5764"gross_data <- gross_data[-c(101,102)]gross_data<-as.numeric(gross_data)## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion#Let's have another look at the length of gross data
length(gross_data)## [1] 100summary(gross_data)## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 18 2711 5817 9960 12625 53218 10Step 11 Now we have successfully scraped all the 11 features for the 100 most popular feature films released in 2016. Let’s combine them to create a dataframe and inspect its structure.
#Combining all the lists to form a data frame
movies_df<-data.frame(Rank = rank_data, Title = title_data,
Description = description_data, Runtime = runtime_data,
Genre = genre_data, Rating = rating_data,
Metascore = metascore_data, Votes = votes_data, Gross_Earning_in_Mil = gross_data,
Director = directors_data, Actor = actors_data)#Structure of the data frame
str(movies_df)## 'data.frame': 100 obs. of 11 variables:
## $ Rank : num 1 2 3 4 5 6 7 8 9 10 ...
## $ Title : chr "Moana" "Moonlight" "Suicide Squad" "Rogue One: A Star Wars Story" ...
## $ Description : chr "In Ancient Polynesia, when a terrible curse incurred by the Demigod Maui reaches Moana's island, she answers th"| __truncated__ "A young African-American man grapples with his identity and sexuality while experiencing the everyday struggles"| __truncated__ "A secret government agency recruits some of the most dangerous incarcerated super-villains to form a defensive "| __truncated__ "The daughter of an Imperial scientist joins the Rebel Alliance in a risky move to steal the Death Star plans." ...
## $ Runtime : num 107 111 123 133 127 128 151 108 116 116 ...
## $ Genre : Factor w/ 8 levels "Action","Adventure",..: 3 7 1 1 2 5 1 3 7 1 ...
## $ Rating : num 7.6 7.4 6 7.8 6.7 8 6.5 8 7.9 7.4 ...
## $ Metascore : num 81 99 40 65 57 94 44 78 81 70 ...
## $ Votes : num 255125 258773 580892 533067 150584 ...
## $ Gross_Earning_in_Mil: num 24876 2785 32510 53218 8724 ...
## $ Director : Factor w/ 98 levels "Alex Proyas",..: 82 11 25 35 92 20 98 14 29 86 ...
## $ Actor : Factor w/ 92 levels "Aamir Khan","Adam Driver",..: 8 52 89 32 31 72 9 34 5 70 ...#'data.frame': 100 obs. of 11 variables:
#$ Rank : num 1 2 3 4 5 6 7 8 9 10 ...
#$ Title : Factor w/ 99 levels "10 Cloverfield Lane",..: 66 53 54 32 58 93 8 43 97 7 ...
#$ Description : Factor w/ 100 levels "19-year-old Billy Lynn is brought home for a victory tour after a harrowing Iraq battle. Through flashbacks the film shows what"| __truncated__,..: 57 59 3 100 21 33 90 14 13 97 ...
#$ Runtime : num 108 107 111 139 116 92 115 128 111 116 ...
#$ Genre : Factor w/ 10 levels "Action","Adventure",..: 3 3 7 4 2 3 1 5 5 7 ...
#$ Rating : num 7.2 7.7 7.6 8.2 7 6.5 6.1 8.4 6.3 8 ...
#$ Metascore : num 59 81 99 71 41 56 36 93 39 81 ...
#$ Votes : num 40603 91333 112609 177229 148467 ...
#$ Gross_Earning_in_Mil: num 269.3 248 27.5 67.1 99.5 ...
#$ Director : Factor w/ 98 levels "Andrew Stanton",..: 17 80 9 64 67 95 56 19 49 28 ...
#$ Actor : Factor w/ 86 levels "Aaron Eckhart",..: 59 7 56 5 42 6 64 71 86 3 ...Step 6. Analyzing scraped data from the web
Once you have the data, you can perform several tasks like analyzing the data, drawing inferences from it, training machine learning models over this data, etc. I have gone on to create some interesting visualization out of the data we have just scraped. Follow the visualizations and answer the questions given below. Post your answers in the comment section below.
library('ggplot2')
qplot(data = movies_df,Runtime,fill = Genre,bins = 30)
runtime_data## [1] 107 111 123 133 127 128 151 108 116 116 139 115 147 108 106 118 132 118
## [19] 145 127 118 106 132 117 106 115 116 110 121 122 144 144 137 117 116 108
## [37] 128 120 88 97 89 118 104 112 102 123 107 94 123 92 107 88 127 129
## [55] 106 123 97 102 141 86 103 134 86 101 161 106 116 114 110 81 96 82
## [73] 91 111 120 123 124 97 100 161 115 107 112 108 118 99 112 163 146 87
## [91] 130 134 156 111 133 98 108 118 139 101max(runtime_data)## [1] 163#maxruntime <-c(runtime_data = 163)# maxruntime# which(runtime_data[163)#By counting I know it is number 88, but how would I find the exact number and the title of film 88 with R?# install.packages("ggplotlyExtra")library(ggplotlyExtra)Question 1 Based on the above data, which movie from which Genre had the longest runtime?
movies_df<-data.frame(Rank = rank_data, Title = title_data,
Description = description_data, Runtime = runtime_data,
Genre = genre_data, Rating = rating_data,
Metascore = metascore_data, Votes = votes_data, Gross_Earning_in_Mil = gross_data,
Director = directors_data, Actor = actors_data)
#Structure of the data frame
str(movies_df)## 'data.frame': 100 obs. of 11 variables:
## $ Rank : num 1 2 3 4 5 6 7 8 9 10 ...
## $ Title : chr "Moana" "Moonlight" "Suicide Squad" "Rogue One: A Star Wars Story" ...
## $ Description : chr "In Ancient Polynesia, when a terrible curse incurred by the Demigod Maui reaches Moana's island, she answers th"| __truncated__ "A young African-American man grapples with his identity and sexuality while experiencing the everyday struggles"| __truncated__ "A secret government agency recruits some of the most dangerous incarcerated super-villains to form a defensive "| __truncated__ "The daughter of an Imperial scientist joins the Rebel Alliance in a risky move to steal the Death Star plans." ...
## $ Runtime : num 107 111 123 133 127 128 151 108 116 116 ...
## $ Genre : Factor w/ 8 levels "Action","Adventure",..: 3 7 1 1 2 5 1 3 7 1 ...
## $ Rating : num 7.6 7.4 6 7.8 6.7 8 6.5 8 7.9 7.4 ...
## $ Metascore : num 81 99 40 65 57 94 44 78 81 70 ...
## $ Votes : num 255125 258773 580892 533067 150584 ...
## $ Gross_Earning_in_Mil: num 24876 2785 32510 53218 8724 ...
## $ Director : Factor w/ 98 levels "Alex Proyas",..: 82 11 25 35 92 20 98 14 29 86 ...
## $ Actor : Factor w/ 92 levels "Aamir Khan","Adam Driver",..: 8 52 89 32 31 72 9 34 5 70 ...# p1 <- qplot(data = movies_df,Runtime,fill = Genre,bins = 30)
p1 <- movies_df %>%
ggplot(aes(x=Runtime, fill = Genre)) +
geom_histogram(position="identity", alpha=0.5, binwidth = 5, color = "white")+
scale_fill_discrete(name = "Genre") +
labs(title = "Top 100 Movies of 2016 Runtime by Genre")
plot(p1)
# tried ggplotly but not available for R version 4.0movies_df %>%
rownames_to_column(var = "Name") %>%
filter(Runtime == max(Runtime))## Name Rank Title
## 1 88 88 American Honey
## Description
## 1 A teenage girl with nothing to lose joins a traveling magazine sales crew, and gets caught up in a whirlwind of hard partying, law bending and young love as she criss-crosses the Midwest with a band of misfits.
## Runtime Genre Rating Metascore Votes Gross_Earning_in_Mil Director
## 1 163 Drama 7 79 35944 18 Andrea Arnold
## Actor
## 1 Sasha LaneAnswer Based on the above data, the longest runtime was a drama, American Honey, which lasted 163 minutes.
ggplot(movies_df,aes(x=Runtime,y=Rating))+
geom_point(aes(size=Votes,col=Genre))
Question 2 Based on the above data, in the Runtime of 130-160 mins, which genre has the highest votes?
p2 <- movies_df %>%
ggplot(aes(x=Runtime,y=Rating))+
geom_point(aes(size=Votes,col=Genre, text = paste("Movie Title:", title_data)), alpha = 0.7) +
labs(title = "Top 100 Movies of 2016 Runtime by Ratings") ## Warning: Ignoring unknown aesthetics: textp2
movies_df %>%
rownames_to_column(var = "Name") %>%
filter(Runtime == c(130,160)) %>%
filter(Votes == max(Votes))## Name Rank Title
## 1 91 91 A Silent Voice: The Movie
## Description
## 1 A young man is ostracized by his classmates after he bullies a deaf girl to the point where she moves away. Years later, he sets off on a path for redemption.
## Runtime Genre Rating Metascore Votes Gross_Earning_in_Mil Director
## 1 130 Animation 8.2 78 38139 66 Naoko Yamada
## Actor
## 1 Miyu Irino# trying to get answer in Genre rather than title name.
movies_df %>%
rownames_to_column(var = "Name") %>%
filter(Runtime == c(130,160)) %>%
group_by(Genre) %>%
filter(Votes == max(Votes))## # A tibble: 1 x 12
## # Groups: Genre [1]
## Name Rank Title Description Runtime Genre Rating Metascore Votes
## <chr> <dbl> <chr> <chr> <dbl> <fct> <dbl> <dbl> <dbl>
## 1 91 91 A Si~ A young ma~ 130 Anim~ 8.2 78 38139
## # ... with 3 more variables: Gross_Earning_in_Mil <dbl>, Director <fct>,
## # Actor <fct># trying to get answer in Genre rather than title name.
movies_df %>%
rownames_to_column(var = "Name") %>%
filter(Runtime == c(130,160)) %>%
group_by(Genre) %>%
summarize(Votes = mean(Votes)) %>%
filter(Votes == max(Votes))## `summarise()` ungrouping output (override with `.groups` argument)## # A tibble: 1 x 2
## Genre Votes
## <fct> <dbl>
## 1 Animation 38139# I think the answer above is giving me the genre of the movie with the highest number of votes. How do I get the votes sorted by genre and then averaged? Particularly because the error given seems to say that the summarize function overrides the group by function.# trying to get answer in Genre rather than title name.
#movies_df %>%
#rownames_to_column(var = "Name") %>%
#filter(Runtime == c(130,160)) %>%
#summarize(Votes = mean(Votes)) %>%
#filter(Votes == max(Votes)) %>%
#group_by(Genre)# trying to get answer in Genre rather than title name.
#movies_df %>%
#rownames_to_column(var = "Name") %>%
#filter(Runtime == c(130,160)) %>%
#group_by(Genre)
#summarize(votes_data == mean(votes_data)) %>%
#filter(votes_data == max(votes_data))Answer Based on the above data, in the Runtime of 130-160 minutes, the animation genre has the highest votes.
ggplot(movies_df,aes(x=Runtime,y=Gross_Earning_in_Mil))+
geom_point(aes(size=Rating,col=Genre))## Warning: Removed 10 rows containing missing values (geom_point).
Question 3 Based on the above data, across all genres which genre has the highest average gross earnings in runtime 100 to 120.
# attempted ggplot with error
#p3 <- movies_df %>%
#ggplot(aes(x=Runtime,y=Gross_Earning_in_Mil))+
#geom_point(aes(size = Rating,col = Genre), alpha = 0.5) +
#labs(title = "Top 100 Movies of 2016 Runtime by Gross Earnings in Millions") +
#scale_y_continuous("Gross Earnings in Millions", limits =c(-10, 600))
#ggplot(p3)#p3
p3 that produced output but not knit
# according to the plot above it seems to be comedy?# step 1 runtime >=100 - I'm going step by step to get this, I know there is a way to do it all with piping in one go, but I need to make logical sense of the steps first!
runtime_over100 <- movies_df %>%
filter(Runtime >= 100)# unsuccessful group by genre - results provided movie names
runtime_100_120 <- runtime_over100 %>%
filter(Runtime <= 120)
runtime_100_120## Rank Title
## 1 1 Moana
## 2 2 Moonlight
## 3 8 Zootopia
## 4 9 Arrival
## 5 10 The Nice Guys
## 6 12 Doctor Strange
## 7 14 Deadpool
## 8 15 Your Name.
## 9 16 Jack Reacher: Never Go Back
## 10 18 Captain Fantastic
## 11 21 Train to Busan
## 12 22 The Invisible Guest
## 13 24 Split
## 14 25 Me Before You
## 15 26 Assassin's Creed
## 16 27 Nocturnal Animals
## 17 28 The Legend of Tarzan
## 18 34 The Neon Demon
## 19 35 Passengers
## 20 36 Sing
## 21 38 Independence Day: Resurgence
## 22 42 Lion
## 23 43 The Edge of Seventeen
## 24 44 The Girl on the Train
## 25 45 Hell or High Water
## 26 47 Deepwater Horizon
## 27 51 Central Intelligence
## 28 55 Hail, Caesar!
## 29 58 Dirty Grandpa
## 30 61 10 Cloverfield Lane
## 31 64 Hunt for the Wilderpeople
## 32 66 The Jungle Book
## 33 67 Ghostbusters
## 34 68 War Dogs
## 35 69 How to Be Single
## 36 74 Why Him?
## 37 75 Allegiant
## 38 79 Bad Moms
## 39 81 The Founder
## 40 82 Resident Evil: The Final Chapter
## 41 83 Teenage Mutant Ninja Turtles: Out of the Shadows
## 42 84 The Do-Over
## 43 85 The Bad Batch
## 44 87 The 5th Wave
## 45 94 The Girl with All the Gifts
## 46 97 Pride and Prejudice and Zombies
## 47 98 Paterson
## 48 100 Special Correspondents
## Description
## 1 In Ancient Polynesia, when a terrible curse incurred by the Demigod Maui reaches Moana's island, she answers the Ocean's call to seek out the Demigod to set things right.
## 2 A young African-American man grapples with his identity and sexuality while experiencing the everyday struggles of childhood, adolescence, and burgeoning adulthood.
## 3 In a city of anthropomorphic animals, a rookie bunny cop and a cynical con artist fox must work together to uncover a conspiracy.
## 4 A linguist works with the military to communicate with alien lifeforms after twelve mysterious spacecraft appear around the world.
## 5 In 1970s Los Angeles, a mismatched pair of private eyes investigate a missing girl and the mysterious death of a porn star.
## 6 While on a journey of physical and spiritual healing, a brilliant neurosurgeon is drawn into the world of the mystic arts.
## 7 A wisecracking mercenary gets experimented on and becomes immortal but ugly, and sets out to track down the man who ruined his looks.
## 8 Two strangers find themselves linked in a bizarre way. When a connection forms, will distance be the only thing to keep them apart?
## 9 Jack Reacher must uncover the truth behind a major government conspiracy in order to clear his name while on the run as a fugitive from the law.
## 10 In the forests of the Pacific Northwest, a father devoted to raising his six kids with a rigorous physical and intellectual education is forced to leave his paradise and enter the world, challenging his idea of what it means to be a parent.
## 11 While a zombie virus breaks out in South Korea, passengers struggle to survive on the train from Seoul to Busan.
## 12 A successful entrepreneur accused of murder and a witness preparation expert have less than three hours to come up with an impregnable defence.
## 13 Three girls are kidnapped by a man with a diagnosed 23 distinct personalities. They must try to escape before the apparent emergence of a frightful new 24th.
## 14 A girl in a small town forms an unlikely bond with a recently-paralyzed man she's taking care of.
## 15 Callum Lynch explores the memories of his ancestor Aguilar de Nerha and gains the skills of a Master Assassin, before taking on the secret Templar society.
## 16 A wealthy art gallery owner is haunted by her ex-husband's novel, a violent thriller she interprets as a symbolic revenge tale.
## 17 Tarzan, having acclimated to life in London, is called back to his former home in the jungle to investigate the activities at a mining encampment.
## 18 An aspiring model, Jesse, is new to Los Angeles. However, her beauty and youth, which generate intense fascination and jealousy within the fashion industry, may prove themselves sinister.
## 19 A spacecraft traveling to a distant colony planet and transporting thousands of people has a malfunction in its sleep chambers. As a result, two passengers are awakened 90 years early.
## 20 In a city of humanoid animals, a hustling theater impresario's attempt to save his theater with a singing competition becomes grander than he anticipates even as its finalists find that their lives will never be the same.
## 21 Two decades after the first Independence Day invasion, Earth is faced with a new extra-Solar threat. But will mankind's new space defenses be enough?
## 22 A five-year-old Indian boy is adopted by an Australian couple after getting lost hundreds of kilometers from home. 25 years later, he sets out to find his lost family.
## 23 High-school life gets even more unbearable for Nadine when her best friend, Krista, starts dating her older brother.
## 24 A divorcee becomes entangled in a missing persons investigation that promises to send shockwaves throughout her life.
## 25 A divorced father and his ex-con older brother resort to a desperate scheme in order to save their family's ranch in West Texas.
## 26 A dramatization of the disaster in April 2010, when the offshore drilling rig called the Deepwater Horizon exploded, resulting in the worst oil spill in American history.
## 27 After he reconnects with an awkward pal from high school through Facebook, a mild-mannered accountant is lured into the world of international espionage.
## 28 A Hollywood fixer in the 1950s works to keep the studio's stars in line.
## 29 Right before his wedding, an uptight guy is tricked into driving his grandfather, a lecherous former Army Lieutenant Colonel, to Florida for Spring Break.
## 30 After getting in a car accident, a woman is held in a shelter with two men, who claim the outside world is affected by a widespread chemical attack.
## 31 A national manhunt is ordered for a rebellious kid and his foster uncle who go missing in the wild New Zealand bush.
## 32 After a threat from the tiger Shere Khan forces him to flee the jungle, a man-cub named Mowgli embarks on a journey of self discovery with the help of panther Bagheera and free-spirited bear Baloo.
## 33 Following a ghost invasion of Manhattan, paranormal enthusiasts Erin Gilbert and Abby Yates, nuclear engineer Jillian Holtzmann, and subway worker Patty Tolan band together to stop the otherworldly threat.
## 34 Loosely based on the true story of two young men, David Packouz and Efraim Diveroli, who won a three hundred million dollar contract from the Pentagon to arm America's allies in Afghanistan.
## 35 A group of young adults navigate love and relationships in New York City.
## 36 A holiday gathering threatens to go off the rails when Ned Fleming realizes that his daughter's Silicon Valley millionaire boyfriend is about to pop the question.
## 37 After the earth-shattering revelations of Insurgent, Tris must escape with Four beyond the wall that encircles Chicago, to finally discover the shocking truth of the world around them.
## 38 When three overworked and under-appreciated moms are pushed beyond their limits, they ditch their conventional responsibilities for a jolt of long overdue freedom, fun and comedic self-indulgence.
## 39 The story of Ray Kroc, a salesman who turned two brothers' innovative fast food eatery, McDonald's, into the biggest restaurant business in the world, with a combination of ambition, persistence, and ruthlessness.
## 40 Alice returns to where the nightmare began: The Hive in Raccoon City, where the Umbrella Corporation is gathering its forces for a final strike against the only remaining survivors of the apocalypse.
## 41 The Turtles get into another battle with their enemy the Shredder, who has acquired new allies: the mutant thugs Bebop and Rocksteady and the alien being Krang.
## 42 Two down-on-their-luck guys decide to fake their own deaths and start over with new identities, only to find the people they're pretending to be are in even deeper trouble.
## 43 In a desert dystopia, a young woman is kidnapped by cannibals.
## 44 Four waves of increasingly deadly alien attacks have left most of Earth in ruin. Cassie is on the run, desperately trying to save her younger brother.
## 45 A scientist and a teacher living in a dystopian future embark on a journey of survival with a special young girl named Melanie.
## 46 Five sisters in 19th century England must cope with the pressures to marry while protecting themselves from a growing population of zombies.
## 47 A quiet observation of the triumphs and defeats of daily life, along with the poetry evident in its smallest details.
## 48 A radio journalist and his technician get in over their heads when they hatch a scheme to fake their own kidnapping during a rebel uprising in South America and hide out in New York instead.
## Runtime Genre Rating Metascore Votes Gross_Earning_in_Mil
## 1 107 Animation 7.6 81 255125 24876
## 2 111 Drama 7.4 99 258773 2785
## 3 108 Animation 8.0 78 416128 34127
## 4 116 Drama 7.9 81 567996 10055
## 5 116 Action 7.4 70 265589 3626
## 6 115 Action 7.5 72 571548 23264
## 7 108 Action 8.0 65 874655 36307
## 8 106 Animation 8.4 79 171687 502
## 9 118 Action 6.1 47 132601 5870
## 10 118 Comedy 7.9 72 179095 588
## 11 118 Action 7.5 72 142259 213
## 12 106 Crime 8.1 NA 121271 9343
## 13 117 Horror 7.3 62 403409 5625
## 14 106 Drama 7.4 51 196681 5465
## 15 115 Action 5.7 36 179478 1064
## 16 116 Drama 7.5 67 226199 12664
## 17 110 Action 6.2 44 159263 3434
## 18 117 Horror 6.2 51 80128 4770
## 19 116 Drama 7.0 41 336255 133
## 20 108 Animation 7.1 59 127922 10001
## 21 120 Action 5.2 32 162278 8626
## 22 118 Biography 8.0 69 203555 8922
## 23 104 Comedy 7.3 77 98322 NA
## 24 112 Crime 6.5 48 163139 3582
## 25 102 Action 7.6 88 195170 9769
## 26 107 Action 7.1 68 144924 1443
## 27 107 Action 6.3 52 152839 6143
## 28 106 Comedy 6.3 72 120028 3115
## 29 102 Comedy 5.9 21 109391 4737
## 30 103 Drama 7.2 76 280260 858
## 31 101 Adventure 7.9 81 102470 10247
## 32 106 Adventure 7.4 77 252925 710
## 33 116 Action 5.2 60 198216 36400
## 34 114 Biography 7.1 57 169732 12834
## 35 110 Comedy 6.1 51 78857 4303
## 36 111 Comedy 6.2 39 97433 NA
## 37 120 Action 5.7 33 104224 NA
## 38 100 Comedy 6.2 60 109869 6618
## 39 115 Biography 7.2 66 113498 4010
## 40 107 Action 5.5 49 81048 48630
## 41 112 Action 6.0 40 81919 11326
## 42 108 Action 5.7 22 37967 1239
## 43 118 Drama 5.3 62 24774 1279
## 44 112 Action 5.2 33 96588 8205
## 45 111 Drama 6.6 67 52115 2159
## 46 108 Action 5.8 45 49637 1091
## 47 118 Comedy 7.4 90 65597 214
## 48 101 Comedy 5.8 36 19652 NA
## Director Actor
## 1 Ron Clements Auli'i Cravalho
## 2 Barry Jenkins Mahershala Ali
## 3 Byron Howard Ginnifer Goodwin
## 4 Denis Villeneuve Amy Adams
## 5 Shane Black Russell Crowe
## 6 Scott Derrickson Benedict Cumberbatch
## 7 Tim Miller Ryan Reynolds
## 8 Makoto Shinkai Ryûnosuke Kamiki
## 9 Edward Zwick Tom Cruise
## 10 Matt Ross Viggo Mortensen
## 11 Sang-ho Yeon Yoo Gong
## 12 Oriol Paulo Mario Casas
## 13 M. Night Shyamalan James McAvoy
## 14 Thea Sharrock Emilia Clarke
## 15 Justin Kurzel Michael Fassbender
## 16 Tom Ford Amy Adams
## 17 David Yates Alexander Skarsgård
## 18 Nicolas Winding Refn Elle Fanning
## 19 Morten Tyldum Jennifer Lawrence
## 20 Garth Jennings Matthew McConaughey
## 21 Roland Emmerich Liam Hemsworth
## 22 Garth Davis Dev Patel
## 23 Kelly Fremon Craig Hailee Steinfeld
## 24 Tate Taylor Emily Blunt
## 25 David Mackenzie Chris Pine
## 26 Peter Berg Mark Wahlberg
## 27 Rawson Marshall Thurber Dwayne Johnson
## 28 Ethan Coen Josh Brolin
## 29 Dan Mazer Robert De Niro
## 30 Dan Trachtenberg John Goodman
## 31 Taika Waititi Sam Neill
## 32 Jon Favreau Neel Sethi
## 33 Paul Feig Melissa McCarthy
## 34 Todd Phillips Jonah Hill
## 35 Christian Ditter Dakota Johnson
## 36 John Hamburg Zoey Deutch
## 37 Robert Schwentke Shailene Woodley
## 38 Jon Lucas Mila Kunis
## 39 John Lee Hancock Michael Keaton
## 40 Paul W.S. Anderson Milla Jovovich
## 41 Dave Green Megan Fox
## 42 Steven Brill Adam Sandler
## 43 Ana Lily Amirpour Suki Waterhouse
## 44 J Blakeson Chloë Grace Moretz
## 45 Colm McCarthy Sennia Nanua
## 46 Burr Steers Lily James
## 47 Jim Jarmusch Adam Driver
## 48 Ricky Gervais Ricky Gervais#attempting to get highest gross amounts
gross_data## [1] 24876 2785 32510 53218 8724 15110 33036 34127 10055 3626 6721 23264
## [13] 40808 36307 502 5870 23404 588 201 16961 213 9343 13829 5625
## [25] 5465 1064 12664 3434 NA 15885 5285 15544 NA 4770 133 10001
## [37] 27040 8626 10314 NA NA 8922 NA 3582 9769 5174 1443 7540
## [49] 2686 770 6143 16243 15371 12744 3115 6508 3008 4737 421 3559
## [61] 858 5512 7208 10247 520 710 36400 12834 4303 4684 6727 NA
## [73] 12507 NA NA NA 3035 6032 6618 2641 4010 48630 11326 1239
## [85] 1279 2683 8205 18 6268 3492 66 811 36838 2159 3189 4601
## [97] 1091 214 5764 NA# average gross by genre
average_gross_action <- movies_df %>%
group_by(Genre) %>%
arrange(desc(Gross_Earning_in_Mil))
average_gross_action## # A tibble: 100 x 11
## # Groups: Genre [8]
## Rank Title Description Runtime Genre Rating Metascore Votes
## <dbl> <chr> <chr> <dbl> <fct> <dbl> <dbl> <dbl>
## 1 4 Rogu~ The daught~ 133 Acti~ 7.8 65 533067
## 2 82 Resi~ Alice retu~ 107 Acti~ 5.5 49 81048
## 3 13 Capt~ Political ~ 147 Acti~ 7.8 75 637167
## 4 93 The ~ Soon after~ 156 Horr~ 7.5 81 45052
## 5 67 Ghos~ Following ~ 116 Acti~ 5.2 60 198216
## 6 14 Dead~ A wisecrac~ 108 Acti~ 8 65 874655
## 7 8 Zoot~ In a city ~ 108 Anim~ 8 78 416128
## 8 7 Batm~ Fearing th~ 151 Acti~ 6.5 44 607303
## 9 3 Suic~ A secret g~ 123 Acti~ 6 40 580892
## 10 37 The ~ As a math ~ 128 Acti~ 7.3 51 250898
## # ... with 90 more rows, and 3 more variables: Gross_Earning_in_Mil <dbl>,
## # Director <fct>, Actor <fct># step 2: runtime between 100 and 120
runtime_100_120 <- runtime_over100 %>%
filter(Runtime <= 120)
runtime_100_120## Rank Title
## 1 1 Moana
## 2 2 Moonlight
## 3 8 Zootopia
## 4 9 Arrival
## 5 10 The Nice Guys
## 6 12 Doctor Strange
## 7 14 Deadpool
## 8 15 Your Name.
## 9 16 Jack Reacher: Never Go Back
## 10 18 Captain Fantastic
## 11 21 Train to Busan
## 12 22 The Invisible Guest
## 13 24 Split
## 14 25 Me Before You
## 15 26 Assassin's Creed
## 16 27 Nocturnal Animals
## 17 28 The Legend of Tarzan
## 18 34 The Neon Demon
## 19 35 Passengers
## 20 36 Sing
## 21 38 Independence Day: Resurgence
## 22 42 Lion
## 23 43 The Edge of Seventeen
## 24 44 The Girl on the Train
## 25 45 Hell or High Water
## 26 47 Deepwater Horizon
## 27 51 Central Intelligence
## 28 55 Hail, Caesar!
## 29 58 Dirty Grandpa
## 30 61 10 Cloverfield Lane
## 31 64 Hunt for the Wilderpeople
## 32 66 The Jungle Book
## 33 67 Ghostbusters
## 34 68 War Dogs
## 35 69 How to Be Single
## 36 74 Why Him?
## 37 75 Allegiant
## 38 79 Bad Moms
## 39 81 The Founder
## 40 82 Resident Evil: The Final Chapter
## 41 83 Teenage Mutant Ninja Turtles: Out of the Shadows
## 42 84 The Do-Over
## 43 85 The Bad Batch
## 44 87 The 5th Wave
## 45 94 The Girl with All the Gifts
## 46 97 Pride and Prejudice and Zombies
## 47 98 Paterson
## 48 100 Special Correspondents
## Description
## 1 In Ancient Polynesia, when a terrible curse incurred by the Demigod Maui reaches Moana's island, she answers the Ocean's call to seek out the Demigod to set things right.
## 2 A young African-American man grapples with his identity and sexuality while experiencing the everyday struggles of childhood, adolescence, and burgeoning adulthood.
## 3 In a city of anthropomorphic animals, a rookie bunny cop and a cynical con artist fox must work together to uncover a conspiracy.
## 4 A linguist works with the military to communicate with alien lifeforms after twelve mysterious spacecraft appear around the world.
## 5 In 1970s Los Angeles, a mismatched pair of private eyes investigate a missing girl and the mysterious death of a porn star.
## 6 While on a journey of physical and spiritual healing, a brilliant neurosurgeon is drawn into the world of the mystic arts.
## 7 A wisecracking mercenary gets experimented on and becomes immortal but ugly, and sets out to track down the man who ruined his looks.
## 8 Two strangers find themselves linked in a bizarre way. When a connection forms, will distance be the only thing to keep them apart?
## 9 Jack Reacher must uncover the truth behind a major government conspiracy in order to clear his name while on the run as a fugitive from the law.
## 10 In the forests of the Pacific Northwest, a father devoted to raising his six kids with a rigorous physical and intellectual education is forced to leave his paradise and enter the world, challenging his idea of what it means to be a parent.
## 11 While a zombie virus breaks out in South Korea, passengers struggle to survive on the train from Seoul to Busan.
## 12 A successful entrepreneur accused of murder and a witness preparation expert have less than three hours to come up with an impregnable defence.
## 13 Three girls are kidnapped by a man with a diagnosed 23 distinct personalities. They must try to escape before the apparent emergence of a frightful new 24th.
## 14 A girl in a small town forms an unlikely bond with a recently-paralyzed man she's taking care of.
## 15 Callum Lynch explores the memories of his ancestor Aguilar de Nerha and gains the skills of a Master Assassin, before taking on the secret Templar society.
## 16 A wealthy art gallery owner is haunted by her ex-husband's novel, a violent thriller she interprets as a symbolic revenge tale.
## 17 Tarzan, having acclimated to life in London, is called back to his former home in the jungle to investigate the activities at a mining encampment.
## 18 An aspiring model, Jesse, is new to Los Angeles. However, her beauty and youth, which generate intense fascination and jealousy within the fashion industry, may prove themselves sinister.
## 19 A spacecraft traveling to a distant colony planet and transporting thousands of people has a malfunction in its sleep chambers. As a result, two passengers are awakened 90 years early.
## 20 In a city of humanoid animals, a hustling theater impresario's attempt to save his theater with a singing competition becomes grander than he anticipates even as its finalists find that their lives will never be the same.
## 21 Two decades after the first Independence Day invasion, Earth is faced with a new extra-Solar threat. But will mankind's new space defenses be enough?
## 22 A five-year-old Indian boy is adopted by an Australian couple after getting lost hundreds of kilometers from home. 25 years later, he sets out to find his lost family.
## 23 High-school life gets even more unbearable for Nadine when her best friend, Krista, starts dating her older brother.
## 24 A divorcee becomes entangled in a missing persons investigation that promises to send shockwaves throughout her life.
## 25 A divorced father and his ex-con older brother resort to a desperate scheme in order to save their family's ranch in West Texas.
## 26 A dramatization of the disaster in April 2010, when the offshore drilling rig called the Deepwater Horizon exploded, resulting in the worst oil spill in American history.
## 27 After he reconnects with an awkward pal from high school through Facebook, a mild-mannered accountant is lured into the world of international espionage.
## 28 A Hollywood fixer in the 1950s works to keep the studio's stars in line.
## 29 Right before his wedding, an uptight guy is tricked into driving his grandfather, a lecherous former Army Lieutenant Colonel, to Florida for Spring Break.
## 30 After getting in a car accident, a woman is held in a shelter with two men, who claim the outside world is affected by a widespread chemical attack.
## 31 A national manhunt is ordered for a rebellious kid and his foster uncle who go missing in the wild New Zealand bush.
## 32 After a threat from the tiger Shere Khan forces him to flee the jungle, a man-cub named Mowgli embarks on a journey of self discovery with the help of panther Bagheera and free-spirited bear Baloo.
## 33 Following a ghost invasion of Manhattan, paranormal enthusiasts Erin Gilbert and Abby Yates, nuclear engineer Jillian Holtzmann, and subway worker Patty Tolan band together to stop the otherworldly threat.
## 34 Loosely based on the true story of two young men, David Packouz and Efraim Diveroli, who won a three hundred million dollar contract from the Pentagon to arm America's allies in Afghanistan.
## 35 A group of young adults navigate love and relationships in New York City.
## 36 A holiday gathering threatens to go off the rails when Ned Fleming realizes that his daughter's Silicon Valley millionaire boyfriend is about to pop the question.
## 37 After the earth-shattering revelations of Insurgent, Tris must escape with Four beyond the wall that encircles Chicago, to finally discover the shocking truth of the world around them.
## 38 When three overworked and under-appreciated moms are pushed beyond their limits, they ditch their conventional responsibilities for a jolt of long overdue freedom, fun and comedic self-indulgence.
## 39 The story of Ray Kroc, a salesman who turned two brothers' innovative fast food eatery, McDonald's, into the biggest restaurant business in the world, with a combination of ambition, persistence, and ruthlessness.
## 40 Alice returns to where the nightmare began: The Hive in Raccoon City, where the Umbrella Corporation is gathering its forces for a final strike against the only remaining survivors of the apocalypse.
## 41 The Turtles get into another battle with their enemy the Shredder, who has acquired new allies: the mutant thugs Bebop and Rocksteady and the alien being Krang.
## 42 Two down-on-their-luck guys decide to fake their own deaths and start over with new identities, only to find the people they're pretending to be are in even deeper trouble.
## 43 In a desert dystopia, a young woman is kidnapped by cannibals.
## 44 Four waves of increasingly deadly alien attacks have left most of Earth in ruin. Cassie is on the run, desperately trying to save her younger brother.
## 45 A scientist and a teacher living in a dystopian future embark on a journey of survival with a special young girl named Melanie.
## 46 Five sisters in 19th century England must cope with the pressures to marry while protecting themselves from a growing population of zombies.
## 47 A quiet observation of the triumphs and defeats of daily life, along with the poetry evident in its smallest details.
## 48 A radio journalist and his technician get in over their heads when they hatch a scheme to fake their own kidnapping during a rebel uprising in South America and hide out in New York instead.
## Runtime Genre Rating Metascore Votes Gross_Earning_in_Mil
## 1 107 Animation 7.6 81 255125 24876
## 2 111 Drama 7.4 99 258773 2785
## 3 108 Animation 8.0 78 416128 34127
## 4 116 Drama 7.9 81 567996 10055
## 5 116 Action 7.4 70 265589 3626
## 6 115 Action 7.5 72 571548 23264
## 7 108 Action 8.0 65 874655 36307
## 8 106 Animation 8.4 79 171687 502
## 9 118 Action 6.1 47 132601 5870
## 10 118 Comedy 7.9 72 179095 588
## 11 118 Action 7.5 72 142259 213
## 12 106 Crime 8.1 NA 121271 9343
## 13 117 Horror 7.3 62 403409 5625
## 14 106 Drama 7.4 51 196681 5465
## 15 115 Action 5.7 36 179478 1064
## 16 116 Drama 7.5 67 226199 12664
## 17 110 Action 6.2 44 159263 3434
## 18 117 Horror 6.2 51 80128 4770
## 19 116 Drama 7.0 41 336255 133
## 20 108 Animation 7.1 59 127922 10001
## 21 120 Action 5.2 32 162278 8626
## 22 118 Biography 8.0 69 203555 8922
## 23 104 Comedy 7.3 77 98322 NA
## 24 112 Crime 6.5 48 163139 3582
## 25 102 Action 7.6 88 195170 9769
## 26 107 Action 7.1 68 144924 1443
## 27 107 Action 6.3 52 152839 6143
## 28 106 Comedy 6.3 72 120028 3115
## 29 102 Comedy 5.9 21 109391 4737
## 30 103 Drama 7.2 76 280260 858
## 31 101 Adventure 7.9 81 102470 10247
## 32 106 Adventure 7.4 77 252925 710
## 33 116 Action 5.2 60 198216 36400
## 34 114 Biography 7.1 57 169732 12834
## 35 110 Comedy 6.1 51 78857 4303
## 36 111 Comedy 6.2 39 97433 NA
## 37 120 Action 5.7 33 104224 NA
## 38 100 Comedy 6.2 60 109869 6618
## 39 115 Biography 7.2 66 113498 4010
## 40 107 Action 5.5 49 81048 48630
## 41 112 Action 6.0 40 81919 11326
## 42 108 Action 5.7 22 37967 1239
## 43 118 Drama 5.3 62 24774 1279
## 44 112 Action 5.2 33 96588 8205
## 45 111 Drama 6.6 67 52115 2159
## 46 108 Action 5.8 45 49637 1091
## 47 118 Comedy 7.4 90 65597 214
## 48 101 Comedy 5.8 36 19652 NA
## Director Actor
## 1 Ron Clements Auli'i Cravalho
## 2 Barry Jenkins Mahershala Ali
## 3 Byron Howard Ginnifer Goodwin
## 4 Denis Villeneuve Amy Adams
## 5 Shane Black Russell Crowe
## 6 Scott Derrickson Benedict Cumberbatch
## 7 Tim Miller Ryan Reynolds
## 8 Makoto Shinkai Ryûnosuke Kamiki
## 9 Edward Zwick Tom Cruise
## 10 Matt Ross Viggo Mortensen
## 11 Sang-ho Yeon Yoo Gong
## 12 Oriol Paulo Mario Casas
## 13 M. Night Shyamalan James McAvoy
## 14 Thea Sharrock Emilia Clarke
## 15 Justin Kurzel Michael Fassbender
## 16 Tom Ford Amy Adams
## 17 David Yates Alexander Skarsgård
## 18 Nicolas Winding Refn Elle Fanning
## 19 Morten Tyldum Jennifer Lawrence
## 20 Garth Jennings Matthew McConaughey
## 21 Roland Emmerich Liam Hemsworth
## 22 Garth Davis Dev Patel
## 23 Kelly Fremon Craig Hailee Steinfeld
## 24 Tate Taylor Emily Blunt
## 25 David Mackenzie Chris Pine
## 26 Peter Berg Mark Wahlberg
## 27 Rawson Marshall Thurber Dwayne Johnson
## 28 Ethan Coen Josh Brolin
## 29 Dan Mazer Robert De Niro
## 30 Dan Trachtenberg John Goodman
## 31 Taika Waititi Sam Neill
## 32 Jon Favreau Neel Sethi
## 33 Paul Feig Melissa McCarthy
## 34 Todd Phillips Jonah Hill
## 35 Christian Ditter Dakota Johnson
## 36 John Hamburg Zoey Deutch
## 37 Robert Schwentke Shailene Woodley
## 38 Jon Lucas Mila Kunis
## 39 John Lee Hancock Michael Keaton
## 40 Paul W.S. Anderson Milla Jovovich
## 41 Dave Green Megan Fox
## 42 Steven Brill Adam Sandler
## 43 Ana Lily Amirpour Suki Waterhouse
## 44 J Blakeson Chloë Grace Moretz
## 45 Colm McCarthy Sennia Nanua
## 46 Burr Steers Lily James
## 47 Jim Jarmusch Adam Driver
## 48 Ricky Gervais Ricky Gervais# Step 3: separating genre and highest average gross earning
genre_gross_100_120 <- runtime_100_120 %>%
group_by(Genre) %>%
head(5)
genre_gross_100_120## # A tibble: 5 x 11
## # Groups: Genre [3]
## Rank Title Description Runtime Genre Rating Metascore Votes Gross_Earning_i~
## <dbl> <chr> <chr> <dbl> <fct> <dbl> <dbl> <dbl> <dbl>
## 1 1 Moana In Ancient~ 107 Anim~ 7.6 81 255125 24876
## 2 2 Moon~ A young Af~ 111 Drama 7.4 99 258773 2785
## 3 8 Zoot~ In a city ~ 108 Anim~ 8 78 416128 34127
## 4 9 Arri~ A linguist~ 116 Drama 7.9 81 567996 10055
## 5 10 The ~ In 1970s L~ 116 Acti~ 7.4 70 265589 3626
## # ... with 2 more variables: Director <fct>, Actor <fct>movies_df %>%
rownames_to_column(var = "Name") %>%
filter(Runtime == c(100,120)) %>%
group_by(Genre) %>%
summarize(averageGross = mean(Gross_Earning_in_Mil)) %>%
filter(averageGross == max(averageGross))## `summarise()` ungrouping output (override with `.groups` argument)## # A tibble: 1 x 2
## Genre averageGross
## <fct> <dbl>
## 1 Action 8626# Question here is is this similar to my query in question 2 - does this answer only give us the genre of the single most grossing film, but not give us the genre that when all movies in each genre are taken together has an average highest gross?Answer Based on the above data, across all genres the genre that has the highest average gross earnings in runtime 100 to 120 is Action (given lines 781 - 788, but might be comedy given p3).