Setup

Load the packages used throughout this assignment.

library(rvest)
library(robotstxt)
library(tidyverse)

Question 5: Is scraping allowed?

I used the robotstxt package to check whether scraping the Wikipedia page is permitted by the robots exclusion protocol.

url_2026 <- "https://en.wikipedia.org/wiki/2026_in_film"

allowed <- paths_allowed(url_2026)
allowed
## [1] TRUE

Answer: Scraping this page is allowed: TRUE.

Question 6: Scrape the “Highest-grossing films” table

I read the page, extracted every table with the wikitable class, parsed them into tibbles, and then picked the one with a “gross” column.

page <- read_html(url_2026)

tables <- page |>
  html_elements("table.wikitable") |>
  html_table()

# Find the table that has a "gross" column
idx <- which(map_lgl(tables, ~ any(str_detect(names(.x), regex("gross", ignore_case = TRUE)))))

hg_2026 <- tables[[idx[1]]]
hg_2026
## # A tibble: 10 × 4
##     Rank Title                        Distributor           `Worldwide gross`
##    <int> <chr>                        <chr>                 <chr>            
##  1     1 Spider-Man: Brand New Day †  Sony                  $2,523,147,515   
##  2     2 The Odyssey                  Universal             $1,804,356,298   
##  3     3 Toy Story 5                  Disney                $1,149,474,196   
##  4     4 Michael                      Lionsgate / Universal $1,028,703,388   
##  5     5 The Super Mario Galaxy Movie Universal             $1,013,278,851   
##  6     6 The Devil Wears Prada 2      20th Century          $693,492,689     
##  7     7 Project Hail Mary            Amazon MGM / Sony     $685,040,805     
##  8     8 Pegasus 3                    Maoyan                $656,459,523     
##  9     9 Minions & Monsters           Universal             $530,319,698     
## 10    10 Obsession                    Focus / Universal     $524,535,537

I obtained a structured table with 10 rows and 4 columns, so the answer is True.

Question 8 (Bonus): Highest-grossing films, 2020 to 2025

I wrote a function that scrapes the highest-grossing films table for a given year, then used map_dfr() to apply it to each year from 2020 to 2025 and combine the results into one data frame with a year column.

get_hg <- function(year) {
  Sys.sleep(1)  # be polite to the server
  url <- paste0("https://en.wikipedia.org/wiki/", year, "_in_film")

  tbls <- read_html(url) |>
    html_elements("table.wikitable") |>
    html_table()

  idx <- which(map_lgl(tbls, ~ any(str_detect(names(.x), regex("gross", ignore_case = TRUE)))))

  tbls[[idx[1]]] |>
    mutate(across(everything(), as.character)) |>
    mutate(year = year)
}

hg_films <- map_dfr(2020:2025, get_hg)

Quick check: each year should contribute 10 rows.

table(hg_films$year)
## 
## 2020 2021 2022 2023 2024 2025 
##   10   10   10   10   10   10

The combined data frame:

print(hg_films, n = Inf)
## # A tibble: 60 × 5
##    Rank  Title                               Distributor `Worldwide gross`  year
##    <chr> <chr>                               <chr>       <chr>             <int>
##  1 1     Demon Slayer: Kimetsu no Yaiba Mug… Toho / Ani… $507,127,293[4]    2020
##  2 2     The Eight Hundred                   CMC Pictur… $461,421,559       2020
##  3 3     My People, My Homeland              China Lion  $433,241,288[5]    2020
##  4 4     Bad Boys for Life                   Sony        $426,505,244       2020
##  5 5     Tenet                               Warner Bro… $365,309,519       2020
##  6 6     Sonic the Hedgehog                  Paramount   $319,715,683       2020
##  7 7     Dolittle                            Universal   $251,410,631       2020
##  8 8     Jiang Ziya                          Beijing En… $243,883,429       2020
##  9 9     A Little Red Flower                 HG Enterta… $238,600,000[6][…  2020
## 10 10    Shock Wave 2                        Universe F… $226,400,000       2020
## 11 1     Spider-Man: No Way Home             Sony        $1,921,426,073[n…  2021
## 12 2     The Battle at Lake Changjin         Bona        $909,596,236       2021
## 13 3     Hi, Mom                             Lian Ray    $841,674,419       2021
## 14 4     No Time to Die                      MGM / Univ… $774,253,007       2021
## 15 5     F9                                  Universal   $726,229,501       2021
## 16 6     Detective Chinatown 3               Warner Bro… $686,257,563       2021
## 17 7     Venom: Let There Be Carnage         Sony        $506,863,592       2021
## 18 8     Godzilla vs. Kong                   Warner Bro… $470,116,094       2021
## 19 9     Shang-Chi and the Legend of the Te… Disney      $432,243,292       2021
## 20 10    Dune                                Warner Bro… $411,000,000       2021
## 21 1     Avatar: The Way of Water            20th Centu… $2,320,250,281[n…  2022
## 22 2     Top Gun: Maverick                   Paramount   $1,495,696,292[n…  2022
## 23 3     Jurassic World Dominion             Universal   $1,001,978,080     2022
## 24 4     Doctor Strange in the Multiverse o… Disney      $955,775,804       2022
## 25 5     Minions: The Rise of Gru            Universal   $940,628,210       2022
## 26 6     Black Panther: Wakanda Forever      Disney      $859,208,836       2022
## 27 7     The Batman                          Warner Bro… $772,245,583       2022
## 28 8     Thor: Love and Thunder              Disney      $760,928,081       2022
## 29 9     The Battle at Lake Changjin II      Bona        $626,571,697       2022
## 30 10    Puss in Boots: The Last Wish        Universal   $485,261,639       2022
## 31 1     Barbie                              Warner Bro… $1,448,232,531     2023
## 32 2     The Super Mario Bros. Movie         Universal   $1,360,847,665     2023
## 33 3     Oppenheimer                         Universal   $975,811,333       2023
## 34 4     Guardians of the Galaxy Vol. 3      Disney      $845,555,777       2023
## 35 5     Fast X                              Universal   $714,875,015       2023
## 36 6     Spider-Man: Across the Spider-Verse Sony        $690,824,738       2023
## 37 7     Full River Red                      China Film… $670,491,959       2023
## 38 8     Wonka                               Warner Bro… $634,681,815       2023
## 39 9     The Wandering Earth 2               China Film… $615,023,132       2023
## 40 10    Mission: Impossible – Dead Reckoni… Paramount   $571,125,435       2023
## 41 1     Inside Out 2                        Disney      $1,698,863,816     2024
## 42 2     Deadpool & Wolverine                Disney      $1,338,073,645     2024
## 43 3     Moana 2                             Disney      $1,059,242,164     2024
## 44 4     Despicable Me 4                     Universal   $972,021,410       2024
## 45 5     Wicked                              Universal   $758,729,195       2024
## 46 6     Mufasa: The Lion King               Disney      $723,060,982       2024
## 47 7     Dune: Part Two                      Warner Bro… $715,409,065       2024
## 48 8     Godzilla x Kong: The New Empire     Warner Bro… $572,505,338       2024
## 49 9     Kung Fu Panda 4                     Universal   $547,689,492       2024
## 50 10    Sonic the Hedgehog 3                Paramount   $492,162,604       2024
## 51 1     Ne Zha 2                            Beijing En… $2,215,690,000[3…  2025
## 52 2     Zootopia 2                          Disney      $1,870,309,291     2025
## 53 3     Avatar: Fire and Ash                20th Centu… $1,490,386,712     2025
## 54 4     Lilo & Stitch                       Disney      $1,038,027,526     2025
## 55 5     A Minecraft Movie                   Warner Bro… $961,287,780       2025
## 56 6     Jurassic World Rebirth              Universal   $872,428,220       2025
## 57 7     Demon Slayer: Kimetsu no Yaiba – T… Aniplex / … $793,594,078       2025
## 58 8     How to Train Your Dragon            Universal   $639,866,451       2025
## 59 9     F1                                  Warner Bro… $634,142,436       2025
## 60 10    Superman                            Warner Bro… $618,723,803       2025

I completed this task successfully, so the answer is True.