Assignment 8

Author

Sam Barbaro

Assignment 8

Pulling one bestseller list from 3/4, I noticed that 4 books on the mass market paperback bestsellers contain the word “Amish.” These are by all different authors, so it’s not a series. This suggests a popularity for Amish romances that I’d like to investigate further. I would like to use the books API find weekly the count of books on the mass market paperback bestsellers list that have the word “Amish” in them for the current year and what percentage of the MMPB bestsellers list they compose. I would also like to perform a search for articles that discuss this phenomenon using the articles API.

Pulling the bestseller list from 3/4

I test the API by pulling the bestseller list from March 3.

library(httr2)
Warning: package 'httr2' was built under R version 4.5.2
library(jsonlite)
library(tidyverse)
Warning: package 'tidyverse' was built under R version 4.5.2
Warning: package 'ggplot2' was built under R version 4.5.2
Warning: package 'tibble' was built under R version 4.5.2
Warning: package 'readr' was built under R version 4.5.2
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.4     ✔ readr     2.1.6
✔ forcats   1.0.1     ✔ stringr   1.5.2
✔ ggplot2   4.0.1     ✔ tibble    3.3.1
✔ lubridate 1.9.4     ✔ tidyr     1.3.1
✔ purrr     1.1.0     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter()  masks stats::filter()
✖ purrr::flatten() masks jsonlite::flatten()
✖ dplyr::lag()     masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(httr2)
library(purrr)
library(lubridate)

#My API keys are stored elsewhere in the R environment
api_key <- Sys.getenv("NYT_API_KEY1")


resp <- request("https://api.nytimes.com/svc/books/v3/lists/overview.json") |>
  req_url_query(
    `api-key` = api_key,
    published_date = "2026-03-04"
  ) |>
  req_perform()

# Parse
raw_data <- resp |> resp_body_json()

#flatten and format in a tibble
books_list <- raw_data$results$lists |> 
    map_df(function(lst) {
        lst$books |> 
            map_df(function(book) {
                tibble(
                    list_name = lst$display_name,
                    rank = book$rank,
                    title = book$title,
                    author = book$author,
                    description = book$description
                )
            })
    })

I used Gemini to figure out how to parse and flatten this data and then mixed and matched the variables. map_df unests the data frame. The Books API sends back several lists (e.g. mass market, hardcover fiction), with tons of information (ISBNs , links to buy at various booksellers, rank last week). This selects a few important details (list name, rank, title, author, and description), and places them in a tibble.

Pulling multiple bestseller lists

Trying to pull all the bestseller lists for an entire year, of course, resulted in a throttle. Here are three for this year. There’s a built-in 12-second delay before each query.

# Dates for bestseller lists
comparison_dates <- c("2026-01-04", "2026-02-01", "2026-03-01")


get_monthly_overview <- function(target_date) {
  Sys.sleep(12) # Prevent "Too Many Requests" errors (12 seconds between queries)
  
  resp <- request("https://api.nytimes.com/svc/books/v3/lists/overview.json") |>
    req_url_query(
      `api-key` = api_key,
      published_date = target_date
    ) |>
    req_perform()
  
  raw_data <- resp |> resp_body_json()
  
  # Flatten logic
  raw_data$results$lists |> 
    map_df(function(lst) {
      map_df(lst$books, ~ tibble(
        published_date = target_date,
        list_name = lst$display_name,
        rank = .x$rank,
        title = .x$title,
        author = .x$author,
        description = .x$description
      ))
    })
}


books_comparison <- map_df(comparison_dates, get_monthly_overview)

# View results
print(books_comparison)
# A tibble: 690 × 6
   published_date list_name                        rank title author description
   <chr>          <chr>                           <int> <chr> <chr>  <chr>      
 1 2026-01-04     Combined Print & E-Book Fiction     1 THE … John … When Simon…
 2 2026-01-04     Combined Print & E-Book Fiction     2 THE … Freid… Troubles s…
 3 2026-01-04     Combined Print & E-Book Fiction     3 THE … Virgi… Letters fr…
 4 2026-01-04     Combined Print & E-Book Fiction     4 THE … Dan B… As he sear…
 5 2026-01-04     Combined Print & E-Book Fiction     5 THEO… Allen… A man trav…
 6 2026-01-04     Combined Print & E-Book Fiction     6 THE … Liz M… When a 13-…
 7 2026-01-04     Combined Print & E-Book Fiction     7 BRIM… Calli… The second…
 8 2026-01-04     Combined Print & E-Book Fiction     8 PROJ… Andy … Ryland Gra…
 9 2026-01-04     Combined Print & E-Book Fiction     9 MONA… Thoma… A 10-year-…
10 2026-01-04     Combined Print & E-Book Fiction    10 ALCH… SenLi… After the …
# ℹ 680 more rows

Filter for books about Amish romance

books_filtered <- books_comparison |>
  filter(grepl("AMISH", title, ignore.case = TRUE))
books_filtered
# A tibble: 6 × 6
  published_date list_name    rank title                      author description
  <chr>          <chr>       <int> <chr>                      <chr>  <chr>      
1 2026-02-01     Mass Market     8 HER AMISH CHRISTMAS WISH   Jocel… A woman, w…
2 2026-02-01     Mass Market     9 AN AMISH BOOKSHOP COURTSH… Patri… The second…
3 2026-03-01     Mass Market     3 AN AMISH WIDOW&#39;S HOPE  Pamel… When a wid…
4 2026-03-01     Mass Market     4 HER AMISH WINTER MATCH     Rebec… Pain from …
5 2026-03-01     Mass Market     5 FALLING FOR THE AMISH RIV… Virgi… The second…
6 2026-03-01     Mass Market    13 THEIR AMISH COURTSHIP SEC… Jo An… The third …
#there are 6, all mass market paperbacks

#let's check the description, too
books_description <- books_comparison |>
  filter(grepl("AMISH", description, ignore.case = TRUE))

What percentage of the MMPB bestsellers list do these books compose?

count(books_filtered) / (count(books_comparison |> filter(list_name == "Mass Market"))
  )
          n
1 0.1333333
#Amish romances account for ~13.3% of mass market bestsellers YTD in 2026.
books_description
# A tibble: 4 × 6
  published_date list_name    rank title                      author description
  <chr>          <chr>       <int> <chr>                      <chr>  <chr>      
1 2026-02-01     Mass Market     9 AN AMISH BOOKSHOP COURTSH… Patri… The second…
2 2026-03-01     Mass Market     3 AN AMISH WIDOW&#39;S HOPE  Pamel… When a wid…
3 2026-03-01     Mass Market     4 HER AMISH WINTER MATCH     Rebec… Pain from …
4 2026-03-01     Mass Market    13 THEIR AMISH COURTSHIP SEC… Jo An… The third …
#only 4 books include the word "Amish" in their description

#let's see how many times the Rachel Reid (Heated Rivalry) books show up
heated_rivalry <- books_comparison |> 
  filter(grepl("Rachel Reid", author, ignore.case = TRUE))

heated_rivalry
# A tibble: 14 × 6
   published_date list_name                        rank title author description
   <chr>          <chr>                           <int> <chr> <chr>  <chr>      
 1 2026-01-04     Combined Print & E-Book Fiction    12 HEAT… Rache… The second…
 2 2026-02-01     Combined Print & E-Book Fiction     2 HEAT… Rache… The second…
 3 2026-02-01     Combined Print & E-Book Fiction     7 THE … Rache… The sixth …
 4 2026-02-01     Combined Print & E-Book Fiction     9 GAME… Rache… The captai…
 5 2026-02-01     Paperback Trade Fiction             3 HEAT… Rache… The second…
 6 2026-02-01     Paperback Trade Fiction            10 GAME… Rache… The captai…
 7 2026-02-01     Paperback Trade Fiction            11 THE … Rache… The sixth …
 8 2026-02-01     Audio Fiction                       8 HEAT… Rache… The second…
 9 2026-03-01     Combined Print & E-Book Fiction     6 HEAT… Rache… The second…
10 2026-03-01     Combined Print & E-Book Fiction    13 THE … Rache… The sixth …
11 2026-03-01     Paperback Trade Fiction             3 HEAT… Rache… The second…
12 2026-03-01     Paperback Trade Fiction             9 THE … Rache… The sixth …
13 2026-03-01     Paperback Trade Fiction            11 GAME… Rache… The captai…
14 2026-03-01     Audio Fiction                      14 HEAT… Rache… The second…
#which book appears the most often

book_frequency <- books_comparison |>
    count(title) |>
  arrange(desc(n)) |>
  slice_head(n = 50)

book_frequency
# A tibble: 50 × 2
   title                               n
   <chr>                           <int>
 1 1929                               12
 2 HOW TO TEST NEGATIVE FOR STUPID     9
 3 NOBODY'S GIRL                       9
 4 PROJECT HAIL MARY                   9
 5 THE ANXIOUS GENERATION              9
 6 THE BODY KEEPS THE SCORE            9
 7 THE CORRESPONDENT                   9
 8 DUNGEON CRAWLER CARL                8
 9 THE WIDOW                           8
10 THEO OF GOLDEN                      8
# ℹ 40 more rows
#which author appears the most often?
author_frequency <- books_comparison |>
    count(author) |>
  arrange(desc(n)) |>
  slice_head(n = 50)

author_frequency
# A tibble: 50 × 2
   author                  n
   <chr>               <int>
 1 Freida McFadden        24
 2 Dav Pilkey             20
 3 Matt Dinniman          17
 4 Rachel Reid            14
 5 Andrew Ross Sorkin     12
 6 Lynn Painter           11
 7 Callie Hart            10
 8 Matthew McConaughey    10
 9 Andy Weir               9
10 Bessel van der Kolk     9
# ℹ 40 more rows
#who has more than one book on the list (not just one book that appears a bunch
#of times)?

unique_books <- books_comparison |>
  distinct(title, author)

popular_authors <- unique_books |>
  count(author) |>
  arrange(desc(n)) |>
  filter(n > 1)

popular_authors
# A tibble: 30 × 2
   author                                      n
   <chr>                                   <int>
 1 Dav Pilkey                                  8
 2 Freida McFadden                             7
 3 William W. Johnstone and J.A. Johnstone     5
 4 Adam Wallace                                4
 5 Matt Dinniman                               4
 6 Erin Hunter                                 3
 7 J.K. Rowling                                3
 8 Lynn Painter                                3
 9 Rachel Reid                                 3
10 Tui T. Sutherland                           3
# ℹ 20 more rows

Searching for articles about Amish romance

api_key_2 <- Sys.getenv("NYT_API_KEY2")
query_2 <- "Amish romance"

resp <- request("https://api.nytimes.com/svc/search/v2/articlesearch.json") |>
  req_url_query(q = query_2, `api-key` = api_key_2) |>
  req_perform()

data <- resp |> 
  resp_body_json()

headlines <- sapply(data$response$docs, function(x) x$headline$main)
print(headlines)
 [1] "8 New Books We Love This Week"                                               
 [2] "When Did Chicken Become the Most Expensive Thing on the Menu?"               
 [3] "Home-Schooled Kids Are Not All Right"                                        
 [4] "Coqodaq, From the Cote Team, Is a Temple of Fried Chicken"                   
 [5] "Taylor and Travis Might Save Romance, but Posh and Becks Were Here All Along"
 [6] "How Does a Baby Bust End?"                                                   
 [7] "Kate Baer Is Speaking Truth. From Her Minivan."                              
 [8] "The 10 Funniest Movies on Netflix"                                           
 [9] "How This 71-Year-Old Video Art Pioneer Became a TikTok Star"                 
[10] "What to Remember Before You Watch ‘Orange Is the New Black’ Season 6"        

Only one of these articles is likely relevant (8 New Books…). The article about Kate Baer talks about how she read Amish Romances. Let’s try just looking for “Amish” and “Amish books.”

Sys.sleep(20) 

query_3 <- "Amish"
query_4 <- "Amish books"

resp_2 <- request("https://api.nytimes.com/svc/search/v2/articlesearch.json") |>
  req_url_query(q = query_3, `api-key` = api_key_2) |>
  req_perform()

data_2 <- resp_2 |> 
  resp_body_json()

headlines_2 <- sapply(data_2$response$docs, function(x) x$headline$main)
print(headlines_2)
 [1] "Sue Bender, Who Wrote About Living With the Amish, Dies at 91"                          
 [2] "Woman Who Tried to Let Her Twin Take Blame for Fatal Buggy Crash Gets 4 Years in Prison"
 [3] "An Amish Avatar and an A.I. Monk Are Pitching Supplements on Social Media"              
 [4] "Supreme Court Questions N.Y. Ban on Religious Exemptions for Vaccines"                  
 [5] "A Family of 7 Looked for a Wreck to Restore in Pennsylvania Dutch Country"              
 [6] "‘Spiritual Delusion’ Led to Father and Son’s Apparent Drownings, Sheriff Says"          
 [7] "Chasing the Perfect Photograph in Amish Country"                                        
 [8] "A Killing Spree in Utah Rattles a Region and Puzzles Officials"                         
 [9] "Stephen Colbert Sounds the Alarm on ‘Death by Florida’"                                 
[10] "No, Young Men Are Not Returning to Church"                                              
#searching for "Amish books"
Sys.sleep(30) 

resp_3 <- request("https://api.nytimes.com/svc/search/v2/articlesearch.json") |>
  req_url_query(q = query_4, `api-key` = api_key_2) |>
  req_perform()

data_3 <- resp_3 |> 
  resp_body_json()

headlines_3 <- sapply(data_3$response$docs, function(x) x$headline$main)
print(headlines_3)
 [1] "Supreme Court Questions N.Y. Ban on Religious Exemptions for Vaccines"         
 [2] "8 New Books We Love This Week"                                                 
 [3] "No, Young Men Are Not Returning to Church"                                     
 [4] "The Moral Cost of Our Spectacles of Violence"                                  
 [5] "‘War Is Too Serious to Take Seriously All the Time,’ So He Wrote a Comic Novel"
 [6] "A Writer With a Divine Touch Captures Life in a Christian Commune"             
 [7] "22 Books Coming in August"                                                     
 [8] "Autism Has Always Existed. We Haven’t Always Called It Autism."                
 [9] "George Tice, ‘Bard of New Jersey’ With a Camera, Dies at 86"                   
[10] "Home-Schooled Kids Are Not All Right"                                          

At this point it became clear that there was a limit of 10 results. These queries request pages 2 and 3.

Sys.sleep(20) 

resp_page_2 <- request("https://api.nytimes.com/svc/search/v2/articlesearch.json") |>
  req_url_query(
    q = query_4, 
    `api-key` = api_key_2,
    page = 1  
  ) |>
  req_perform()

data_4 <- resp_page_2 |> 
  resp_body_json()

headlines_4 <- sapply(data_4$response$docs, function(x) x$headline$main)
print(headlines_4)
 [1] "What the Look of Your Favorite Podcast Is Trying to Tell You"           
 [2] "Trump Takes a Red Pen to History"                                       
 [3] "They Barter and Trade to Survive. How Will They Vote?"                  
 [4] "Yiddish Is a Supposedly Dying Language That’s Thrillingly Alive"        
 [5] "‘Pod Save America’ Won’t Quit"                                          
 [6] "Is the Internet the Enemy of Progress?"                                 
 [7] "A Lakeside Restaurant Reopens in Paris’s Bois de Boulogne"              
 [8] "Few Smartphones, Some Beer: A Christian Village Grapples With Modernity"
 [9] "Coqodaq, From the Cote Team, Is a Temple of Fried Chicken"              
[10] "When a Breakup Is an Act of Love"                                       
#let's look at Amish Romance again
Sys.sleep(20) 

resp_page_2 <- request("https://api.nytimes.com/svc/search/v2/articlesearch.json") |>
  req_url_query(
    q = query_2, 
    `api-key` = api_key_2,
    page = 1  
  ) |>
  req_perform()

data_4 <- resp_page_2 |> 
  resp_body_json()

headlines_5 <- sapply(data_4$response$docs, function(x) x$headline$main)
print(headlines_5)
 [1] "A Crime Writer Finds the ‘Perfect Setting’ in Amish Country"        
 [2] "Running With the Herd"                                              
 [3] "In ‘Love Finds You in Charm,’ an Amish Woman Ponders a Bigger World"
 [4] "A Farm Family’s Century: Crops, Weather, War "                      
 [5] "Drawing Ire, Not Artists"                                           
 [6] "Nationwide Theater Listings"                                        
 [7] "Biloxi Bound"                                                       
 [8] "Paperback Highlights"                                               
 [9] "Paperback Highlights"                                               
[10] "No Cyclone or Hot Dogs, but Still the Boardwalk"                    
Sys.sleep(20) 
#page 3
resp_page_3 <- request("https://api.nytimes.com/svc/search/v2/articlesearch.json") |>
  req_url_query(
    q = query_2, 
    `api-key` = api_key_2,
    page = 2  
  ) |>
  req_perform()

data_5 <- resp_page_3 |> 
  resp_body_json()

headlines_6 <- sapply(data_5$response$docs, function(x) x$headline$main)
print(headlines_6)
 [1] "WEEKEND EXCURSION; Steaming Through a World That Time Left Behind"         
 [2] "Theater Listings: June 19 — 25"                                            
 [3] "ON THE TOWNS"                                                              
 [4] "Children, Crooks and Capers"                                               
 [5] "Garages for Chardonnays, Not Camrys"                                       
 [6] "Taking  the high road to romance - Style - International Herald Tribune"   
 [7] "Easygoing Tailoring and Tweeds in Prefall Collections"                     
 [8] "Carriage Horse Is Electrocuted on Street"                                  
 [9] "CABLE TV NOTES; FROM MANY SOURCES, CELEBRATIONS OF THE COUNTRY'S DIVERSITY"
[10] "New York fashion week gets the red carpet blues"                           

Here we see a little more evidence of these books – paperback highlights, a crime writer, “Love Finds You in Charm.”

Google Gemini. (2026). Gemini 3 Flash [Large language model].
https://gemini.google.com. Accessed March 24-27, 2026.