For this assignment, I plan to use the New York Times Books API to explore bestseller rankings and investigate which authors appear most frequently across different bestseller lists. My main research question is: Which authors have the most appearances across the New York Times Best Sellers lists, and how are books distributed across the different list categories? I chose this topic because I am interested in examining patterns in book popularity and whether certain authors have a stronger presence across multiple categories or formats.
I will use the /lists/overview.json endpoint, which provides information about bestseller lists, including book titles, authors, publishers, rankings, and the number of weeks each book has remained on a list. I plan to retrieve the latest available bestseller data rather than focus on a specific historical date. To access the API, I will register an application through the NYT Developer Portal and store my API key as an environment variable rather than including it directly in my code.
In R, I will use the httr package to make the API request and jsonlite to parse the JSON response. Since the data is organized into nested lists containing individual books, I anticipate needing to restructure it into a tidy data frame using purrr, tibble, and dplyr. I will select the variables relevant to my analysis, check for missing values, and ensure that numerical fields are stored in the appropriate formats.
Once the data is cleaned, I plan to identify the ten authors with the most bestseller-list appearances and compare the number of book entries across categories. I will also consider how books appearing on multiple lists might affect these counts. My overall goal is to practice retrieving and transforming data from a public API while using the resulting dataset to answer a meaningful research question.
API: New York Times Books API
Endpoint: https://api.nytimes.com/svc/books/v3/lists/overview.json
This endpoint provides an overview of the New York Times bestseller lists for a given publication date.
Request parameters:
api-key: Authenticates the request using an environment variable.library(httr)
library(jsonlite)
library(dplyr)
Attaching package: 'dplyr'The following objects are masked from 'package:stats':
filter, lagThe following objects are masked from 'package:base':
intersect, setdiff, setequal, unionlibrary(tidyr)
library(purrr)
Attaching package: 'purrr'The following object is masked from 'package:jsonlite':
flattenlibrary(tibble)I used httr to make the API request, jsonlite to parse the JSON response, and dplyr, tidyr, purrr, and tibble to organize and analyze the resulting data.
api_key <- Sys.getenv("NYT_API_KEY")
if (!nzchar(api_key)) {
stop("NYT_API_KEY is not set.")
}
url <- "https://api.nytimes.com/svc/books/v3/lists/overview.json"
response <- GET(
url,
query = list(`api-key` = api_key)
)
stop_for_status(response)
nyt_data <- fromJSON(
content(response, as = "text", encoding = "UTF-8"),
simplifyVector = FALSE
)
nyt_data$status[1] "OK"The API returns nested JSON, with multiple bestseller lists containing individual books. I used map_dfr() to extract the book information and combine it into one data frame.
books_df <- map_dfr(nyt_data$results$lists, function(lst) {
map_dfr(lst$books, function(book) {
tibble(
list_name = lst$list_name,
rank = book$rank,
title = book$title,
author = book$author,
publisher = book$publisher,
weeks_on_list = book$weeks_on_list
)
})
})
books_df <- books_df %>%
mutate(
rank = as.integer(rank),
weeks_on_list = as.integer(weeks_on_list)
) %>%
distinct()
head(books_df, 10)# A tibble: 10 × 6
list_name rank title author publisher weeks_on_list
<chr> <int> <chr> <chr> <chr> <int>
1 Combined Print & E-Book Fiction 1 THRESHI… Rebec… Red Tower 3
2 Combined Print & E-Book Fiction 2 THE WIT… Freid… Poisoned… 2
3 Combined Print & E-Book Fiction 3 THE FRE… John … Doubleday 3
4 Combined Print & E-Book Fiction 4 THEO OF… Allen… Atria 46
5 Combined Print & E-Book Fiction 5 THE CAL… Kathr… Spiegel … 24
6 Combined Print & E-Book Fiction 6 PECK & … Bonni… Scribner 1
7 Combined Print & E-Book Fiction 7 DUNGEON… Matt … Ace 41
8 Combined Print & E-Book Fiction 8 VERITY Colle… Grand Ce… 105
9 Combined Print & E-Book Fiction 9 PARTITA Barba… Harper 2
10 Combined Print & E-Book Fiction 10 HOLLOW … Jodi … Ballanti… 5dim(books_df)[1] 240 6str(books_df)tibble [240 × 6] (S3: tbl_df/tbl/data.frame)
$ list_name : chr [1:240] "Combined Print & E-Book Fiction" "Combined Print & E-Book Fiction" "Combined Print & E-Book Fiction" "Combined Print & E-Book Fiction" ...
$ rank : int [1:240] 1 2 3 4 5 6 7 8 9 10 ...
$ title : chr [1:240] "THRESHING DAY" "THE WITCH" "THE FRENCH ILLUSION" "THEO OF GOLDEN" ...
$ author : chr [1:240] "Rebecca Yarros" "Freida McFadden" "John Grisham" "Allen Levi" ...
$ publisher : chr [1:240] "Red Tower" "Poisoned Pen" "Doubleday" "Atria" ...
$ weeks_on_list: int [1:240] 3 2 3 46 24 1 41 105 2 5 ...The original JSON response contains nested bestseller lists, each with its own collection of books. I flattened this structure so that each row represents one book appearing on one bestseller list.
I selected the variables most relevant to my research question: list name, rank, title, author, publisher, and weeks on the list.
I also converted ranking and duration fields into integers and removed exact duplicate rows. Missing values, if present, were retained rather than replaced with potentially misleading values.
books_by_list <- books_df %>%
count(list_name, sort = TRUE, name = "book_count")
books_by_list# A tibble: 19 × 2
list_name book_count
<chr> <int>
1 Audio Children's 15
2 Audio Fiction 15
3 Audio Nonfiction 15
4 Combined Print & E-Book Fiction 15
5 Combined Print & E-Book Nonfiction 15
6 Graphic Books and Manga 15
7 Hardcover Fiction 15
8 Hardcover Nonfiction 15
9 Paperback Nonfiction 15
10 Paperback Trade Fiction 15
11 Advice, How-To & Miscellaneous 10
12 Audio Advice, How-To & Misc 10
13 Business 10
14 Children’s & Young Adult Series 10
15 Children’s Middle Grade Hardcover 10
16 Children’s Picture Books 10
17 Middle Grade Paperback 10
18 Young Adult Hardcover 10
19 Young Adult Paperback 10This table summarizes the number of book entries returned for each bestseller list.
The New York Times Books API request was successful, returning bestseller information across 19 lists and 240 book entries. I was able to extract relevant information from the nested JSON response and organize it into a tidy data frame containing book titles, authors, publishers, rankings, and weeks on the bestseller list.
To answer my research question, I counted how frequently each author appeared across the retrieved lists. Matt Dinniman had the highest number of appearances with 14, followed by J.K. Rowling with 9 and Suzanne Collins with 6. These results suggest that certain authors have a particularly strong presence across multiple bestseller categories.
I also compared the number of entries in each bestseller list. The lists contained either 10 or 15 entries, depending on the category. However, these differences reflect the number of rankings returned by the API rather than a direct measure of reader popularity.
One limitation is that the same book may appear across multiple lists or formats. Therefore, the author counts represent bestseller-list appearances rather than unique books or total sales.