I used the New York Times Books API to compare two Hardcover Fiction best-seller lists, 52 weeks apart. I wanted to know whether the latest list had more books that had stayed on the list for several weeks, or more recent entries.
The code requests both lists, saves the JSON responses, and turns the book records into a table. I compare the median weeks on the list, count recent and longer-running entries, and look at the top-ranked books. The saved responses let me render the report again without repeating the requests.
GET https://api.nytimes.com/svc/books/v3/lists/{date}/{list}.json| Parameter | Value | Where it goes |
|---|---|---|
date |
current, then the same week one year earlier (YYYY-MM-DD) |
in the URL path |
list |
hardcover-fiction |
in the URL path |
api-key |
my key, read from the NYT_API_KEY environment variable |
in the query string |
I use the list date returned by the API and subtract 364 days, or 52 weeks, for the earlier comparison. This keeps the weekday the same, although the calendar date may differ. The publication date of a list is also different from the sales week it covers.
The key is stored in .Renviron as NYT_API_KEY and read with Sys.getenv(). It is not included in this document. The .gitignore file excludes .Renviron from version control.
library(dplyr)
library(tidyr)
library(readr)
library(stringr)
library(ggplot2)
library(httr) # GET(), status_code(), content()
library(jsonlite) # fromJSON()
api_key <- Sys.getenv("NYT_API_KEY")
has_key <- nzchar(api_key)
has_key[1] TRUEhas_key shows whether R found the key without displaying it. Once both JSON files are saved, the report can run without the key.
cache_path <- function(date) {
file.path("data", paste0("nyt_hardcover-fiction_", date, ".json"))
}
validate_list <- function(txt, expected_date = NULL) {
parsed <- tryCatch(fromJSON(txt), error = function(e) {
stop("The response is not valid JSON.", call. = FALSE)
})
result <- parsed$results
if (!identical(parsed$status, "OK") || !is.list(result) ||
!identical(result$list_name, "Hardcover Fiction") ||
!is.data.frame(result$books) || nrow(result$books) == 0) {
stop("Expected a nonempty Hardcover Fiction response.", call. = FALSE)
}
published <- result$published_date
if (length(published) != 1L || is.na(published) ||
!grepl("^[0-9]{4}-[0-9]{2}-[0-9]{2}$", published) ||
is.na(as.Date(published))) {
stop("The response has no valid publication date.", call. = FALSE)
}
if (!is.null(expected_date) && published != expected_date) {
stop("The returned publication date does not match the requested week.",
call. = FALSE)
}
parsed
}
# Return only JSON, never the response object or a URL containing the key.
request_list <- function(date) {
if (!has_key) stop("Set NYT_API_KEY in .Renviron first.", call. = FALSE)
url <- paste0("https://api.nytimes.com/svc/books/v3/lists/",
date, "/hardcover-fiction.json")
resp <- tryCatch(
GET(url, query = list(`api-key` = api_key), timeout(30)),
error = function(e) stop("NYT request failed. Check your connection and retry.",
call. = FALSE)
)
if (status_code(resp) != 200L) {
stop("NYT returned HTTP ", status_code(resp),
". Check the Books API subscription/key or retry later.", call. = FALSE)
}
txt <- content(resp, as = "text", encoding = "UTF-8")
validate_list(txt, if (date == "current") NULL else date)
Sys.sleep(12) # Space requests conservatively; account quotas still apply.
txt
}
dir.create("data", showWarnings = FALSE)
saved <- sort(list.files("data",
pattern = "^nyt_hardcover-fiction_[0-9]{4}-[0-9]{2}-[0-9]{2}\\.json$"))
# Reuse the saved snapshot even when a key is present.
# Set params.refresh_data to true only when intentionally updating the analysis.
if (length(saved) > 0L && !isTRUE(params$refresh_data)) {
latest_file <- file.path("data", tail(saved, 1))
published_now <- validate_list(read_file(latest_file))$results$published_date
stopifnot(basename(latest_file) == basename(cache_path(published_now)))
} else {
current_txt <- request_list("current")
published_now <- validate_list(current_txt)$results$published_date
write_file(current_txt, cache_path(published_now))
}
published_then <- as.character(as.Date(published_now) - 364)
if (!file.exists(cache_path(published_then))) {
write_file(request_list(published_then), cache_path(published_then))
}
invisible(validate_list(read_file(cache_path(published_then)), published_then))
c(latest_snapshot = published_now, year_earlier = published_then)latest_snapshot year_earlier
"2026-11-08" "2025-11-09" The responses are saved in data/, and later renders reuse those files. To update the comparison, I can set refresh_data: true at the top of the document, render, and change it back to false. Both JSON files should be included with the report so the results can be reproduced without a key.
fromJSON() reads the response. The book records are in results$books; I convert these to a tibble and keep the columns used below.
tidy_list <- function(date) {
parsed <- validate_list(read_file(cache_path(date)), date)
books <- as_tibble(parsed$results$books)
# The columns I need. Everything else (cover image links, review links,
# the nested isbns and buy_links lists) gets dropped.
needed <- c("rank", "rank_last_week", "weeks_on_list", "title", "author",
"publisher", "description", "primary_isbn13")
stopifnot(all(needed %in% names(books)))
books |>
select(all_of(needed)) |>
mutate(
published_date = as.Date(parsed$results$published_date),
across(where(is.character), \(x) na_if(str_squish(x), "")) # "" -> NA
) |>
relocate(published_date)
}
books_now <- tidy_list(published_now)
books_then <- tidy_list(published_then)
books_now |> select(rank, title, author, weeks_on_list)# A tibble: 15 × 4
rank title author weeks_on_list
<int> <chr> <chr> <int>
1 1 THRESHING DAY Rebecca Yarros 4
2 2 MISS WOLCOTT'S GHOST Louise Penny 1
3 3 A MARRIED LITTLE CHRISTMAS Abby Jimenez 1
4 4 THE FAMILY CROSS James Patterson 1
5 5 CHAIN REACTION Lee Child and Andrew Ch… 1
6 6 THE FRENCH ILLUSION John Grisham 4
7 7 THE WITCH Freida McFadden 3
8 8 WITH HEARTS OF FLAME Briar Boleyn 1
9 9 THE CALAMITY CLUB Kathryn Stockett 25
10 10 PARTITA Barbara Kingsolver 3
11 11 HOLLOW BONES Jodi Picoult 6
12 12 PECK & PECK Bonnie Garmus 2
13 13 THE DUNGEON ANARCHIST'S COOKBOOK Matt Dinniman 32
14 14 THE GATE OF THE FERAL GODS Matt Dinniman 21
15 15 MERCY'S CHILD Richard Paul Evans 1isbns and buy_links because this comparison does not use them.NA.stopifnot(nrow(books_now) > 0, nrow(books_then) > 0)
stopifnot(!anyDuplicated(books_now$rank), !anyDuplicated(books_then$rank))
stopifnot(is.numeric(books_now$weeks_on_list), is.numeric(books_then$weeks_on_list))
stopifnot(!anyNA(books_now$rank), !anyNA(books_then$rank))
stopifnot(!anyNA(books_now$weeks_on_list), !anyNA(books_then$weeks_on_list))
stopifnot(all(books_now$weeks_on_list >= 0), all(books_then$weeks_on_list >= 0))
both <- bind_rows(
books_now |> mutate(week = "This week"),
books_then |> mutate(week = "A year earlier")
) |>
mutate(week = factor(week, levels = c("This week", "A year earlier")))
nrow(both)[1] 30For this comparison, I call books with at least 10 weeks on the list long-running entries, and books with 2 weeks or fewer recent arrivals. Books with 3-9 weeks fall in between. These are working definitions for this report. They describe time on the list, not how recently a book was published.
compare <- both |>
summarise(
books = n(),
median_weeks = median(weeks_on_list, na.rm = TRUE),
longest_run = max(weeks_on_list, na.rm = TRUE),
long_running = sum(weeks_on_list >= 10, na.rm = TRUE),
recent_arrivals = sum(weeks_on_list <= 2, na.rm = TRUE),
.by = week
)
compare |>
rename(Week = week, Books = books, `Median weeks on list` = median_weeks,
`Longest run (weeks)` = longest_run, `Long-running (10+ weeks)` = long_running,
`Recent arrivals (2 or fewer)` = recent_arrivals) |>
knitr::kable()| Week | Books | Median weeks on list | Longest run (weeks) | Long-running (10+ weeks) | Recent arrivals (2 or fewer) |
|---|---|---|---|---|---|
| This week | 15 | 3 | 32 | 3 | 7 |
| A year earlier | 15 | 3 | 98 | 3 | 7 |
books_now |>
slice_min(rank, n = 5) |>
select(Rank = rank, Title = title, Author = author, `Weeks on list` = weeks_on_list) |>
knitr::kable()| Rank | Title | Author | Weeks on list |
|---|---|---|---|
| 1 | THRESHING DAY | Rebecca Yarros | 4 |
| 2 | MISS WOLCOTT’S GHOST | Louise Penny | 1 |
| 3 | A MARRIED LITTLE CHRISTMAS | Abby Jimenez | 1 |
| 4 | THE FAMILY CROSS | James Patterson | 1 |
| 5 | CHAIN REACTION | Lee Child and Andrew Child | 1 |
The table above shows the five highest-ranked books in the latest saved list.
both |>
group_by(week) |>
slice_max(weeks_on_list, n = 3, with_ties = FALSE) |>
ungroup() |>
select(Week = week, Title = title, Author = author, `Weeks on list` = weeks_on_list) |>
knitr::kable()| Week | Title | Author | Weeks on list |
|---|---|---|---|
| This week | THE DUNGEON ANARCHIST’S COOKBOOK | Matt Dinniman | 32 |
| This week | THE CALAMITY CLUB | Kathryn Stockett | 25 |
| This week | THE GATE OF THE FERAL GODS | Matt Dinniman | 21 |
| A year earlier | FOURTH WING | Rebecca Yarros | 98 |
| A year earlier | IRON FLAME | Rebecca Yarros | 82 |
| A year earlier | ONYX STORM | Rebecca Yarros | 38 |
The next chart shows weeks on the list for every book, ordered by rank within each week.
plot_data <- both |>
arrange(week, desc(rank)) |>
mutate(
plot_id = paste(week, rank, sep = "::"),
plot_id = factor(plot_id, levels = unique(plot_id)),
label = str_wrap(paste0(rank, ". ", title), width = 30)
)
ggplot(plot_data, aes(x = weeks_on_list, y = plot_id)) +
scale_y_discrete(labels = setNames(plot_data$label, plot_data$plot_id)) +
geom_col(fill = "#1F4E79") +
facet_wrap(~ week, scales = "free_y") +
labs(title = "Weeks on the NYT Hardcover Fiction list, by book",
subtitle = "Books ordered by rank (rank 1 at the top)",
x = "Weeks on list", y = NULL) +
theme_minimal()
now_row <- compare |> filter(week == "This week")
then_row <- compare |> filter(week == "A year earlier")
top1 <- books_now |> slice_min(rank, n = 1)On the list dated 2026-11-08, the median book had been on the list for 3 weeks, and 3 of 15 books were long-running (10+ weeks), while 7 were recent arrivals (2 weeks or fewer). The longest-running title had been on the list for 32 weeks. The number-one book was THRESHING DAY by Rebecca Yarros.
A year earlier (2025-11-09) the median was 3 weeks, with 3 long-running books and 7 recent arrivals. The latest list therefore has a similar median time on the list than the same week a year ago.
In the latest list, recent arrivals account for 7 of 15 books, compared with 3 long-running entries. Recent arrivals therefore make up a larger group. They do not form a majority of the list. The two weeks have the same counts in these two categories. The longest time on the list is 32 weeks in the latest week, compared with 98 weeks a year earlier.
This comparison has a few limits:
A useful next step would be to collect several months of lists and see whether the same pattern holds.
I started with a draft from a collaborator that included Claude assistance with planning, code, and writing. ChatGPT helped revise the code and explanations. The tables and numerical results come from running the R code on the NYT responses.