The following project utilizes New York Times Books API analyze best-selling books in the Advice, How-To & Miscellaneous category, concentrating on self-help and personal development.
Research query I’m answering is pinpointing which books rank highest, and how long have they stayed on the best-seller list ?
I chose New York Times Books API, with the endpoint being https://api.nytimes.com/svc/books/v3/lists/current/advice-how-to-and-miscellaneous.json
The function of endpoint is to obtain the present best-seller rankings for the Advice, How-to & Miscellaneous category, which covers my desired genre.
We will use NY TIMES API to obtain the list.
library(httr)
library(jsonlite)
library(dplyr)
library(tibble)
library(knitr)url <- "https://api.nytimes.com/svc/books/v3/lists/current/advice-how-to-and-miscellaneous.json"
api_key <- Sys.getenv("NYT_API_KEY")
response <- GET(
url,
query = list("api-key" = api_key)
)
status_code(response)## [1] 200json_text <- content(response, as = "text", encoding = "UTF-8")
nyt_data <- fromJSON(json_text)
names(nyt_data)## [1] "status" "copyright"
## [3] "num_results" "last_modified"
## [5] "results"The JSON response has nested book info within the results field. I plan to extract the book records and select the variables important to my research query.
books_raw <- nyt_data$results$books
names(books_raw)## [1] "age_group"
## [2] "amazon_product_url"
## [3] "article_chapter_link"
## [4] "asterisk"
## [5] "author"
## [6] "book_image"
## [7] "book_image_height"
## [8] "book_image_width"
## [9] "book_review_link"
## [10] "book_uri"
## [11] "contributor"
## [12] "contributor_note"
## [13] "created_date"
## [14] "dagger"
## [15] "description"
## [16] "first_chapter_link"
## [17] "price"
## [18] "primary_isbn10"
## [19] "primary_isbn13"
## [20] "publisher"
## [21] "rank"
## [22] "rank_last_week"
## [23] "sunday_review_link"
## [24] "title"
## [25] "updated_date"
## [26] "weeks_on_list"
## [27] "isbns"
## [28] "buy_links"head(books_raw)## age_group
## 1
## 2
## 3
## 4
## 5
## 6
## amazon_product_url
## 1 https://www.amazon.com/dp/0593234170?tag=thenewyorktim-20
## 2 https://www.amazon.com/dp/1400356822?tag=thenewyorktim-20
## 3 https://www.amazon.com/dp/B0H2GJJVDW?tag=thenewyorktim-20
## 4 https://www.amazon.com/dp/1401971369?tag=thenewyorktim-20
## 5 https://www.amazon.com/dp/1668032147?tag=thenewyorktim-20
## 6 https://www.amazon.com/Atomic-Habits-Proven-Build-Break/dp/0735211299?tag=thenewyorktim-20
## article_chapter_link asterisk
## 1 0
## 2 0
## 3 0
## 4 0
## 5 0
## 6 0
## author
## 1 Ina Garten
## 2 Philip Anthony Mitchell
## 3 Ethan Mollick
## 4 Mel Robbins and Sawyer Robbins
## 5 Andrew D. Huberman with Jessica Wapner
## 6 James Clear
## book_image
## 1 https://static01.nyt.com/bestsellers/images/9780593234174.jpg
## 2 https://static01.nyt.com/bestsellers/images/9781400356829.jpg
## 3 https://static01.nyt.com/bestsellers/images/9798217181391.jpg
## 4 https://static01.nyt.com/bestsellers/images/9781401971366.jpg
## 5 https://static01.nyt.com/bestsellers/images/9781668032145.jpg
## 6 https://static01.nyt.com/bestsellers/images/9780735211292.jpg
## book_image_height book_image_width
## 1 500 381
## 2 500 333
## 3 500 331
## 4 400 312
## 5 500 318
## 6 400 312
## book_review_link
## 1
## 2
## 3
## 4
## 5
## 6
## book_uri
## 1 nyt://book/48b8ec97-110c-57dc-8e69-6389607bfbb0
## 2 nyt://book/a9febde8-094b-5f49-b850-8fca759d615c
## 3 nyt://book/6b3788b5-0328-5560-859c-ef60624ac1fe
## 4 nyt://book/9639c9a8-e41f-50f7-beca-289f3fdd52af
## 5 nyt://book/3a986f8b-484f-58ec-a583-4d4b64c04d45
## 6 nyt://book/0398a355-c032-534e-a0af-647b06f0840d
## contributor
## 1 by Ina Garten
## 2 by Philip Anthony Mitchell
## 3 by Ethan Mollick
## 4 by Mel Robbins and Sawyer Robbins
## 5 by Andrew D. Huberman with Jessica Wapner
## 6 by James Clear
## contributor_note created_date
## 1 2026-10-28T17:27:55.954Z
## 2 2026-10-21T17:29:29.474Z
## 3 2026-10-28T17:27:56.101Z
## 4 2026-10-22T19:42:06.651Z
## 5 2026-09-23T18:21:02.85Z
## 6 2026-10-21T18:00:43.614Z
## dagger description first_chapter_link
## 1 0
## 2 0
## 3 0
## 4 0
## 5 0
## 6 1
## price primary_isbn10 primary_isbn13
## 1 0.00 9780593234174
## 2 0.00 9781400356829
## 3 0.00 9798217181391
## 4 0.00 9781401971366
## 5 0.00 9781668032145
## 6 0.00 9780735211292
## publisher rank rank_last_week
## 1 Clarkson Potter 1 0
## 2 Thomas Nelson 2 1
## 3 Portfolio 3 0
## 4 Hay House 4 6
## 5 Simon Element 5 3
## 6 Avery 6 4
## sunday_review_link title
## 1 SIMPLY INA
## 2 CONTEND
## 3 CO-EXISTENCE
## 4 THE LET THEM THEORY
## 5 PROTOCOLS
## 6 ATOMIC HABITS
## updated_date weeks_on_list
## 1 2026-10-28T17:27:55.954Z 1
## 2 2026-10-21T17:29:29.474Z 2
## 3 2026-10-28T17:27:56.101Z 1
## 4 2026-10-22T19:42:06.651Z 96
## 5 2026-09-23T18:21:02.85Z 6
## 6 2026-10-21T18:00:43.614Z 360
## isbns
## 1 , 9780593234174
## 2 , 9781400356829
## 3 , 9798217181391
## 4 , 9781401971366
## 5 , 9781668032145
## 6 , 9780735211292
## buy_links
## 1 Amazon, Apple Books, Barnes & Noble, Books-A-Million, Bookshop.org, https://www.amazon.com/dp/0593234170?tag=thenewyorktim-20, https://goto.applebooks.apple/9780593234174?at=10lIEQ, https://barnesandnoble.sjv.io/c/197432/1961179/22009?prodsku=9780593234174&u=https%3A%2F%2Fwww.barnesandnoble.com%2Fw%2Fsimply-ina-ina-garten%2F1149724863%3Fean%3D9780593234174&intsrc=APIG_15890, https://www.anrdoezrs.net/click-7990613-35140?url=https%3A%2F%2Fwww.booksamillion.com%2Fp%2FSimply-Ina%2FIna-Garten%2F9780593234174, https://bookshop.org/a/3546/9780593234174
## 2 Amazon, Apple Books, Barnes & Noble, Books-A-Million, Bookshop.org, https://www.amazon.com/dp/1400356822?tag=thenewyorktim-20, https://goto.applebooks.apple/9781400356829?at=10lIEQ, https://barnesandnoble.sjv.io/c/197432/1961179/22009?prodsku=9781400356829&u=https%3A%2F%2Fwww.barnesandnoble.com%2Fw%2Fcontend-philip-anthony-mitchell%2F1149894242%3Fean%3D9781400356829&intsrc=APIG_15890, https://www.anrdoezrs.net/click-7990613-35140?url=https%3A%2F%2Fwww.booksamillion.com%2Fp%2FContend%2FPhilip-Anthony-Mitchell%2F9781400356829, https://bookshop.org/a/3546/9781400356829
## 3 Amazon, Apple Books, Barnes & Noble, Books-A-Million, Bookshop.org, https://www.amazon.com/dp/B0H2GJJVDW?tag=thenewyorktim-20, https://goto.applebooks.apple/9798217181391?at=10lIEQ, https://barnesandnoble.sjv.io/c/197432/1961179/22009?prodsku=9798217181391&u=https%3A%2F%2Fwww.barnesandnoble.com%2Fw%2Fco-existence-ethan-mollick%2F1150229796%3Fean%3D9798217181391&intsrc=APIG_15890, https://www.anrdoezrs.net/click-7990613-35140?url=https%3A%2F%2Fwww.booksamillion.com%2Fp%2FCo-Existence%2FEthan-Mollick%2F9798217181391, https://bookshop.org/a/3546/9798217181391
## 4 Amazon, Apple Books, Barnes and Noble, Books-A-Million, Bookshop.org, https://www.amazon.com/dp/1401971369?tag=thenewyorktim-20, https://goto.applebooks.apple/9781401971366?at=10lIEQ, https://barnesandnoble.sjv.io/c/197432/1961179/22009?prodsku=9781401971366&u=https%3A%2F%2Fwww.barnesandnoble.com%2Fw%2Fthe-let-them-theory-mel-robbins%2F1146342595%3Fean%3D9781401971366&intsrc=APIG_15890, https://www.anrdoezrs.net/click-7990613-35140?url=https%3A%2F%2Fwww.booksamillion.com%2Fp%2FTHE%2BLET%2BTHEM%2BTHEORY%2FMel%2BRobbins%2F9781401971366, https://bookshop.org/a/3546/9781401971366
## 5 Amazon, Apple Books, Barnes & Noble, Books-A-Million, Bookshop.org, https://www.amazon.com/dp/1668032147?tag=thenewyorktim-20, https://goto.applebooks.apple/9781668032145?at=10lIEQ, https://barnesandnoble.sjv.io/c/197432/1961179/22009?prodsku=9781668032145&u=https%3A%2F%2Fwww.barnesandnoble.com%2Fw%2Fprotocols-andrew-d-huberman-phd%2F1145846225%3Fean%3D9781668032145&intsrc=APIG_15890, https://www.anrdoezrs.net/click-7990613-35140?url=https%3A%2F%2Fwww.booksamillion.com%2Fp%2FProtocols%2FAndrew-D-Huberman%2F9781668032145, https://bookshop.org/a/3546/9781668032145
## 6 Amazon, Apple Books, Barnes and Noble, Books-A-Million, Bookshop.org, https://www.amazon.com/Atomic-Habits-Proven-Build-Break/dp/0735211299?tag=thenewyorktim-20, https://goto.applebooks.apple/9781847941831?at=10lIEQ, https://barnesandnoble.sjv.io/c/197432/1961179/22009?prodsku=9780735211292&u=https%3A%2F%2Fwww.barnesandnoble.com%2Fw%2Fatomic-habits-james-clear%2F1129201155%3Fean%3D9780735211292&intsrc=APIG_15890, https://www.anrdoezrs.net/click-7990613-35140?url=https%3A%2F%2Fwww.booksamillion.com%2Fp%2FATOMIC%2BHABITS%2FJames%2BClear%2F9780735211292, https://bookshop.org/a/3546/9780735211292books_tidy <- books_raw %>%
as_tibble() %>%
select(
rank,
title,
author,
publisher,
weeks_on_list,
rank_last_week,
description
) %>%
arrange(rank)
print(books_tidy)## # A tibble: 10 × 7
## rank title author publisher weeks_on_list
## <int> <chr> <chr> <chr> <int>
## 1 1 SIMP… Ina G… Clarkson… 1
## 2 2 CONT… Phili… Thomas N… 2
## 3 3 CO-E… Ethan… Portfolio 1
## 4 4 THE … Mel R… Hay House 96
## 5 5 PROT… Andre… Simon El… 6
## 6 6 ATOM… James… Avery 360
## 7 7 MEDI… Ayesh… Clarkson… 1
## 8 8 SIMP… Sadia… Clarkson… 1
## 9 9 WELC… Taryn… 37 Ink 1
## 10 10 MAKI… Lysa … Thomas N… 4
## # ℹ 2 more variables: rank_last_week <int>,
## # description <chr>glimpse(books_tidy)## Rows: 10
## Columns: 7
## $ rank <int> 1, 2, 3, 4, 5, 6, 7,…
## $ title <chr> "SIMPLY INA", "CONTE…
## $ author <chr> "Ina Garten", "Phili…
## $ publisher <chr> "Clarkson Potter", "…
## $ weeks_on_list <int> 1, 2, 1, 96, 6, 360,…
## $ rank_last_week <int> 0, 1, 0, 6, 3, 4, 0,…
## $ description <chr> "", "", "", "", "", …colSums(is.na(books_tidy))## rank title author
## 0 0 0
## publisher weeks_on_list rank_last_week
## 0 0 0
## description
## 0books_tidy <- books_tidy %>%
mutate(
description = na_if(trimws(description), "")
)
colSums(is.na(books_tidy))## rank title author
## 0 0 0
## publisher weeks_on_list rank_last_week
## 0 0 0
## description
## 10This portion outlines the 5 highest-ranked books and looks at the amount of weeks they have shown on the New York Times best-seller list.
top_five <- books_tidy %>%
arrange(rank) %>%
slice_head(n = 5) %>%
select(rank, title, author, weeks_on_list)
knitr::kable(top_five)| rank | title | author | weeks_on_list |
|---|---|---|---|
| 1 | SIMPLY INA | Ina Garten | 1 |
| 2 | CONTEND | Philip Anthony Mitchell | 2 |
| 3 | CO-EXISTENCE | Ethan Mollick | 1 |
| 4 | THE LET THEM THEORY | Mel Robbins and Sawyer Robbins | 96 |
| 5 | PROTOCOLS | Andrew D. Huberman with Jessica Wapner | 6 |
books_summary <- books_tidy %>%
summarise(
total_books = n(),
average_weeks = mean(weeks_on_list),
median_weeks = median(weeks_on_list),
longest_weeks = max(weeks_on_list)
)
knitr::kable(books_summary, digits = 2)| total_books | average_weeks | median_weeks | longest_weeks |
|---|---|---|---|
| 10 | 47.3 | 1.5 | 360 |
longest_running <- books_tidy %>%
arrange(desc(weeks_on_list)) %>%
slice_head(n = 5) %>%
select(title, author, rank, weeks_on_list)
knitr::kable(longest_running)| title | author | rank | weeks_on_list |
|---|---|---|---|
| ATOMIC HABITS | James Clear | 6 | 360 |
| THE LET THEM THEORY | Mel Robbins and Sawyer Robbins | 4 | 96 |
| PROTOCOLS | Andrew D. Huberman with Jessica Wapner | 5 | 6 |
| MAKING PEACE WITH WHAT I CAN’T CONTROL | Lysa TerKeurst | 10 | 4 |
| CONTEND | Philip Anthony Mitchell | 2 | 2 |
THe NY Times Book API yielded data in JSON format with nested book records. I took the books from the results field and changed them into a tidy data frame implementing dplyr and tibble,
Having selected 7 variables important to analysis: rank, title, author, publisher, weeks on the list, previous week’s rank, and description. Every row represents one book.
The beginning missing-value check produced zero missing values. But, some descriptions had empty strings, which were adjusted to NA to showcase missing info more precisely. The ranking variables were already stored as integers, removing the need for more type conversion.
The NY Times Books API properly gave best-seller data from the Advice, How-To & Miscellaneous category. Parsing and cleaningthe JSON response allowed me to create a tidy data frame containing 10 books and seven variables.
The analytical results showed that the present rankings don’t actually reflect long-term popularity. For instance, “The Let Them Theory ranked 4th in spite of staying on the list for 96 weeks ! The category also included books outside the self-help-specific genre, which restricts conclusions about the personal development names.