api-key: My authentication key, retrieved from my environment variable.
hardcover-fiction: The bestseller list requested, specified in the endpoint URL.
The API returns information about books currently appearing on the hardcover fiction bestseller list.
Approach:
I started out by creating and account on https://developer.nytimes.com/. Getting an API key meant I needed to register an app. I named my app FavBooks. I wanted to get the basic requirements to start the homework out of the way, so after some refreshing on some of my skills, I was able to store the API Key in system environment and then load it here using request and req_url_query function.
Once I have an ok status, as can be seen from the following code, I’d now work on creating and cleaning the dataset.
library(httr2)library(tidyverse)
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.2.1 ✔ readr 2.2.0
✔ forcats 1.0.1 ✔ stringr 1.6.0
✔ ggplot2 4.0.3 ✔ tibble 3.3.1
✔ lubridate 1.9.5 ✔ tidyr 1.3.2
✔ purrr 1.2.1
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library("DT")#Reading the Sys.env file where the API_KEY is savedreadRenviron("Sys.env")#making sure the key isn't nullSys.getenv("API_KEY") !=""
[1] TRUE
#getting the request ready, NYTimes gives example code for this!req<-request("https://api.nytimes.com/svc/books/v3/lists/current/hardcover-fiction.json") |>req_url_query(`api-key`=Sys.getenv("API_KEY"))#actually performing the requestresp<-req_perform(req)#resp holds the actual json with results and books, untidy databooks <-resp_body_json(resp)$results$books #Creating a function to call run through each book, which will pick the#called upon entity from the datareturn_tidy<-function(x,field){x[[field]]}
Dataframe creation using sapply function to run through every book
books_df <-data.frame(Rank =sapply(books, return_tidy, field ="rank"),Title =sapply(books, return_tidy, field ="title"),Author =sapply(books, return_tidy, field ="author"),Description =sapply(books, return_tidy, field ="description"),Publisher =sapply(books, return_tidy, field ="publisher"),Weeks_on_List =sapply(books, return_tidy, field ="weeks_on_list"))
Top 5 books in descending order that have been on the list.
Rank Title Author Weeks_on_List
1 13 THE DUNGEON ANARCHIST'S COOKBOOK Matt Dinniman 32
2 9 THE CALAMITY CLUB Kathryn Stockett 25
3 14 THE GATE OF THE FERAL GODS Matt Dinniman 21
4 11 HOLLOW BONES Jodi Picoult 6
5 1 THRESHING DAY Rebecca Yarros 4
Second API Call:
Making another API call to NYTimes that would return overview list. Upon investigating a little bit, I realized that it returns a list of lists (list of best selling books under Hard-fiction, or how-to books etc).
I ran the following commands to investigate:
Following code, returned a column name List, which is NYTimes’ way of showing genre, Average_Weeks, and Number_of_Books. I felt like this data was more insteresting. and I decided to plot this.
req2<-request("https://api.nytimes.com/svc/books/v3/lists/overview.json?") |>req_url_query(`api-key`=Sys.getenv("API_KEY"))resp2<-req_perform(req2)books_overview <-resp_body_json(resp2)$results$listsbooks_df_overview <-map_dfr(books_overview, function(lst) {data.frame(List = lst$list_name,Rank =sapply(lst$books, return_tidy, field ="rank"),Title =sapply(lst$books, return_tidy, field ="title"),Author =sapply(lst$books, return_tidy, field ="author"),Weeks_on_List =sapply(lst$books, return_tidy, field ="weeks_on_list") )})books_df_overview |>group_by(List) |>summarise(Average_Weeks =mean(Weeks_on_List, na.rm =TRUE),Number_of_Books =n() )
ggplot(books_df_overview, aes(x = List, y = Weeks_on_List)) +geom_boxplot() +labs(title ="Weeks on NYT Bestseller Lists",x ="Bestseller Category",y ="Weeks on List" ) +theme_minimal() +theme(axis.text.x =element_text(angle =45, hjust =1))
#Looking at why Children's books have such a huge >750 weeks streak.books_df_overview |>filter(List =="Children’s Middle Grade Hardcover") |>arrange(desc(Weeks_on_List)) |>select(Title, Weeks_on_List)
Title Weeks_on_List
1 WONDER 566
2 THE COMPLETE COOKBOOK FOR YOUNG CHEFS 292
3 WOMBAT WAITING 24
4 KIKI'S DELIVERY SERVICE 9
5 KIKI AND THE NEW MAGIC 9
6 THE WILD ZONE 4
7 BUILDING 903 4
8 GAME OF PETS: THE BOY WHO KNEW TOO MUCH 4
9 DRAGONBORN: THE TWILIGHT CHILD 1
10 LICORICE 1
Data Cleaning:
The API response was returned in JSON format, with book infomation stored in nested lists. I extracted the books portion of the response and created a function called return_tidy to extract the data I needed i.e. publisher, weeks_on_the_list, etc.
I also used the sapply() function to iterate through each book and extract it’s rank, title, author, description, publisher, and number of weeks on the best seller list.
I then used data.frame() to organize these extracted fields into a structured table with each row representing an individual books.
When I plotted the overview api call dataframe, I noticed that children’s books plot was huge.
It was >750 weeks. I looked at the data (with the help of ChatGPT: books_df_overview |> filter(List == “Children’s Middle Grade Hardcover”) |> arrange(desc(Weeks_on_List)) |> select(Title, Weeks_on_List)
The above code gave me the outliers: WONDER and The Complete Cookbook for Young Chefs. Been the best sellers in their own categories for around 11 years.
ggplot(books_df_overview,aes(x =reorder(List, Weeks_on_List, FUN = median),y = Weeks_on_List)) +geom_boxplot() +coord_flip(ylim =c(0, 100)) +labs(title ="Weeks on NYT Bestseller Lists",subtitle ="Focused view: 0–100 weeks",x ="Bestseller Category",y ="Weeks on List" ) +theme_minimal()
