The New York Times web site provides a rich set of APIs, as described here: https://developer.nytimes.com/apis You’ll need to start by signing up for an API key. Your task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it into an R DataFrame.
#load the packages
library(tidyverse)
library(httr2)
library(httr)
library(jsonlite)
library(dplyr)
library(stringr)
library(knitr)
library(kableExtra)I first accessed the NY Times API and registered for an API key.We will use “apikey” to store the key, and “books” to store the url for the NY Times Books API. We will be accessing the Books API.
# API Key
apikey <- "k9CxclOeI6dxOGdMEf7x73gHhYXIVgNS"
# Get the URL
books <- GET(paste("https://api.nytimes.com/svc/books/v3/lists/best-sellers/history.json?api-key=", apikey,sep=""))
# See if connection is successful
books$status_code## [1] 200summary(books)## Length Class Mode
## url 1 -none- character
## status_code 1 -none- numeric
## headers 21 insensitive list
## all_headers 1 -none- list
## cookies 7 data.frame list
## content 15170 -none- raw
## date 1 POSIXct numeric
## times 6 -none- numeric
## request 7 request list
## handle 1 curl_handle externalptr#store as text
books <- content(books, as = "text")From using the Books API data, we will determine what are the top 3 best selling books.
#place response in a object using fromJSNON() function
booklist <-fromJSON(books)
#results
bestsellers <- booklist$results
#view the data
head(bestsellers)## title
## 1 "I GIVE YOU MY BODY ..."
## 2 "MOST BLESSED OF THE PATRIARCHS"
## 3 "YOU JUST NEED TO LOSE WEIGHT"
## 4 #ASKGARYVEE
## 5 #GIRLBOSS
## 6 #IMOMSOHARD
## description
## 1 The author of the Outlander novels gives tips on writing sex scenes, drawing on examples from the books.
## 2 A character study that attempts to make sense of Jefferson’s contradictions.
## 3 The co-host of the podcast “Maintenance Phase” examines myths about gaining and losing weight to dismantle anti-fat bias.
## 4 The entrepreneur expands on subjects addressed on his Internet show, like marketing, management and social media.
## 5 An online fashion retailer traces her path to success.
## 6
## contributor author
## 1 by Diana Gabaldon Diana Gabaldon
## 2 by Annette Gordon-Reed and Peter S. Onuf Annette Gordon-Reed and Peter S Onuf
## 3 by Aubrey Gordon Aubrey Gordon
## 4 by Gary Vaynerchuk Gary Vaynerchuk
## 5 by Sophia Amoruso Sophia Amoruso
## 6 by Kristin Hensley and Jen Smedley Kristin Hensley and Jen Smedley
## contributor_note price age_group publisher
## 1 0.00 Dell
## 2 0.00 Liveright
## 3 0.00 Beacon
## 4 0.00 HarperCollins
## 5 0.00 Portfolio/Penguin/Putnam
## 6 0.00 HarperOne
## isbns
## 1 0399178570, 9780399178573
## 2 0871404427, 9780871404428
## 3 0807006475, 0807006483, 9780807006474, 9780807006481
## 4 0062273124, 0062273132, 9780062273123, 9780062273130
## 5 039916927X, 1591847931, 9780399169274, 9781591847939
## 6 006285769X, 9780062857699
## ranks_history
## 1 0399178570, 9780399178573, 8, Advice How-To and Miscellaneous, Advice, How-To & Miscellaneous, 2016-09-04, 2016-08-20, 1, 0, 0, 0
## 2 0871404427, 9780871404428, 16, Hardcover Nonfiction, Hardcover Nonfiction, 2016-05-01, 2016-04-16, 1, 0, 1, 0
## 3 0807006475, 0807006475, 9780807006474, 9780807006474, 2, 6, Paperback Nonfiction, Combined Print and E-Book Nonfiction, Paperback Nonfiction, Combined Print & E-Book Nonfiction, 2023-01-29, 2023-01-29, 2023-01-14, 2023-01-14, 1, 1, 0, 0, 0, 0, 0, 0
## 4 0062273124, 0062273124, 9780062273123, 9780062273123, 5, 6, Business Books, Advice How-To and Miscellaneous, Business, Advice, How-To & Miscellaneous, 2016-04-10, 2016-03-27, 2016-03-26, 2016-03-12, 0, 1, 0, 0, 0, 0, 1, 1
## 5 1591847931, 1591847931, 1591847931, 1591847931, 039916927X, 9781591847939, 9781591847939, 9781591847939, 9781591847939, 9780399169274, 8, 9, 9, 8, 10, Business Books, Business Books, Business Books, Business Books, Business Books, Business, Business, Business, Business, Business, 2016-03-13, 2016-01-17, 2015-12-13, 2015-11-15, 2014-11-09, 2016-02-27, 2016-01-02, 2015-11-28, 2015-10-31, 2014-10-25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
## 6 006285769X, 9780062857699, 10, Advice How-To and Miscellaneous, Advice, How-To & Miscellaneous, 2019-04-21, 2019-04-06, 1, 0, 0, 1
## reviews
## 1 , , ,
## 2 , , ,
## 3 , , ,
## 4 , , ,
## 5 , , ,
## 6 , , ,#Check if it is a data frame
class(bestsellers)## [1] "data.frame"#drop column names
colnames(bestsellers)## [1] "title" "description" "contributor" "author"
## [5] "contributor_note" "price" "age_group" "publisher"
## [9] "isbns" "ranks_history" "reviews"bestsellers## title
## 1 "I GIVE YOU MY BODY ..."
## 2 "MOST BLESSED OF THE PATRIARCHS"
## 3 "YOU JUST NEED TO LOSE WEIGHT"
## 4 #ASKGARYVEE
## 5 #GIRLBOSS
## 6 #IMOMSOHARD
## 7 #NEVERAGAIN
## 8 $100 STARTUP
## 9 $20 PER GALLON
## 10 '57, Chicago
## 11 'ROCK OF AGES: ''ROLLING STONE'' HISTORY OF ROCK AND ROLL'
## 12 'THE HIGH ROAD TO CHINA: GEORGE BOGLE, THE PANCHEN LAMA AND THE FIRST BRITISH EXPEDITION TO TIBET'
## 13 'TIL DEATH
## 14 'TIL DEATH DO US PART
## 15 'Til Faith Do Us Part: How Interfaith Marriage is Transforming America
## 16 'TIS THE SEASON
## 17 (RE)BORN IN THE USA
## 18 ------, THAT'S DELICIOUS
## 19 ...and the Horse He Rode In On: The People V. Kenneth Starr
## 20 .HACK G.U. , VOL. 5
## description
## 1 The author of the Outlander novels gives tips on writing sex scenes, drawing on examples from the books.
## 2 A character study that attempts to make sense of Jefferson’s contradictions.
## 3 The co-host of the podcast “Maintenance Phase” examines myths about gaining and losing weight to dismantle anti-fat bias.
## 4 The entrepreneur expands on subjects addressed on his Internet show, like marketing, management and social media.
## 5 An online fashion retailer traces her path to success.
## 6
## 7 Students from Marjory Stoneman Douglas High School describe the Valentine's Day mass shooting and outline ways to prevent similar incidents.
## 8 How to build a profitable start up for $100 or less and be your own boss.
## 9
## 10 <NA>
## 11 <NA>
## 12 <NA>
## 13
## 14 A matchmaker in Victorian England turns to a crime novelist for help when she starts receiving a series of disturbingly personalized trinkets.
## 15 <NA>
## 16 Two classic holiday stories — "Under the Christmas Tree" (2009) and "Midnight Confessions" (2010) — plus the novella "Backward Glance" (1991).
## 17 The soccer commentator describes how he embraced American popular culture while growing up in Liverpool.
## 18
## 19 <NA>
## 20 This series, set in the future, is about an online, multiplayer game run amok. This volume concludes the tale.
## contributor
## 1 by Diana Gabaldon
## 2 by Annette Gordon-Reed and Peter S. Onuf
## 3 by Aubrey Gordon
## 4 by Gary Vaynerchuk
## 5 by Sophia Amoruso
## 6 by Kristin Hensley and Jen Smedley
## 7 by David Hogg and Lauren Hogg
## 8 by Chris Guillebeau
## 9 by Christopher Steiner
## 10 <NA>
## 11 <NA>
## 12 <NA>
## 13 by Sharon Sala
## 14 by Amanda Quick
## 15 <NA>
## 16 by Robyn Carr
## 17 by Roger Bennett
## 18 by Action Bronson with Rachel Wharton
## 19 <NA>
## 20 by Hamazaki Tatsuya
## author contributor_note price age_group
## 1 Diana Gabaldon 0.00
## 2 Annette Gordon-Reed and Peter S Onuf 0.00
## 3 Aubrey Gordon 0.00
## 4 Gary Vaynerchuk 0.00
## 5 Sophia Amoruso 0.00
## 6 Kristin Hensley and Jen Smedley 0.00
## 7 David Hogg and Lauren Hogg 0.00
## 8 Chris Guillebeau 23.00
## 9 Christopher Steiner 0.00
## 10 Steve Monroe <NA> 0.00 <NA>
## 11 GEOFFREY STOKES, KEN TUCKER' 'ED WARD <NA> 0.00 <NA>
## 12 KATE TELTSCHER <NA> 0.00 <NA>
## 13 Sharon Sala 0.00
## 14 Amanda Quick 0.00
## 15 Naomi Schaefer Riley <NA> 0.00 <NA>
## 16 Ron Carr 0.00
## 17 Roger Bennett 0.00
## 18 Action Bronson with Rachel Wharton 0.00
## 19 James Carville <NA> 0.00 <NA>
## 20 Hamazaki Tatsuya 10.99
## publisher
## 1 Dell
## 2 Liveright
## 3 Beacon
## 4 HarperCollins
## 5 Portfolio/Penguin/Putnam
## 6 HarperOne
## 7 Random House
## 8 Crown Business
## 9 Grand Central
## 10 <NA>
## 11 <NA>
## 12 <NA>
## 13 Harlequin Mira
## 14 Berkley
## 15 <NA>
## 16 Harlequin Mira
## 17 Dey Street
## 18 Abrams
## 19 <NA>
## 20 TOKYOPOP
## isbns
## 1 0399178570, 9780399178573
## 2 0871404427, 9780871404428
## 3 0807006475, 0807006483, 9780807006474, 9780807006481
## 4 0062273124, 0062273132, 9780062273123, 9780062273130
## 5 039916927X, 1591847931, 9780399169274, 9781591847939
## 6 006285769X, 9780062857699
## 7 198480183X, 9781984801838
## 8 0307951529, 9780307951526
## 9 NULL
## 10 0786867302, 9780786867301
## 11 0671630687, 9780671630683
## 12 0374217009, 9780374217006
## 13 0778314278, 9780778314271
## 14 069819361X, 039917446X, 9780698193611, 9780399174469
## 15 0199873747, 9780199873746
## 16 0778316645, 9780778316640
## 17 0062958690, 0062958720, 9780062958693, 9780062958723
## 18 1419726552, 9781419726552
## 19 0684857340, 9780684857343
## 20 NULL
## ranks_history
## 1 0399178570, 9780399178573, 8, Advice How-To and Miscellaneous, Advice, How-To & Miscellaneous, 2016-09-04, 2016-08-20, 1, 0, 0, 0
## 2 0871404427, 9780871404428, 16, Hardcover Nonfiction, Hardcover Nonfiction, 2016-05-01, 2016-04-16, 1, 0, 1, 0
## 3 0807006475, 0807006475, 9780807006474, 9780807006474, 2, 6, Paperback Nonfiction, Combined Print and E-Book Nonfiction, Paperback Nonfiction, Combined Print & E-Book Nonfiction, 2023-01-29, 2023-01-29, 2023-01-14, 2023-01-14, 1, 1, 0, 0, 0, 0, 0, 0
## 4 0062273124, 0062273124, 9780062273123, 9780062273123, 5, 6, Business Books, Advice How-To and Miscellaneous, Business, Advice, How-To & Miscellaneous, 2016-04-10, 2016-03-27, 2016-03-26, 2016-03-12, 0, 1, 0, 0, 0, 0, 1, 1
## 5 1591847931, 1591847931, 1591847931, 1591847931, 039916927X, 9781591847939, 9781591847939, 9781591847939, 9781591847939, 9780399169274, 8, 9, 9, 8, 10, Business Books, Business Books, Business Books, Business Books, Business Books, Business, Business, Business, Business, Business, 2016-03-13, 2016-01-17, 2015-12-13, 2015-11-15, 2014-11-09, 2016-02-27, 2016-01-02, 2015-11-28, 2015-10-31, 2014-10-25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
## 6 006285769X, 9780062857699, 10, Advice How-To and Miscellaneous, Advice, How-To & Miscellaneous, 2019-04-21, 2019-04-06, 1, 0, 0, 1
## 7 198480183X, 9781984801838, 9, Paperback Nonfiction, Paperback Nonfiction, 2018-07-08, 2018-06-23, 1, 0, 0, 0
## 8 NULL
## 9 NULL
## 10 NULL
## 11 NULL
## 12 NULL
## 13 NULL
## 14 069819361X, 069819361X, 9780698193611, 9780698193611, 15, 9, Combined Print and E-Book Fiction, E-Book Fiction, Combined Print & E-Book Fiction, E-Book Fiction, 2016-05-08, 2016-05-08, 2016-04-23, 2016-04-23, 1, 1, 0, 0, 0, 0, 0, 0
## 15 NULL
## 16 0778316645, 0778316645, 9780778316640, 9780778316640, 18, 6, Mass Market Paperback, Mass Market Paperback, Paperback Mass-Market Fiction, Paperback Mass-Market Fiction, 2014-11-30, 2014-11-16, 2014-11-15, 2014-11-01, 0, 1, 0, 0, 0, 1, 0, 0
## 17 0062958690, 0062958690, 9780062958693, 9780062958693, 3, 1, Combined Print and E-Book Nonfiction, Hardcover Nonfiction, Combined Print & E-Book Nonfiction, Hardcover Nonfiction, 2021-07-18, 2021-07-18, 2021-07-03, 2021-07-03, 1, 1, 0, 0, 0, 0, 0, 0
## 18 1419726552, 9781419726552, 9, Advice How-To and Miscellaneous, Advice, How-To & Miscellaneous, 2017-10-01, 2017-09-16, 1, 0, 0, 1
## 19 NULL
## 20 NULL
## reviews
## 1 , , ,
## 2 , , ,
## 3 , , ,
## 4 , , ,
## 5 , , ,
## 6 , , ,
## 7 , , ,
## 8 , , ,
## 9 , , ,
## 10 , NA, https://www.nytimes.com/2001/07/29/books/books-in-brief-fiction-poetry-319660.html, NA
## 11 , NA, https://www.nytimes.com/1986/12/28/books/three-chord-music-in-a-three-piece-suit.html, NA
## 12 , NA, https://www.nytimes.com/2007/04/22/books/review/Stuart.t.html, NA
## 13 , , ,
## 14 , , ,
## 15 , NA, https://www.nytimes.com/2013/05/26/books/review/til-faith-do-us-part-by-naomi-schaefer-riley.html, NA
## 16 , , ,
## 17 , , ,
## 18 , , ,
## 19 , NA, https://www.nytimes.com/1998/10/18/books/preaching-to-the-converted.html, NA
## 20 , , ,head(bestsellers)## title
## 1 "I GIVE YOU MY BODY ..."
## 2 "MOST BLESSED OF THE PATRIARCHS"
## 3 "YOU JUST NEED TO LOSE WEIGHT"
## 4 #ASKGARYVEE
## 5 #GIRLBOSS
## 6 #IMOMSOHARD
## description
## 1 The author of the Outlander novels gives tips on writing sex scenes, drawing on examples from the books.
## 2 A character study that attempts to make sense of Jefferson’s contradictions.
## 3 The co-host of the podcast “Maintenance Phase” examines myths about gaining and losing weight to dismantle anti-fat bias.
## 4 The entrepreneur expands on subjects addressed on his Internet show, like marketing, management and social media.
## 5 An online fashion retailer traces her path to success.
## 6
## contributor author
## 1 by Diana Gabaldon Diana Gabaldon
## 2 by Annette Gordon-Reed and Peter S. Onuf Annette Gordon-Reed and Peter S Onuf
## 3 by Aubrey Gordon Aubrey Gordon
## 4 by Gary Vaynerchuk Gary Vaynerchuk
## 5 by Sophia Amoruso Sophia Amoruso
## 6 by Kristin Hensley and Jen Smedley Kristin Hensley and Jen Smedley
## contributor_note price age_group publisher
## 1 0.00 Dell
## 2 0.00 Liveright
## 3 0.00 Beacon
## 4 0.00 HarperCollins
## 5 0.00 Portfolio/Penguin/Putnam
## 6 0.00 HarperOne
## isbns
## 1 0399178570, 9780399178573
## 2 0871404427, 9780871404428
## 3 0807006475, 0807006483, 9780807006474, 9780807006481
## 4 0062273124, 0062273132, 9780062273123, 9780062273130
## 5 039916927X, 1591847931, 9780399169274, 9781591847939
## 6 006285769X, 9780062857699
## ranks_history
## 1 0399178570, 9780399178573, 8, Advice How-To and Miscellaneous, Advice, How-To & Miscellaneous, 2016-09-04, 2016-08-20, 1, 0, 0, 0
## 2 0871404427, 9780871404428, 16, Hardcover Nonfiction, Hardcover Nonfiction, 2016-05-01, 2016-04-16, 1, 0, 1, 0
## 3 0807006475, 0807006475, 9780807006474, 9780807006474, 2, 6, Paperback Nonfiction, Combined Print and E-Book Nonfiction, Paperback Nonfiction, Combined Print & E-Book Nonfiction, 2023-01-29, 2023-01-29, 2023-01-14, 2023-01-14, 1, 1, 0, 0, 0, 0, 0, 0
## 4 0062273124, 0062273124, 9780062273123, 9780062273123, 5, 6, Business Books, Advice How-To and Miscellaneous, Business, Advice, How-To & Miscellaneous, 2016-04-10, 2016-03-27, 2016-03-26, 2016-03-12, 0, 1, 0, 0, 0, 0, 1, 1
## 5 1591847931, 1591847931, 1591847931, 1591847931, 039916927X, 9781591847939, 9781591847939, 9781591847939, 9781591847939, 9780399169274, 8, 9, 9, 8, 10, Business Books, Business Books, Business Books, Business Books, Business Books, Business, Business, Business, Business, Business, 2016-03-13, 2016-01-17, 2015-12-13, 2015-11-15, 2014-11-09, 2016-02-27, 2016-01-02, 2015-11-28, 2015-10-31, 2014-10-25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
## 6 006285769X, 9780062857699, 10, Advice How-To and Miscellaneous, Advice, How-To & Miscellaneous, 2019-04-21, 2019-04-06, 1, 0, 0, 1
## reviews
## 1 , , ,
## 2 , , ,
## 3 , , ,
## 4 , , ,
## 5 , , ,
## 6 , , ,After connecting to the API and transforming the JSON data into an R dataframe, we can see that the top 3 selling books in history are “I give you my body”, “Most Blessed of the Patriarchs” and “You just need to lose weight”.
I believe using API can be very useful for sourcing a variety of data, though I had a bit of trouble initially connecting to the API and bringing in the response (it took a few different methods to get the status code from 401 to 202), but afterwards, using the fromJSON function from the jsonlite package made the process to converting the response into a data frame was more straightforward.