In this assignment, we are required to access an API from the New York Times and scrape a JSON formatted table that we then convert into a data frame. To get access to an API key, one needs to make an account with NYT developers. I chose to use the book API. APIs can be easily accessed using the httr package
library(httr)
library(jsonlite)While some API require a separate command to enter the key, the NYT API allows the key inside of the URL. I decided to get the book reviews for the author Steven Pinker, a psychology professor and bestselling author at Harvard University.
res = GET("https://api.nytimes.com/svc/books/v3/reviews.json?author=Steven+Pinker&api-key=zcx9ad4Vhxkw8E3sTKxdkYKnAVUE6mql")
res## Response [https://api.nytimes.com/svc/books/v3/reviews.json?author=Steven+Pinker&api-key=zcx9ad4Vhxkw8E3sTKxdkYKnAVUE6mql]
## Date: 2024-03-23 15:48
## Status: 200
## Content-Type: application/json; charset=UTF-8
## Size: 4.17 kBInvestigating the res variable, we can see that the status is 200, which means it returned data successfully. Using the jsonlite package we can efficently work with the JSON data and extract it from the response variable to use it in R. First we need to convert the string that is the variable content in the res list to JSON, upon which we can then convert it to a dataframe.
data = fromJSON(rawToChar(res$content))
pinker = as.data.frame(data$results)
pinker## url
## 1 http://www.nytimes.com/1999/11/28/books/washington-sleeped-here.html
## 2 http://www.nytimes.com/1994/03/10/books/books-of-the-times-how-the-mind-translates-thoughts-into-words.html
## 3 http://www.nytimes.com/2011/10/09/books/review/the-better-angels-of-our-nature-by-steven-pinker-book-review.html
## 4 http://www.nytimes.com/2014/10/19/books/review/steven-pinker-the-sense-of-style-review.html
## 5 http://www.nytimes.com/2007/09/23/books/review/Saletan-t.html
## 6 http://www.nytimes.com/1997/10/05/books/the-brain-s-software.html
## 7 http://www.nytimes.com/1997/11/24/books/books-of-the-times-thinking-deeply-about-thinking-and-having-fun-too.html
## 8 http://www.nytimes.com/2002/10/13/books/the-evolutionary-war.html
## 9 https://www.nytimes.com/2018/03/02/books/review/steven-pinker-enlightenment-now.html
## publication_dt byline
## 1 1999-11-28 MARK ARONOFF
## 2 1994-03-10 CHRISTOPHER LEHMANN-HAUPT
## 3 2011-10-09 PETER SINGER
## 4 2014-10-19 CHARLES McGRATH
## 5 2007-09-23 WILLIAM SALETAN
## 6 1997-10-05 MARK RIDLEY
## 7 1997-11-24
## 8 2002-10-13 ROBERT J. RICHARDS
## 9 2018-03-02 Sarah Bakewell
## book_title
## 1 Words and Rules : The Ingredients of Language
## 2 Language Instinct : How the Mind Creates Language
## 3 The Better Angels of Our Nature
## 4 The Sense of Style
## 5 The Stuff of Thought: Language as a Window Into Human Nature
## 6 How the Mind Works
## 7 How the Mind Works
## 8 The Blank Slate: The Modern Denial of Human Nature
## 9 Enlightenment Now: The Case for Reason, Science, Humanism, and Progress
## book_author
## 1 Steven Pinker
## 2 Steven Pinker
## 3 Steven Pinker
## 4 Steven Pinker
## 5 Steven Pinker
## 6 Steven Pinker
## 7 Steven Pinker
## 8 Steven Pinker
## 9 Steven Pinker
## summary
## 1
## 2
## 3 In his new book, Steven Pinker argues that our current era is less violent, less cruel and more peaceful than any previous period of human existence.
## 4 In his writing guide, the Harvard polymath Steven Pinker favors looser, more easygoing grammatical usage.
## 5 Steven Pinker brings his theory of human nature and his obsession with words together in one book.
## 6
## 7
## 8
## 9 In “Enlightenment Now,” the Harvard professor offers much evidence that the world, our feelings notwithstanding, is definitely getting better.
## uuid
## 1 00000000-0000-0000-0000-000000000000
## 2 00000000-0000-0000-0000-000000000000
## 3 00000000-0000-0000-0000-000000000000
## 4 00000000-0000-0000-0000-000000000000
## 5 00000000-0000-0000-0000-000000000000
## 6 00000000-0000-0000-0000-000000000000
## 7 00000000-0000-0000-0000-000000000000
## 8 00000000-0000-0000-0000-000000000000
## 9 00000000-0000-0000-0000-000000000000
## uri
## 1 nyt://book/00000000-0000-0000-0000-000000000000
## 2 nyt://book/00000000-0000-0000-0000-000000000000
## 3 nyt://book/00000000-0000-0000-0000-000000000000
## 4 nyt://book/00000000-0000-0000-0000-000000000000
## 5 nyt://book/00000000-0000-0000-0000-000000000000
## 6 nyt://book/00000000-0000-0000-0000-000000000000
## 7 nyt://book/00000000-0000-0000-0000-000000000000
## 8 nyt://book/00000000-0000-0000-0000-000000000000
## 9 nyt://book/00000000-0000-0000-0000-000000000000
## isbn13
## 1 9780756756116
## 2 9780060976514
## 3 9780143122012
## 4 9780143127796, 9780670025855, 9780698170308
## 5 9780143114246
## 6 9780393045352
## 7 9780393045352
## 8 9780670031511
## 9 9780525427575The pinker dataframe contains the scraped information: Book reviews previously published in the NYT for books of him. There are less reviews than I expected (only 9), given he’s published more bestselling books than the ones listed. I had hoped that there would be some type of quantitative score for each review, but there is not, so one could do a sentiment analysis of the reviews, then standardize it to recieve such a score.