Problem Statement
The New York Times web site provides a rich set of APIs, as described here: https://developer.nytimes.com/apis
You’ll need to start by signing up for an API key. Your task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it into an R DataFrame.
Git-Hub
https://github.com/forhadakbar/data607fall2019/tree/master/Week%2009 https://github.com/ShovanBiswas/DATA607/tree/master/Week9
Load Packages
Approach
We selected two APIs from NYT’s site, namely Books API and Articles Search API. The former is simpler and the latter is relatively complex. After creating the accounts (of each member), we obtained the the key. In the RMD, We stored the key in a variable and hid it with echo=FALSE.
I Books API
The BOOKS API provides information about book reviews and the NYT best seller list.
Extraction of books by Yuval Noah Harari
From Books API’s best seller lis, we filtered out author Harari’s books, and recasted into R dataframe.
url <- paste0("https://api.nytimes.com/svc/books/v3/lists/best-sellers/history.json?author=Yuval+Noah+Harari&api-key=",NYTKEY)
harari <- fromJSON(url) %>%
as.data.frame() %>%
select(4, 5, 7, 11)
names(harari) <- str_replace_all(names(harari), "results.", "")
kable(harari)| title | description | author | publisher |
|---|---|---|---|
| 21 LESSONS FOR THE 21ST CENTURY | Technological, political and social issues in the modern era, and the choices individuals might consider in facing them. | Yuval Noah Harari | Spiegel & Grau |
| HOMO DEUS | A look into the future by the author of “Sapiens.” Read by Derek Perkins. 14 hours, 53 minutes unabridged. | Yuval Noah Harari | HarperAudio |
| SAPIENS | How Homo sapiens became Earth’s dominant species. | Yuval Noah Harari | Harper Perennial |
Extraction of current best sellers
From Books API’s we selected the current best sellers of NYT.
url <- paste0("https://api.nytimes.com/svc/books/v3/lists/current/hardcover-fiction.json?api-key=",NYTKEY)
current <- fromJSON(url,flatten = TRUE)
x <- current$results # Having scrutinized the content of "fromJSON(url,flatten = TRUE)", we located that a usable data structure was nested inside of current$results.
y <- x$books # The exact data frame, where it reided was x$books.
kable(y)| rank | rank_last_week | weeks_on_list | asterisk | dagger | primary_isbn10 | primary_isbn13 | publisher | description | price | title | author | contributor | contributor_note | book_image | book_image_width | book_image_height | amazon_product_url | age_group | book_review_link | first_chapter_link | sunday_review_link | article_chapter_link | isbns | buy_links | book_uri |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 0 | 1 | 0 | 0 | 0385544189 | 9780385544184 | Doubleday | Cullen Post, a lawyer and Episcopal minister, antagonizes some ruthless killers when he takes on a wrongful conviction case. | 0 | THE GUARDIANS | John Grisham | by John Grisham | https://s1.nyt.com/du/books/images/9780385544184.jpg | 330 | 481 | https://www.amazon.com/Guardians-Novel-John-Grisham/dp/0385544189?tag=NYTBS-20 | list(isbn10 = “0385544189”, isbn13 = “9780385544184”) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Guardians-Novel-John-Grisham/dp/0385544189?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=THE+GUARDIANS&author=John+Grisham”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780385544184”, “http://www.indiebound.org/book/9780385544184?aff=NYT”)) | nyt://book/34587156-73ec-5121-946b-482b51beaf6c | ||||||
| 2 | 2 | 59 | 0 | 0 | 0735219095 | 9780735219090 | Putnam | In a quiet town on the North Carolina coast in 1969, a young woman who survived alone in the marsh becomes a murder suspect. | 0 | WHERE THE CRAWDADS SING | Delia Owens | by Delia Owens | https://s1.nyt.com/du/books/images/9780735219090.jpg | 328 | 495 | https://www.amazon.com/Where-Crawdads-Sing-Delia-Owens/dp/0735219095?tag=NYTBS-20 | list(isbn10 = c(“0735219095”, “0735219117”, “0525640371”, “0593105419”), isbn13 = c(“9780735219090”, “9780735219113”, “9780525640370”, “9780593105412”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Where-Crawdads-Sing-Delia-Owens/dp/0735219095?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=WHERE+THE+CRAWDADS+SING&author=Delia+Owens”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780735219090”, “http://www.indiebound.org/book/9780735219090?aff=NYT”)) | nyt://book/053b4109-4555-5aa1-9b39-cc40549bcdf0 | ||||||
| 3 | 1 | 2 | 0 | 0 | 0316420271 | 9780316420273 | Little, Brown | In the 19th installment of the Women’s Murder Club series, detective Lindsay Boxer and company take on a fearsome criminal known only as “Loman.” | 0 | THE 19TH CHRISTMAS | James Patterson and Maxine Paetro | by James Patterson and Maxine Paetro | https://s1.nyt.com/du/books/images/9780316494021.jpg | 321 | 495 | https://www.amazon.com/19th-Christmas-Womens-Murder-Club-ebook/dp/B07P1SSF9S?tag=NYTBS-20 | list(isbn10 = c(“0316420271”, “031649402X”), isbn13 = c(“9780316420273”, “9780316494021”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/19th-Christmas-Womens-Murder-Club-ebook/dp/B07P1SSF9S?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=THE+19TH+CHRISTMAS&author=James+Patterson+and+Maxine+Paetro”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780316420273”, “http://www.indiebound.org/book/9780316420273?aff=NYT”)) | nyt://book/8a6cd41d-3325-5e3d-a20e-7e051a28014e | ||||||
| 4 | 6 | 4 | 0 | 0 | 0399590595 | 9780399590597 | One World | A young man who was gifted with a mysterious power becomes part of a war between slavers and the enslaved. | 0 | THE WATER DANCER | Ta-Nehisi Coates | by Ta-Nehisi Coates | https://s1.nyt.com/du/books/images/9780399590597.jpg | 326 | 495 | https://www.amazon.com/Water-Dancer-Oprahs-Book-Club/dp/0399590595?tag=NYTBS-20 | https://www.nytimes.com/2019/09/20/books/review-water-dancer-ta-nehisi-coates.html | list(isbn10 = c(“0399590595”, “0525494847”), isbn13 = c(“9780399590597”, “9780525494843”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Water-Dancer-Oprahs-Book-Club/dp/0399590595?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=THE+WATER+DANCER&author=Ta-Nehisi+Coates”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780399590597”, “http://www.indiebound.org/book/9780399590597?aff=NYT”)) | nyt://book/30d61b66-0e7d-582c-a3ab-ddce005d5361 | |||||
| 5 | 5 | 6 | 0 | 0 | 1982110562 | 9781982110567 | Scribner | Children with special talents are abducted and sequestered in an institution where the sinister staff seeks to extract their gifts through harsh methods. | 0 | THE INSTITUTE | Stephen King | by Stephen King | https://s1.nyt.com/du/books/images/9781982110567.jpg | 328 | 495 | https://www.amazon.com/Institute-Novel-Stephen-King/dp/1982110562?tag=NYTBS-20 | https://www.nytimes.com/2019/09/08/books/review-institute-stephen-king.html | list(isbn10 = c(“1982110562”, “1982110597”, “1508279071”, “1508279063”, “1432870122”), isbn13 = c(“9781982110567”, “9781982110598”, “9781508279075”, “9781508279068”, “9781432870126”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Institute-Novel-Stephen-King/dp/1982110562?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=THE+INSTITUTE&author=Stephen+King”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9781982110567”, “http://www.indiebound.org/book/9781982110567?aff=NYT”)) | nyt://book/ef08eed0-f900-53fe-9764-da9152bd5e8f | |||||
| 6 | 0 | 1 | 0 | 0 | 0812996542 | 9780812996548 | Random House | In a follow-up to the Pulitzer Prize-winning novel “Olive Kitteridge,” new relationships, including a second marriage, are encountered in a seaside town in Maine. | 0 | OLIVE, AGAIN | Elizabeth Strout | by Elizabeth Strout | https://s1.nyt.com/du/books/images/9780812996548.jpg | 327 | 495 | https://www.amazon.com/Olive-Again-Novel-Elizabeth-Strout/dp/0812996542?tag=NYTBS-20 | list(isbn10 = “0812996542”, isbn13 = “9780812996548”) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Olive-Again-Novel-Elizabeth-Strout/dp/0812996542?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=OLIVE%2C+AGAIN&author=Elizabeth+Strout”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780812996548”, “http://www.indiebound.org/book/9780812996548?aff=NYT”)) | nyt://book/738ec7a9-1834-5cbf-a9e0-d53d9b7dd6e0 | ||||||
| 7 | 7 | 4 | 0 | 0 | 0062963678 | 9780062963673 | Harper | A sibling relationship is impacted when the family goes from poverty to wealth and back again over the course of many decades. | 0 | THE DUTCH HOUSE | Ann Patchett | by Ann Patchett | https://s1.nyt.com/du/books/images/9780062963673.jpg | 328 | 495 | https://www.amazon.com/Dutch-House-Novel-Ann-Patchett/dp/0062963678?tag=NYTBS-20 | https://www.nytimes.com/2019/09/21/books/review-dutch-house-ann-patchett.html | list(isbn10 = c(“0062963678”, “0062963694”), isbn13 = c(“9780062963673”, “9780062963697”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Dutch-House-Novel-Ann-Patchett/dp/0062963678?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=THE+DUTCH+HOUSE&author=Ann+Patchett”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780062963673”, “http://www.indiebound.org/book/9780062963673?aff=NYT”)) | nyt://book/0ad5db6c-4d15-5624-bef3-1385f93f78e2 | |||||
| 8 | 8 | 6 | 0 | 0 | 0385543786 | 9780385543781 | Nan A. Talese/Doubleday | In a sequel to “The Handmaid’s Tale,” old secrets bring three women together as the Republic of Gilead’s theocratic regime shows signs of decay. | 0 | THE TESTAMENTS | Margaret Atwood | by Margaret Atwood | https://s1.nyt.com/du/books/images/9780385543781.jpg | 326 | 495 | https://www.amazon.com/Testaments-Sequel-Handmaids-Tale/dp/0385543786?tag=NYTBS-20 | list(isbn10 = c(“0385543786”, “0385543794”, “0525590455”), isbn13 = c(“9780385543781”, “9780385543798”, “9780525590453”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Testaments-Sequel-Handmaids-Tale/dp/0385543786?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=THE+TESTAMENTS&author=Margaret+Atwood”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780385543781”, “http://www.indiebound.org/book/9780385543781?aff=NYT”)) | nyt://book/e1e18717-04bb-51a2-942a-45743507535f | ||||||
| 9 | 4 | 2 | 0 | 0 | 1250313074 | 9781250313072 | Flatiron | After mysteriously surviving a multiple homicide, Galaxy Stern comes face to face with dark magic, murder and more at Yale University. | 0 | NINTH HOUSE | Leigh Bardugo | by Leigh Bardugo | https://s1.nyt.com/du/books/images/9781250313072.jpg | 325 | 495 | https://www.amazon.com/Ninth-House-Leigh-Bardugo/dp/1250313074?tag=NYTBS-20 | https://www.nytimes.com/2019/10/08/books/review/ninth-house-leigh-bardugo.html | list(isbn10 = c(“1250313074”, “1250266459”), isbn13 = c(“9781250313072”, “9781250266453”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Ninth-House-Leigh-Bardugo/dp/1250313074?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=NINTH+HOUSE&author=Leigh+Bardugo”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9781250313072”, “http://www.indiebound.org/book/9781250313072?aff=NYT”)) | nyt://book/65f94f17-7cfb-508e-99c4-ec09ead9948c | |||||
| 10 | 10 | 2 | 0 | 0 | 0399562486 | 9780399562488 | Pamela Dorman/Viking | In Depression-era America, five women refuse to be cowed by men or convention as they deliver books throughout the mountains of Kentucky. | 0 | THE GIVER OF STARS | Jojo Moyes | by Jojo Moyes | https://s1.nyt.com/du/books/images/9780399562501.jpg | 328 | 495 | https://www.amazon.com/Giver-Stars-Novel-Jojo-Moyes/dp/0399562486?tag=NYTBS-20 | list(isbn10 = c(“0399562486”, “0399562508”), isbn13 = c(“9780399562488”, “9780399562501”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Giver-Stars-Novel-Jojo-Moyes/dp/0399562486?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=THE+GIVER+OF+STARS&author=Jojo+Moyes”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780399562488”, “http://www.indiebound.org/book/9780399562488?aff=NYT”)) | nyt://book/0365e66e-8090-5ec1-8d8c-2ed89027d374 | ||||||
| 11 | 11 | 3 | 0 | 0 | 0525536612 | 9780525536611 | Putnam | The 12th book in the Virgil Flowers series. A fight between university departments turns deadly. | 0 | BLOODY GENIUS | John Sandford | by John Sandford | https://s1.nyt.com/du/books/images/9780525536611.jpg | 328 | 495 | https://www.amazon.com/Bloody-Genius-Virgil-Flowers-Novel/dp/0525536612?tag=NYTBS-20 | list(isbn10 = “0525536612”, isbn13 = “9780525536611”) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Bloody-Genius-Virgil-Flowers-Novel/dp/0525536612?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=BLOODY+GENIUS&author=John+Sandford”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780525536611”, “http://www.indiebound.org/book/9780525536611?aff=NYT”)) | nyt://book/fea821b6-6392-50b8-b5ae-048621b34a82 | ||||||
| 12 | 3 | 2 | 0 | 0 | 0316435570 | 9780316435574 | Little, Brown | In the sequel to “Winter in Paradise,” Irene Steele visits the island of St. John to get to the bottom of the mysterious life and death of her husband. | 0 | WHAT HAPPENS IN PARADISE | Elin Hilderbrand | by Elin Hilderbrand | https://s1.nyt.com/du/books/images/9780316435574.jpg | 319 | 495 | https://www.amazon.com/What-Happens-Paradise-Elin-Hilderbrand/dp/0316435570?tag=NYTBS-20 | list(isbn10 = c(“0316435570”, “0316426075”), isbn13 = c(“9780316435574”, “9780316426077”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/What-Happens-Paradise-Elin-Hilderbrand/dp/0316435570?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=WHAT+HAPPENS+IN+PARADISE&author=Elin+Hilderbrand”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780316435574”, “http://www.indiebound.org/book/9780316435574?aff=NYT”)) | nyt://book/8ba462d1-3a6a-5728-8922-532bbcf58680 | ||||||
| 13 | 12 | 4 | 0 | 0 | 1501190628 | 9781501190629 | Emily Bestler/Atria | Mexican cartels, ISIS and a possible pandemic bring Mitch Rapp back into action. | 0 | VINCE FLYNN: LETHAL AGENT | Kyle Mills | by Kyle Mills | https://s1.nyt.com/du/books/images/9781501190643.jpg | 326 | 495 | https://www.amazon.com/Lethal-Agent-Mitch-Rapp-Novel-ebook/dp/B07NBJSFFJ?tag=NYTBS-20 | list(isbn10 = c(“1501190628”, “1982140127”, “1501190644”, “1508287716”, “1508287708”), isbn13 = c(“9781501190629”, “9781982140120”, “9781501190643”, “9781508287711”, “9781508287704”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Lethal-Agent-Mitch-Rapp-Novel-ebook/dp/B07NBJSFFJ?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=VINCE+FLYNN%3A+LETHAL+AGENT&author=Kyle+Mills”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9781501190629”, “http://www.indiebound.org/book/9781501190629?aff=NYT”)) | nyt://book/86e28126-9c5c-5a00-b21f-80352e72ca2d | ||||||
| 14 | 13 | 3 | 0 | 0 | 1250265703 | 9781250265708 | St. Martin’s | A 16-year-old, who sleeps with a concentration camp commandant in order to survive, is sentenced to a Siberian prison camp where she cares for the ill. | 0 | CILKA’S JOURNEY | Heather Morris | by Heather Morris | https://s1.nyt.com/du/books/images/9781250265708.jpg | 326 | 495 | https://www.amazon.com/Cilkas-Journey-Novel-Tattooist-Auschwitz/dp/1250265703?tag=NYTBS-20 | list(isbn10 = c(“1250265703”, “1250265797”), isbn13 = c(“9781250265708”, “9781250265791”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Cilkas-Journey-Novel-Tattooist-Auschwitz/dp/1250265703?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=CILKA%27S+JOURNEY&author=Heather+Morris”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9781250265708”, “http://www.indiebound.org/book/9781250265708?aff=NYT”)) | nyt://book/014cbbe5-9d31-59d6-9f09-dd39d04421a0 | ||||||
| 15 | 9 | 2 | 0 | 0 | 039917950X | 9780399179501 | Delacorte | A prestigious New York lawyer learns that the life she constructed for her children and herself isn’t exactly what she thought it was. | 0 | CHILD’S PLAY | Danielle Steel | by Danielle Steel | https://s1.nyt.com/du/books/images/9780399179518.jpg | 326 | 495 | https://www.amazon.com/Childs-Play-Novel-Danielle-Steel-ebook/dp/B07MGDJT1N?tag=NYTBS-20 | list(isbn10 = c(“039917950X”, “0399179518”), isbn13 = c(“9780399179501”, “9780399179518”)) | list(name = c(“Amazon”, “Apple Books”, “Barnes and Noble”, “Local Booksellers”), url = c(“https://www.amazon.com/Childs-Play-Novel-Danielle-Steel-ebook/dp/B07MGDJT1N?tag=NYTBS-20”, “http://du-gae-books-dot-nyt-du-prd.appspot.com/buy?title=CHILD%27S+PLAY&author=Danielle+Steel”, “http://www.anrdoezrs.net/click-7990613-11819508?url=http%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780399179501”, “http://www.indiebound.org/book/9780399179501?aff=NYT”)) | nyt://book/7425ca98-5374-5a6c-8381-890b33710a6b |
| rank | publisher | description | title | author | contributor | amazon_product_url |
|---|---|---|---|---|---|---|
| 1 | Doubleday | Cullen Post, a lawyer and Episcopal minister, antagonizes some ruthless killers when he takes on a wrongful conviction case. | THE GUARDIANS | John Grisham | by John Grisham | https://www.amazon.com/Guardians-Novel-John-Grisham/dp/0385544189?tag=NYTBS-20 |
| 2 | Putnam | In a quiet town on the North Carolina coast in 1969, a young woman who survived alone in the marsh becomes a murder suspect. | WHERE THE CRAWDADS SING | Delia Owens | by Delia Owens | https://www.amazon.com/Where-Crawdads-Sing-Delia-Owens/dp/0735219095?tag=NYTBS-20 |
| 3 | Little, Brown | In the 19th installment of the Women’s Murder Club series, detective Lindsay Boxer and company take on a fearsome criminal known only as “Loman.” | THE 19TH CHRISTMAS | James Patterson and Maxine Paetro | by James Patterson and Maxine Paetro | https://www.amazon.com/19th-Christmas-Womens-Murder-Club-ebook/dp/B07P1SSF9S?tag=NYTBS-20 |
| 4 | One World | A young man who was gifted with a mysterious power becomes part of a war between slavers and the enslaved. | THE WATER DANCER | Ta-Nehisi Coates | by Ta-Nehisi Coates | https://www.amazon.com/Water-Dancer-Oprahs-Book-Club/dp/0399590595?tag=NYTBS-20 |
| 5 | Scribner | Children with special talents are abducted and sequestered in an institution where the sinister staff seeks to extract their gifts through harsh methods. | THE INSTITUTE | Stephen King | by Stephen King | https://www.amazon.com/Institute-Novel-Stephen-King/dp/1982110562?tag=NYTBS-20 |
| 6 | Random House | In a follow-up to the Pulitzer Prize-winning novel “Olive Kitteridge,” new relationships, including a second marriage, are encountered in a seaside town in Maine. | OLIVE, AGAIN | Elizabeth Strout | by Elizabeth Strout | https://www.amazon.com/Olive-Again-Novel-Elizabeth-Strout/dp/0812996542?tag=NYTBS-20 |
| 7 | Harper | A sibling relationship is impacted when the family goes from poverty to wealth and back again over the course of many decades. | THE DUTCH HOUSE | Ann Patchett | by Ann Patchett | https://www.amazon.com/Dutch-House-Novel-Ann-Patchett/dp/0062963678?tag=NYTBS-20 |
| 8 | Nan A. Talese/Doubleday | In a sequel to “The Handmaid’s Tale,” old secrets bring three women together as the Republic of Gilead’s theocratic regime shows signs of decay. | THE TESTAMENTS | Margaret Atwood | by Margaret Atwood | https://www.amazon.com/Testaments-Sequel-Handmaids-Tale/dp/0385543786?tag=NYTBS-20 |
| 9 | Flatiron | After mysteriously surviving a multiple homicide, Galaxy Stern comes face to face with dark magic, murder and more at Yale University. | NINTH HOUSE | Leigh Bardugo | by Leigh Bardugo | https://www.amazon.com/Ninth-House-Leigh-Bardugo/dp/1250313074?tag=NYTBS-20 |
| 10 | Pamela Dorman/Viking | In Depression-era America, five women refuse to be cowed by men or convention as they deliver books throughout the mountains of Kentucky. | THE GIVER OF STARS | Jojo Moyes | by Jojo Moyes | https://www.amazon.com/Giver-Stars-Novel-Jojo-Moyes/dp/0399562486?tag=NYTBS-20 |
| 11 | Putnam | The 12th book in the Virgil Flowers series. A fight between university departments turns deadly. | BLOODY GENIUS | John Sandford | by John Sandford | https://www.amazon.com/Bloody-Genius-Virgil-Flowers-Novel/dp/0525536612?tag=NYTBS-20 |
| 12 | Little, Brown | In the sequel to “Winter in Paradise,” Irene Steele visits the island of St. John to get to the bottom of the mysterious life and death of her husband. | WHAT HAPPENS IN PARADISE | Elin Hilderbrand | by Elin Hilderbrand | https://www.amazon.com/What-Happens-Paradise-Elin-Hilderbrand/dp/0316435570?tag=NYTBS-20 |
| 13 | Emily Bestler/Atria | Mexican cartels, ISIS and a possible pandemic bring Mitch Rapp back into action. | VINCE FLYNN: LETHAL AGENT | Kyle Mills | by Kyle Mills | https://www.amazon.com/Lethal-Agent-Mitch-Rapp-Novel-ebook/dp/B07NBJSFFJ?tag=NYTBS-20 |
| 14 | St. Martin’s | A 16-year-old, who sleeps with a concentration camp commandant in order to survive, is sentenced to a Siberian prison camp where she cares for the ill. | CILKA’S JOURNEY | Heather Morris | by Heather Morris | https://www.amazon.com/Cilkas-Journey-Novel-Tattooist-Auschwitz/dp/1250265703?tag=NYTBS-20 |
| 15 | Delacorte | A prestigious New York lawyer learns that the life she constructed for her children and herself isn’t exactly what she thought it was. | CHILD’S PLAY | Danielle Steel | by Danielle Steel | https://www.amazon.com/Childs-Play-Novel-Danielle-Steel-ebook/dp/B07MGDJT1N?tag=NYTBS-20 |
II Articles Search API
Article Search API provides faclity to look up articles by keyword, which can be refined by using filters and facets.
Setting up parameters
Extraction of chosen articles
There were 36 hits, with 10 hits per page. In max_pages, we computed the number of pages to be 3.
article_url <- paste0("http://api.nytimes.com/svc/search/v2/articlesearch.json?q=",search_item,"&begin_date=",begin_date,"&end_date=",end_date,"&facet_filter=true&api-key=", NYTKEY, sep="")
query <- fromJSON(article_url)
max_pages <- round((query$response$meta$hits[1] / 10) - 1) # Computing the number of pagesExtraction of chosen articles
Now, we create a dictionary, called pages, and store the individual pages in it.
pages <- list()
for(i in 0:max_pages){
nyt_search <- fromJSON(paste0(article_url, "&page=", i), flatten = TRUE) %>% data.frame()
message("Retrieving page ", i + 1)
pages[[i + 1]] <- nyt_search
Sys.sleep(2)
}## Retrieving page 1## Retrieving page 2## Retrieving page 3## Retrieving page 4r-binding all pages into one big data frame, whose class is shown below. At this point, we achieved our aim of extracting news into a data frame.
all_pages <- rbind_pages(pages) # Now, we rbind the individual pages into one big data frame
class(all_pages) # This is a data frame## [1] "data.frame"Data Visualization
Now, we’ll explore what data visualization we can do.
all_pages %>%
group_by(response.docs.type_of_material) %>%
summarize(count = n()) %>%
mutate(percent = (count / sum(count)) * 100) %>%
ggplot() +
geom_bar(aes(y = percent, x = reorder(response.docs.type_of_material, percent), fill = response.docs.type_of_material), stat = "identity") + coord_flip()
We observe that majority of the articles are in news category, followed by Op-Ed.
In the following we’ll explore a section wise analysis of news about Iran Nuclear deal.
all_pages %>%
group_by(response.docs.section_name) %>%
summarize(count = n()) %>%
mutate(percent = (count / sum(count)) * 100) %>%
ggplot() +
geom_bar(aes(y = percent, x = reorder(response.docs.section_name, percent), fill = response.docs.section_name), stat = "identity") + coord_flip()
We conclude that the World section contains the maximum amount of responses.