The New York Times web site provides a rich set of APIs, as described here: http://developer.nytimes.com/docs You’ll need to start by signing up for an API key.
Your task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it to an R dataframe.
API Key = 473651b17665485b98798261318ca388
Pavan was kind enough to share a resource with the rest of the class. There was a specific example listed here with the NY Times. A lot of my information came from this cran webpage. https://cran.r-project.org/web/packages/jsonlite/vignettes/json-apis.html
Load libraries.
library(httr)
library(stringr)
library(jsonlite)## Warning: package 'jsonlite' was built under R version 3.3.2# We'll use a query search for "Donald Trump" for our Top Stories NY Times API listing.
query <- "Donald Trump"
# Will need to modify the query so that way the API understands our search. This is done by deleting the space and adding in the "+". This is done through the gsub function via stringr package.
query1 <- gsub("\\s", "+", query)
# Create an api.key (that was obtained from the NY Times website
api.key <- "&api-key=473651b17665485b98798261318ca388"
# Paste together the URL with the query and with the API key
url <- paste("http://api.nytimes.com/svc/topstories/v2/home.json?",query1, sep="")
url.withkey <- paste(url,api.key, sep="")
# Let's make sure that the information we are getting from this API is in the JSON format (as asked in the question stem). We will use the GET function from the httr package.
nyt.info <- GET("http://api.nytimes.com/svc/topstories/v2/home.json")
paste("The information is saved as:",nyt.info$headers$`content-type`)## [1] "The information is saved as: application/json; charset=utf-8"# As you notice, this API is in the JSON format. Therefore, we can take advantage of the jsonlite package to create a data frame, using this simple function of fromJSON()
nyt.json <- fromJSON(url.withkey)
names(nyt.json)## [1] "status" "copyright" "section" "last_updated"
## [5] "num_results" "results"# The results section will contain the pertinant informaiont to create our data frame.
paste("There are", nyt.json$num_results, "articles in the Top Stories Section of the NY Times.")## [1] "There are 34 articles in the Top Stories Section of the NY Times."# Create the data frame.
nyt.df <- nyt.json$results
head(nyt.df)## section subsection
## 1 U.S. Politics
## 2 U.S. Politics
## 3 U.S. Politics
## 4 Business Day Energy & Environment
## 5 U.S.
## 6 Business Day
## title
## 1 2 White House Officials Helped Give Nunes Intelligence Reports
## 2 Trump’s Threats Against Freedom Caucus Cause Few Shivers of Fear
## 3 The Roots of the Battle Over Neil Gorsuch: ‘They Started It’
## 4 Trump Critics on Climate Policy Hope Executives Can Sway Him
## 5 Trump University Suit Settlement Approved by Judge
## 6 Also Made in Mexico: Lifesaving Medical Devices
## abstract
## 1 The revelation fueled criticism that Representative Devin Nunes has been too eager to do the White House’s bidding, not run an independent investigation
## 2 The president’s attack on fellow Republicans was only the latest and most high profile in the increasingly open warfare within the party.
## 3 The fight over President Trump’s Supreme Court nominee reflects years of creeping institutional shifts, partisan tensions and hypocritical arguments.
## 4 Companies that have embraced environmental goals are represented on White House councils, offering influence that advocacy groups hope they’ll use.
## 5 The approval of the settlement, assuming it stands, brings to a close a case that garnered outsize national attention during the presidential race.
## 6 The medical supply industry makes a particularly revelatory case study of the difficulties of untangling global trade.
## url
## 1 https://www.nytimes.com/2017/03/30/us/politics/devin-nunes-intelligence-reports.html
## 2 https://www.nytimes.com/2017/03/30/us/politics/donald-trump-freedom-caucus.html
## 3 https://www.nytimes.com/2017/03/31/us/politics/supreme-court-neil-gorsuch-senate.html
## 4 https://www.nytimes.com/2017/03/30/business/energy-environment/trump-business-climate-policy.html
## 5 https://www.nytimes.com/2017/03/31/us/trump-university-settlement.html
## 6 https://www.nytimes.com/2017/03/31/business/us-mexico-trade-medical-devices.html
## byline item_type
## 1 By MATTHEW ROSENBERG, MAGGIE HABERMAN and ADAM GOLDMAN Article
## 2 By JONATHAN MARTIN and JENNIFER STEINHAUER Article
## 3 By MATT FLEGENHEIMER Article
## 4 By HIROKO TABUCHI and DIANE CARDWELL Article
## 5 By STEVE EDER and JENNIFER MEDINA Article
## 6 By SARAH VARNEY Article
## updated_date created_date
## 1 2017-03-30T18:20:22-04:00 2017-03-30T13:16:51-04:00
## 2 2017-03-30T16:21:08-04:00 2017-03-30T16:21:10-04:00
## 3 2017-03-31T05:00:03-04:00 2017-03-31T05:00:04-04:00
## 4 2017-03-30T20:52:30-04:00 2017-03-30T20:52:31-04:00
## 5 2017-03-31T12:33:15-04:00 2017-03-31T12:33:16-04:00
## 6 2017-03-31T07:00:26-04:00 2017-03-31T07:00:27-04:00
## published_date material_type_facet kicker
## 1 2017-03-30T13:16:51-04:00
## 2 2017-03-30T16:21:10-04:00
## 3 2017-03-31T05:00:04-04:00
## 4 2017-03-30T20:52:31-04:00
## 5 2017-03-31T12:33:16-04:00
## 6 2017-03-31T07:00:27-04:00
## des_facet
## 1 United States Politics and Government, Classified Information and State Secrets, Espionage and Intelligence Services
## 2 United States Politics and Government, Law and Legislation
## 3 United States Politics and Government, Appointments and Executive Changes, Filibusters and Debate Curbs
## 4 Greenhouse Gas Emissions, United States Politics and Government, Global Warming, Regulation and Deregulation of Industry, United Nations Framework Convention on Climate Change
## 5 Suits and Litigation (Civil), Decisions and Verdicts
## 6 Medical Devices, International Trade and World Market, Factories and Manufacturing, Protectionism (Trade), North American Free Trade Agreement
## org_facet
## 1 House Committee on Intelligence
## 2 House Freedom Caucus, Republican Party, Twitter
## 3 Supreme Court (US), Senate
## 4 General Motors
## 5 Trump University
## 6
## per_facet geo_facet
## 1 Nunes, Devin G, Trump, Donald J Russia
## 2 Ryan, Paul D Jr, Sanford, Mark United States
## 3 Gorsuch, Neil M, Garland, Merrick B, Trump, Donald J
## 4 Immelt, Jeffrey R, Trump, Donald J
## 5 Curiel, Gonzalo P
## 6 Mexico
## multimedia
## 1 https://static01.nyt.com/images/2017/03/31/automobiles/31NUNES-01/31NUNES-01-thumbStandard.jpg, https://static01.nyt.com/images/2017/03/31/automobiles/31NUNES-01/31NUNES-01-thumbLarge.jpg, https://static01.nyt.com/images/2017/03/31/automobiles/31NUNES-01/31NUNES-01-articleInline.jpg, https://static01.nyt.com/images/2017/03/31/automobiles/31NUNES-01/31NUNES-01-mediumThreeByTwo210.jpg, https://static01.nyt.com/images/2017/03/31/automobiles/31NUNES-01/31NUNES-01-superJumbo.jpg, Standard Thumbnail, thumbLarge, Normal, mediumThreeByTwo210, superJumbo, 75, 150, 127, 140, 1365, 75, 150, 190, 210, 2048, image, image, image, image, image, photo, photo, photo, photo, photo, Representative Devin Nunes of California, a Republican and the chairman of the House Intelligence Committee, arrived for a meeting Tuesday on Capitol Hill., Representative Devin Nunes of California, a Republican and the chairman of the House Intelligence Committee, arrived for a meeting Tuesday on Capitol Hill., Representative Devin Nunes of California, a Republican and the chairman of the House Intelligence Committee, arrived for a meeting Tuesday on Capitol Hill., Representative Devin Nunes of California, a Republican and the chairman of the House Intelligence Committee, arrived for a meeting Tuesday on Capitol Hill., Representative Devin Nunes of California, a Republican and the chairman of the House Intelligence Committee, arrived for a meeting Tuesday on Capitol Hill., Doug Mills/The New York Times, Doug Mills/The New York Times, Doug Mills/The New York Times, Doug Mills/The New York Times, Doug Mills/The New York Times
## 2 https://static01.nyt.com/images/2017/03/31/us/31freedom/31freedom-thumbStandard.jpg, https://static01.nyt.com/images/2017/03/31/us/31freedom/31freedom-thumbLarge.jpg, https://static01.nyt.com/images/2017/03/31/us/31freedom/31freedom-articleInline.jpg, https://static01.nyt.com/images/2017/03/31/us/31freedom/31freedom-mediumThreeByTwo210.jpg, https://static01.nyt.com/images/2017/03/31/us/31freedom/31freedom-superJumbo.jpg, Standard Thumbnail, thumbLarge, Normal, mediumThreeByTwo210, superJumbo, 75, 150, 127, 140, 1365, 75, 150, 190, 210, 2048, image, image, image, image, image, photo, photo, photo, photo, photo, Representatives Mike Johnson, left, and Mark Sanford, members of the House Freedom Caucus., Representatives Mike Johnson, left, and Mark Sanford, members of the House Freedom Caucus., Representatives Mike Johnson, left, and Mark Sanford, members of the House Freedom Caucus., Representatives Mike Johnson, left, and Mark Sanford, members of the House Freedom Caucus., Representatives Mike Johnson, left, and Mark Sanford, members of the House Freedom Caucus., Evan Vucci/Associated Press, Evan Vucci/Associated Press, Evan Vucci/Associated Press, Evan Vucci/Associated Press, Evan Vucci/Associated Press
## 3 https://static01.nyt.com/images/2017/04/01/us/01GORSUCH-flags/31GORSUCH-flags-thumbStandard.jpg, https://static01.nyt.com/images/2017/04/01/us/01GORSUCH-flags/31GORSUCH-flags-thumbLarge.jpg, https://static01.nyt.com/images/2017/04/01/us/01GORSUCH-flags/31GORSUCH-flags-articleInline.jpg, https://static01.nyt.com/images/2017/04/01/us/01GORSUCH-flags/31GORSUCH-flags-mediumThreeByTwo210.jpg, https://static01.nyt.com/images/2017/04/01/us/01GORSUCH-flags/31GORSUCH-flags-superJumbo.jpg, Standard Thumbnail, thumbLarge, Normal, mediumThreeByTwo210, superJumbo, 75, 150, 127, 140, 1367, 75, 150, 190, 210, 2048, image, image, image, image, image, photo, photo, photo, photo, photo, Backers of Judge Neil M. Gorsuch, President Trump’s Supreme Court pick, showed their support outside the court on Wednesday., Backers of Judge Neil M. Gorsuch, President Trump’s Supreme Court pick, showed their support outside the court on Wednesday., Backers of Judge Neil M. Gorsuch, President Trump’s Supreme Court pick, showed their support outside the court on Wednesday., Backers of Judge Neil M. Gorsuch, President Trump’s Supreme Court pick, showed their support outside the court on Wednesday., Backers of Judge Neil M. Gorsuch, President Trump’s Supreme Court pick, showed their support outside the court on Wednesday., Gabriella Demczuk for The New York Times, Gabriella Demczuk for The New York Times, Gabriella Demczuk for The New York Times, Gabriella Demczuk for The New York Times, Gabriella Demczuk for The New York Times
## 4 https://static01.nyt.com/images/2017/03/31/us/31CLIMATEBIZ/31CLIMATEBIZ-thumbStandard.jpg, https://static01.nyt.com/images/2017/03/31/us/31CLIMATEBIZ/31CLIMATEBIZ-thumbLarge.jpg, https://static01.nyt.com/images/2017/03/31/us/31CLIMATEBIZ/31CLIMATEBIZ-articleInline.jpg, https://static01.nyt.com/images/2017/03/31/us/31CLIMATEBIZ/31CLIMATEBIZ-mediumThreeByTwo210.jpg, https://static01.nyt.com/images/2017/03/31/us/31CLIMATEBIZ/31CLIMATEBIZ-superJumbo.jpg, Standard Thumbnail, thumbLarge, Normal, mediumThreeByTwo210, superJumbo, 75, 150, 127, 140, 1365, 75, 150, 190, 210, 2048, image, image, image, image, image, photo, photo, photo, photo, photo, Air and fuel heaters at General Electric’s Gas Turbine Technology Lab in Greenville, S.C. G.E.’s chief has been outspoken about clean energy., Air and fuel heaters at General Electric’s Gas Turbine Technology Lab in Greenville, S.C. G.E.’s chief has been outspoken about clean energy., Air and fuel heaters at General Electric’s Gas Turbine Technology Lab in Greenville, S.C. G.E.’s chief has been outspoken about clean energy., Air and fuel heaters at General Electric’s Gas Turbine Technology Lab in Greenville, S.C. G.E.’s chief has been outspoken about clean energy., Air and fuel heaters at General Electric’s Gas Turbine Technology Lab in Greenville, S.C. G.E.’s chief has been outspoken about clean energy., Jeremy M. Lange for The New York Times, Jeremy M. Lange for The New York Times, Jeremy M. Lange for The New York Times, Jeremy M. Lange for The New York Times, Jeremy M. Lange for The New York Times
## 5 https://static01.nyt.com/images/2017/04/01/us/01trumpuniversity/01trumpuniversity-thumbStandard.jpg, https://static01.nyt.com/images/2017/04/01/us/01trumpuniversity/01trumpuniversity-thumbLarge.jpg, https://static01.nyt.com/images/2017/04/01/us/01trumpuniversity/01trumpuniversity-articleInline.jpg, https://static01.nyt.com/images/2017/04/01/us/01trumpuniversity/01trumpuniversity-mediumThreeByTwo210.jpg, https://static01.nyt.com/images/2017/04/01/us/01trumpuniversity/01trumpuniversity-superJumbo.jpg, Standard Thumbnail, thumbLarge, Normal, mediumThreeByTwo210, superJumbo, 75, 150, 127, 140, 1365, 75, 150, 190, 210, 2048, image, image, image, image, image, photo, photo, photo, photo, photo, Jason Forge, center, with members of the legal team representing the plaintiffs in a lawsuit against Trump University, in San Diego on Thursday., Jason Forge, center, with members of the legal team representing the plaintiffs in a lawsuit against Trump University, in San Diego on Thursday., Jason Forge, center, with members of the legal team representing the plaintiffs in a lawsuit against Trump University, in San Diego on Thursday., Jason Forge, center, with members of the legal team representing the plaintiffs in a lawsuit against Trump University, in San Diego on Thursday., Jason Forge, center, with members of the legal team representing the plaintiffs in a lawsuit against Trump University, in San Diego on Thursday., Andrew Cullen for The New York Times, Andrew Cullen for The New York Times, Andrew Cullen for The New York Times, Andrew Cullen for The New York Times, Andrew Cullen for The New York Times
## 6 https://static01.nyt.com/images/2017/03/24/business/25DEVICES1/25DEVICES1-thumbStandard.jpg, https://static01.nyt.com/images/2017/03/24/business/25DEVICES1/25DEVICES1-thumbLarge.jpg, https://static01.nyt.com/images/2017/03/24/business/25DEVICES1/25DEVICES1-articleInline.jpg, https://static01.nyt.com/images/2017/03/24/business/25DEVICES1/25DEVICES1-mediumThreeByTwo210.jpg, https://static01.nyt.com/images/2017/03/24/business/25DEVICES1/25DEVICES1-superJumbo.jpg, Standard Thumbnail, thumbLarge, Normal, mediumThreeByTwo210, superJumbo, 75, 150, 152, 140, 1637, 75, 150, 190, 210, 2048, image, image, image, image, image, photo, photo, photo, photo, photo, Workers at a Greatbatch Medical plant in Tijuana, Mexico. It is one of a number of American-run companies in Tijuana that make millions of medical devices each day., Workers at a Greatbatch Medical plant in Tijuana, Mexico. It is one of a number of American-run companies in Tijuana that make millions of medical devices each day., Workers at a Greatbatch Medical plant in Tijuana, Mexico. It is one of a number of American-run companies in Tijuana that make millions of medical devices each day., Workers at a Greatbatch Medical plant in Tijuana, Mexico. It is one of a number of American-run companies in Tijuana that make millions of medical devices each day., Workers at a Greatbatch Medical plant in Tijuana, Mexico. It is one of a number of American-run companies in Tijuana that make millions of medical devices each day., John Francis Peters for The New York Times, John Francis Peters for The New York Times, John Francis Peters for The New York Times, John Francis Peters for The New York Times, John Francis Peters for The New York Times
## short_url
## 1 https://nyti.ms/2nEkrFF
## 2 https://nyti.ms/2oDIORR
## 3 https://nyti.ms/2nGyvi4
## 4 https://nyti.ms/2oEFJk7
## 5 https://nyti.ms/2nE45LI
## 6 https://nyti.ms/2nGRGIHclass(nyt.df)## [1] "data.frame"We have successfully used an API, downloaded the JSON information, and converted into a data frame.
# If you were looking for a specific article title, we can use this data frame like any other data frame.
# For example:
nyt.df$title[1]## [1] "2 White House Officials Helped Give Nunes Intelligence Reports"