Read and parse JSON data from NY Times Newswire API.

library(rjson)
library(RCurl)
library(rjson)
library(urltools)
library(dplyr)

Times Newswire API

Get the top 20 articles from the Times Newswire Service.

Compose the url string according to the format supported in the NYT API query. This is described on the NYT API FAQ pages. See http://developer.nytimes.com/timeswire_v3.json. A valid key for API queries must be obtained from the site prior to this.

apikey <- "e30dee57d7e44a4d8ee930e1cc06b927"
baseurl <- "http://api.nytimes.com/svc/news/v3/content/all/all.json?"
limit <- 20
url <- paste0(baseurl, "&api-key=", apikey, "&limit=", limit) 

# Issue the query:
jsondata <- fromJSON(file = url)

class(jsondata)
## [1] "list"
length(jsondata)
## [1] 4
names(jsondata)
## [1] "status"      "copyright"   "num_results" "results"

The data returned is in form of a list with 4 elements. The “status” element contains the status of the query. The “copyright” element contains the copyright notice. The payload is in the “results” element.

# Print the status.
jsondata$status
## [1] "OK"
# Print the copyright
jsondata$copyright
## [1] "Copyright (c) 2017 The New York Times Company.  All Rights Reserved."
# Print the length of "results"
length(jsondata$results)
## [1] 20

We now attempt to form a data frame containing each of the items in the “results” element of the list returned by the query. In the data frame, we include only the title, abstract, and url of each news item in the Newswire service.

items <- list()
ul <- list()
jdf <- data.frame("title" = character(),
                  "abstract" = character(),
                  "url" = character(),
                  stringsAsFactors = FALSE)

for (i in 1:length(jsondata$results)) {
    items[i] <- jsondata$results[i]
    ul <- unlist(items[i])
    
    df <- data.frame(ul[["title"]], ul[["abstract"]], ul[["url"]],
                     stringsAsFactors = FALSE)
    names(df) <- c("title", "abstract", "url")
    jdf <- rbind(jdf, df)
}

str(jdf)
## 'data.frame':    20 obs. of  3 variables:
##  $ title   : chr  "What to Ask About Hypertension" "What<U+0092>s on TV Saturday: Black Sabbath<U+0092>s Final Performance and <U+0091>Arrival<U+0092>" "Astros Rout Yu Darvish and Hold On to Beat the Dodgers in Game 3" "In Seoul, Mattis Accuses North Korea of <U+0091>Outlaw Behavior<U+0092>" ...
##  $ abstract: chr  "Talking points for patients and physicians." "Ozzy Osbourne, the Prince of Darkness, sings swan songs on Showtime. And Denis Villeneuve<U+0092>s sci-fi film makes a"| __truncated__ "Houston knocked Darvish out of the game before the second inning was over, then relied on strong defense and so"| __truncated__ "Defense Secretary Jim Mattis said that Pyongyang was accelerating the threat of nuclear war through missile tes"| __truncated__ ...
##  $ url     : chr  "https://www.nytimes.com/ref/health/healthguide/esn-hypertension-ask.html" "https://www.nytimes.com/2017/10/28/arts/television/whats-on-tv-saturday-black-sabbaths-final-performance-and-arrival.html" "https://www.nytimes.com/2017/10/28/sports/baseball/astros-dodgers-world-series.html" "https://www.nytimes.com/2017/10/28/world/asia/mattis-south-north-korea.html" ...
# Examine the first few elements.

head(jdf)
##                                                                    title
## 1                                         What to Ask About Hypertension
## 2 What<U+0092>s on TV Saturday: Black Sabbath<U+0092>s Final Performance and <U+0091>Arrival<U+0092>
## 3       Astros Rout Yu Darvish and Hold On to Beat the Dodgers in Game 3
## 4              In Seoul, Mattis Accuses North Korea of <U+0091>Outlaw Behavior<U+0092>
## 5                                          Corrections: October 28, 2017
## 6               In Oregon, a Little Hotel to Bring Out Your Inner Artist
##                                                                                                                                                                 abstract
## 1                                                                                                                            Talking points for patients and physicians.
## 2                         Ozzy Osbourne, the Prince of Darkness, sings swan songs on Showtime. And Denis Villeneuve<U+0092>s sci-fi film makes a timely case for communication.
## 3                Houston knocked Darvish out of the game before the second inning was over, then relied on strong defense and solid relief work to take the Series lead.
## 4 Defense Secretary Jim Mattis said that Pyongyang was accelerating the threat of nuclear war through missile tests and that the United States would protect its allies.
## 5                                                                                                          Corrections appearing in print on Saturday, October 28, 2017.
## 6                 Promoted as the <U+0093>World<U+0092>s First Kickstarter-Funded Hotel,<U+0094> the Jennings Hotel in Joseph, Ore., is the next best thing to staying in an artist<U+0092>s studio.
##                                                                                                                         url
## 1                                                  https://www.nytimes.com/ref/health/healthguide/esn-hypertension-ask.html
## 2 https://www.nytimes.com/2017/10/28/arts/television/whats-on-tv-saturday-black-sabbaths-final-performance-and-arrival.html
## 3                                       https://www.nytimes.com/2017/10/28/sports/baseball/astros-dodgers-world-series.html
## 4                                               https://www.nytimes.com/2017/10/28/world/asia/mattis-south-north-korea.html
## 5                                           https://www.nytimes.com/2017/10/28/pageoneplus/corrections-october-28-2017.html
## 6                                 https://www.nytimes.com/2017/10/28/travel/where-to-stay-joseph-oregon-jennings-hotel.html