API

The New York Times web site provides a rich set of APIs, as described here: http://developer.nytimes.com/docs You’ll need to start by signing up for an API key. Your task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it to an R dataframe.

I have selected article search API with search query - Brexit as it is very “hot” topic in the UK now.

Loading necessary library

library(jsonlite)

Signing up for an API key

api_key<-"?api-key=ec35a8674ec64bc2b24217d98784448a"

Setting up search criterias

q<-"&q=brexit"
begin_date<-"&begin_date=20180901"
end_date<-"&end_date=20181020"
page<-"&page="

Reading JSON and transforming data to R data frame

df<-data.frame()
 # getting 30 page (10 results per page)
for (i in 1:30){
  # constructing URL from search criterias
    url <- paste0('https://api.nytimes.com/svc/search/v2/articlesearch.json',api_key,q,begin_date,end_date,page,toString(i))
    data<-fromJSON(url,flatten = TRUE)
    # selecting only necessary columns from JSON file
    docs.df <- data$response$docs
    for(k in 1:nrow(docs.df)){
      temp_df <- data.frame(headline=docs.df[k,"headline.main"],source=docs.df[k,"source"],section=docs.df[k,"section_name"],news_description=docs.df[k,"news_desk"],document_type=docs.df[k,"document_type"])
      df <- rbind(df,temp_df)
    }
    Sys.sleep(2)
    print(paste("finished page ",i))
}

## [1] "finished page  1"
## [1] "finished page  2"
## [1] "finished page  3"
## [1] "finished page  4"
## [1] "finished page  5"
## [1] "finished page  6"
## [1] "finished page  7"
## [1] "finished page  8"
## [1] "finished page  9"
## [1] "finished page  10"
## [1] "finished page  11"
## [1] "finished page  12"
## [1] "finished page  13"
## [1] "finished page  14"
## [1] "finished page  15"
## [1] "finished page  16"
## [1] "finished page  17"
## [1] "finished page  18"
## [1] "finished page  19"
## [1] "finished page  20"
## [1] "finished page  21"
## [1] "finished page  22"
## [1] "finished page  23"
## [1] "finished page  24"
## [1] "finished page  25"
## [1] "finished page  26"
## [1] "finished page  27"
## [1] "finished page  28"
## [1] "finished page  29"
## [1] "finished page  30"

head(df)

##                                                       headline  source
## 1            Britain's Labour Will Not Vote for 'Blind Brexit' Reuters
## 2            EU Leaders Prepare Hardball Brexit Summit for May Reuters
## 3            Ireland Insists on Brexit 'Backstop': PM Varadkar Reuters
## 4    The Latest: Austria's Kurz Sees Border as Brexit 'Tactic'      AP
## 5 EU Negotiator Calls for 'Orderly Brexit' With Irish Backstop Reuters
## 6      DUP Brexit Spokesman Says No Deal 'Probably Inevitable' Reuters
##   section news_description document_type
## 1  Europe             None       article
## 2  Europe             None       article
## 3  Europe             None       article
## 4  Europe             None       article
## 5  Europe             None       article
## 6  Europe             None       article