Using the Article Search API from New York Times APIs.
Constructed an interface in R to read in the JSON data, and transformed it to an R dataframe.
library(httr)
library(jsonlite)
library(knitr)
library(XML)
library(plyr)
my_url = "https://api.nytimes.com/svc/search/v2/articlesearch.json"
# From API documentation, we can pass fields we're interested in
myjson <- GET(my_url,query = list(api_key = "06fa94b122274699adb1f46dedad1be2", order = "by-title", q= "\"data science\"", begin_date = "20160101" , sort = "newest",
offset = 20 , fl="web_url,snippet,headline,pub_date,pub_date, news_desk"))
# convert the data into a readable format
json_data <- fromJSON(paste(myjson, collapse=""))
#put into data fram
jsonDF <- data.frame(json_data)
# get the columns were interested in
jsonDF <- jsonDF[3:7]
jsonDF## response.docs.web_url
## 1 https://www.nytimes.com/reuters/2017/10/22/business/22reuters-ecommerce-seasia.html
## 2 https://www.nytimes.com/2017/10/22/fashion/weddings/rebecca-martin-jared-lander.html
## 3 https://www.nytimes.com/2017/10/19/business/dealbook/ken-chenault-american-express.html
## 4 https://www.nytimes.com/2017/10/05/opinion/privacy-rights-security-breaches.html
## 5 https://www.nytimes.com/2017/09/03/fashion/weddings/bhavna-changrani-sameer-punyani.html
## 6 https://www.nytimes.com/2017/08/24/technology/coding-boot-camps-close.html
## 7 https://www.nytimes.com/2017/08/10/science/instagram-mental-health-depression.html
## 8 https://www.nytimes.com/2017/07/29/pageoneplus/corrections-july-30-2017.html
## 9 https://www.nytimes.com/2017/06/27/technology/education-partovi-computer-science-coding-apple-microsoft.html
## 10 https://www.nytimes.com/2017/06/07/education/with-innovation-colleges-fill-the-skills-gap.html
## response.docs.snippet
## 1 When diaper maker DSG International (Thailand) wants to know what its customers are thinking, it often turns to Lazada, an e-commerce firm majority-owned by Alibaba Group Holding.
## 2 The couple met in New York in May 2014 at a meet-up about statistical programming organized by the groom.
## 3 When the American Express C.E.O. steps down in February, there may be just three black men running Fortune 500 companies.
## 4 The Yahoo breach is another reminder that the right to privacy is dying, if not already dead.
## 5 The couple met on the eve of the huge snowstorm in 2016 that shut down Washington for days.
## 6 Successful schools, analysts say, will increasingly be the ones that expand their programs to suit the changing needs of employers.
## 7 From the colors in photos to the filters chosen, Instagram users with a history of depression present the world differently, a study suggests.
## 8 Corrections appearing in print on Sunday, July 30, 2017.
## 9 Code.org, a tech-backed nonprofit, is pressing schools to teach computer science. But are tech firms swaying education to serve their own interests?
## 10 Many employers say college graduates often do not have the skills required for a job. Some colleges are working to solve that problem.
## response.docs.headline.main
## 1 Consumer Goods Firms Harness Online Data to Tap Southeast Asia e-Commerce Boom
## 2 Rebecca Martin, Jared Lander
## 3 Ken Chenaults Retirement Makes a Small Club Smaller: DealBook Briefing
## 4 The End of Privacy
## 5 Bhavna Changrani, Sameer Punyani
## 6 As Coding Boot Camps Close, the Field Faces a Reality Check
## 7 Your Instagram Posts May Hold Clues to Your Mental Health
## 8 Corrections: July 30, 2017
## 9 How Silicon Valley Pushed Coding Into American Classrooms
## 10 With Innovation, Colleges Fill the Skills Gap
## response.docs.headline.print_headline
## 1 Consumer Goods Firms Harness Online Data to Tap Southeast Asia e-Commerce Boom
## 2 Rebecca Martin, Jared Lander
## 3 Ken Chenaults Retirement Makes a Small Club Smaller: DealBook Briefing
## 4 How Privacy as We Knew It Died
## 5 Bhavna Changrani, Sameer Punyani
## 6 Beyond Basic Training
## 7 Your Instagram Posts May Hold Clues to Your Mental Health
## 8 Corrections
## 9 Silicon Valley Plugs Coding Into Schools
## 10 Skills-Gap Fixes
## response.docs.headline.kicker response.docs.pub_date
## 1 <NA> 2017-10-22T23:15:27+0000
## 2 <NA> 2017-10-22T04:17:00+0000
## 3 <NA> 2017-10-19T11:00:21+0000
## 4 Op-Ed Contributors 2017-10-05T23:26:59+0000
## 5 <NA> 2017-09-03T04:15:03+0000
## 6 <NA> 2017-08-24T17:05:12+0000
## 7 <NA> 2017-08-10T13:41:07+0000
## 8 <NA> 2017-07-30T02:52:21+0000
## 9 Education Disrupted 2017-06-27T09:00:38+0000
## 10 Closing the gap 2017-06-07T10:00:41+0000
## response.docs.new_desk
## 1 None
## 2 Society
## 3 Business
## 4 OpEd
## 5 Society
## 6 Business
## 7 Science
## 8 Corrections
## 9 Business
## 10 SpecialSections