The New York Times web site provides a rich set of APIs, as described here: http://developer.nytimes.com/docs You’ll need to start by signing up for an API key. The task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it to an R dataframe. I chose the “Article Search API” and queried articles writted about motorcycles since I recently got my motorcycle license.
suppressMessages(suppressWarnings(library(RCurl)))
suppressMessages(suppressWarnings(library(jsonlite)))
suppressMessages(suppressWarnings(library(tidyr)))
suppressMessages(suppressWarnings(library(dplyr)))
suppressMessages(suppressWarnings(library(ggplot2)))
suppressMessages(suppressWarnings(library(DT)))api_key <- '&api-key=fce90863096a4e99bebb64b7daded8e9'
search_word <- 'motorcycle'
nyt_url <- 'https://api.nytimes.com/svc/search/v2/articlesearch.json?q='query <- getURL(paste0(nyt_url, search_word, api_key))
json_nyt <- fromJSON(query)
json_nyt_df <- data.frame(json_nyt)
json_nyt_df## status
## 1 OK
## 2 OK
## 3 OK
## 4 OK
## 5 OK
## 6 OK
## 7 OK
## 8 OK
## 9 OK
## 10 OK
## copyright
## 1 Copyright (c) 2017 The New York Times Company. All Rights Reserved.
## 2 Copyright (c) 2017 The New York Times Company. All Rights Reserved.
## 3 Copyright (c) 2017 The New York Times Company. All Rights Reserved.
## 4 Copyright (c) 2017 The New York Times Company. All Rights Reserved.
## 5 Copyright (c) 2017 The New York Times Company. All Rights Reserved.
## 6 Copyright (c) 2017 The New York Times Company. All Rights Reserved.
## 7 Copyright (c) 2017 The New York Times Company. All Rights Reserved.
## 8 Copyright (c) 2017 The New York Times Company. All Rights Reserved.
## 9 Copyright (c) 2017 The New York Times Company. All Rights Reserved.
## 10 Copyright (c) 2017 The New York Times Company. All Rights Reserved.
## response.docs.web_url
## 1 https://www.nytimes.com/2017/09/18/sports/autoracing/ana-carrasco-motorcyle-racing.html
## 2 https://www.nytimes.com/2017/08/10/sports/autoracing/angel-nieto-dead-spanish-champion-motorcyclist.html
## 3 https://www.nytimes.com/2017/08/03/automobiles/harley-davidson-indian-motorcycles.html
## 4 https://www.nytimes.com/2017/08/14/movies/deadpool-2-stuntwoman-dies-motorcycle.html
## 5 https://www.nytimes.com/video/world/europe/100000005302787/vladimir-putin-combat-school-ukraine.html
## 6 https://www.nytimes.com/2017/06/19/insider/capturing-the-scene-at-200-mph.html
## 7 https://www.nytimes.com/aponline/2017/10/23/world/europe/ap-eu-italy-tree-on-taxi.html
## 8 https://www.nytimes.com/aponline/2017/10/23/us/ap-us-illinois-governor-2018.html
## 9 https://www.nytimes.com/aponline/2017/10/18/business/ap-us-tv-american-chopper.html
## 10 https://www.nytimes.com/interactive/2017/06/07/sports/Isle-of-man-TT-motorcycle-race.html
## response.docs.snippet
## 1 Ana Carrasco, a 20-year-old from Spain, has been riding since she was 3.
## 2 Nieto rose from poverty to hobnob with a king, while doing for motorcycle racing in Spain what Seve Ballesteros did for golf there.
## 3 Sales of Indian motorcycles, a revered brand that disappeared for decades, are rising while Harley sales are falling.
## 4 The superhero sequel was filming in Vancouver, British Columbia, where the police department confirmed the death and said it would investigate the incident.
## 5 As skirmishes in Ukraine continue, the Night Wolves — a Russian motorcycle club that is on friendly terms with President Vladimir V. Putin — run a combat training center just a few hours away.
## 6 Neil Collier on the Isle of Man TT motorcycle race and the logistics of filming a nerve-racking, fast-action sport.
## 7 A pine tree has crushed a taxi in Rome, injuring the driver but leaving two passengers uninjured.
## 8 Illinois Gov. Bruce Rauner has made official what's already widely known: He's seeking re-election for a second term.
## 9 "American Chopper" will return to the Discovery Channel next year, five years after its first run on the network ended.
## 10 The Isle of Man TT motorcycle race has claimed the lives of 146 riders since 1907. Here are the deadly turns, colorful characters and rich history along the course.
## response.docs.print_page response.docs.source
## 1 12 The New York Times
## 2 22 The New York Times
## 3 4 The New York Times
## 4 <NA> The New York Times
## 5 <NA> The New York Times
## 6 <NA> The New York Times
## 7 <NA> AP
## 8 <NA> AP
## 9 <NA> AP
## 10 0 The New York Times
## response.docs.multimedia
## 1 image, image, image, xlarge, wide, thumbnail, images/2017/09/19/sports/19MOTORCYCLEweb1/19MOTORCYCLEweb1-articleLarge.jpg, images/2017/09/19/sports/19MOTORCYCLEweb1/19MOTORCYCLEweb1-thumbWide.jpg, images/2017/09/19/sports/19MOTORCYCLEweb1/19MOTORCYCLEweb1-thumbStandard.jpg, 400, 126, 75, 600, 190, 75, 0, 0, 0, 600, NA, NA, images/2017/09/19/sports/19MOTORCYCLEweb1/19MOTORCYCLEweb1-articleLarge.jpg, NA, NA, 400, NA, NA, NA, images/2017/09/19/sports/19MOTORCYCLEweb1/19MOTORCYCLEweb1-thumbWide.jpg, NA, NA, 190, NA, NA, 126, NA, NA, NA, 75, NA, NA, images/2017/09/19/sports/19MOTORCYCLEweb1/19MOTORCYCLEweb1-thumbStandard.jpg, NA, NA, 75
## 2 image, image, image, xlarge, wide, thumbnail, images/2017/08/11/sports/11nieto-obit-1/09nieto-obit-1-articleLarge.jpg, images/2017/08/11/sports/11nieto-obit-1/09nieto-obit-1-thumbWide.jpg, images/2017/08/11/sports/11nieto-obit-1/09nieto-obit-1-thumbStandard.jpg, 420, 126, 75, 600, 190, 75, 0, 0, 0, 600, NA, NA, images/2017/08/11/sports/11nieto-obit-1/09nieto-obit-1-articleLarge.jpg, NA, NA, 420, NA, NA, NA, images/2017/08/11/sports/11nieto-obit-1/09nieto-obit-1-thumbWide.jpg, NA, NA, 190, NA, NA, 126, NA, NA, NA, 75, NA, NA, images/2017/08/11/sports/11nieto-obit-1/09nieto-obit-1-thumbStandard.jpg, NA, NA, 75
## 3 image, image, image, xlarge, wide, thumbnail, images/2017/08/04/business/04WHEELS1/04WHEELS1-articleLarge.jpg, images/2017/08/04/business/04WHEELS1/04WHEELS1-thumbWide.jpg, images/2017/08/04/business/04WHEELS1/04WHEELS1-thumbStandard.jpg, 400, 126, 75, 600, 190, 75, 0, 0, 0, 600, NA, NA, images/2017/08/04/business/04WHEELS1/04WHEELS1-articleLarge.jpg, NA, NA, 400, NA, NA, NA, images/2017/08/04/business/04WHEELS1/04WHEELS1-thumbWide.jpg, NA, NA, 190, NA, NA, 126, NA, NA, NA, 75, NA, NA, images/2017/08/04/business/04WHEELS1/04WHEELS1-thumbStandard.jpg, NA, NA, 75
## 4 image, image, image, xlarge, wide, thumbnail, images/2017/08/15/arts/15deadpool1/15deadpool1-articleLarge.jpg, images/2017/08/15/arts/15deadpool1/15deadpool1-thumbWide.jpg, images/2017/08/15/arts/15deadpool1/15deadpool1-thumbStandard.jpg, 421, 126, 75, 600, 190, 75, 0, 0, 0, 600, NA, NA, images/2017/08/15/arts/15deadpool1/15deadpool1-articleLarge.jpg, NA, NA, 421, NA, NA, NA, images/2017/08/15/arts/15deadpool1/15deadpool1-thumbWide.jpg, NA, NA, 190, NA, NA, 126, NA, NA, NA, 75, NA, NA, images/2017/08/15/arts/15deadpool1/15deadpool1-thumbStandard.jpg, NA, NA, 75
## 5 image, image, image, xlarge, wide, thumbnail, images/2017/08/26/insider/night-wolves-still/night-wolves-still-articleLarge.jpg, images/2017/08/26/insider/night-wolves-still/night-wolves-still-thumbWide.jpg, images/2017/08/26/insider/night-wolves-still/night-wolves-still-thumbStandard.jpg, 377, 126, 75, 600, 190, 75, 0, 0, 0, 600, NA, NA, images/2017/08/26/insider/night-wolves-still/night-wolves-still-articleLarge.jpg, NA, NA, 377, NA, NA, NA, images/2017/08/26/insider/night-wolves-still/night-wolves-still-thumbWide.jpg, NA, NA, 190, NA, NA, 126, NA, NA, NA, 75, NA, NA, images/2017/08/26/insider/night-wolves-still/night-wolves-still-thumbStandard.jpg, NA, NA, 75
## 6 NULL
## 7 NULL
## 8 NULL
## 9 NULL
## 10 image, image, image, thumbnail, xlarge, wide, images/2017/06/11/sports/11TT-promo/11TT-promo-thumbStandard.jpg, images/2017/06/11/sports/11TT-promo/11TT-promo-articleLarge.jpg, images/2017/06/11/sports/11TT-promo/11TT-promo-thumbWide-v2.jpg, 75, 405, 126, 75, 600, 190, 0, 0, 0, 75, NA, NA, images/2017/06/11/sports/11TT-promo/11TT-promo-thumbStandard.jpg, NA, NA, 75, NA, NA, NA, 600, NA, NA, images/2017/06/11/sports/11TT-promo/11TT-promo-articleLarge.jpg, NA, NA, 405, NA, NA, NA, images/2017/06/11/sports/11TT-promo/11TT-promo-thumbWide-v2.jpg, NA, NA, 190, NA, NA, 126
## response.docs.headline.main
## 1 Woman Wins Championship Motorcycle Race for First Time
## 2 Angel Nieto, Motorcyclist Who Won 13 Grand Prix Championships, Dies at 70
## 3 Harley-Davidson’s Newest Rival Is an Old Brand
## 4 ‘Deadpool 2’ Stuntwoman Dies in Motorcycle Accident on Set
## 5 Inside a Putin-Linked Biker Club — and Its Combat School
## 6 Capturing the Scene at 200 M.P.H.
## 7 Falling Pine Tree in Rome Crushes Taxi, Injuring Driver
## 8 Illinois Governor Makes Re-election Bid Official With Video
## 9 Discovery Channel Revives 'American Chopper' After 5 Years
## 10 Take a Lap in the World’s Most Dangerous Race
## response.docs.headline.print_headline
## 1 Woman Wins World Championship Race
## 2 Angel Nieto, Motorcyclist With 13 Titles, Dies at 70
## 3 Harley (b. 1903) Is Challenged by New Version of an Even Older Brand
## 4 ‘Deadpool 2’ Stuntwoman Dies in Motorcycle Accident on Set
## 5 <NA>
## 6 Capturing the Scene at 200 M.P.H.
## 7 Falling Pine Tree in Rome Crushes Taxi, Injuring Driver
## 8 Illinois Governor Makes Re-election Bid Official With Video
## 9 Discovery Channel Revives 'American Chopper' After 5 Years
## 10 <NA>
## response.docs.headline.kicker
## 1 <NA>
## 2 <NA>
## 3 Wheels
## 4 <NA>
## 5 <NA>
## 6 <NA>
## 7 <NA>
## 8 <NA>
## 9 <NA>
## 10
## response.docs.keywords
## 1 N, N, 1, 2, subject, persons, Motorcycle Racing, Carrasco, Ana
## 2 N, N, N, N, 1, 2, 3, 4, persons, subject, subject, glocations, Nieto, Angel (1947-2017), Deaths (Obituaries), Motorcycle Racing, Spain
## 3 N, N, N, N, 1, 2, 3, 4, subject, organizations, organizations, organizations, Motorcycles, Motor Bikes and Motorscooters, Harley-Davidson Inc, Polaris Industries, Indian Motorcycle Co
## 4 N, N, N, N, N, N, N, 1, 2, 3, 4, 5, 6, 7, subject, subject, subject, creative_works, subject, organizations, glocations, Movies, Stunts and Stuntmen, Deaths (Fatalities), Deadpool 2 (Movie), Motorcycles, Motor Bikes and Motorscooters, Twentieth Century Fox, Vancouver (British Columbia)
## 5 N, N, N, N, N, N, N, 1, 2, 3, 4, 5, 6, 7, glocations, glocations, organizations, subject, persons, subject, subject, Ukraine, Russia, Night Wolves (Motorcycle Club), Motorcycles, Motor Bikes and Motorscooters, Putin, Vladimir V, Embargoes and Sanctions, Defense and Military Forces
## 6 N, N, N, N, N, 1, 2, 3, 4, 5, glocations, subject, subject, subject, glocations, Isle of Man, Motorcycles, Motor Bikes and Motorscooters, Deaths (Fatalities), Motorcycle Racing, Northern Ireland
## 7 NULL
## 8 NULL
## 9 NULL
## 10 N, N, N, N, 1, 2, 3, 4, subject, subject, subject, glocations, Deaths (Fatalities), Motorcycle Racing, Sports Injuries, Isle of Man
## response.docs.pub_date response.docs.document_type
## 1 2017-09-18T14:35:15+0000 article
## 2 2017-08-10T19:13:00+0000 article
## 3 2017-08-03T22:15:20+0000 article
## 4 2017-08-14T22:14:54+0000 article
## 5 2017-08-24T21:30:01+0000 multimedia
## 6 2017-06-19T20:57:35+0000 article
## 7 2017-10-23T14:21:47+0000 article
## 8 2017-10-23T18:01:31+0000 article
## 9 2017-10-18T17:21:47+0000 article
## 10 2017-06-07T12:15:34+0000 multimedia
## response.docs.new_desk response.docs.section_name
## 1 Sports Auto Racing
## 2 Obits Auto Racing
## 3 Business <NA>
## 4 Culture <NA>
## 5 World / Europe Europe
## 6 Insider <NA>
## 7 None Europe
## 8 None <NA>
## 9 None <NA>
## 10 Sports <NA>
## original
## 1 By VICTOR MATHER
## 2 By RICHARD SANDOMIR
## 3 By NORMAN MAYERSOHN
## 4 By SOPHIE HAIGNEY
## 5 By NATALIA V. OSIPOVA and NAIRA DAVLASHYAN
## 6 By NEIL COLLIER
## 7 By THE ASSOCIATED PRESS
## 8 By THE ASSOCIATED PRESS
## 9 By THE ASSOCIATED PRESS
## 10 By ANDREW KEH
## response.docs.type_of_material response.docs._id
## 1 News 59bfd9b195d0e0246f20906a
## 2 Obituary (Obit) 598cb04295d0e0246f1ff1c9
## 3 News 5983a07c95d0e0246f1fe022
## 4 News 599220e495d0e0246f1ffb5b
## 5 Video 599f456095d0e0246f20160c
## 6 News 59483ac495d0e02550995bcb
## 7 News 59edfb0595d0e0246f2119a9
## 8 News 59ee2e9c95d0e0246f211a3a
## 9 News 59e78db895d0e0246f210c46
## 10 Interactive Feature 5937ee6195d0e024b5875203
## response.docs.word_count response.docs.score
## 1 391 5.1989710
## 2 897 2.4859512
## 3 1102 1.4699018
## 4 259 1.2330924
## 5 34 0.4377208
## 6 651 0.4343883
## 7 118 0.4066832
## 8 132 0.3506509
## 9 131 0.3455038
## 10 0 0.3441349
## response.docs.uri response.meta.hits
## 1 nyt://article/bea41aac-8a41-529d-8f69-1a57e793cfd6 23244
## 2 nyt://article/5af6c322-2b15-5103-aad8-28971f71271e 23244
## 3 nyt://article/ea8a3747-654d-568a-a49d-d6b33dfb5c73 23244
## 4 nyt://article/393af4d3-7199-564b-b092-44d90f7df69a 23244
## 5 nyt://video/710269ff-a21f-5960-a231-0d64c269382a 23244
## 6 nyt://article/b49d7fb9-e965-5ff5-b1da-c3b3f0540c7e 23244
## 7 nyt://article/1f196482-34bc-5e96-a031-f8c47bafe21e 23244
## 8 nyt://article/50a4c4dd-888a-5cfe-b914-1b3d8bc73f35 23244
## 9 nyt://article/9b1b5919-f6b0-5187-9748-c3b867c94721 23244
## 10 <NA> 23244
## response.meta.offset response.meta.time
## 1 0 9
## 2 0 9
## 3 0 9
## 4 0 9
## 5 0 9
## 6 0 9
## 7 0 9
## 8 0 9
## 9 0 9
## 10 0 9