We are tasked with creating an interface with the NY Times API and read in the json data to extract some information to be stored in a data frame.
Load required libraries
A helper function for displaying tables
We will use the url and api_key below along with a query to stitch together a request to the NYT API.
url <- "https://api.nytimes.com/svc/search/v2/articlesearch.json?"
key <- str_c("&api-key=", api_key)A function that takes in a a parsed response from the API and extracts the information. There are 10 articles per page. Not all fields have values so we test for that case with the ifelse() function and assign missing values as NA.
get_article_info <- function(parsed_response) {
# initialize dataframe
df <- data.frame(headline=character(),
author=character(),
source=character(),
pubdate=as.Date(character()),
url=character(),
stringsAsFactors=FALSE)
# for each article per page, extract the information
for (i in 1:10) {
# get article
article <- parsed_response$response$docs[[i]]
# extract info
headline <- ifelse(is.null(article$headline$main), "NA", article$headline$main)
author <- ifelse(is.null(article$byline$original), "NA", article$byline$original)
source <- article$source
pubdate <- article$pub_date
weburl <- article$web_url
# store in dataframe format
info <- data.frame(headline, author,source, pubdate, weburl)
names(info) <- names(df)
# append to dataframe
df <- rbind(df, info)
}
return(df)
}Iterate and call the get_article_info() function and feed it the parsed_response for each page. Here we arbitrarily limit our extraction to the first 10 pages of responses. Our query will simply be the newest articles.
# initialize dataframe that will contain the 10 extracted article info for each response page
articles <- data.frame(headline=character(),
author=character(),
source=character(),
pubdate=as.Date(character()),
url=character(),
stringsAsFactors=FALSE)
# iterate
for (i in 1:10) {
# string the query and request together and send a request
query <- str_c("q=","new+york+times&page=", i, "&sort=newest")
request <- str_c(url, query, key)
nyt <- GET(request)
# parse the request
parsed_response <- content(nyt, "parse")
# call get_article_info function
article_info <- get_article_info(parsed_response)
# append to articles data frame
articles <- rbind(articles, article_info)
}The information has been combined into a data frame. Here are the top results sorted by recency
| headline | author | source | pubdate | url |
|---|---|---|---|---|
| Leader’s Death Will Damage ISIS, but Not Destroy It | By Ben Hubbard, Rukmini Callimachi and Alissa J. Rubin | The New York Times | 2019-10-27T21:21:07+0000 | https://www.nytimes.com/2019/10/27/world/middleeast/isis-leader-dead-impact.html |
| Corrections: October 27, 2019 | NA | The New York Times | 2019-10-27T21:13:38+0000 | https://www.nytimes.com/2019/10/27/pageoneplus/corrections-october-27-2019.html |
| ISIS, Boeing, Pakistan: Your Monday Briefing | By Alisha Haridasani Gupta | The New York Times | 2019-10-27T20:49:14+0000 | https://www.nytimes.com/2019/10/27/briefing/isis-boeing-pakistan.html |
| John Conyers Jr., Longest-Serving African-American in Congressional History, Dies at 90 | By Adam Clymer | The New York Times | 2019-10-27T20:38:04+0000 | https://www.nytimes.com/2019/10/27/obituaries/john-conyers-jr-dead.html |
| Jets’ Darnold Throws 3 More Picks in 29-15 Loss at Jaguars | By The Associated Press | AP | 2019-10-27T20:35:11+0000 | https://www.nytimes.com/aponline/2019/10/27/sports/football/ap-fbn-jets-jaguars.html |
| ‘Seinfeld’ Star Alexander, Southside Johnny in NJ Fame Hall | By The Associated Press | AP | 2019-10-27T20:32:13+0000 | https://www.nytimes.com/aponline/2019/10/27/us/ap-us-new-jersey-hall-of-fame.html |
Perform some basic tidying and transformations.
# remove By and split the data into year, month and day.
articles$author <- str_remove(articles$author, "By ")
articles <- articles %>% mutate(year = str_sub(pubdate,1,4), month = str_sub(pubdate,6,7), day = str_sub(pubdate,9,10))
articles<- articles[,c(1:3,6:8,4,5)]Display the results of the transformation.
| headline | author | source | year | month | day | pubdate | url |
|---|---|---|---|---|---|---|---|
| Leader’s Death Will Damage ISIS, but Not Destroy It | Ben Hubbard, Rukmini Callimachi and Alissa J. Rubin | The New York Times | 2019 | 10 | 27 | 2019-10-27T21:21:07+0000 | https://www.nytimes.com/2019/10/27/world/middleeast/isis-leader-dead-impact.html |
| Corrections: October 27, 2019 | NA | The New York Times | 2019 | 10 | 27 | 2019-10-27T21:13:38+0000 | https://www.nytimes.com/2019/10/27/pageoneplus/corrections-october-27-2019.html |
| ISIS, Boeing, Pakistan: Your Monday Briefing | Alisha Haridasani Gupta | The New York Times | 2019 | 10 | 27 | 2019-10-27T20:49:14+0000 | https://www.nytimes.com/2019/10/27/briefing/isis-boeing-pakistan.html |
| John Conyers Jr., Longest-Serving African-American in Congressional History, Dies at 90 | Adam Clymer | The New York Times | 2019 | 10 | 27 | 2019-10-27T20:38:04+0000 | https://www.nytimes.com/2019/10/27/obituaries/john-conyers-jr-dead.html |
| Jets’ Darnold Throws 3 More Picks in 29-15 Loss at Jaguars | The Associated Press | AP | 2019 | 10 | 27 | 2019-10-27T20:35:11+0000 | https://www.nytimes.com/aponline/2019/10/27/sports/football/ap-fbn-jets-jaguars.html |
| ‘Seinfeld’ Star Alexander, Southside Johnny in NJ Fame Hall | The Associated Press | AP | 2019 | 10 | 27 | 2019-10-27T20:32:13+0000 | https://www.nytimes.com/aponline/2019/10/27/us/ap-us-new-jersey-hall-of-fame.html |
| See Where the Raid on al-Baghdadi Took Place in Syria | Allison McCann, Anjali Singhvi and Weiyi Cai | The New York Times | 2019 | 10 | 27 | 2019-10-27T20:26:25+0000 | https://www.nytimes.com/interactive/2019/10/27/world/middleeast/baghdadi-isis-leader-maps.html |
| Al-Baghdadi Is Dead. The Story Doesn’t End Here. | Thomas L. Friedman | The New York Times | 2019 | 10 | 27 | 2019-10-27T19:40:12+0000 | https://www.nytimes.com/2019/10/27/opinion/trump-al-baghdadi-dead.html |
| When Poor People Are Beaten for Seeking Help | Mara Gay and Emma Goldberg | The New York Times | 2019 | 10 | 27 | 2019-10-27T19:00:16+0000 | https://www.nytimes.com/2019/10/27/opinion/hra-nyc-abuse.html |
| The Kincade Fire in Pictures | The New York Times | The New York Times | 2019 | 10 | 27 | 2019-10-27T18:55:08+0000 | https://www.nytimes.com/2019/10/27/us/california-fires-photos.html |
| La edad de oro del cine mexicano sobrevive en línea | J. Hoberman | The New York Times | 2019 | 10 | 27 | 2019-10-27T17:01:09+0000 | https://www.nytimes.com/es/2019/10/27/espanol/cultura/cine-oro-mexicano-youtube.html |
| The Latest: Patriots, 49ers Stay Perfect Going Into Week 9 | The Associated Press | AP | 2019 | 10 | 27 | 2019-10-27T16:50:01+0000 | https://www.nytimes.com/aponline/2019/10/27/sports/football/ap-fbn-nfl-week-8-the-latest.html |
| Transcript of Trump’s Remarks on the Death of Abu Bakr al-Baghdadi | The New York Times | The New York Times | 2019 | 10 | 27 | 2019-10-27T16:42:32+0000 | https://www.nytimes.com/2019/10/27/us/trump-transcript-isis-al-baghdadi.html |
| Abu Bakr al-Baghdadi, ISIS Leader Known for His Brutality, Is Dead at 48 | Rukmini Callimachi and Falih Hassan | The New York Times | 2019 | 10 | 27 | 2019-10-27T15:31:34+0000 | https://www.nytimes.com/2019/10/27/world/middleeast/al-baghdadi-dead.html |
| California Fire Updates: Emergency Declared as Residents Flee and Power Goes Out | Lauren Hepler and Nicholas Bogel-Burroughs | The New York Times | 2019 | 10 | 27 | 2019-10-27T14:30:20+0000 | https://www.nytimes.com/2019/10/27/us/kincade-fire-california.html |
| ISIS Leader al-Baghdadi Is Dead, Trump Says | Peter Baker, Eric Schmitt and Helene Cooper | The New York Times | 2019 | 10 | 27 | 2019-10-27T13:25:12+0000 | https://www.nytimes.com/2019/10/27/us/politics/isis-leader-al-baghdadi-dead.html |
| Bros to Pros: The N.F.L.’s Quinnen and Quincy Williams | Dave Caldwell | The New York Times | 2019 | 10 | 27 | 2019-10-27T11:00:08+0000 | https://www.nytimes.com/2019/10/27/sports/bros-to-pros-the-nfls-quinnen-and-quincy-williams.html |
| Impeachment, California, Halloween: Your Weekend Briefing | Remy Tumin and Lance Booth | The New York Times | 2019 | 10 | 27 | 2019-10-27T10:00:19+0000 | https://www.nytimes.com/2019/10/27/briefing/impeachment-california-halloween.html |
| Before Deadly Crashes, Boeing Pushed for Law That Undercut Oversight | Natalie Kitroeff and David Gelles | The New York Times | 2019 | 10 | 27 | 2019-10-27T09:00:15+0000 | https://www.nytimes.com/2019/10/27/business/boeing-737-max-crashes.html |
| Planning for the Unexpected by Taking Charge of Her Finances | Sara Aridi | The New York Times | 2019 | 10 | 27 | 2019-10-27T09:00:09+0000 | https://www.nytimes.com/2019/10/27/neediest-cases/taking-charge-of-her-finances.html |
| The Week in Books | NA | The New York Times | 2019 | 10 | 27 | 2019-10-27T09:00:03+0000 | https://www.nytimes.com/2019/10/27/books/the-week-in-books.html |
| 1.5 Million Packages a Day: The Internet Brings Chaos to N.Y. Streets | Matthew Haag and Winnie Hu | The New York Times | 2019 | 10 | 27 | 2019-10-27T07:00:12+0000 | https://www.nytimes.com/2019/10/27/nyregion/nyc-amazon-delivery.html |
| What’s on TV Sunday: ‘Mrs. Fletcher’ and ‘The Witch’ | Lauren Messman | The New York Times | 2019 | 10 | 27 | 2019-10-27T05:00:06+0000 | https://www.nytimes.com/2019/10/27/arts/television/whats-on-tv-sunday-mrs-fletcher-and-the-witch.html |
| Former Browns Pro Bowl DB Parrish Dies at 83 | Reuters | Reuters | 2019 | 10 | 27 | 2019-10-27T04:35:06+0000 | https://www.nytimes.com/reuters/2019/10/27/sports/football/27reuters-football-nfl-cle-parrish-obit.html |
| Special Operations Raid Said to Kill Senior Terrorist Leader in Syria | Eric Schmitt, Maggie Haberman and Rukmini Callimachi | The New York Times | 2019 | 10 | 27 | 2019-10-27T04:25:09+0000 | https://www.nytimes.com/2019/10/27/us/politics/special-operations-raid-terrorist-syria.html |
| DeRozan’s Late Shot Sends Spurs to 124-122 Win Over Wizards | The Associated Press | AP | 2019 | 10 | 27 | 2019-10-27T03:23:02+0000 | https://www.nytimes.com/aponline/2019/10/26/sports/basketball/bc-bkn-wizards-spurs.html |
| Van Riemsdyk Scores 2, Flyers Beat Blue Jackets 7-4 | The Associated Press | AP | 2019 | 10 | 27 | 2019-10-27T02:14:05+0000 | https://www.nytimes.com/aponline/2019/10/26/sports/hockey/ap-hkn-blue-jackets-flyers.html |
| The Latest: Astros Pound Nats 8-1, Even World Series 2-2 | The Associated Press | AP | 2019 | 10 | 26 | 2019-10-26T23:32:08+0000 | https://www.nytimes.com/aponline/2019/10/26/sports/baseball/ap-bbo-world-series-the-latest.html |
| A Player With Shoulder Pain, and a League Happy to Turn Its Back | Michael Powell | The New York Times | 2019 | 10 | 26 | 2019-10-26T22:23:07+0000 | https://www.nytimes.com/2019/10/26/sports/football/jets-osemele-injury.html |
| The ‘Deep State’ Exists to Battle People Like Trump | Margaret O’Mara | The New York Times | 2019 | 10 | 26 | 2019-10-26T21:45:18+0000 | https://www.nytimes.com/2019/10/26/opinion/sunday/trump-civil-service-deep-state.html |
| How Sweet it Is! | Caitlin Lovinger | The New York Times | 2019 | 10 | 26 | 2019-10-26T21:40:03+0000 | https://www.nytimes.com/2019/10/26/crosswords/daily-puzzle-2019-10-27.html |
| ‘Clueless’ and ‘Saved by the Bell’ Are How We Got Trump | Lindy West | The New York Times | 2019 | 10 | 26 | 2019-10-26T19:40:29+0000 | https://www.nytimes.com/2019/10/26/opinion/sunday/90s-movies-tv-activism.html |
| Pyongyang’s Retro Sci-Fi Architecture | Cristiano Bianchi and Kristina Drapic | The New York Times | 2019 | 10 | 26 | 2019-10-26T19:37:45+0000 | https://www.nytimes.com/2019/10/26/opinion/sunday/architecture-north-korea-pyongyang.html |
| IoG: Internet of Ghouls | Rachel Siemens | The New York Times | 2019 | 10 | 26 | 2019-10-26T18:30:08+0000 | https://www.nytimes.com/2019/10/26/opinion/sunday/halloween-internet-haunted.html |
| Extra, Extra! Prez Won’t Read All About It | Maureen Dowd | The New York Times | 2019 | 10 | 26 | 2019-10-26T18:30:07+0000 | https://www.nytimes.com/2019/10/26/opinion/sunday/trump-cancels-subscription.html |
| Against the Superhero Regime | Ross Douthat | The New York Times | 2019 | 10 | 26 | 2019-10-26T18:30:04+0000 | https://www.nytimes.com/2019/10/26/opinion/sunday/joker-movie-martin-scorsese.html |
| Who Is Bill Barr? | Emily Bazelon | The New York Times | 2019 | 10 | 26 | 2019-10-26T18:30:01+0000 | https://www.nytimes.com/interactive/2019/10/26/opinion/william-barr-trump.html |
| Choosing to Be Vulnerable With My Patients | Helen Ouyang | The New York Times | 2019 | 10 | 26 | 2019-10-26T18:26:20+0000 | https://www.nytimes.com/2019/10/26/opinion/sunday/doctors-addiction-trust.html |
| U.K. Truck Driver Charged With Manslaughter in Deaths of 39 | Yonette Joseph and Ed O’Loughlin | The New York Times | 2019 | 10 | 26 | 2019-10-26T17:31:35+0000 | https://www.nytimes.com/2019/10/26/world/europe/container-39-bodies-charges.html |
| The E.P.A., on Asbestos | NA | The New York Times | 2019 | 10 | 26 | 2019-10-26T16:00:03+0000 | https://www.nytimes.com/2019/10/26/opinion/letters/epa-asbestos-wnyc.html |
| Waiting for Bolton: A Capital Speculates on What He Will Say | Peter Baker | The New York Times | 2019 | 10 | 26 | 2019-10-26T15:52:16+0000 | https://www.nytimes.com/2019/10/26/us/politics/bolton-trump-impeachment.html |
| Thanks, Whistle-Blower, Your Work Is Done | The Editorial Board | The New York Times | 2019 | 10 | 26 | 2019-10-26T15:00:01+0000 | https://www.nytimes.com/interactive/2019/10/26/opinion/trump-whistleblower-letter.html |
| Man Out for Night on the Town Is Stabbed in Times Square | The Associated Press | AP | 2019 | 10 | 26 | 2019-10-26T14:35:01+0000 | https://www.nytimes.com/aponline/2019/10/26/us/ap-us-times-square-stabbing.html |
| Official Says Attempts to Rally Top State Officials Behind Ukraine Ambassador Failed | Nicholas Fandos and Sharon LaFraniere | The New York Times | 2019 | 10 | 26 | 2019-10-26T14:19:22+0000 | https://www.nytimes.com/2019/10/26/us/philip-reeker-testimony.html |
| Boris Johnson Is Heading for a Scorched-Earth Election | Jenni Russell | The New York Times | 2019 | 10 | 26 | 2019-10-26T14:06:05+0000 | https://www.nytimes.com/2019/10/26/opinion/boris-brexit-conservatives.html |
| Economic Incentives Don’t Always Do What We Want Them To | Esther Duflo and Abhijit Banerjee | The New York Times | 2019 | 10 | 26 | 2019-10-26T14:04:02+0000 | https://www.nytimes.com/2019/10/26/opinion/sunday/duflo-banerjee-economic-incentives.html |
| A Racist Attack Shows How Whiteness Evolves | Nell Irvin Painter | The New York Times | 2019 | 10 | 26 | 2019-10-26T14:00:10+0000 | https://www.nytimes.com/2019/10/26/opinion/new-jersey-high-school-racism.html |
| Pan Am Remains Grounded. In Culture, That Is. | Laura M. Holson | The New York Times | 2019 | 10 | 26 | 2019-10-26T13:00:55+0000 | https://www.nytimes.com/2019/10/26/us/pan-am-travel.html |
| Getting Ready for the New York City Marathon | Jen A. Miller | The New York Times | 2019 | 10 | 26 | 2019-10-26T12:00:05+0000 | https://www.nytimes.com/2019/10/26/well/move/getting-ready-for-the-new-york-city-marathon.html |
| 10 Watches That Changed What’s on Your Wrist | Victoria Gomelsky | International New York Times | 2019 | 10 | 26 | 2019-10-26T09:00:29+0000 | https://www.nytimes.com/2019/10/26/fashion/watches-design-audemars-piguet-apple-watch.html |
| ‘We’re Asking You to Dig Deep’: Biden Seeks to Steady Finances as Allies Fret | Shane Goldmacher | The New York Times | 2019 | 10 | 26 | 2019-10-26T09:00:21+0000 | https://www.nytimes.com/2019/10/26/us/politics/joe-biden-campaign-fundraising.html |
| Thanksgiving Getaway: Going Someplace Other Than Grandmother’s House | Sarah Firshein | The New York Times | 2019 | 10 | 26 | 2019-10-26T09:00:20+0000 | https://www.nytimes.com/2019/10/26/travel/Thanksgiving-travel-vacations.html |
| Running Away, and Finding Solace in Art | Sara Aridi | The New York Times | 2019 | 10 | 26 | 2019-10-26T09:00:17+0000 | https://www.nytimes.com/2019/10/26/neediest-cases/leaving-ghana-art-therapy.html |
| The Impeachment Inquiry This Week in 6 Developments | Kaly Soto | The New York Times | 2019 | 10 | 26 | 2019-10-26T09:00:16+0000 | https://www.nytimes.com/2019/10/26/briefing/the-impeachment-inquiry-this-week-in-6-developments.html |
| Secret Deal Helped Housing Industry Stop Tougher Rules on Climate Change | Christopher Flavelle | The New York Times | 2019 | 10 | 26 | 2019-10-26T09:00:15+0000 | https://www.nytimes.com/2019/10/26/climate/building-codes-secret-deal.html |
| Symbol of ’80s Greed Stands to Profit From Trump Tax Break for Poor Areas | Eric Lipton and Jesse Drucker | The New York Times | 2019 | 10 | 26 | 2019-10-26T09:00:14+0000 | https://www.nytimes.com/2019/10/26/business/michael-milken-trump-opportunity-zones.html |
| In Hungary, Viktor Orban Showers Money on Stadiums, Less So on Hospitals | Patrick Kingsley and Benjamin Novak | The New York Times | 2019 | 10 | 26 | 2019-10-26T08:45:26+0000 | https://www.nytimes.com/2019/10/26/world/europe/viktor-orban-soccer-health-care.html |
| Panic in Pakistani City After 900 Children Test Positive for H.I.V. | Zia ur-Rehman and Maria Abi-Habib | The New York Times | 2019 | 10 | 26 | 2019-10-26T07:22:03+0000 | https://www.nytimes.com/2019/10/26/world/asia/hiv-aids-pakistan-ratodero.html |
| How a Rooftop Meadow of Bees and Butterflies Shows N.Y.C.’s Future | Anne Barnard | The New York Times | 2019 | 10 | 26 | 2019-10-26T07:00:07+0000 | https://www.nytimes.com/2019/10/26/nyregion/green-roofs-nyc.html |
| What’s on TV Saturday: ‘BoJack Horseman’ and ‘Saturday Night Live’ | Lauren Messman | The New York Times | 2019 | 10 | 26 | 2019-10-26T05:00:11+0000 | https://www.nytimes.com/2019/10/26/arts/television/whats-on-tv-saturday-bojack-horseman-and-saturday-night-live.html |
| Rachel Maddow Confronts Her NBC News Bosses Live, on the Air | John Koblin and Michael M. Grynbaum | The New York Times | 2019 | 10 | 26 | 2019-10-26T03:49:52+0000 | https://www.nytimes.com/2019/10/25/business/rachel-maddow-ronan-farrow-nbc.html |
| Irving Helps Nets Beat Knicks 113-109 After Blowing Big Lead | The Associated Press | AP | 2019 | 10 | 26 | 2019-10-26T02:35:06+0000 | https://www.nytimes.com/aponline/2019/10/25/sports/basketball/ap-bkn-knicks-nets.html |
| Elizabeth Warren Wants to Lose Your Vote | Bret Stephens | The New York Times | 2019 | 10 | 26 | 2019-10-26T00:45:12+0000 | https://www.nytimes.com/2019/10/25/opinion/elizabeth-warren-2020.html |
| Giuliani Is Drawing Attention to Hunter Biden’s Work in Romania. But There’s a Problem. | Kenneth P. Vogel | The New York Times | 2019 | 10 | 26 | 2019-10-26T00:28:11+0000 | https://www.nytimes.com/2019/10/25/us/politics/giuliani-hunter-biden-romania.html |
| Why Will Breitbart Be Included in ‘Facebook News’? | Charlie Warzel | The New York Times | 2019 | 10 | 26 | 2019-10-26T00:26:46+0000 | https://www.nytimes.com/2019/10/25/opinion/mark-zuckerberg-facebook.html |
| E.A. Carmean,Who Forsook a Life in Art for the Church, Dies at 74 | Roberta Smith | The New York Times | 2019 | 10 | 26 | 2019-10-26T00:14:34+0000 | https://www.nytimes.com/2019/10/25/arts/ea-carmean-dead.html |
| Lebanon Battles to Be Born at Last | Roger Cohen | The New York Times | 2019 | 10 | 25 | 2019-10-25T23:25:40+0000 | https://www.nytimes.com/2019/10/25/opinion/lebanon-protests.html |
| Leroy Johnson, Barrier-Breaking Georgia Politician, Dies at 91 | Neil Genzlinger | The New York Times | 2019 | 10 | 25 | 2019-10-25T23:24:42+0000 | https://www.nytimes.com/2019/10/25/us/politics/leroy-johnson-barrier-breaking-georgia-politician-dies-at-91.html |
| Bernie Parrish, All-Pro Cornerback Who Took on the N.F.L., Dies at 83 | Richard Sandomir | The New York Times | 2019 | 10 | 25 | 2019-10-25T22:51:51+0000 | https://www.nytimes.com/2019/10/25/sports/football/bernie-parrish-dead.html |
| Halloween: Nooses and Blackface Decorations Prompt Backlash | Azi Paybarah | The New York Times | 2019 | 10 | 25 | 2019-10-25T22:45:41+0000 | https://www.nytimes.com/2019/10/25/nyregion/nooses-blackface-halloween-decorations.html |
| Applications Open for the 2020 New York Portfolio Review | James Estrin | The New York Times | 2019 | 10 | 25 | 2019-10-25T21:42:50+0000 | https://www.nytimes.com/2019/10/25/reader-center/applications-open-for-the-2020-new-york-portfolio-review.html |
| Health Official: No ‘Short List’ of Vaping Illness Suspects | The Associated Press | AP | 2019 | 10 | 25 | 2019-10-25T21:41:03+0000 | https://www.nytimes.com/aponline/2019/10/25/health/ap-us-med-vaping-illnesses.html |
| The Latest: Astros Cut Their Deficit to 2-1 | The Associated Press | AP | 2019 | 10 | 25 | 2019-10-25T21:26:17+0000 | https://www.nytimes.com/aponline/2019/10/25/sports/baseball/ap-bbo-world-series-the-latest.html |
| The 13 Scariest Horror Movies on Netflix Right Now | Noel Murray | The New York Times | 2019 | 10 | 25 | 2019-10-25T21:11:48+0000 | https://www.nytimes.com/2019/10/25/movies/netflix-horror.html |
| ‘My Grandchild Is Not a Terrorist’ | Livia Albeck-Ripka | The New York Times | 2019 | 10 | 25 | 2019-10-25T20:37:35+0000 | https://www.nytimes.com/2019/10/25/podcasts/daily-newsletter-isis-syria.html |
| In Canada’s North, Art Emerges In the Face of Difficulty | NA | The New York Times | 2019 | 10 | 25 | 2019-10-25T20:05:05+0000 | https://www.nytimes.com/2019/10/25/world/canada/Cape-Dorset-Inuit-Artists.html |
| My Family Cared for My Sick Aunt. Who’s Caring for Us? | Lorene Cary | The New York Times | 2019 | 10 | 25 | 2019-10-25T19:17:03+0000 | https://www.nytimes.com/2019/10/25/opinion/health-care-old-age-disability.html |
| 5 Ballot Proposals for New York | The Editorial Board | The New York Times | 2019 | 10 | 25 | 2019-10-25T19:11:03+0000 | https://www.nytimes.com/2019/10/25/opinion/new-york-ballot-questions-2019.html |
| 11 of Our Best Weekend Reads | Kaly Soto | The New York Times | 2019 | 10 | 25 | 2019-10-25T19:00:25+0000 | https://www.nytimes.com/2019/10/25/briefing/11-of-our-best-weekend-reads.html |
| Thar Be Witches, and Us | Whitney Curry Wimbish | The New York Times | 2019 | 10 | 25 | 2019-10-25T19:00:14+0000 | https://www.nytimes.com/2019/10/25/opinion/sunday/thar-be-witches-and-us.html |
| The Playlist: Frank Ocean’s Pillow Talk, and 12 More New Songs | Jon Pareles, Jon Caramanica, Caryn Ganz and Giovanni Russonello | The New York Times | 2019 | 10 | 25 | 2019-10-25T18:49:31+0000 | https://www.nytimes.com/2019/10/25/arts/music/playlist-frank-ocean-selena-gomez-coldplay.html |
| University of Missouri Apologizes After Tweet on Diversity | The Associated Press | AP | 2019 | 10 | 25 | 2019-10-25T18:41:10+0000 | https://www.nytimes.com/aponline/2019/10/25/us/ap-us-university-of-missouri-diversity-tweet.html |
| Maurice Nadjari Dies at 95; Crusading, and Criticized, Prosecutor | Robert D. McFadden | The New York Times | 2019 | 10 | 25 | 2019-10-25T17:47:55+0000 | https://www.nytimes.com/2019/10/25/nyregion/maurice-nadjari-dead.html |
| Jeers for Harvey Weinstein at NYC Actors Showcase | The Associated Press | AP | 2019 | 10 | 25 | 2019-10-25T17:41:04+0000 | https://www.nytimes.com/aponline/2019/10/25/us/ap-us-sexual-misconduct-harvey-weinstein.html |
| NFL At 100: Arrival of 1970s Ushers in NFL’s Modern Era | The Associated Press | AP | 2019 | 10 | 25 | 2019-10-25T17:38:10+0000 | https://www.nytimes.com/aponline/2019/10/25/sports/football/ap-fbn-nfl-at-100-the-1970s.html |
| Marta Kurtag Dies at 92, Sundering a Profound Musical Partnership | David Allen | The New York Times | 2019 | 10 | 25 | 2019-10-25T17:37:28+0000 | https://www.nytimes.com/2019/10/25/arts/music/marta-kurtag-dead.html |
| Mark Zuckerberg: Facebook Can Help the News Business | Mark Zuckerberg | The New York Times | 2019 | 10 | 25 | 2019-10-25T17:24:54+0000 | https://www.nytimes.com/2019/10/25/opinion/sunday/mark-zuckerberg-facebook-news.html |
| Actresses Cite Lack of Hollywood Stylists Who Get Black Hair | The Associated Press | AP | 2019 | 10 | 25 | 2019-10-25T17:02:06+0000 | https://www.nytimes.com/aponline/2019/10/25/us/ap-us-black-hair-hollywood.html |
| Clinton Praises Cummings: ‘He Stood Against Corrupt Leadership’ | The New York Times | The New York Times | 2019 | 10 | 25 | 2019-10-25T16:49:17+0000 | https://www.nytimes.com/video/us/100000006789434/hillary-clinton-elijah-cummings.html |
| In Criminal Justice Speech, Trump Belittles Obama’s Efforts for Black People | Astead W. Herndon and Maggie Astor | The New York Times | 2019 | 10 | 25 | 2019-10-25T16:15:32+0000 | https://www.nytimes.com/2019/10/25/us/politics/trump-benedict-college-hbcu.html |
| Why Does Only One Party Play by the Rules? | Jennifer Senior | The New York Times | 2019 | 10 | 25 | 2019-10-25T16:11:07+0000 | https://www.nytimes.com/2019/10/25/opinion/trump-2020-democrats.html |
| In One Afghan District, the Same Promises of Victory, 32 Years Apart | Thomas Gibbons-Neff | The New York Times | 2019 | 10 | 25 | 2019-10-25T16:08:23+0000 | https://www.nytimes.com/2019/10/25/magazine/afghanistan-war-baghlan-austin-miller.html |
| Elijah Cummings Funeral: Bill and Hillary Clinton Urge Defense of Democracy | Liam Stack | The New York Times | 2019 | 10 | 25 | 2019-10-25T16:02:45+0000 | https://www.nytimes.com/2019/10/25/us/politics/clinton-elijah-cummings.html |
| 8 Years After ‘The Night Circus,’ Erin Morgenstern Has a New Novel | Lyndsay Faye | The New York Times | 2019 | 10 | 25 | 2019-10-25T15:58:03+0000 | https://www.nytimes.com/2019/10/25/books/review/starless-sea-erin-morgenstern.html |
| Trouble in Paradise: Chile’s Inequality Explodes | Carlos Peña | The New York Times | 2019 | 10 | 25 | 2019-10-25T15:00:06+0000 | https://www.nytimes.com/2019/10/25/opinion/25chile-pinera-crisis.html |
| California Fires Update: Thousands Evacuated; Governor Declares State of Emergency | Tim Arango and Thomas Fuller | The New York Times | 2019 | 10 | 25 | 2019-10-25T14:50:42+0000 | https://www.nytimes.com/2019/10/25/us/santa-clarita-fire-sonoma-california.html |
| December Democratic Debate Will Be in Los Angeles, With Higher Thresholds | Shane Goldmacher | The New York Times | 2019 | 10 | 25 | 2019-10-25T14:36:11+0000 | https://www.nytimes.com/2019/10/25/us/politics/democratic-debate-los-angeles.html |
| Desserts Over Darkness | Sam Sifton | The New York Times | 2019 | 10 | 25 | 2019-10-25T14:30:01+0000 | https://www.nytimes.com/2019/10/25/dining/desserts-over-darkness.html |
| ‘This Is Who We’ve Been Waiting For’: A Diva on the Precipice | Zachary Woolfe | The New York Times | 2019 | 10 | 25 | 2019-10-25T14:00:08+0000 | https://www.nytimes.com/2019/10/25/arts/music/julia-bullock-zauberland-lincoln-center.html |
| Elijah Cummings Is Remembered as ‘a Great Man, a Moral Leader and a Friend’ | The New York Times | The New York Times | 2019 | 10 | 25 | 2019-10-25T13:56:12+0000 | https://www.nytimes.com/video/us/100000006789008/elijah-cummings-funeral.html |
As shown in the table above, using the httr package we were able to make specific paginated requests to the NYT API containing a query for the newest articles. We then parsed the JSON responses and extracted the information of interest: headlines, authors, source, publication date and web url. We applied some basic transformations to clean up the author column and split the publication date into year, month and day for further filtering.
Next steps could involve reading input from the user in order to modify the query to the user’s interest.