library(tidyverse)
library(jsonlite)The objective is to create an interface with one of the NY Times APIs, read in the JSON response, and convert it into a data frame. I chose to call the archive API, specifically querying articles from April 1921, then filtering the results further for articles from April 10th 1921. Finally, to make the output slightly easier to read, I glued the headline and abstract to gather.
NYT_KEY <- rstudioapi::askForPassword("Enter API Key")
year <- "1921"
month <- "4"
# Call API for articles from April 1921, flatten into data frame
nineteentwentyone <- fromJSON(str_glue("https://api.nytimes.com/svc/archive/v1/{year}/{month}.json?api-key={NYT_KEY}"), flatten = TRUE) %>% data.frame()
glimpse(nineteentwentyone)## Rows: 8,549
## Columns: 29
## $ copyright <chr> "Copyright (c) 2020 The New York…
## $ response.docs.abstract <chr> "MOBILE, Ala., March 31.--The Gi…
## $ response.docs.web_url <chr> "https://www.nytimes.com/1921/04…
## $ response.docs.snippet <chr> "", "", "", "", "", "", "", "", …
## $ response.docs.lead_paragraph <chr> "", "", "", "", "", "", "", "", …
## $ response.docs.print_section <chr> "S", NA, NA, NA, NA, "S", NA, NA…
## $ response.docs.print_page <chr> "22", "2", "15", "1", "12", "22"…
## $ response.docs.source <chr> "The New York Times", "The New Y…
## $ response.docs.multimedia <list> [], [], [], [], [], [], [], [],…
## $ response.docs.keywords <list> [<data.frame[0 x 0]>], [<data.f…
## $ response.docs.pub_date <chr> "1921-04-01T05:00:00+0000", "192…
## $ response.docs.document_type <chr> "article", "article", "article",…
## $ response.docs.news_desk <chr> "None", "None", "None", "None", …
## $ response.docs.section_name <chr> "Archives", "Archives", "Archive…
## $ response.docs.type_of_material <chr> "Archives", "Archives", "Archive…
## $ response.docs._id <chr> "nyt://article/011de24b-e64e-54c…
## $ response.docs.word_count <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ response.docs.uri <chr> "nyt://article/011de24b-e64e-54c…
## $ response.docs.headline.main <chr> "RAIN AGAIN KEEPS GIANTS' BATS I…
## $ response.docs.headline.kicker <chr> "1", "1", "1", "2", "1", "1", "1…
## $ response.docs.headline.content_kicker <lgl> NA, NA, NA, NA, NA, NA, NA, NA, …
## $ response.docs.headline.print_headline <chr> "RAIN AGAIN KEEPS GIANTS' BATS I…
## $ response.docs.headline.name <lgl> NA, NA, NA, NA, NA, NA, NA, NA, …
## $ response.docs.headline.seo <lgl> NA, NA, NA, NA, NA, NA, NA, NA, …
## $ response.docs.headline.sub <lgl> NA, NA, NA, NA, NA, NA, NA, NA, …
## $ response.docs.byline.original <chr> "Special to The New York Times",…
## $ response.docs.byline.person <list> [<data.frame[1 x 8]>], [<data.f…
## $ response.docs.byline.organization <chr> NA, NA, NA, NA, NA, NA, NA, NA, …
## $ response.hits <int> 8549, 8549, 8549, 8549, 8549, 85…# Select only a subset of useful columns
nyt_1921_april <- nineteentwentyone %>% select(c("response.docs.abstract",
"response.docs.web_url",
"response.docs.headline.main",
"response.docs.pub_date"))
# Rename columns to be easier to read
nyt_1921_april <- nyt_1921_april %>% rename(abstract = "response.docs.abstract",
url = "response.docs.web_url",
headline = "response.docs.headline.main",
pub_date = "response.docs.pub_date")
# Glue headline and abstract to make it easier to read
headline_abs_apr <- nyt_1921_april %>% str_glue_data("{rownames(.)} Headline: {headline} //// Abstract: {abstract}")
head(headline_abs_apr)## 1 Headline: RAIN AGAIN KEEPS GIANTS' BATS IDLE; Louisville Game Called Off, but Squad Has Workout--Bancroft and Rapp Better. //// Abstract: MOBILE, Ala., March 31.--The Giants will have to get along as best they can without a Spring game with the Kentucky Colonels this year, for once again today the elements, which have shown so much hostility since the arrival of ...
## 2 Headline: CHANCELLOR MAYR DUBIOUS.; Says Charles Has Lost Any Chance for Throne He Had. //// Abstract:
## 3 Headline: HARDING TO RECEIVE AMNESTY COMMITTEE; On the Same Date, April 13, a Petition for Release of Prisoners Goes to Congress. //// Abstract: WASHINGTON, March 31.--President Harding informed the Political Amnesty Committee today he would receive the committee on April 13, the second anniversary of the date that Debts was sent to prison.
## 4 Headline: HARDING TAKES UP RAILROAD QUESTION; Message to Congress or More Immediate Action May Follow Talk With Barton and Clark. EMPLOYES ASK CONFERENCE Would Meet Executives to Deal With Wages and Rules--Central Resents Warfield's Action. //// Abstract: WASHINGTON, March 31.--President Harding was in conference more than two hours on the railroad crisis today with Chairman Barton of the Railroad Labor Board and Chairman Clark of the Interstate Commerce Commission, and it was inferred from this conference that he would make the railroad situation the subject of an address to the special session of Congress. The conference may also result in more direct action to bring relief.
## 5 Headline: MORE PACKERS ACCEPT.; Twelve Independent Concerns Align Themselves for Wage Agreement. //// Abstract:
## 6 Headline: COLUMBIA'S OPENER TODAY.; Rain Causes One-Day Postponement of Game With Bowdoin. //// Abstract:# Filter for articles from April 10th 1921
nyt_1921_april_10 <- nyt_1921_april %>% filter(pub_date == "1921-04-10T05:00:00+0000")
# Glue headline and abstract to make it easier to read
headline_abs_apr10 <- nyt_1921_april_10 %>% str_glue_data("{rownames(.)} Headline: {headline} //// Abstract: {abstract}")
head(headline_abs_apr10, 15)## 1 Headline: TRAINING THE YOUNG.; A Plan to Bring Out Best Mental and Moral Qualities. //// Abstract: A new idea in the training of children has been evolved by a St. Louis school teacher, Miss Mabel Evans. She believes that many mothers fal to bring out their children's best mental and moral qualities in pre-school days, and wants ...
## 2 Headline: Julia Culp Ill, Her Recital Put Off. //// Abstract:
## 3 Headline: Article 3 -- No Title //// Abstract:
## 4 Headline: Rachel, One Hundred Years After //// Abstract: THE centennial of the birth of Rachel has just passed. Time was when rules came down from their thrones to pay tribute to the genius of Rachel--as acknowledgedly the greatest tragedienne of the first half of the nine teenth century as Bernhardt is of today.
## 5 Headline: TO OFFER GRAIN TRADE BILL.; Capper Revises Measure on Speculation, Which Failed Last Season. //// Abstract: WASHINGTON, April 9.--An AntiGrain Gambling gill will be introduced in the new Congress by Senator Capper and Representative Tincher, both of Kansas.
## 6 Headline: CLAIMS OIL FIELDS FOR THE CHEROKEES; Suit in Federal Supreme Court Says Old Patent Gives Right to 14,000,000 Acres. IN TEXAS AND OKLAHOMA But Cherokee Leaders Say Suit Is Unauthorized--Oklahoma Official Doubts Claim. //// Abstract: WASHINGTON, April 9.--Claim to 14,000,000 acres of land in Texas and Oklahoma, including the cities of Tulsa and Oklahoma City and practically the entire Burk-Burnett oil field, was filed in the Supreme Court today in behalf of the Cherokee Indiana Nation, supported by a government patent and by President Van Buren and certified as authentic by the Interior Department.
## 7 Headline: ENJOINS SHIPPING TIE-UP.; Baltimore Judge Orders Strikers Not to Interfere With Southern Co. //// Abstract:
## 8 Headline: FUGITIVE NAVAL LIEUTENANT FOUND; Lynam, Accused of $25,000 Embezzlement, Recognized in Montreal. TAKEN BACK TO BOSTON Denies Theft, Declaring Money Was Stolen From Him on a Railroad Train. //// Abstract: BOSTON, April 9.--Lieutenant Joseph J. Lynam, the naval paymaster whose disappearance three weeks ago was followed by charges of embezzlement of $25,000, was returned here from Montreal today, arraigned before a Federal Commissioner, and held in $25,000 bonds for the Grand Jury.
## 9 Headline: GERMANY FROM WITHIN; Riotous Living of War Profiteers Contrasted With Misery of Poor //// Abstract: THERE is no question but that the future of Germany depends upon the solution of the food problem. Insufficient nourishment and the lack of fertilizers constitute the greatest obstacles to the rehabilitation of German agricultural life.
## 10 Headline: TWO IN COSTUME RECITALS.; Lydia Lipkowska and Nina Tarasova in Native Dress. //// Abstract:
## 11 Headline: RICE IS BILLIARD VICTOR.; Handicap Helps Him to Defeat Klinger in Poggenburg Tourney. //// Abstract:
## 12 Headline: LABOR WILL FIGHT LUSK SCHOOL BILLS; Delegation Will Be Sent to Albany to Protest Against Proposed Measures. //// Abstract:
## 13 Headline: NEW YORK ASSOCIATED BANKS //// Abstract:
## 14 Headline: Obituary 6 -- No Title //// Abstract:
## 15 Headline: MISS MOFFAT BRIDE OF JOHN C. WHITE; Daughter of Mrs. R.B. Moffat Marries Ex-Ambassador's Son in St. James's Church. KATHERINE T. GALVIN WEDS Marries Major P.C. Kalloch, U.S. A., at Her Home--Nuptial of W. H. Bamford Jr. and Miss Ward. //// Abstract: The wedding of John Campbell White, a son of Henry White, formerly United States Ambassador to France and to Italy, and a delegate to the Versailles Peace Conference, and Miss Elizabeth Barclay Moffat, daughter of Mrs. R. Burnham Moffat, ...While the names are different we can see the newsworthy events of a hundred years ago are not so different from the events we read about today. There was wage issues, land claim disputes, white-collar crime, and sports updates. Overall, finding out about this API is inspiring as this has the potential to streamline research projects. The archive API is also a great compliment to the NY Times TimesMachine which provides images of the actual printed versions of past papers, a feature that is the main reason for keeping my NY Times subscription at the moment.
Jonathan D. Fitzgerald (2018) Working with The New York Times API in R