Web APIs

The New York Times web site provides a rich set of APIs, as described here: http://developer.nytimes.com/docs. You’ll need to start by signing up for an API key. Your task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it to an R dataframe.

I will begin with an articles API search at the NY Times for articles containing the key word saturn between the dates of October 1, 2018 to October 31, 2018 and read in the first page.

library(jsonlite)
library(httr)
library(tidyverse)

API_url <- paste0("https://api.nytimes.com/svc/search/v2/articlesearch.json?q=saturn?begin_date=20181001?end_date=20181031&api-key=", API_Key)

NY_search <- fromJSON(API_url, flatten = TRUE) %>% data.frame()

#str(NY_search)
ncol(NY_search)
## [1] 29
nrow((NY_search))
## [1] 10

So, now I have a dataframe with 10 observations of 29 variables.

baseurl <- API_url
initialQuery <- fromJSON(baseurl)
maxPages <- ((initialQuery$response$meta$hits[1] / 10)-1) 
maxPages
## [1] 783.6

The query only read in the first of 784 pages returned in the search and each page has 10 articles. In total there are 7836 articles returned by the keyword search for Saturn. That’s an amazing number of articles on Saturn from 10-01-18 to 10-31-18.

Now read in 4 pages total to get a few more articles:

pages <- list()
for(i in 0:3){
    nytSearch <- fromJSON(paste0(baseurl, "&page=", i), flatten = TRUE) %>% data.frame() 
    message("Retrieving page ", i)
    pages[[i+1]] <- nytSearch 
    Sys.sleep(1) 
}
## Retrieving page 0
## Retrieving page 1
## Retrieving page 2
## Retrieving page 3
allNYTSearch <- rbind_pages(pages)

Now that I have the dataframe with approximately 40 articles, I will select just the information about articles and display the headlines, snippets and authors.

NYTimes_saturn_articles <- allNYTSearch %>% 
    filter(response.docs.document_type == "article") %>%
    select(Headline = response.docs.headline.main, Author = response.docs.byline.original, Summary = response.docs.snippet)

I now we have a dataframe with 31 observations of 3 variables printed as a table.

str(NYTimes_saturn_articles)
## 'data.frame':    31 obs. of  3 variables:
##  $ Headline: chr  "Letter of Recommendation: Saturn" "Cassini Flies Toward a Fiery Death on Saturn" "Back to Saturn? Five Missions Proposed to Follow Cassini" "Cassini Vanishes Into Saturn, Its Mission Celebrated and Mourned" ...
##  $ Author  : chr  "By MARTA BAUSELLS" "By DENNIS OVERBYE" "By KENNETH CHANG" "By KENNETH CHANG" ...
##  $ Summary : chr  "The climate is cold as hell, but, oh, those rings." "Launched in 1997, the Cassini-Huygens mission has reshaped scientific understanding of the solar system’s most "| __truncated__ "Although NASA does not have yet a follow-up mission to Saturn on its schedule, scientists are dreaming up ideas for one." "Orbiting the ringed planet since 2004, the spacecraft solved some mysteries and made discoveries that upended o"| __truncated__ ...
knitr::kable(NYTimes_saturn_articles)
Headline Author Summary
Letter of Recommendation: Saturn By MARTA BAUSELLS The climate is cold as hell, but, oh, those rings.
Cassini Flies Toward a Fiery Death on Saturn By DENNIS OVERBYE Launched in 1997, the Cassini-Huygens mission has reshaped scientific understanding of the solar system’s most exotic planet and its mysterious moons.
Back to Saturn? Five Missions Proposed to Follow Cassini By KENNETH CHANG Although NASA does not have yet a follow-up mission to Saturn on its schedule, scientists are dreaming up ideas for one.
Cassini Vanishes Into Saturn, Its Mission Celebrated and Mourned By KENNETH CHANG Orbiting the ringed planet since 2004, the spacecraft solved some mysteries and made discoveries that upended our notions about the solar system.
The ‘Sounds’ of Space as NASA’s Cassini Dives by Saturn By KENNETH CHANG The spacecraft recorded some light patter as it passed between Saturn and its innermost ring, when scientists had expected the sound of “driving through Iowa in a hailstorm.”
Cassini’s Grand Finale: A Dive Between Saturn and Its Rings By DENNIS OVERBYE The spacecraft is set to venture into the gap between Saturn and its innermost ring 22 times until Sept. 15, then crash into the planet.
Pan, Moon of Saturn, Looks Like a Cosmic Ravioli (or Maybe a Walnut) By KENNETH CHANG In photographs by NASA’s Cassini spacecraft, the closest images ever taken, the tiny, wrinkly moon has a deep ridge that could be a couple of miles high.
Electronics Retailer Ceconomy Blames Hot Weather for Poor Sales By REUTERS Ceconomy, Europe’s biggest consumer electronics retailer, reported a 3.8 percent drop in sales in its fiscal fourth quarter, which it blamed on unusually hot weather in July and August in its home market of Germany.
Finalists in NASA’s Spacecraft Sweepstakes: A Drone on Titan, and a Comet-Chaser By KENNETH CHANG The missions, known as Dragonfly and Caesar, were selected from a dozen proposals made in the agency’s New Frontiers competition.
How Cassini Will Begin Its Date With Death on Saturn By DENNIS OVERBYE On Wednesday, the spacecraft that has studied Saturn for a dozen years will commence a series of maneuvers that will graze the planet’s rings.
79 Moons of Jupiter and Counting By KENNETH CHANG The latest survey of the region around the gas giant turned up a dozen new moons, including an oddball that was going in the wrong direction.
Plumes From Saturn’s Moon Enceladus Hint That It Could Support Life By KENNETH CHANG Data from the Cassini spacecraft suggest that hydrothermal vents could provide ingredients for microbes or other forms of alien life to exist.
Bradford Smith, Who Showed Postcards From Outer Space, Dies at 86 By DENNIS OVERBYE As head of the camera team for the Voyager mission, Dr. Smith was humanity’s tour guide to the solar system.
Etymological Origins By DEB AMLEN This is the dawning of another solving week, led by Alex Eaton-Salners.
The Paradoxes and the Glory of Apollo 8’s Journey Around the Moon By M. G. LORD Fifty years after the spacecraft became the first to leave Earth’s orbit, Robert Kurson tells the story of the remarkable odyssey in “Rocket Men.”
Mars Is Frigid, Rusty and Haunted. We Can’t Stop Looking at It. By DENNIS OVERBYE An oasis in the sky inspires the imagination. A series of discoveries refreshes our yearning for the red planet.
A Goblin World That Points Toward Hidden Planet Nine in the Solar System By KENNETH CHANG What astronomers have found about the curious orbit of a small ice world far away reinforces the idea that a large world is hidden out in the solar system.
Louis Vuitton’s Future Shock By VANESSA FRIEDMAN The spring collections are a wrap, and there was not a hoodie in sight. Instead there were Miu Miu’s glamour pusses and Nicolas Ghesquière’s intergalactic power players.
Mission to Saturn’s Moon By NATALIE PROULX What’s the story behind this image?
New Dive Into Old Data Finds Plumes Erupt From Jupiter’s Moon Europa By KENNETH CHANG A re-examination of old data shows a NASA spacecraft may have flown through a plume in 1997. The plumes could offer hints of life on the ice-encrusted moon.
NASA’s Jupiter Mission Reveals the ‘Brand-New and Unexpected’ By KENNETH CHANG Observations taken from the first few orbits of the Juno spacecraft provide a glimpse of the interior, the poles and the equator of the solar system’s largest planet.
A Helicopter on Mars? NASA Wants to Try By KENNETH CHANG The space agency’s next Martian rover, currently scheduled for a 2020 launch, is to carry a four-pound helicopter.
Led by a Vietnam War Opus, a Rich Season of Documentaries By NEIL GENZLINGER The fall season’s documentaries show the form’s strengths and versatility, with the Ken Burns opus on Vietnam dominating the schedule.
Watch Saturn Shine on Friday, No Equipment Required By NICHOLAS ST. FLEUR Our solar system’s sixth planet will be at opposition, providing one of the best opportunities to see it with the naked eye.
Keep America’s Roadside Weird By MARGARET RENKL We brake for giant chickens. And peaches. And Vulcans. You probably should, too.
Cassini’s Final Mission: Obliteration By NICHOLAS ST. FLEUR The spacecraft is preparing to plunge into Saturn’s atmosphere to ensure that nothing from Earth is left on the planets moons, which could host life.
This Asteroid Shouldn’t Be Where Astronomers Found It By NICHOLAS ST. FLEUR Space rocks like 2004 EW95 are common between Mars and Jupiter. The discovery of the object near Neptune may provide insights into how the planets formed.
Distant Ringed Object Could Be ‘Saturn on Steroids’ By NICHOLAS ST. FLEUR A potential planet detected by astronomers probably has rings that spin in the opposite direction of the object’s spin around its sun.
NASA Is Cooler Than Ryan Gosling By SHANNON STIRONE Heralded in film for its past, but underfunded and underappreciated, the agency continues to give us insight and breakthroughs.
Settling Arguments About Hydrogen With 168 Giant Lasers By KENNETH CHANG Scientists at Lawrence Livermore National Laboratory said they were “converging on the truth” in an experiment to understand hydrogen in its liquid metallic state.
Kilauea Volcano’s Lava Fields Offer Scientists a Portal to Mars By MIKE IVES Scientists are studying the Hawaiian volcano as part of a NASA-led project to answer questions like how life on Mars could have developed — if it ever did.