The New York Times web site provides a rich set of APIs, as described here: http://developer.nytimes.com/docs You’ll need to start by signing up for an API key.
Your task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it to an R dataframe.
API Key = LcgRcKkHVsLH2Hmj6La1HTACd0CR7eAn
Load libraries.
library(httr)
library(stringr)
library(jsonlite)
library(kableExtra)I’ll use a query search for our Top Stories NY Times API listing.
# Create an api.key (that was obtained from the NY Times website)
api.key <- "&api-key=LcgRcKkHVsLH2Hmj6La1HTACd0CR7eAn"
# Paste together the URL with the query and with the API key
url <- paste("http://api.nytimes.com/svc/topstories/v2/home.json?")
url.withkey <- paste(url,api.key, sep="")# Let's make sure that the information we are getting from this API is
# in the JSON format We will use the GET function from the httr package.
nyt.info <- GET("http://api.nytimes.com/svc/topstories/v2/home.json")
nyt.json <- fromJSON(url.withkey)
names(nyt.json)## [1] "status" "copyright" "section" "last_updated"
## [5] "num_results" "results"nyt.df <- nyt.json$results
kable(head(nyt.df,3))| section | subsection | title | abstract | url | byline | item_type | updated_date | created_date | published_date | material_type_facet | kicker | des_facet | org_facet | per_facet | geo_facet | multimedia | short_url |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| World | Africa | Russias Military Mission Creep Advances to a New Front: Africa | In an influence campaign that is worrying the Pentagon, Russias weapons sales, military training programs and security agreements are growing across Africa. | https://www.nytimes.com/2019/03/31/world/africa/russia-military-africa.html | By ERIC SCHMITT | Article | 2019-03-31T15:41:08-04:00 | 2019-03-31T15:41:08-04:00 | 2019-03-31T15:41:08-04:00 | c(“United States Defense and Military Forces”, “United States International Relations”, “Defense and Military Forces”, “Arms Trade”, “United States Politics and Government”, “Mercenaries and Private Military Contractors”, “Cold War Era”) | Defense Department | Waldhauser, Thomas D | c(“Africa”, “USSR (Former Soviet Union)”, “Russia”) | list(url = c(“https://static01.nyt.com/images/2019/03/28/us/politics/00dc-military1-promo/00dc-military1-promo-thumbStandard-v2.jpg",”https://static01.nyt.com/images/2019/03/28/us/politics/00dc-military1-promo/00dc-military1-promo-thumbLarge-v2.jpg", “https://static01.nyt.com/images/2019/03/28/us/politics/00dc-military1-promo/00dc-military1-promo-articleInline-v2.jpg",”https://static01.nyt.com/images/2019/03/28/us/politics/00dc-military1-promo/00dc-military1-promo-mediumThreeByTwo210-v2.jpg", “https://static01.nyt.com/images/2019/03/28/us/politics/00dc-military1-promo/00dc-military1-promo-superJumbo-v2.jpg" ), format = c(”Standard Thumbnail“,”thumbLarge“,”Normal“,”mediumThreeByTwo210“,”superJumbo“), height = c(75, 150, 127, 140, 1365), width = c(75, 150, 190, 210, 2048), type = c(”image“,”image“,”image“,”image“,”image“), subtype = c(”photo“,”photo“,”photo“,”photo“,”photo“), caption = c(”New recruits for the Central African Republics armed forces. Members have trained with Russian consultants as part of Moscows growing influence across Africa.“,”New recruits for the Central African Republics armed forces. Members have trained with Russian consultants as part of Moscows growing influence across Africa.“,”New recruits for the Central African Republics armed forces. Members have trained with Russian consultants as part of Moscows growing influence across Africa.“,”New recruits for the Central African Republics armed forces. Members have trained with Russian consultants as part of Moscows growing influence across Africa.“,”New recruits for the Central African Republics armed forces. Members have trained with Russian consultants as part of Moscows growing influence across Africa.“), copyright = c(”Florent Vergnes/Agence France-Presse Getty Images“,”Florent Vergnes/Agence France-Presse Getty Images“,”Florent Vergnes/Agence France-Presse Getty Images“,”Florent Vergnes/Agence France-Presse Getty Images“,”Florent Vergnes/Agence France-Presse Getty Images“)) | https://nyti.ms/2FMh34S | ||
| World | Africa | Political Handlers With Trump Ties Take Their Election Playbooks to Africa | American campaign consultants, including one mentored by Paul Manafort, are increasingly being hired by African candidates to help them win at the polls in still fragile democracies. | https://www.nytimes.com/2019/03/31/world/africa/africa-trump-manafort-political-consultants.html | By DIONNE SEARCEY | Article | 2019-03-31T13:03:59-04:00 | 2019-03-31T13:03:59-04:00 | 2019-03-31T13:03:59-04:00 | c(“Politics and Government”, “Elections”, “Lobbying and Lobbyists”, “Consultants”, “Public Relations and Publicity”) | character(0) | c(“Manafort, Paul J”, “Abubakar, Atiku”, “Riva Levinson”) | c(“Nigeria”, “Cameroon”, “Gabon”, “United States”) | list(url = c(“https://static01.nyt.com/images/2019/03/14/world/Nigeria-Manafort5/Nigeria-Manafort5-thumbStandard.jpg",”https://static01.nyt.com/images/2019/03/14/world/Nigeria-Manafort5/Nigeria-Manafort5-thumbLarge.jpg", “https://static01.nyt.com/images/2019/03/14/world/Nigeria-Manafort5/fileupload-1552590747680-articleInline.jpg",”https://static01.nyt.com/images/2019/03/14/world/Nigeria-Manafort5/Nigeria-Manafort5-mediumThreeByTwo210.jpg", “https://static01.nyt.com/images/2019/03/14/world/Nigeria-Manafort5/fileupload-1552590747680-superJumbo.jpg" ), format = c(”Standard Thumbnail“,”thumbLarge“,”Normal“,”mediumThreeByTwo210“,”superJumbo“), height = c(75, 150, 126, 140, 1363), width = c(75, 150, 190, 210, 2048), type = c(”image“,”image“,”image“,”image“,”image“), subtype = c(”photo“,”photo“,”photo“,”photo“,”photo“), caption = c(”Riva Levinson, a political consultant once employed by Paul Manafort, has worked for both opposition parties and governments in Africa.“,”Riva Levinson, a political consultant once employed by Paul Manafort, has worked for both opposition parties and governments in Africa.“,”Riva Levinson, a political consultant once employed by Paul Manafort, has worked for both opposition parties and governments in Africa.“,”Riva Levinson, a political consultant once employed by Paul Manafort, has worked for both opposition parties and governments in Africa.“,”Riva Levinson, a political consultant once employed by Paul Manafort, has worked for both opposition parties and governments in Africa.“), copyright = c(”Tom Williams/CQ Roll Call“,”Tom Williams/CQ Roll Call“,”Tom Williams/CQ Roll Call“,”Tom Williams/CQ Roll Call“,”Tom Williams/CQ Roll Call“)) | https://nyti.ms/2FMFjDL | ||
| World | Europe | Russia Ordered a Killing That Made No Sense. Then the Assassin Started Talking. | It turned out the killer of a Ukrainian electrician was working for Russian intelligence agents. | https://www.nytimes.com/2019/03/31/world/europe/russian-assassinations-putin-ukraine.html | By MICHAEL SCHWIRTZ | Article | 2019-03-31T02:59:11-04:00 | 2019-03-31T02:59:11-04:00 | 2019-03-31T02:59:11-04:00 | c(“Espionage and Intelligence Services”, “Assassinations and Attempted Assassinations”, “Defense and Military Forces”, “Politics and Government”) | c(“Federal Security Service”, “GRU (Russia)”, “KGB”) | character(0) | c(“Russia”, “Ukraine”) | list(url = c(“https://static01.nyt.com/images/2019/03/31/world/SUB31Assassin3/xxassassin2-thumbStandard.jpg",”https://static01.nyt.com/images/2019/03/31/world/SUB31Assassin3/xxassassin2-thumbLarge.jpg", “https://static01.nyt.com/images/2019/03/31/world/SUB31Assassin3/merlin_147801636_79b50f4e-25c1-4983-9c8e-90b6ed046d2d-articleInline.jpg",”https://static01.nyt.com/images/2019/03/31/world/SUB31Assassin3/SUB31Assassin3-mediumThreeByTwo210-v2.jpg", “https://static01.nyt.com/images/2019/03/31/world/SUB31Assassin3/merlin_147801636_79b50f4e-25c1-4983-9c8e-90b6ed046d2d-superJumbo.jpg" ), format = c(”Standard Thumbnail“,”thumbLarge“,”Normal“,”mediumThreeByTwo210“,”superJumbo“), height = c(75, 150, 127, 140, 1365), width = c(75, 150, 190, 210, 2048), type = c(”image“,”image“,”image“,”image“,”image“), subtype = c(”photo“,”photo“,”photo“,”photo“,”photo“), caption = c(”The headstone of Ivan Mamchur, a Ukrainian prison guard and military veteran who was murdered in Rivne, a city in western Ukraine.“,”The headstone of Ivan Mamchur, a Ukrainian prison guard and military veteran who was murdered in Rivne, a city in western Ukraine.“,”The headstone of Ivan Mamchur, a Ukrainian prison guard and military veteran who was murdered in Rivne, a city in western Ukraine.“,”The headstone of Ivan Mamchur, a Ukrainian prison guard and military veteran who was murdered in Rivne, a city in western Ukraine.“,”The headstone of Ivan Mamchur, a Ukrainian prison guard and military veteran who was murdered in Rivne, a city in western Ukraine.“), copyright = c(”Joseph Sywenkyj for The New York Times“,”Joseph Sywenkyj for The New York Times“,”Joseph Sywenkyj for The New York Times“,”Joseph Sywenkyj for The New York Times“,”Joseph Sywenkyj for The New York Times“)) | https://nyti.ms/2UpxxI0 |
class(nyt.df)## [1] "data.frame"I have successfully used an API, downloaded the JSON information, and converted into a data frame.
kable(nyt.df$title)| x |
|---|
| Russias Military Mission Creep Advances to a New Front: Africa |
| Political Handlers With Trump Ties Take Their Election Playbooks to Africa |
| Russia Ordered a Killing That Made No Sense. Then the Assassin Started Talking. |
| ‘Breaches Everywhere’: Flooding Bursts Midwest Levees, and Tough Questions Follow |
| Joe Biden Says He Did Not Act Inappropriately with Lucy Flores |
| Who Is Lucy Flores, the Woman Accusing Joe Biden of Kissing Her? |
| Elizabeth Warren Loses Finance Director as She Struggles in 2020 Race |
| New York State Budget Deal Brings Congestion Pricing, Plastic Bag Ban and Mansion Tax |
| Plastic Bags to Be Banned in New York; Second Statewide Ban, After California |
| Over $10 to Drive in Manhattan? What We Know About the Congestion Pricing Plan |
| Michael Avenattis Biggest Case Yet: His Own |
| 11 of Our Best Weekend Reads |
| Brexit, Offshore Drilling, N.C.A.A.: Your Weekend Briefing |
| Brexit, Indian Elections, the Kardashians: Your Monday Briefing |
| Rwandas Children of Rape Have Come of Age |
| The Best Restaurant if Youre Over 50 |
| Why Are We Still Slaughtering the American Bison? |
| New Yorks Best Schools Need to Do Better |
| Bret Easton Ellis Has Calmed Down. He Thinks You Should, Too. |
| The Legend of Zion |
| On ‘S.N.L.,’ Trump, Mueller and Barr Interpret the Final Report Very Differently |
| Migrant Camp Under El Paso Bridge Has Been Cleared Out |
| Teenager Fatally Shot After Knocking on Wrong Apartment Door in Atlanta, Police Say |
| ‘Dumbo’ Falls Short of Box-Office Expectations but Still Comes in First |
| Huge Crowds of Algerians Protest Armys Latest Compromise Offer |
| Mark Zuckerbergs Call to Regulate Facebook, Explained |
| Freed From ISIS, Syrians Struggle to Rebuild Raqqa |
| Kenneth Gibson, 86, Dies; Newark Mayor Broke Race Barrier in Northeast |
| ‘Race-Biased Dragnet’: DNA From 360 Black Men Was Collected to Solve Vetrano Murder, Defense Lawyers Say |
| The Landlord Wants Facial Recognition in Its Rent-Stabilized Buildings. Why? |
| What to Do When Youre Bored With Your Routines |
| Productivity Isnt About Time Management. Its About Attention Management. |
| Stop Letting Modern Distractions Steal Your Attention |
| 5 Cheap(ish) Things to Help Get Rid of Distractions |
| The Playlist: Sky Ferreiras Moody Re-emergence, and 13 More New Songs |
| For Two Longtime Acquaintances, a Second Chance at Love |
| How the Brewmaster of Brooklyn Brewery Spends His Sundays |
| Power and Love: What We Can All Learn From Wagners ‘Ring’ |
| Easy, Versatile, Fantastic |
| I Detest the N.R.A. What Should I Do With My Gun? |
| Plastic Bags, or Paper? Heres What to Consider When You Hit the Grocery Store |
| Saturns Rings Are Sculpted by a Crew of Mini-Moons |
| How Pain Tolerance and Anxiety Seem to Be Connected |
| Eyes on the Road! (Your Car Is Watching) |
# get article sections/segments
kable(unique(nyt.df$section))| x |
|---|
| World |
| U.S. |
| New York |
| Sports |
| Reader Center |
| Briefing |
| Opinion |
| Books |
| Arts |
| Movies |
| Technology |
| Obituaries |
| Smarter Living |
| Fashion |
| Food |
| Magazine |
| Climate |
| Science |
| Health |
| Business |
Here I choose section that I’m interested in .Let’s choose US articles
# Titles
subset(nyt.df$title,nyt.df$section == 'U.S.')## [1] "Breaches Everywhere: Flooding Bursts Midwest Levees, and Tough Questions Follow"
## [2] "Joe Biden Says He Did Not Act Inappropriately with Lucy Flores"
## [3] "Who Is Lucy Flores, the Woman Accusing Joe Biden of Kissing Her?"
## [4] "Elizabeth Warren Loses Finance Director as She Struggles in 2020 Race"
## [5] "Migrant Camp Under El Paso Bridge Has Been Cleared Out"
## [6] "Teenager Fatally Shot After Knocking on Wrong Apartment Door in Atlanta, Police Say"# Abstracts
subset(nyt.df$abstract,nyt.df$section == 'U.S.')## [1] "Hundreds of miles of levees in the Midwest have been overwhelmed by the floods, leaving Swiss cheese infrastructure and reigniting a flood control debate."
## [2] "The former vice president, accused by Ms. Flores, a Nevada Democrat, of kissing and touching her, did acknowledge expressions of affection on campaign trail."
## [3] "Ms. Flores, 39, is a former Nevada state assemblywoman who lost campaigns to become the states lieutenant governor and to represent the Fourth District in Congress."
## [4] "Money is a problem for the Democratic senator, who is likely to be substantially out-raised in the first quarter. And shes not the fresh candidate anymore. But much time remains."
## [5] "Authorities appear to have moved families out of a makeshift enclosure under a bridge in El Paso, with much of the site cleared on Sunday."
## [6] "Omarian Banks, 19, thought he was outside an apartment he shared with his girlfriend, but a man inside confronted him and opened fire, the police said."