Webscraping to Find Key DS Skills

Kai Lukowiak

2017-10-28

Data 607 Week 9 Assignment

webscraping from an API

Your task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it to an R dataframe.

  • Data can be collected here
library(knitr)
library(dplyr)
library(stringr)
library(jsonlite)
library(httr)
library(pander)

The API key and url to extract the most popular opinion peices over the last thirty days are here.

apiKey <-  "&api-key=b95585fd6b264ab08e28fb0e96998aae"
urlBase <-"https://api.nytimes.com/svc/mostpopular/v2/mostshared/Opinion/30.json?"
nytURL <- paste(urlBase, apiKey, sep = "")

We use the GET command to extract a json like object from the API

nytData <- GET(nytURL)
content(nytData, 'text')
## [1] "{\"status\":\"OK\",\"copyright\":\"Copyright (c) 2017 The New York Times Company.  All Rights Reserved.\",\"num_results\":276,\"results\":[{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/09\\/30\\/opinion\\/hugh-hefner.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By ROSS DOUTHAT\",\"title\":\"Speaking Ill of Hugh Hefner\",\"abstract\":\"An honest obituary for a wicked American.\",\"published_date\":\"2017-09-30\",\"source\":\"The New York Times\",\"asset_id\":100000005467201,\"total_shares\":1,\"des_facet\":[\"LIBERALISM (US POLITICS)\",\"WOMEN AND GIRLS\",\"SEX\"],\"org_facet\":[\"PLAYBOY MAGAZINE\",\"PORNOGRAPHY\",\"CELEBRITIES\"],\"per_facet\":[\"HEFNER, HUGH\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Hugh Hefner attending a Playmate of the Year luncheon in 2013 at the Playboy Mansion in Los Angeles. Mr. Hefner died on Wednesday.\",\"copyright\":\"Charley Gallay\\/Getty Images, for Playboy\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/01\\/opinion\\/sunday\\/01douthat\\/01douthat-thumbStandard-v2.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/01\\/opinion\\/sunday\\/01douthat\\/01douthat-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/01\\/opinion\\/sunday\\/01douthat\\/01douthat-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/interactive\\/2017\\/10\\/08\\/opinion\\/editorials\\/republican-etiquette-guide.html\",\"column\":\"\",\"section\":\"Opinion\",\"byline\":\"By THE EDITORIAL BOARD\",\"title\":\"The Republican\\u2019s Guide to Presidential Etiquette\",\"abstract\":\"Republicans used to care a whole lot about how a president comported himself. Here is what they now condone.\",\"published_date\":\"2017-10-08\",\"source\":\"The New York Times\",\"asset_id\":100000005483715,\"total_shares\":2,\"des_facet\":[\"WHITE HOUSE BUILDING (WASHINGTON, DC)\",\"UNITED STATES POLITICS AND GOVERNMENT\",\"ETHICS AND OFFICIAL MISCONDUCT\",\"CUSTOMS, ETIQUETTE AND MANNERS\"],\"org_facet\":\"\",\"per_facet\":[\"TRUMP, DONALD J\"],\"geo_facet\":\"\",\"media\":\"\"},{\"url\":\"https:\\/\\/www.nytimes.com\\/interactive\\/2017\\/10\\/04\\/opinion\\/thoughts-prayers-nra-funding-senators.html\",\"column\":\"\",\"section\":\"Opinion\",\"byline\":\"By DAVID LEONHARDT, IAN PRASAD PHILBRICK and STUART A. THOMPSON\",\"title\":\"The Congress Members Receiving the Most N.R.A. Funding\",\"abstract\":\"Thoughts and prayers and the N.R.A.: Here are the top career recipients of N.R.A. campaign money.\",\"published_date\":\"2017-10-04\",\"source\":\"The New York Times\",\"asset_id\":100000005478110,\"total_shares\":3,\"des_facet\":[\"GUN CONTROL\"],\"org_facet\":[\"NATIONAL RIFLE ASSN\",\"SENATE\",\"HOUSE OF REPRESENTATIVES\"],\"per_facet\":\"\",\"geo_facet\":\"\",\"media\":\"\"},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/13\\/opinion\\/mayim-bialik-feminist-harvey-weinstein.html\",\"column\":\"Op-Ed Contributor\",\"section\":\"Opinion\",\"byline\":\"By MAYIM BIALIK\",\"title\":\"Mayim Bialik: Being a Feminist in Harvey Weinstein\\u2019s World\",\"abstract\":\"I have always had an uncomfortable relationship with being employed in an industry built on the objectification of women.\",\"published_date\":\"2017-10-13\",\"source\":\"The New York Times\",\"asset_id\":100000005493485,\"total_shares\":4,\"des_facet\":[\"WOMEN AND GIRLS\",\"ACTORS AND ACTRESSES\",\"TELEVISION\",\"WOMEN'S RIGHTS\"],\"org_facet\":\"\",\"per_facet\":[\"WEINSTEIN, HARVEY\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Mayim Bialik as Blossom Russo in season one of \\u201cBlossom.\\u201d\",\"copyright\":\"Alice S. Hall\\/NBC, via Getty Images\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/13\\/opinion\\/13bialik3web\\/13bialik3web-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/13\\/opinion\\/13bialik3web\\/13bialik3web-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/13\\/opinion\\/13bialik3web\\/13bialik3web-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/03\\/opinion\\/stephen-paddock-las-vegas-terrorism.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By THOMAS L. FRIEDMAN\",\"title\":\"If Only Stephen Paddock Were a Muslim\",\"abstract\":\"What happens when the killer\\u2019s country of origin is us?\",\"published_date\":\"2017-10-03\",\"source\":\"The New York Times\",\"asset_id\":100000005475422,\"total_shares\":5,\"des_facet\":[\"GUN CONTROL\",\"TERRORISM\",\"UNITED STATES POLITICS AND GOVERNMENT\"],\"org_facet\":\"\",\"per_facet\":[\"PADDOCK, STEPHEN C (1953-2017)\"],\"geo_facet\":[\"LAS VEGAS (NEV)\"],\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Las Vegas, Monday.\",\"copyright\":\"Hilary Swift for The New York Times\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/04\\/opinion\\/04friedmanWeb\\/04friedmanWeb-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/04\\/opinion\\/04friedmanWeb\\/04friedmanWeb-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/04\\/opinion\\/04friedmanWeb\\/04friedmanWeb-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/interactive\\/2017\\/10\\/02\\/opinion\\/editorials\\/mass-shootings-congress.html\",\"column\":\"\",\"section\":\"Opinion\",\"byline\":\"By THE EDITORIAL BOARD\",\"title\":\"477 Days. 521 Mass Shootings. Zero Action From Congress.\",\"abstract\":\"Political leaders have done nothing to curb the bloodshed since the Pulse nightclub massacre in Orlando last year.\",\"published_date\":\"2017-10-02\",\"source\":\"The New York Times\",\"asset_id\":100000005474681,\"total_shares\":6,\"des_facet\":[\"MURDERS, ATTEMPTED MURDERS AND HOMICIDES\",\"LAS VEGAS, NEV, SHOOTING (OCTOBER, 2017)\",\"ORLANDO, FLA, SHOOTING (2016)\"],\"org_facet\":[\"HOUSE OF REPRESENTATIVES\",\"SENATE\"],\"per_facet\":\"\",\"geo_facet\":[\"LAS VEGAS (NEV)\"],\"media\":\"\"},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/17\\/opinion\\/columnists\\/weinstein-harassment-witchunt.html\",\"column\":\"Contributing Op-Ed Writer\",\"section\":\"Opinion\",\"byline\":\"By LINDY WEST\",\"title\":\"Yes, This Is a Witch Hunt. I\\u2019m a Witch and I\\u2019m Hunting You.\",\"abstract\":\"Sorry, Woody Allen. I know you hate rumor mills, but sometimes they\\u2019re our only recourse.\",\"published_date\":\"2017-10-17\",\"source\":\"The New York Times\",\"asset_id\":100000005501461,\"total_shares\":7,\"des_facet\":[\"SEXUAL HARASSMENT\",\"WOMEN AND GIRLS\",\"SEX CRIMES\"],\"org_facet\":\"\",\"per_facet\":[\"TRUMP, DONALD J\",\"WEINSTEIN, HARVEY\",\"ALLEN, WOODY\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Women on Friday protesting the Manhattan district attorney\\u2019s handling of accusations against Harvey Weinstein.\",\"copyright\":\"Holly Pickett for The New York Times\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/18\\/opinion\\/18westSubs\\/18westSubs-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/18\\/opinion\\/18westSubs\\/18westSubs-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/18\\/opinion\\/18westSubs\\/18westSubs-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/03\\/opinion\\/rosanne-cash-country-musicians-nra.html\",\"column\":\"Op-Ed Contributor\",\"section\":\"Opinion\",\"byline\":\"By ROSANNE CASH\",\"title\":\"Rosanne Cash: Country Musicians, Stand Up to the N.R.A.\",\"abstract\":\"Patriotism is not antithetical to gun control.\",\"published_date\":\"2017-10-03\",\"source\":\"The New York Times\",\"asset_id\":100000005474509,\"total_shares\":8,\"des_facet\":[\"GUN CONTROL\",\"LOBBYING AND LOBBYISTS\"],\"org_facet\":[\"NATIONAL RIFLE ASSN\"],\"per_facet\":\"\",\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Vince Gill and Amy Grant pray during a candlelight vigil in Nashville for the victims of the shooting in Las Vegas.\",\"copyright\":\"Rick Diamond\\/Getty Images \",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/03\\/opinion\\/03cashSubs\\/03cashSubs-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/03\\/opinion\\/03cashSubs\\/03cashSubs-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/03\\/opinion\\/03cashSubs\\/03cashSubs-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/17\\/opinion\\/college-advice-professor.html\",\"column\":\"On Campus\",\"section\":\"Opinion\",\"byline\":\"By SUSAN SHAPIRO\",\"title\":\"College Advice I Wish I\\u2019d Taken\",\"abstract\":\"I\\u2019m a teacher, but as an undergraduate, I was the type of mediocre student I now disdain.\",\"published_date\":\"2017-10-17\",\"source\":\"The New York Times\",\"asset_id\":100000005493469,\"total_shares\":9,\"des_facet\":[\"COLLEGES AND UNIVERSITIES\",\"SOCIAL MEDIA\"],\"org_facet\":[\"TEACHERS AND SCHOOL EMPLOYEES\"],\"per_facet\":\"\",\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"\",\"copyright\":\"Kearin Ever Cook\\/Pratt Institute\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/17\\/opinion\\/17oncampus\\/17oncampus-thumbStandard-v2.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/17\\/opinion\\/17oncampus\\/17oncampus-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/17\\/opinion\\/17oncampus\\/17oncampus-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/15\\/opinion\\/columnists\\/trump-spite-obama-legacy.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By CHARLES M. BLOW\",\"title\":\"Trump, Chieftain of Spite\",\"abstract\":\"Trump can\\u2019t hold a candle to Obama, so he\\u2019s taking a tiki torch to Obama\\u2019s legacy.\",\"published_date\":\"2017-10-15\",\"source\":\"The New York Times\",\"asset_id\":100000005495325,\"total_shares\":10,\"des_facet\":[\"UNITED STATES POLITICS AND GOVERNMENT\",\"PATIENT PROTECTION AND AFFORDABLE CARE ACT (2010)\",\"FEDERAL AID (US)\"],\"org_facet\":\"\",\"per_facet\":[\"TRUMP, DONALD J\",\"OBAMA, BARACK\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Donald Trump and Barack Obama after Mr. Trump\\u2019s inauguration on January 20.\",\"copyright\":\"Ruth Fremson\\/The New York Times\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/16\\/opinion\\/16blowWEB\\/16blowWEB-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/16\\/opinion\\/16blowWEB\\/16blowWEB-mediumThreeByTwo210-v3.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/16\\/opinion\\/16blowWEB\\/16blowWEB-mediumThreeByTwo440-v3.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/09\\/opinion\\/harvey-weinstein-lena-dunham-silence-.html\",\"column\":\"Op-Ed Contributor\",\"section\":\"Opinion\",\"byline\":\"By LENA DUNHAM\",\"title\":\"Lena Dunham: Harvey Weinstein and the Silence of the Men\",\"abstract\":\"The hush in Hollywood about the megaproducer only reinforces a culture that keeps women from speaking up.\",\"published_date\":\"2017-10-09\",\"source\":\"The New York Times\",\"asset_id\":100000005486608,\"total_shares\":11,\"des_facet\":[\"SEXUAL HARASSMENT\",\"WOMEN AND GIRLS\",\"MOVIES\"],\"org_facet\":[\"WEINSTEIN CO\"],\"per_facet\":[\"WEINSTEIN, HARVEY\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Harvey Weinstein at a party after the 2016 Academy Awards.\",\"copyright\":\"Axel Koester\\/Corbis, via Getty Images\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/09dunhamWeb\\/09dunhamWeb-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/09dunhamWeb\\/09dunhamWeb-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/09dunhamWeb\\/09dunhamWeb-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/09\\/25\\/opinion\\/colin-kaepernick-football-protests.html\",\"column\":\"Op-Ed Contributor\",\"section\":\"Opinion\",\"byline\":\"By ERIC REID\",\"title\":\"Eric Reid: Why Colin Kaepernick and I Decided to Take a Knee\",\"abstract\":\"We chose the gesture purposefully. Our bodies were like flags flown at half-mast to mark a tragedy.\",\"published_date\":\"2017-09-25\",\"source\":\"The New York Times\",\"asset_id\":100000005457221,\"total_shares\":12,\"des_facet\":[\"DEMONSTRATIONS, PROTESTS AND RIOTS\",\"POLICE BRUTALITY, MISCONDUCT AND SHOOTINGS\",\"BLACKS\"],\"org_facet\":[\"NATIONAL FOOTBALL LEAGUE\",\"SAN FRANCISCO 49ERS\"],\"per_facet\":[\"KAEPERNICK, COLIN\",\"REID, ERIC (1991- )\",\"TRUMP, DONALD J\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Colin Kaepernick, right, and Eric Reid kneeling during the national anthem before an N.F.L. game last year.\",\"copyright\":\"Marcio Jose Sanchez\\/Associated Press\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/09\\/27\\/insider\\/25reidSub\\/25reidSub-thumbStandard-v2.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/09\\/27\\/insider\\/25reidSub\\/25reidSub-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/09\\/27\\/insider\\/25reidSub\\/25reidSub-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/02\\/opinion\\/catalonia-spain-referendum.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By PATRICK CHAPPATTE\",\"title\":\"After Catalonia\\u2019s Chaotic Vote for Independence\",\"abstract\":\"Clashes with police during Sunday\\u2019s referendum left supporters on both sides even more polarized.\",\"published_date\":\"2017-10-02\",\"source\":\"The New York Times\",\"asset_id\":100000005474636,\"total_shares\":13,\"des_facet\":[\"REFERENDUMS\"],\"org_facet\":\"\",\"per_facet\":\"\",\"geo_facet\":[\"CATALONIA (SPAIN)\"],\"media\":\"\"},{\"url\":\"https:\\/\\/www.nytimes.com\\/interactive\\/2017\\/10\\/13\\/opinion\\/sunday\\/Silicon-Valley-Is-Not-Your-Friend.html\",\"column\":\"Opinion\",\"section\":\"Opinion\",\"byline\":\"By NOAM COHEN\",\"title\":\"Silicon Valley Is Not Your Friend\",\"abstract\":\"We are beginning to understand that tech companies don\\u2019t have our best interests at heart. Did they ever?\",\"published_date\":\"2017-10-13\",\"source\":\"The New York Times\",\"asset_id\":100000005494137,\"total_shares\":14,\"des_facet\":[\"COMPUTERS AND THE INTERNET\",\"ONLINE ADVERTISING\",\"SEARCH ENGINES\"],\"org_facet\":[\"AMAZON.COM INC\",\"FACEBOOK INC\",\"GOOGLE INC\"],\"per_facet\":\"\",\"geo_facet\":\"\",\"media\":\"\"},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/05\\/opinion\\/guns-second-amendment-nra.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By BRET STEPHENS\",\"title\":\"Repeal the Second Amendment\",\"abstract\":\"It sounds like political Mission Impossible. But it\\u2019s the only meaningful way to stop the killing.\",\"published_date\":\"2017-10-05\",\"source\":\"The New York Times\",\"asset_id\":100000005478925,\"total_shares\":15,\"des_facet\":[\"GUN CONTROL\",\"FIREARMS\",\"SECOND AMENDMENT (US CONSTITUTION)\"],\"org_facet\":[\"NATIONAL RIFLE ASSN\",\"LAS VEGAS, NEV, SHOOTING (OCTOBER, 2017)\",\"MURDERS, ATTEMPTED MURDERS AND HOMICIDES\"],\"per_facet\":[\"PADDOCK, STEPHEN C (1953-2017)\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"\",\"copyright\":\"Victor J. Blue\\/Bloomberg \",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/05\\/opinion\\/05stephensSub\\/05stephensSub-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/05\\/opinion\\/05stephensSub\\/05stephensSub-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/05\\/opinion\\/05stephensSub\\/05stephensSub-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/10\\/opinion\\/auschwitz-israel-holocaust.html\",\"column\":\"Op-Docs\",\"section\":\"Opinion\",\"byline\":\"By MATAN ROCHLITZ\",\"title\":\"I Have a Message for You\",\"abstract\":\"To save her own life, she left her father to die. Decades later, she got a message from him.\",\"published_date\":\"2017-10-10\",\"source\":\"The New York Times\",\"asset_id\":100000005486932,\"total_shares\":16,\"des_facet\":[\"HOLOCAUST AND THE NAZI ERA\",\"WORLD WAR II (1939-45)\",\"WAR CRIMES, GENOCIDE AND CRIMES AGAINST HUMANITY\",\"FAMILIES AND FAMILY LIFE\"],\"org_facet\":[\"AUSCHWITZ CONCENTRATION CAMP\"],\"per_facet\":\"\",\"geo_facet\":[\"TEL AVIV (ISRAEL)\",\"BELGIUM\"],\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"\",\"copyright\":\"Matan Rochlitz\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/10opdoc-message-alt-img\\/10opdoc-message-alt-img-thumbStandard-v3.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/10opdoc-message-alt-img\\/10opdoc-message-alt-img-mediumThreeByTwo210-v3.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/10opdoc-message-alt-img\\/10opdoc-message-alt-img-mediumThreeByTwo440-v3.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/13\\/opinion\\/us-soccer-world-cup.html\",\"column\":\"Sporting\",\"section\":\"Opinion\",\"byline\":\"By BRIAN PHILLIPS\",\"title\":\"The Real Failure of U.S. Men\\u2019s Soccer\",\"abstract\":\"The team flubbed its World Cup chance, but its leaders are more focused on spin control than improving the level of play.\",\"published_date\":\"2017-10-13\",\"source\":\"The New York Times\",\"asset_id\":100000005494571,\"total_shares\":17,\"des_facet\":[\"SOCCER\",\"WORLD CUP (SOCCER)\"],\"org_facet\":[\"UNITED STATES SOCCER FEDERATION\"],\"per_facet\":[\"ARENA, BRUCE\",\"GULATI, SUNIL\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Jozy Altidore (No. 17) missed a scoring chance against Trinidad and Tobago on Oct. 10.\",\"copyright\":\"Rebecca Blackwell\\/Associated Press\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/15\\/opinion\\/sunday\\/15sportingSUB\\/15sportingSUB-thumbStandard-v4.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/15\\/opinion\\/sunday\\/15sportingSUB\\/15sportingSUB-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/15\\/opinion\\/sunday\\/15sportingSUB\\/15sportingSUB-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/05\\/opinion\\/sunday\\/nuclear-north-korea.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By NICHOLAS KRISTOF\",\"title\":\"Inside North Korea, and Feeling the Drums of War\",\"abstract\":\"A visit to the reclusive country reveals the perilous moment we\\u2019re in.\",\"published_date\":\"2017-10-05\",\"source\":\"The New York Times\",\"asset_id\":100000005477022,\"total_shares\":18,\"des_facet\":[\"UNITED STATES INTERNATIONAL RELATIONS\",\"UNITED STATES DEFENSE AND MILITARY FORCES\",\"PROPAGANDA\"],\"org_facet\":[\"NUCLEAR WEAPONS\"],\"per_facet\":[\"WARMBIER, OTTO FREDERICK (1994- )\",\"KIM JONG-UN\"],\"geo_facet\":[\"NORTH KOREA\"],\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"A propaganda poster on a main street in the North Korean capital, Pyongyang, depicts a missile striking the United States Capitol. It reads: &ldquo;The response of the Democratic People&rsquo;s Republic of Korea.&rdquo;\",\"copyright\":\"Jonah M. Kessel\\/The New York Times\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/08\\/opinion\\/sunday\\/08Kristof-print\\/08Kristof1-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/08\\/opinion\\/sunday\\/08Kristof-print\\/08Kristof1-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/08\\/opinion\\/sunday\\/08Kristof-print\\/08Kristof1-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/19\\/opinion\\/lupita-nyongo-harvey-weinstein.html\",\"column\":\"Op-Ed Contributor\",\"section\":\"Opinion\",\"byline\":\"By LUPITA NYONG\\u2019O\",\"title\":\"Lupita Nyong\\u2019o: Speaking Out About Harvey Weinstein\",\"abstract\":\"I don\\u2019t feel alone anymore. Let us never shut up about this.\",\"published_date\":\"2017-10-19\",\"source\":\"The New York Times\",\"asset_id\":100000005505402,\"total_shares\":19,\"des_facet\":[\"MOVIES\",\"ACTORS AND ACTRESSES\",\"WOMEN AND GIRLS\"],\"org_facet\":[\"SEXUAL HARASSMENT\"],\"per_facet\":[\"WEINSTEIN, HARVEY\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Lupita Nyong\\u2019o in 2015.\",\"copyright\":\"Jesse Dittmar for The New York Times\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/20\\/opinion\\/19nyongoWeb\\/19nyongoWeb-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/20\\/opinion\\/19nyongoWeb\\/19nyongoWeb-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/20\\/opinion\\/19nyongoWeb\\/19nyongoWeb-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/09\\/08\\/opinion\\/sunday\\/what-the-rich-wont-tell-you.html\",\"column\":\"Opinion\",\"section\":\"Opinion\",\"byline\":\"By RACHEL SHERMAN\",\"title\":\"What the Rich Won\\u2019t Tell You\",\"abstract\":\"\\u201cThere\\u2019s nobody who knows how much we spend. You\\u2019re the only person I ever said those numbers to out loud.\\u201d\",\"published_date\":\"2017-09-08\",\"source\":\"The New York Times\",\"asset_id\":100000005412992,\"total_shares\":20,\"des_facet\":[\"HIGH NET WORTH INDIVIDUALS\",\"INCOME INEQUALITY\"],\"org_facet\":\"\",\"per_facet\":\"\",\"geo_facet\":\"\",\"media\":\"\"}]}"

And convert it into a data frame with relevant columns.

json_raw <- content(nytData, type = "text", encoding = "UTF-8")
json1 <- fromJSON(json_raw)
df <- as.data.frame(json1$results)
df1 <- df %>% select(byline, title, abstract, published_date)
df1$byline <-  str_replace(df1$byline, 'By ', "")
df1 %>% kable("html") 
byline title abstract published_date
ROSS DOUTHAT Speaking Ill of Hugh Hefner An honest obituary for a wicked American. 2017-09-30
THE EDITORIAL BOARD The Republican’s Guide to Presidential Etiquette Republicans used to care a whole lot about how a president comported himself. Here is what they now condone. 2017-10-08
DAVID LEONHARDT, IAN PRASAD PHILBRICK and STUART A. THOMPSON The Congress Members Receiving the Most N.R.A. Funding Thoughts and prayers and the N.R.A.: Here are the top career recipients of N.R.A. campaign money. 2017-10-04
MAYIM BIALIK Mayim Bialik: Being a Feminist in Harvey Weinstein’s World I have always had an uncomfortable relationship with being employed in an industry built on the objectification of women. 2017-10-13
THOMAS L. FRIEDMAN If Only Stephen Paddock Were a Muslim What happens when the killer’s country of origin is us? 2017-10-03
THE EDITORIAL BOARD 477 Days. 521 Mass Shootings. Zero Action From Congress. Political leaders have done nothing to curb the bloodshed since the Pulse nightclub massacre in Orlando last year. 2017-10-02
LINDY WEST Yes, This Is a Witch Hunt. I’m a Witch and I’m Hunting You. Sorry, Woody Allen. I know you hate rumor mills, but sometimes they’re our only recourse. 2017-10-17
ROSANNE CASH Rosanne Cash: Country Musicians, Stand Up to the N.R.A. Patriotism is not antithetical to gun control. 2017-10-03
SUSAN SHAPIRO College Advice I Wish I’d Taken I’m a teacher, but as an undergraduate, I was the type of mediocre student I now disdain. 2017-10-17
CHARLES M. BLOW Trump, Chieftain of Spite Trump can’t hold a candle to Obama, so he’s taking a tiki torch to Obama’s legacy. 2017-10-15
LENA DUNHAM Lena Dunham: Harvey Weinstein and the Silence of the Men The hush in Hollywood about the megaproducer only reinforces a culture that keeps women from speaking up. 2017-10-09
ERIC REID Eric Reid: Why Colin Kaepernick and I Decided to Take a Knee We chose the gesture purposefully. Our bodies were like flags flown at half-mast to mark a tragedy. 2017-09-25
PATRICK CHAPPATTE After Catalonia’s Chaotic Vote for Independence Clashes with police during Sunday’s referendum left supporters on both sides even more polarized. 2017-10-02
NOAM COHEN Silicon Valley Is Not Your Friend We are beginning to understand that tech companies don’t have our best interests at heart. Did they ever? 2017-10-13
BRET STEPHENS Repeal the Second Amendment It sounds like political Mission Impossible. But it’s the only meaningful way to stop the killing. 2017-10-05
MATAN ROCHLITZ I Have a Message for You To save her own life, she left her father to die. Decades later, she got a message from him. 2017-10-10
BRIAN PHILLIPS The Real Failure of U.S. Men’s Soccer The team flubbed its World Cup chance, but its leaders are more focused on spin control than improving the level of play. 2017-10-13
NICHOLAS KRISTOF Inside North Korea, and Feeling the Drums of War A visit to the reclusive country reveals the perilous moment we’re in. 2017-10-05
LUPITA NYONG’O Lupita Nyong’o: Speaking Out About Harvey Weinstein I don’t feel alone anymore. Let us never shut up about this. 2017-10-19
RACHEL SHERMAN What the Rich Won’t Tell You “There’s nobody who knows how much we spend. You’re the only person I ever said those numbers to out loud.” 2017-09-08