Data 607 Week 9 Assignment
webscraping from an API
Your task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it to an R dataframe.
- Data can be collected here
library(knitr)
library(dplyr)
library(stringr)
library(jsonlite)
library(httr)
library(pander)The API key and url to extract the most popular opinion peices over the last thirty days are here.
apiKey <- "&api-key=b95585fd6b264ab08e28fb0e96998aae"
urlBase <-"https://api.nytimes.com/svc/mostpopular/v2/mostshared/Opinion/30.json?"
nytURL <- paste(urlBase, apiKey, sep = "")We use the GET command to extract a json like object from the API
nytData <- GET(nytURL)
content(nytData, 'text')## [1] "{\"status\":\"OK\",\"copyright\":\"Copyright (c) 2017 The New York Times Company. All Rights Reserved.\",\"num_results\":276,\"results\":[{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/09\\/30\\/opinion\\/hugh-hefner.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By ROSS DOUTHAT\",\"title\":\"Speaking Ill of Hugh Hefner\",\"abstract\":\"An honest obituary for a wicked American.\",\"published_date\":\"2017-09-30\",\"source\":\"The New York Times\",\"asset_id\":100000005467201,\"total_shares\":1,\"des_facet\":[\"LIBERALISM (US POLITICS)\",\"WOMEN AND GIRLS\",\"SEX\"],\"org_facet\":[\"PLAYBOY MAGAZINE\",\"PORNOGRAPHY\",\"CELEBRITIES\"],\"per_facet\":[\"HEFNER, HUGH\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Hugh Hefner attending a Playmate of the Year luncheon in 2013 at the Playboy Mansion in Los Angeles. Mr. Hefner died on Wednesday.\",\"copyright\":\"Charley Gallay\\/Getty Images, for Playboy\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/01\\/opinion\\/sunday\\/01douthat\\/01douthat-thumbStandard-v2.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/01\\/opinion\\/sunday\\/01douthat\\/01douthat-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/01\\/opinion\\/sunday\\/01douthat\\/01douthat-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/interactive\\/2017\\/10\\/08\\/opinion\\/editorials\\/republican-etiquette-guide.html\",\"column\":\"\",\"section\":\"Opinion\",\"byline\":\"By THE EDITORIAL BOARD\",\"title\":\"The Republican\\u2019s Guide to Presidential Etiquette\",\"abstract\":\"Republicans used to care a whole lot about how a president comported himself. Here is what they now condone.\",\"published_date\":\"2017-10-08\",\"source\":\"The New York Times\",\"asset_id\":100000005483715,\"total_shares\":2,\"des_facet\":[\"WHITE HOUSE BUILDING (WASHINGTON, DC)\",\"UNITED STATES POLITICS AND GOVERNMENT\",\"ETHICS AND OFFICIAL MISCONDUCT\",\"CUSTOMS, ETIQUETTE AND MANNERS\"],\"org_facet\":\"\",\"per_facet\":[\"TRUMP, DONALD J\"],\"geo_facet\":\"\",\"media\":\"\"},{\"url\":\"https:\\/\\/www.nytimes.com\\/interactive\\/2017\\/10\\/04\\/opinion\\/thoughts-prayers-nra-funding-senators.html\",\"column\":\"\",\"section\":\"Opinion\",\"byline\":\"By DAVID LEONHARDT, IAN PRASAD PHILBRICK and STUART A. THOMPSON\",\"title\":\"The Congress Members Receiving the Most N.R.A. Funding\",\"abstract\":\"Thoughts and prayers and the N.R.A.: Here are the top career recipients of N.R.A. campaign money.\",\"published_date\":\"2017-10-04\",\"source\":\"The New York Times\",\"asset_id\":100000005478110,\"total_shares\":3,\"des_facet\":[\"GUN CONTROL\"],\"org_facet\":[\"NATIONAL RIFLE ASSN\",\"SENATE\",\"HOUSE OF REPRESENTATIVES\"],\"per_facet\":\"\",\"geo_facet\":\"\",\"media\":\"\"},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/13\\/opinion\\/mayim-bialik-feminist-harvey-weinstein.html\",\"column\":\"Op-Ed Contributor\",\"section\":\"Opinion\",\"byline\":\"By MAYIM BIALIK\",\"title\":\"Mayim Bialik: Being a Feminist in Harvey Weinstein\\u2019s World\",\"abstract\":\"I have always had an uncomfortable relationship with being employed in an industry built on the objectification of women.\",\"published_date\":\"2017-10-13\",\"source\":\"The New York Times\",\"asset_id\":100000005493485,\"total_shares\":4,\"des_facet\":[\"WOMEN AND GIRLS\",\"ACTORS AND ACTRESSES\",\"TELEVISION\",\"WOMEN'S RIGHTS\"],\"org_facet\":\"\",\"per_facet\":[\"WEINSTEIN, HARVEY\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Mayim Bialik as Blossom Russo in season one of \\u201cBlossom.\\u201d\",\"copyright\":\"Alice S. Hall\\/NBC, via Getty Images\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/13\\/opinion\\/13bialik3web\\/13bialik3web-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/13\\/opinion\\/13bialik3web\\/13bialik3web-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/13\\/opinion\\/13bialik3web\\/13bialik3web-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/03\\/opinion\\/stephen-paddock-las-vegas-terrorism.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By THOMAS L. FRIEDMAN\",\"title\":\"If Only Stephen Paddock Were a Muslim\",\"abstract\":\"What happens when the killer\\u2019s country of origin is us?\",\"published_date\":\"2017-10-03\",\"source\":\"The New York Times\",\"asset_id\":100000005475422,\"total_shares\":5,\"des_facet\":[\"GUN CONTROL\",\"TERRORISM\",\"UNITED STATES POLITICS AND GOVERNMENT\"],\"org_facet\":\"\",\"per_facet\":[\"PADDOCK, STEPHEN C (1953-2017)\"],\"geo_facet\":[\"LAS VEGAS (NEV)\"],\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Las Vegas, Monday.\",\"copyright\":\"Hilary Swift for The New York Times\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/04\\/opinion\\/04friedmanWeb\\/04friedmanWeb-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/04\\/opinion\\/04friedmanWeb\\/04friedmanWeb-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/04\\/opinion\\/04friedmanWeb\\/04friedmanWeb-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/interactive\\/2017\\/10\\/02\\/opinion\\/editorials\\/mass-shootings-congress.html\",\"column\":\"\",\"section\":\"Opinion\",\"byline\":\"By THE EDITORIAL BOARD\",\"title\":\"477 Days. 521 Mass Shootings. Zero Action From Congress.\",\"abstract\":\"Political leaders have done nothing to curb the bloodshed since the Pulse nightclub massacre in Orlando last year.\",\"published_date\":\"2017-10-02\",\"source\":\"The New York Times\",\"asset_id\":100000005474681,\"total_shares\":6,\"des_facet\":[\"MURDERS, ATTEMPTED MURDERS AND HOMICIDES\",\"LAS VEGAS, NEV, SHOOTING (OCTOBER, 2017)\",\"ORLANDO, FLA, SHOOTING (2016)\"],\"org_facet\":[\"HOUSE OF REPRESENTATIVES\",\"SENATE\"],\"per_facet\":\"\",\"geo_facet\":[\"LAS VEGAS (NEV)\"],\"media\":\"\"},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/17\\/opinion\\/columnists\\/weinstein-harassment-witchunt.html\",\"column\":\"Contributing Op-Ed Writer\",\"section\":\"Opinion\",\"byline\":\"By LINDY WEST\",\"title\":\"Yes, This Is a Witch Hunt. I\\u2019m a Witch and I\\u2019m Hunting You.\",\"abstract\":\"Sorry, Woody Allen. I know you hate rumor mills, but sometimes they\\u2019re our only recourse.\",\"published_date\":\"2017-10-17\",\"source\":\"The New York Times\",\"asset_id\":100000005501461,\"total_shares\":7,\"des_facet\":[\"SEXUAL HARASSMENT\",\"WOMEN AND GIRLS\",\"SEX CRIMES\"],\"org_facet\":\"\",\"per_facet\":[\"TRUMP, DONALD J\",\"WEINSTEIN, HARVEY\",\"ALLEN, WOODY\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Women on Friday protesting the Manhattan district attorney\\u2019s handling of accusations against Harvey Weinstein.\",\"copyright\":\"Holly Pickett for The New York Times\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/18\\/opinion\\/18westSubs\\/18westSubs-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/18\\/opinion\\/18westSubs\\/18westSubs-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/18\\/opinion\\/18westSubs\\/18westSubs-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/03\\/opinion\\/rosanne-cash-country-musicians-nra.html\",\"column\":\"Op-Ed Contributor\",\"section\":\"Opinion\",\"byline\":\"By ROSANNE CASH\",\"title\":\"Rosanne Cash: Country Musicians, Stand Up to the N.R.A.\",\"abstract\":\"Patriotism is not antithetical to gun control.\",\"published_date\":\"2017-10-03\",\"source\":\"The New York Times\",\"asset_id\":100000005474509,\"total_shares\":8,\"des_facet\":[\"GUN CONTROL\",\"LOBBYING AND LOBBYISTS\"],\"org_facet\":[\"NATIONAL RIFLE ASSN\"],\"per_facet\":\"\",\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Vince Gill and Amy Grant pray during a candlelight vigil in Nashville for the victims of the shooting in Las Vegas.\",\"copyright\":\"Rick Diamond\\/Getty Images \",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/03\\/opinion\\/03cashSubs\\/03cashSubs-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/03\\/opinion\\/03cashSubs\\/03cashSubs-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/03\\/opinion\\/03cashSubs\\/03cashSubs-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/17\\/opinion\\/college-advice-professor.html\",\"column\":\"On Campus\",\"section\":\"Opinion\",\"byline\":\"By SUSAN SHAPIRO\",\"title\":\"College Advice I Wish I\\u2019d Taken\",\"abstract\":\"I\\u2019m a teacher, but as an undergraduate, I was the type of mediocre student I now disdain.\",\"published_date\":\"2017-10-17\",\"source\":\"The New York Times\",\"asset_id\":100000005493469,\"total_shares\":9,\"des_facet\":[\"COLLEGES AND UNIVERSITIES\",\"SOCIAL MEDIA\"],\"org_facet\":[\"TEACHERS AND SCHOOL EMPLOYEES\"],\"per_facet\":\"\",\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"\",\"copyright\":\"Kearin Ever Cook\\/Pratt Institute\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/17\\/opinion\\/17oncampus\\/17oncampus-thumbStandard-v2.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/17\\/opinion\\/17oncampus\\/17oncampus-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/17\\/opinion\\/17oncampus\\/17oncampus-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/15\\/opinion\\/columnists\\/trump-spite-obama-legacy.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By CHARLES M. BLOW\",\"title\":\"Trump, Chieftain of Spite\",\"abstract\":\"Trump can\\u2019t hold a candle to Obama, so he\\u2019s taking a tiki torch to Obama\\u2019s legacy.\",\"published_date\":\"2017-10-15\",\"source\":\"The New York Times\",\"asset_id\":100000005495325,\"total_shares\":10,\"des_facet\":[\"UNITED STATES POLITICS AND GOVERNMENT\",\"PATIENT PROTECTION AND AFFORDABLE CARE ACT (2010)\",\"FEDERAL AID (US)\"],\"org_facet\":\"\",\"per_facet\":[\"TRUMP, DONALD J\",\"OBAMA, BARACK\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Donald Trump and Barack Obama after Mr. Trump\\u2019s inauguration on January 20.\",\"copyright\":\"Ruth Fremson\\/The New York Times\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/16\\/opinion\\/16blowWEB\\/16blowWEB-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/16\\/opinion\\/16blowWEB\\/16blowWEB-mediumThreeByTwo210-v3.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/16\\/opinion\\/16blowWEB\\/16blowWEB-mediumThreeByTwo440-v3.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/09\\/opinion\\/harvey-weinstein-lena-dunham-silence-.html\",\"column\":\"Op-Ed Contributor\",\"section\":\"Opinion\",\"byline\":\"By LENA DUNHAM\",\"title\":\"Lena Dunham: Harvey Weinstein and the Silence of the Men\",\"abstract\":\"The hush in Hollywood about the megaproducer only reinforces a culture that keeps women from speaking up.\",\"published_date\":\"2017-10-09\",\"source\":\"The New York Times\",\"asset_id\":100000005486608,\"total_shares\":11,\"des_facet\":[\"SEXUAL HARASSMENT\",\"WOMEN AND GIRLS\",\"MOVIES\"],\"org_facet\":[\"WEINSTEIN CO\"],\"per_facet\":[\"WEINSTEIN, HARVEY\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Harvey Weinstein at a party after the 2016 Academy Awards.\",\"copyright\":\"Axel Koester\\/Corbis, via Getty Images\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/09dunhamWeb\\/09dunhamWeb-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/09dunhamWeb\\/09dunhamWeb-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/09dunhamWeb\\/09dunhamWeb-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/09\\/25\\/opinion\\/colin-kaepernick-football-protests.html\",\"column\":\"Op-Ed Contributor\",\"section\":\"Opinion\",\"byline\":\"By ERIC REID\",\"title\":\"Eric Reid: Why Colin Kaepernick and I Decided to Take a Knee\",\"abstract\":\"We chose the gesture purposefully. Our bodies were like flags flown at half-mast to mark a tragedy.\",\"published_date\":\"2017-09-25\",\"source\":\"The New York Times\",\"asset_id\":100000005457221,\"total_shares\":12,\"des_facet\":[\"DEMONSTRATIONS, PROTESTS AND RIOTS\",\"POLICE BRUTALITY, MISCONDUCT AND SHOOTINGS\",\"BLACKS\"],\"org_facet\":[\"NATIONAL FOOTBALL LEAGUE\",\"SAN FRANCISCO 49ERS\"],\"per_facet\":[\"KAEPERNICK, COLIN\",\"REID, ERIC (1991- )\",\"TRUMP, DONALD J\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Colin Kaepernick, right, and Eric Reid kneeling during the national anthem before an N.F.L. game last year.\",\"copyright\":\"Marcio Jose Sanchez\\/Associated Press\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/09\\/27\\/insider\\/25reidSub\\/25reidSub-thumbStandard-v2.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/09\\/27\\/insider\\/25reidSub\\/25reidSub-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/09\\/27\\/insider\\/25reidSub\\/25reidSub-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/02\\/opinion\\/catalonia-spain-referendum.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By PATRICK CHAPPATTE\",\"title\":\"After Catalonia\\u2019s Chaotic Vote for Independence\",\"abstract\":\"Clashes with police during Sunday\\u2019s referendum left supporters on both sides even more polarized.\",\"published_date\":\"2017-10-02\",\"source\":\"The New York Times\",\"asset_id\":100000005474636,\"total_shares\":13,\"des_facet\":[\"REFERENDUMS\"],\"org_facet\":\"\",\"per_facet\":\"\",\"geo_facet\":[\"CATALONIA (SPAIN)\"],\"media\":\"\"},{\"url\":\"https:\\/\\/www.nytimes.com\\/interactive\\/2017\\/10\\/13\\/opinion\\/sunday\\/Silicon-Valley-Is-Not-Your-Friend.html\",\"column\":\"Opinion\",\"section\":\"Opinion\",\"byline\":\"By NOAM COHEN\",\"title\":\"Silicon Valley Is Not Your Friend\",\"abstract\":\"We are beginning to understand that tech companies don\\u2019t have our best interests at heart. Did they ever?\",\"published_date\":\"2017-10-13\",\"source\":\"The New York Times\",\"asset_id\":100000005494137,\"total_shares\":14,\"des_facet\":[\"COMPUTERS AND THE INTERNET\",\"ONLINE ADVERTISING\",\"SEARCH ENGINES\"],\"org_facet\":[\"AMAZON.COM INC\",\"FACEBOOK INC\",\"GOOGLE INC\"],\"per_facet\":\"\",\"geo_facet\":\"\",\"media\":\"\"},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/05\\/opinion\\/guns-second-amendment-nra.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By BRET STEPHENS\",\"title\":\"Repeal the Second Amendment\",\"abstract\":\"It sounds like political Mission Impossible. But it\\u2019s the only meaningful way to stop the killing.\",\"published_date\":\"2017-10-05\",\"source\":\"The New York Times\",\"asset_id\":100000005478925,\"total_shares\":15,\"des_facet\":[\"GUN CONTROL\",\"FIREARMS\",\"SECOND AMENDMENT (US CONSTITUTION)\"],\"org_facet\":[\"NATIONAL RIFLE ASSN\",\"LAS VEGAS, NEV, SHOOTING (OCTOBER, 2017)\",\"MURDERS, ATTEMPTED MURDERS AND HOMICIDES\"],\"per_facet\":[\"PADDOCK, STEPHEN C (1953-2017)\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"\",\"copyright\":\"Victor J. Blue\\/Bloomberg \",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/05\\/opinion\\/05stephensSub\\/05stephensSub-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/05\\/opinion\\/05stephensSub\\/05stephensSub-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/05\\/opinion\\/05stephensSub\\/05stephensSub-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/10\\/opinion\\/auschwitz-israel-holocaust.html\",\"column\":\"Op-Docs\",\"section\":\"Opinion\",\"byline\":\"By MATAN ROCHLITZ\",\"title\":\"I Have a Message for You\",\"abstract\":\"To save her own life, she left her father to die. Decades later, she got a message from him.\",\"published_date\":\"2017-10-10\",\"source\":\"The New York Times\",\"asset_id\":100000005486932,\"total_shares\":16,\"des_facet\":[\"HOLOCAUST AND THE NAZI ERA\",\"WORLD WAR II (1939-45)\",\"WAR CRIMES, GENOCIDE AND CRIMES AGAINST HUMANITY\",\"FAMILIES AND FAMILY LIFE\"],\"org_facet\":[\"AUSCHWITZ CONCENTRATION CAMP\"],\"per_facet\":\"\",\"geo_facet\":[\"TEL AVIV (ISRAEL)\",\"BELGIUM\"],\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"\",\"copyright\":\"Matan Rochlitz\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/10opdoc-message-alt-img\\/10opdoc-message-alt-img-thumbStandard-v3.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/10opdoc-message-alt-img\\/10opdoc-message-alt-img-mediumThreeByTwo210-v3.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/10\\/opinion\\/10opdoc-message-alt-img\\/10opdoc-message-alt-img-mediumThreeByTwo440-v3.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/13\\/opinion\\/us-soccer-world-cup.html\",\"column\":\"Sporting\",\"section\":\"Opinion\",\"byline\":\"By BRIAN PHILLIPS\",\"title\":\"The Real Failure of U.S. Men\\u2019s Soccer\",\"abstract\":\"The team flubbed its World Cup chance, but its leaders are more focused on spin control than improving the level of play.\",\"published_date\":\"2017-10-13\",\"source\":\"The New York Times\",\"asset_id\":100000005494571,\"total_shares\":17,\"des_facet\":[\"SOCCER\",\"WORLD CUP (SOCCER)\"],\"org_facet\":[\"UNITED STATES SOCCER FEDERATION\"],\"per_facet\":[\"ARENA, BRUCE\",\"GULATI, SUNIL\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Jozy Altidore (No. 17) missed a scoring chance against Trinidad and Tobago on Oct. 10.\",\"copyright\":\"Rebecca Blackwell\\/Associated Press\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/15\\/opinion\\/sunday\\/15sportingSUB\\/15sportingSUB-thumbStandard-v4.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/15\\/opinion\\/sunday\\/15sportingSUB\\/15sportingSUB-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/15\\/opinion\\/sunday\\/15sportingSUB\\/15sportingSUB-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/05\\/opinion\\/sunday\\/nuclear-north-korea.html\",\"column\":\"Op-Ed Columnist\",\"section\":\"Opinion\",\"byline\":\"By NICHOLAS KRISTOF\",\"title\":\"Inside North Korea, and Feeling the Drums of War\",\"abstract\":\"A visit to the reclusive country reveals the perilous moment we\\u2019re in.\",\"published_date\":\"2017-10-05\",\"source\":\"The New York Times\",\"asset_id\":100000005477022,\"total_shares\":18,\"des_facet\":[\"UNITED STATES INTERNATIONAL RELATIONS\",\"UNITED STATES DEFENSE AND MILITARY FORCES\",\"PROPAGANDA\"],\"org_facet\":[\"NUCLEAR WEAPONS\"],\"per_facet\":[\"WARMBIER, OTTO FREDERICK (1994- )\",\"KIM JONG-UN\"],\"geo_facet\":[\"NORTH KOREA\"],\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"A propaganda poster on a main street in the North Korean capital, Pyongyang, depicts a missile striking the United States Capitol. It reads: “The response of the Democratic People’s Republic of Korea.”\",\"copyright\":\"Jonah M. Kessel\\/The New York Times\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/08\\/opinion\\/sunday\\/08Kristof-print\\/08Kristof1-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/08\\/opinion\\/sunday\\/08Kristof-print\\/08Kristof1-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/08\\/opinion\\/sunday\\/08Kristof-print\\/08Kristof1-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/10\\/19\\/opinion\\/lupita-nyongo-harvey-weinstein.html\",\"column\":\"Op-Ed Contributor\",\"section\":\"Opinion\",\"byline\":\"By LUPITA NYONG\\u2019O\",\"title\":\"Lupita Nyong\\u2019o: Speaking Out About Harvey Weinstein\",\"abstract\":\"I don\\u2019t feel alone anymore. Let us never shut up about this.\",\"published_date\":\"2017-10-19\",\"source\":\"The New York Times\",\"asset_id\":100000005505402,\"total_shares\":19,\"des_facet\":[\"MOVIES\",\"ACTORS AND ACTRESSES\",\"WOMEN AND GIRLS\"],\"org_facet\":[\"SEXUAL HARASSMENT\"],\"per_facet\":[\"WEINSTEIN, HARVEY\"],\"geo_facet\":\"\",\"media\":[{\"type\":\"image\",\"subtype\":\"photo\",\"caption\":\"Lupita Nyong\\u2019o in 2015.\",\"copyright\":\"Jesse Dittmar for The New York Times\",\"approved_for_syndication\":1,\"media-metadata\":[{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/20\\/opinion\\/19nyongoWeb\\/19nyongoWeb-thumbStandard.jpg\",\"format\":\"Standard Thumbnail\",\"height\":75,\"width\":75},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/20\\/opinion\\/19nyongoWeb\\/19nyongoWeb-mediumThreeByTwo210.jpg\",\"format\":\"mediumThreeByTwo210\",\"height\":140,\"width\":210},{\"url\":\"https:\\/\\/static01.nyt.com\\/images\\/2017\\/10\\/20\\/opinion\\/19nyongoWeb\\/19nyongoWeb-mediumThreeByTwo440.jpg\",\"format\":\"mediumThreeByTwo440\",\"height\":293,\"width\":440}]}]},{\"url\":\"https:\\/\\/www.nytimes.com\\/2017\\/09\\/08\\/opinion\\/sunday\\/what-the-rich-wont-tell-you.html\",\"column\":\"Opinion\",\"section\":\"Opinion\",\"byline\":\"By RACHEL SHERMAN\",\"title\":\"What the Rich Won\\u2019t Tell You\",\"abstract\":\"\\u201cThere\\u2019s nobody who knows how much we spend. You\\u2019re the only person I ever said those numbers to out loud.\\u201d\",\"published_date\":\"2017-09-08\",\"source\":\"The New York Times\",\"asset_id\":100000005412992,\"total_shares\":20,\"des_facet\":[\"HIGH NET WORTH INDIVIDUALS\",\"INCOME INEQUALITY\"],\"org_facet\":\"\",\"per_facet\":\"\",\"geo_facet\":\"\",\"media\":\"\"}]}"
And convert it into a data frame with relevant columns.
json_raw <- content(nytData, type = "text", encoding = "UTF-8")
json1 <- fromJSON(json_raw)
df <- as.data.frame(json1$results)
df1 <- df %>% select(byline, title, abstract, published_date)
df1$byline <- str_replace(df1$byline, 'By ', "")
df1 %>% kable("html") | byline | title | abstract | published_date |
|---|---|---|---|
| ROSS DOUTHAT | Speaking Ill of Hugh Hefner | An honest obituary for a wicked American. | 2017-09-30 |
| THE EDITORIAL BOARD | The Republican’s Guide to Presidential Etiquette | Republicans used to care a whole lot about how a president comported himself. Here is what they now condone. | 2017-10-08 |
| DAVID LEONHARDT, IAN PRASAD PHILBRICK and STUART A. THOMPSON | The Congress Members Receiving the Most N.R.A. Funding | Thoughts and prayers and the N.R.A.: Here are the top career recipients of N.R.A. campaign money. | 2017-10-04 |
| MAYIM BIALIK | Mayim Bialik: Being a Feminist in Harvey Weinstein’s World | I have always had an uncomfortable relationship with being employed in an industry built on the objectification of women. | 2017-10-13 |
| THOMAS L. FRIEDMAN | If Only Stephen Paddock Were a Muslim | What happens when the killer’s country of origin is us? | 2017-10-03 |
| THE EDITORIAL BOARD | 477 Days. 521 Mass Shootings. Zero Action From Congress. | Political leaders have done nothing to curb the bloodshed since the Pulse nightclub massacre in Orlando last year. | 2017-10-02 |
| LINDY WEST | Yes, This Is a Witch Hunt. I’m a Witch and I’m Hunting You. | Sorry, Woody Allen. I know you hate rumor mills, but sometimes they’re our only recourse. | 2017-10-17 |
| ROSANNE CASH | Rosanne Cash: Country Musicians, Stand Up to the N.R.A. | Patriotism is not antithetical to gun control. | 2017-10-03 |
| SUSAN SHAPIRO | College Advice I Wish I’d Taken | I’m a teacher, but as an undergraduate, I was the type of mediocre student I now disdain. | 2017-10-17 |
| CHARLES M. BLOW | Trump, Chieftain of Spite | Trump can’t hold a candle to Obama, so he’s taking a tiki torch to Obama’s legacy. | 2017-10-15 |
| LENA DUNHAM | Lena Dunham: Harvey Weinstein and the Silence of the Men | The hush in Hollywood about the megaproducer only reinforces a culture that keeps women from speaking up. | 2017-10-09 |
| ERIC REID | Eric Reid: Why Colin Kaepernick and I Decided to Take a Knee | We chose the gesture purposefully. Our bodies were like flags flown at half-mast to mark a tragedy. | 2017-09-25 |
| PATRICK CHAPPATTE | After Catalonia’s Chaotic Vote for Independence | Clashes with police during Sunday’s referendum left supporters on both sides even more polarized. | 2017-10-02 |
| NOAM COHEN | Silicon Valley Is Not Your Friend | We are beginning to understand that tech companies don’t have our best interests at heart. Did they ever? | 2017-10-13 |
| BRET STEPHENS | Repeal the Second Amendment | It sounds like political Mission Impossible. But it’s the only meaningful way to stop the killing. | 2017-10-05 |
| MATAN ROCHLITZ | I Have a Message for You | To save her own life, she left her father to die. Decades later, she got a message from him. | 2017-10-10 |
| BRIAN PHILLIPS | The Real Failure of U.S. Men’s Soccer | The team flubbed its World Cup chance, but its leaders are more focused on spin control than improving the level of play. | 2017-10-13 |
| NICHOLAS KRISTOF | Inside North Korea, and Feeling the Drums of War | A visit to the reclusive country reveals the perilous moment we’re in. | 2017-10-05 |
| LUPITA NYONG’O | Lupita Nyong’o: Speaking Out About Harvey Weinstein | I don’t feel alone anymore. Let us never shut up about this. | 2017-10-19 |
| RACHEL SHERMAN | What the Rich Won’t Tell You | “There’s nobody who knows how much we spend. You’re the only person I ever said those numbers to out loud.” | 2017-09-08 |