The New York Times web site provides a rich set of APIs, as described here: http://developer.nytimes.com/docs. My task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it to an R data frame
if (!require('jsonlite')) install.packages('jsonlite')
if (!require('dplyr')) install.packages('dplyr')
if (!require('devtools')) install.packages('devtools')
if (!require('stringr')) install.packages('stringr')Let’s use the Top Stories V2 API and look for stories section.
The Top Stories API returns a list of articles and associated links.API key was requested using signup and received email with API key.
# received API key by requesting using email
key <- "a831fcb0f92846d6868c57b9cea31d22"# add url and text required for call
url <- "https://api.nytimes.com/svc/topstories/v2/home.json"
addurl <- paste0(url, "?api-key=")
# fetched using json + key call
data_fetch <- fromJSON(paste0(addurl, key))
# summary
summary(data_fetch)## Length Class Mode
## status 1 -none- character
## copyright 1 -none- character
## section 1 -none- character
## last_updated 1 -none- character
## num_results 1 -none- numeric
## results 18 data.frame listHere, we are interested only on results which is data frame.
clean_result <- data_fetch$results
clean_result#clean published_date
clean_result$publish_date <- unlist(str_extract(clean_result$published_date, ".+?(?=T)"))
clean_result$publish_time <- unlist(str_extract(clean_result$published_date, "(?<=T)(.*)"))
clean_result$publish_time <- unlist(str_extract(clean_result$publish_time, ".+?(?=-)"))
clean_result <- clean_result %>%
select(section, title,byline, abstract,short_url, publish_date, publish_time)%>%
arrange(section)I have generate the TOP Story list of New York Times using API. List has important information like Section, Title, Publish date, Author, and also short_link to go to the article.
knitr::kable(clean_result)| section | title | byline | abstract | short_url | publish_date | publish_time |
|---|---|---|---|---|---|---|
| Arts | Busy Philipps Has Flourished by Being Herself | By ILANA KAPLAN | With a new memoir out and a talk show debuting Sunday on E!, the actress has parlayed social media clout into the most successful stretch of her career. | https://nyti.ms/2JgQ15C | 2018-10-25 | 19:57:12 |
| Arts | Will TV Change the Comedy Cellar? Comics Are About to Find Out | By JASON ZINOMAN | The premier Greenwich Village club is getting into the late-night programming wars at a time when it has been under fire for Louis C.K. appearances. | https://nyti.ms/2yBFj5t | 2018-10-28 | 12:55:18 |
| Arts | The Playlist: Cardi B Hits Pay Dirt, and 11 More New Songs | By JON PARELES, JON CARAMANICA and GIOVANNI RUSSONELLO | Hear tracks from Maggie Rogers, Julia Holter, Tyler, the Creator and others. | https://nyti.ms/2JjqmZY | 2018-10-26 | 12:29:35 |
| Arts | When City Center Was Balanchine’s House | By MARINA HARSS | Jacques d’Amboise, Patricia Wilde, Allegra Kent and Edward Villella talk about the roles they danced at the theater, which is celebrating George Balanchine and its 75th anniversary as a palace of the arts. | https://nyti.ms/2As9KfQ | 2018-10-25 | 20:05:53 |
| Automobiles | The Computer Chauffeur Is Creeping Closer | By NORMAN MAYERSOHN | Fully autonomous cars are years away, but it’s the automobile where artificial intelligence could have a critical role for the greatest number of people. | https://nyti.ms/2R6wrMn | 2018-10-19 | 05:00:07 |
| Books | 6 New Books We Recommend This Week | Suggested reading from critics and editors at The New York Times. | https://nyti.ms/2JmbfPP | 2018-10-25 | 13:21:40 | |
| Books | Indigenous Stories for Young Readers | By NICOLE LAMY | Young adult and middle grade fiction to educate children in the vast terrain of Native American history. | https://nyti.ms/2QVAkU8 | 2018-10-17 | 14:46:07 |
| Briefing | Violence in the U.S., Sri Lanka, Yemen: Your Monday Briefing | By ALISHA HARIDASANI GUPTA | Here’s what you need to know to start your day. | https://nyti.ms/2CKYLQh | 2018-10-28 | 15:02:34 |
| Business Day | IBM to Buy Red Hat, the Top Linux Distributor, for $34 Billion | By STEVE LOHR | The deal is a big move to bring more software developers under IBM’s wing and hints at a bigger push into cloud computing. | https://nyti.ms/2COnWSf | 2018-10-28 | 14:47:46 |
| Business Day | Fox Executive Condemns Remarks by Guest on Lou Dobbs Show | By NELSON D. SCHWARTZ | The show was originally shown on Thursday. But the remarks, which were linked to anti-Semitism, created a firestorm online when the show was shown again on Saturday. | https://nyti.ms/2yHgV2x | 2018-10-28 | 18:11:06 |
| Business Day | Sunrun Gives Tesla a Fight in the Home Solar Business | By IVAN PENN | Tesla is relying on showrooms to sell electric cars, solar roofs and batteries. But a California rival has made inroads into the residential business. | https://nyti.ms/2yBPS8Y | 2018-10-28 | 17:23:06 |
| Movies | ‘Halloween’ Becomes Second-Highest-Grossing Film in the Franchise | By GABE COHN | Universal’s slasher sequel was No. 1 at the box office for a second weekend, and trails only the 1978 original in earnings when adjusted for inflation. | https://nyti.ms/2yGT2Ii | 2018-10-28 | 15:08:51 |
| Movies | From ‘Walking Dead’ Casualty to South Korean Cinema Star | By MARTIN TSAI | Steven Yeun, who played Glenn Rhee on the TV series, rejected the Asian stereotypes Hollywood was offering. Now he’s working with world-class auteurs. | https://nyti.ms/2At1Tie | 2018-10-26 | 12:06:04 |
| Movies | Is This ‘A Star is Born’ Pop Song Supposed to Be Bad or Glorious? | By KYLE BUCHANAN | “Why Did You Do That?” is the film’s most debated song, and co-writer Diane Warren explains its intention. | https://nyti.ms/2R4e2jf | 2018-10-19 | 14:01:41 |
| New York | I Can Help Who’s Next | By JOHN LELAND and NATAN DVIR | New Yorkers wait in line. Lots of lines. For anything. | https://nyti.ms/2JgT0uJ | 2018-10-25 | 13:43:08 |
| New York | Two Powerball Winners Share $688 Million; One Ticket Came From a Bodega in Harlem | By LUIS FERRÉ-SADURNÍ | The other ticket was sold in Iowa. The winners will split the nearly $688 million jackpot, the fourth-largest lottery prize in the country’s history. | https://nyti.ms/2CIZ3Hr | 2018-10-28 | 15:58:59 |
| Opinion | The Hate Poisoning America | By THE EDITORIAL BOARD | After Pittsburgh, Americans need to ask more of their leaders, and of each other. | https://nyti.ms/2CNGrpx | 2018-10-28 | 18:16:27 |
| Opinion | Trump’s Potent Toxicity | By CHARLES M. BLOW | Trump has flirted with the deepest racists and Nazis and it has not gone unnoticed, least of all by them. | https://nyti.ms/2yDRm2u | 2018-10-28 | 19:03:05 |
| Opinion | Trump’s Corruption: The Definitive List | By DAVID LEONHARDT and IAN PRASAD PHILBRICK | The many ways that the president, his family and his aides are lining their own pockets. | https://nyti.ms/2CMeWg5 | 2018-10-28 | 19:01:41 |
| Opinion | When Hate Goes Mainstream | By JONATHAN A. GREENBLATT | The Pittsburgh massacre is only the latest, worst instance of rising anti-Semitism. Americans of conscience must now push back. | https://nyti.ms/2CO2f4K | 2018-10-28 | 15:15:03 |
| Opinion | America — and Judaism — at Its Best | By LEV GOLINKIN | The man accused of the synagogue massacre in Pittsburgh seemed fixated on HIAS, the refugee organization that helped save my family. | https://nyti.ms/2yEl6wd | 2018-10-28 | 08:49:55 |
| Science | The Wildebeest Is One Highly Toned Machine | By KAREN WEINTRAUB | New research shows the animal’s muscles are incredibly efficient for its size, more so than other large animals. | https://nyti.ms/2AqracP | 2018-10-24 | 13:00:04 |
| Smarter Living | Smart-Home Devices to Make Your Holidays Easier | By WIRECUTTER STAFF | These gadgets can take on your most mundane holiday tasks so you can focus on the fun stuff. | https://nyti.ms/2AtcTfw | 2018-10-26 | 14:07:13 |
| Sports | World Series: Red Sox vs. Dodgers Live Game 5 Updates | By BENJAMIN HOFFMAN | The Boston Red Sox are one win away from the franchise’s ninth World Series title. Stay here for inning-by-inning updates and analysis. | https://nyti.ms/2CJjCTY | 2018-10-28 | 18:35:04 |
| Sports | Dodgers’ Dave Roberts Second-Guessed in World Series Loss, Even by Trump | By TYLER KEPNER | The president tweeted his criticism after the Dodgers pulled their starter with a lead in the seventh inning. They ended up losing to the Red Sox, 9-6. | https://nyti.ms/2yFj1jw | 2018-10-28 | 10:08:19 |
| Sports | Dread, Then Grief, Engulfs Leicester After Owner’s Helicopter Crash | By RORY SMITH | A day after Vichai Srivaddhanaprabha died in a crash outside Leicester City’s stadium, fans gathered to pay tribute to a man known for his generosity. | https://nyti.ms/2CNHzd6 | 2018-10-28 | 20:12:32 |
| Sports | Kirk Gibson’s Home Run Ball Is Still Gone, Probably Forever | By JAMES WAGNER | The ball the injured Dodger sent into the stands in the 1988 World Series never turned up publicly. If it did now, who would believe it? | https://nyti.ms/2AtVeEx | 2018-10-26 | 18:00:49 |
| Style | Getting Shoppers Into Stores Takes More Than Inventory | By ARIEL FOXMAN | It would appear that the ‘hangout’ is now as important as what is hanging on the racks. | https://nyti.ms/2Je0eQe | 2018-10-25 | 05:00:12 |
| Technology | Just Embed a Phone Into This Editor’s Mind, Already | By CHOIRE SICHA | Choire Sicha, who runs The New York Times’s Styles desk, has such a close relationship with his smartphone, he says, it may be time to “punch the circuitry into the back of our skull.” | https://nyti.ms/2Je6NSZ | 2018-10-24 | 13:30:13 |
| U.S. | Pittsburgh Shooting Suspect, After Massacre, Said He ‘Wanted All Jews to Die’ | By CAMPBELL ROBERTSON, SABRINA TAVERNISE and SANDRA E. GARCIA | The authorities identified 11 victims of Saturday’s mass shooting at a synagogue in Pittsburgh and searched for clues about the suspect, Robert Bowers. | https://nyti.ms/2CLIYkb | 2018-10-28 | 12:02:02 |
| U.S. | Tree of Life Synagogue Victims Remembered as Guardians of Their Faith | By SIMON ROMERO, JENNIFER MEDINA and TIMOTHY WILLIAMS | Many of the 11 people who died in the synagogue shooting in Pittsburgh were advanced in years, but they were the steady backbone of their congregations. | https://nyti.ms/2yFBBbg | 2018-10-28 | 14:41:29 |
| U.S. | For Trump, Dutiful Words of Grief, Then Off to the Next Fight | By PETER BAKER and MAGGIE HABERMAN | President Trump’s discordant approach to national crises has thrust his leadership into the center of the national debate with about a week until the midterm elections. | https://nyti.ms/2yH8wfo | 2018-10-28 | 20:38:46 |
| U.S. | Pittsburgh Synagogue Massacre Suspect Was ‘Pretty Much a Ghost’ | By TRIP GABRIEL, JACK HEALY and JULIE TURKEWITZ | Robert Bowers had used anti-Semitic slurs on social media before his deadly rampage on a synagogue in Pittsburgh, but much of the rest of his life is only beginning to become clear. | https://nyti.ms/2yBNQ8v | 2018-10-28 | 15:12:54 |
| U.S. | On Gab, an Extremist-Friendly Site, Pittsburgh Shooting Suspect Aired His Hatred in Full | By KEVIN ROOSE | The social media site has become a haven for neo-Nazis, white nationalists and other extremists. It won’t change its policies, its founder says. | https://nyti.ms/2CNSweL | 2018-10-28 | 09:20:30 |
| U.S. | HIAS, the Jewish Agency Criticized by the Shooting Suspect, Has a History of Aiding Refugees | By MIRIAM JORDAN | HIAS has assisted hundreds of thousands of people displaced by conflict or persecution, often on account of their religious or political beliefs, to rebuild their lives. | https://nyti.ms/2CJPjN8 | 2018-10-28 | 16:26:14 |
| U.S. | To Rally Voters, Democrats Focus on Health Care as Their Closing Argument | By TRIP GABRIEL | Democrats in battleground Senate races, including Missouri, Arizona, Florida, West Virginia and Nevada, have embraced health care as they make their final pitches to voters. | https://nyti.ms/2yFcy8e | 2018-10-28 | 08:00:05 |
| U.S. | Gun Violence and Pipe Bombs Jolt Voters as Election Season Ends. ‘Again?’ One Asks. | By ELIZABETH DIAS | Nine days out from an already divisive midterm election, the campaign’s finale has come amid a cascade of horrors that reflected the country’s fault lines. | https://nyti.ms/2CNXlon | 2018-10-28 | 10:09:50 |
| U.S. | Tragedy in Pittsburgh, House and Senate Latest, Beto Backer: 9 Days to Go | By MATT FLEGENHEIMER and PATRICK HEALY | It’s always hard to know how major news events may affect an election. This time is no exception, with crosscurrents of grief, faith, fear and gun politics. | https://nyti.ms/2yC7uSa | 2018-10-28 | 06:30:05 |
| U.S. | Selling Donald Trump: A First-Time Campaign Manager Tries to Defy the Doubters | By MAGGIE HABERMAN | Brad Parscale believes he is uniquely equipped to manage the potentially fractious relationships inside the campaign and sell voters on the idea of sticking with an unorthodox president. | https://nyti.ms/2yEOD93 | 2018-10-28 | 17:09:28 |
| U.S. | Estranged Family of Cesar Sayoc, Accused Bomber, Wants Him to Get Help | By ALAN BLINDER and PATRICIA MAZZEI | Relatives recalled the troubled life of a man who “just started getting weirder,” and sent him a plea to accept mental health care. He is due in court on Monday. | https://nyti.ms/2yDOtyr | 2018-10-28 | 18:47:41 |
| U.S. | Back at Georgetown Prep, Kavanaugh Is Hailed as a Hero | By KATE KELLY | Justice Kavanaugh returned over the weekend for his high school reunion. His conduct when he was a student there was central to his Supreme Court confirmation battle. | https://nyti.ms/2yCu9xw | 2018-10-28 | 20:04:29 |
| Well | When the Hospice Care System Fails | By DANIELA J. LAMAS, M.D. | Your father was gasping for breath. The hospice care providers had not yet trained you in how to respond. So you called 911. | https://nyti.ms/2Aelm6a | 2018-10-17 | 05:00:01 |
| World | Jair Bolsonaro, Far-Right Populist, Elected President of Brazil | By ERNESTO LONDOÑO and SHASTA DARLINGTON | After choosing leftist presidents in four successive elections, Brazilians opted for a radical new course for Latin America’s largest nation. | https://nyti.ms/2CLjH9P | 2018-10-28 | 18:08:47 |
| World | Brazil’s Polarizing New President, Jair Bolsonaro, in His Own Words | By MARIANA SIMÕES | Brazil’s new leader has made countless divisive statements about torture, democracy, women, gay people and others during his rise to power. We look at some of them. | https://nyti.ms/2yyY21N | 2018-10-28 | 19:00:27 |
| World | 50 Years Later, a Victim of Ireland’s ‘Laundries’ Fights for Answers | By ED O’LOUGHLIN | Elizabeth Coppin spent her youth in abusive schools and workhouses. Now, a U.N. committee on torture has agreed to hear her allegations of systematic human rights violations. | https://nyti.ms/2Jgt16N | 2018-10-26 | 12:51:46 |
| World | As Rich Nations Close the Door on Refugees, Uganda Welcomes Them | By JOSEPH GOLDSTEIN | In Uganda, refugees are given land, the right to work, and freedom to move around. And far from prompting a backlash, Ugandans remain welcoming of refugees. | https://nyti.ms/2yCypgB | 2018-10-28 | 10:34:31 |