Web APIs

CUNY MSDS - DATA607 - Home Work-9

James Kuruvilla

October 28, 2017

Assignment

————————————————————————————————————
————————————————————————————————————

Library Definition

library(knitr)
library(httr)
library(RCurl)
library(jsonlite)
library(RCurl)
library(stringr)
library(ggplot2)

Scrape data from the web in json format and convert it to data frame

myData<- GET(url)
json_raw  <- content(myData, type = "text", encoding = "UTF-8")
json_data <- fromJSON(json_raw)
df        <- as.data.frame(json_data$results)
df_final  <- df %>% select(section, byline, type,title,abstract, published_date)
df_final$byline    <-  str_replace(df_final$byline, 'By ', "")
colnames(df_final) <- c("Section","Author", "Type","Title", "Abstract", "Published_Date")

kable(df_final,"html")
Section Author Type Title Abstract Published_Date
U.S. THE NEW YORK TIMES Article Multiple Weapons Found in Las Vegas Gunman’s Hotel Room The police said the gunman killed himself after firing on concertgoers outside, leaving more than 500 people injured. 2017-10-02
U.S. JODI KANTOR and MEGAN TWOHEY Article Harvey Weinstein Paid Off Sexual Harassment Accusers for Decades An investigation by The New York Times found allegations stretching back to 1990 about Mr. Weinstein’s treatment of women in Hollywood. 2017-10-05
Business Day JOHN KOBLIN Article Premature CBS Report of Tom Petty’s Death Sets Off an Outpouring A premature report was attributed to the Los Angeles Police Department, which later said that “initial information was inadvertently provided to some media sources.” 2017-10-02
U.S. JODI KANTOR and RACHEL ABRAMS Article Gwyneth Paltrow, Angelina Jolie and Others Say Weinstein Harassed Them “This way of treating women ends now,” Ms. Paltrow said as she and other actresses accused the producer of casting-couch abuses. 2017-10-10
Opinion ROSS DOUTHAT Article Speaking Ill of Hugh Hefner An honest obituary for a wicked American. 2017-09-30
U.S. JOSE A. DELREAL and JONAH ENGEL BROMWICH Article Stephen Paddock, Las Vegas Suspect, Was a Gambler Who Drew Little Attention The man who police say killed dozens of people and then himself Sunday night was described as a retiree who lived quietly, liked to gamble, and had no criminal record. 2017-10-02
Opinion LUPITA NYONG’O Article Lupita Nyong’o: Speaking Out About Harvey Weinstein I don’t feel alone anymore. Let us never shut up about this. 2017-10-19
U.S. KEN BELSON, JENNIFER MEDINA and RICHARD PÉREZ-PEÑA Article A Burst of Gunfire, a Pause, Then Carnage in Las Vegas That Would Not Stop Thousands of concertgoers found themselves turned into targets, and in the chaos, many tried to lie flat, with no idea they were being fired upon from above. 2017-10-02
U.S. SARAH ALMUKHTAR, AUDREY CARLSEN, KENAN DAVIS, JOHN ELIGON, FORD FESSENDEN, TROY GRIGGS, MIKA GRÖNDAHL, EVAN GROTHJAN, K.K. RE Interactive Las Vegas Shooting: Chaos at a Concert and a Frantic Search at Mandalay Bay How one of the deadliest mass shootings in American history unfolded. 2017-10-02
Business Day LAURA MANSNERUS Article Hugh Hefner, Who Built the Playboy Empire and Embodied It, Dies at 91 Mr. Hefner, the magazine’s very public avatar, styled himself as an emblem of the sexual revolution. 2017-09-27
Magazine BENOIT DENIZET-LEWIS Article Why Are More American Teenagers Than Ever Suffering From Severe Anxiety? Parents, therapists and schools are struggling to figure out whether helping anxious teenagers means protecting them or pushing them to face their fears. 2017-10-11
U.S. JONATHAN MARTIN and MARK LANDLER Article Bob Corker Says Trump’s Recklessness Threatens ‘World War III’ The powerful Republican senator’s remarks in an interview capped a remarkable day of sulfurous insults between him and the president. 2017-10-08
U.S. SARAH ALMUKHTAR, AUDREY CARLSEN, JON HUANG, K.K. REBECCA LAI, JASMINE C. LEE, JUGAL K. PATEL, ANJALI SINGHVI and DEREK WATKIN Interactive Before and After Photos: Fires Tear Through California’s Wine Country Images, including before and after views, reveal the scale of the devastation. 2017-10-10
Opinion MAYIM BIALIK Article Mayim Bialik: Being a Feminist in Harvey Weinstein’s World I have always had an uncomfortable relationship with being employed in an industry built on the objectification of women. 2017-10-13
Movies CARA BUCKLEY and MELENA RYZIK Article Rose McGowan Attacks Ben Affleck Over Harvey Weinstein: ‘You Lie’ Mr. Affleck said Tuesday he was “angry” about Mr. Weinstein’s alleged harassment and assaults of women but did not indicate he was aware of the behavior. Ms. McGowan said he knew. 2017-10-10
World ROD NORDLAND Article ISIS Fighters, Having Pledged to Fight or Die, Surrender en Masse More than 1,000 militants turned themselves in after the latest in a string of humiliating defeats in Iraq and Syria. Many claim to have been only cooks or clerks. 2017-10-08
U.S. THE NEW YORK TIMES Article Gunman’s Girlfriend Arrives in U.S. and Is Expected to Be Questioned Marilou Danley arrived at Los Angeles International Airport on Tuesday night and was met by the authorities, according to a law enforcement official. 2017-10-03
Science SHANNON HALL Article A Surprise From the Supervolcano Under Yellowstone The conditions that lead to supereruptions can emerge within a single human lifetime, according to a study of the volcano’s remnants in the national park. 2017-10-10
Health DIONNE SEARCEY and MATT RICHTEL Article Obesity Was Rising as Ghana Embraced Fast Food. Then Came KFC. The growing popularity of fried chicken and pizza in parts of Africa underscores how fast food is changing habits and expanding waistlines. 2017-10-02
U.S. LARRY BUCHANAN, EVAN GROTHJAN, JON HUANG, YULIYA PARSHINA-KOTTAS, ADAM PEARCE and KAREN YOURISH Interactive What Is a Bump Stock and How Does It Work? Twelve of the rifles the gunman had in his hotel room were outfitted with a “bump stock,” an attachment that enables a semiautomatic rifle to fire faster. 2017-10-04

Graph the 20 Most Viewed New York Times Articles by Section by Time period 30

df_final %>%
   group_by(Section) %>% 
   tally %>% 
   ggplot(aes(Section, n, fill = Section)) +
          geom_bar(stat = "identity", position = "stack") +
          coord_flip()   + theme(legend.position = "none") + 
    ggtitle("20 Most Viewed NY Times Articles by Section") +
    xlab("Section") + ylab("Number of Views") +
    geom_text(aes(label=n), vjust=0.5, hjust=1.1,color="black")