DATA 607 Week 9 Assignment
Ahsanul Choudhury
October 28, 2016
Objective:
To choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it to an R dataframe. ***
For my assignment I chose to work with the New York Times “Most Popular API”. Under “Most Popular API” there are three sections:
Most Emailed by Section & Time Period
Most Shared by Section & Time Period
Most Viewed by Section & Time Period I am interested in using the Most Viewed section to see which Titles are most viewed today. New York Times “Most Popular API” give 20 results per page so I will use the top 20 results and plot the numeber of times each sections appears in top 20 most viewed today and for the past 7 days to get an idea of readers interest.
API Key
The first step for this assignment was to register and get a API key from New York Times. Once the API key was received the following steps were taken to complete the assignment
Load Libraries
library(jsonlite)
library(plyr)
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:plyr':
##
## arrange, count, desc, failwith, id, mutate, rename, summarise,
## summarize## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(knitr)
library(ggplot2)Establish connection with New York Times
uri <- fromJSON("https://api.nytimes.com/svc/mostpopular/v2/mostviewed/all-sections/1.json?api-key=31f8d708d3d9492c906146c5a1215690")Create R Data Frame
df <- ldply(uri[4], data.frame)Get details of data frame
length(df)## [1] 19names(df)## [1] ".id" "url" "adx_keywords" "column"
## [5] "section" "byline" "type" "title"
## [9] "abstract" "published_date" "source" "id"
## [13] "asset_id" "views" "des_facet" "org_facet"
## [17] "per_facet" "geo_facet" "media"Edit data frame
MostViewed <- df %>% select(views,title, section, url, byline, type)
head(MostViewed)## views
## 1 1
## 2 2
## 3 3
## 4 4
## 5 5
## 6 6
## title
## 1 A Scandal Too Far? Huma Abedin, Hillary Clinton, and a Test of Loyalty
## 2 Emails in Anthony Weiner Inquiry Jolt Hillary Clinton<U+0092>s Campaign
## 3 Hillary Clinton Assails James Comey, Calling Email Decision <U+0091>Deeply Troubling<U+0092>
## 4 Latest Upshot Poll Shows Trump With a Lead in Florida
## 5 Latest Election Polls 2016
## 6 <U+0091>This Changes Everything<U+0092>: Donald Trump Exults as Hillary Clinton<U+0092>s Team Scrambles
## section
## 1 U.S.
## 2 U.S.
## 3 U.S.
## 4 The Upshot
## 5 U.S.
## 6 U.S.
## url
## 1 http://www.nytimes.com/2016/10/30/us/politics/huma-abedin-anthony-weiner-clinton.html
## 2 http://www.nytimes.com/2016/10/29/us/politics/fbi-hillary-clinton-email.html
## 3 http://www.nytimes.com/2016/10/30/us/politics/hillary-clinton-emails-fbi-anthony-weiner.html
## 4 http://www.nytimes.com/interactive/2016/10/30/upshot/florida-poll.html
## 5 http://www.nytimes.com/interactive/2016/us/elections/polls.html
## 6 http://www.nytimes.com/2016/10/29/us/politics/donald-trump-hillary-clinton.html
## byline type
## 1 By AMY CHOZICK and MARK LANDLER Article
## 2 By ADAM GOLDMAN and ALAN RAPPEPORT Article
## 3 By PATRICK HEALY and JONATHAN MARTIN Article
## 4 By NATE COHN Interactive
## 5 Interactive
## 6 By AMY CHOZICK and PATRICK HEALY ArticleFinal data frame
kable(MostViewed)| views | title | section | url | byline | type |
|---|---|---|---|---|---|
| 1 | A Scandal Too Far? Huma Abedin, Hillary Clinton, and a Test of Loyalty | U.S. | http://www.nytimes.com/2016/10/30/us/politics/huma-abedin-anthony-weiner-clinton.html | By AMY CHOZICK and MARK LANDLER | Article |
| 2 | Emails in Anthony Weiner Inquiry Jolt Hillary Clintons Campaign | U.S. | http://www.nytimes.com/2016/10/29/us/politics/fbi-hillary-clinton-email.html | By ADAM GOLDMAN and ALAN RAPPEPORT | Article |
| 3 | Hillary Clinton Assails James Comey, Calling Email Decision ‘Deeply Troubling’ | U.S. | http://www.nytimes.com/2016/10/30/us/politics/hillary-clinton-emails-fbi-anthony-weiner.html | By PATRICK HEALY and JONATHAN MARTIN | Article |
| 4 | Latest Upshot Poll Shows Trump With a Lead in Florida | The Upshot | http://www.nytimes.com/interactive/2016/10/30/upshot/florida-poll.html | By NATE COHN | Interactive |
| 5 | Latest Election Polls 2016 | U.S. | http://www.nytimes.com/interactive/2016/us/elections/polls.html | Interactive | |
| 6 | ‘This Changes Everything’: Donald Trump Exults as Hillary Clintons Team Scrambles | U.S. | http://www.nytimes.com/2016/10/29/us/politics/donald-trump-hillary-clinton.html | By AMY CHOZICK and PATRICK HEALY | Article |
| 7 | 2016 Election Forecast: Who Will Be President? | The Upshot | http://www.nytimes.com/interactive/2016/upshot/presidential-polls-forecast.html | By JOSH KATZ | Interactive |
| 8 | Justice Dept. Strongly Discouraged Comey on Move in Clinton Email Case | U.S. | http://www.nytimes.com/2016/10/30/us/politics/comey-clinton-email-justice.html | By MATT APUZZO, ADAM GOLDMAN, MICHAEL S. SCHMIDT and WILLIAM K. RASHBAUM | Article |
| 9 | 10 Questions (and Answers) About New Email Trove | U.S. | http://www.nytimes.com/2016/10/31/us/politics/fbi-hillary-clinton-emails.html | By MICHAEL S. SCHMIDT and MATT APUZZO | Article |
| 10 | On Clinton Emails, Did the F.B.I. Director Abuse His Power? | Opinion | http://www.nytimes.com/2016/10/30/opinion/on-clinton-emails-did-the-fbi-director-abuse-his-power.html | By RICHARD W. PAINTER | Article |
| 11 | No One Is Looking at This Headline | Sports | http://www.nytimes.com/2016/10/29/sports/phil-heath-mr-olympia-bodybuilder.html | By JOHN BRANCH | Article |
| 12 | ‘Trumps Going to Bake an Inauguration Cake Out of This’ | NYT Now | http://www.nytimes.com/2016/10/29/nytnow/top-10-comments-trumps-going-to-bake-an-inauguration-cake-out-of-this.html | By LELA MOORE and SONA PATEL | Article |
| 13 | World Series: Heres What Happened in Game 3 | Sports | http://www.nytimes.com/2016/10/28/sports/world-series-score-cubs-indians.html | By VICTOR MATHER and DAVID WALDSTEIN | Article |
| 14 | 3 TVs and No Food: Growing Up Poor in America | Opinion | http://www.nytimes.com/2016/10/30/opinion/sunday/3-tvs-and-no-food-growing-up-poor-in-america.html | By NICHOLAS KRISTOF | Article |
| 15 | ‘I Live in a Lie’: Saudi Women Speak Up | World | http://www.nytimes.com/2016/10/29/world/middleeast/saudi-arabia-women.html | By MONA EL-NAGGAR | Article |
| 16 | WikiLeaks Lays Bare a Clinton Insiders Emphatic Cheers and Jeers | U.S. | http://www.nytimes.com/2016/10/30/us/politics/neera-tanden-wikileaks.html | By STEVE EDER and NICHOLAS CONFESSORE | Article |
| 17 | For Democrats, Anthony Weiner Makes an Unwelcome Return | U.S. | http://www.nytimes.com/2016/10/31/us/politics/anthony-weiner-democratic-reaction.html | By MAGGIE HABERMAN and ALEXANDER BURNS | Article |
| 18 | Peter Thiel Defends His Most Contrarian Move Yet: Supporting Trump | Technology | http://www.nytimes.com/2016/10/31/technology/peter-thiel-defends-his-most-contrarian-move-yet-supporting-trump.html | By DAVID STREITFELD | Article |
| 19 | Letter to Congress From F.B.I. Director on Clinton Email Case | U.S. | http://www.nytimes.com/interactive/2016/10/28/us/politics/fbi-letter.html | Interactive | |
| 20 | Indians Defeat Sloppy Cubs to Take a 3-1 World Series Lead | Sports | http://www.nytimes.com/2016/10/30/sports/baseball/world-series-cleveland-indians-chicago-cubs.html | By DAVID WALDSTEIN | Article |
I wanted to see how many times each section appears in the top 20 most viewed today and in past 7 days
Section count in top 20 most viewed today
SectionCount <- MostViewed %>%count(section)
SectionCount## # A tibble: 7 × 2
## section n
## <chr> <int>
## 1 NYT Now 1
## 2 Opinion 2
## 3 Sports 3
## 4 Technology 1
## 5 The Upshot 2
## 6 U.S. 10
## 7 World 1Plot section count in top 20 most viewed today
ggplot(data = SectionCount, aes(x = section, y = n, fill = section))+ geom_bar(stat="identity", position="dodge") + ggtitle("Most Viewed by Section (Today)") + ylab("Frequency")+ coord_flip() + geom_text(aes(label = n))
Plot section count in top 20 most viewed for the past 7 days
uriWeek <- fromJSON("https://api.nytimes.com/svc/mostpopular/v2/mostviewed/all-sections/7.json?api-key=31f8d708d3d9492c906146c5a1215690")
dfWeek <- ldply(uriWeek[4], data.frame)
SectionCountWeek <- dfWeek %>%count(section)
ggplot(data = SectionCountWeek, aes(x = section, y = n, fill = section))+ geom_bar(stat="identity", position="dodge") + ggtitle("Most Viewed by Section(1 Week)") + ylab("Frequency")+ coord_flip() + geom_text(aes(label = n))
Conclusion
With API New York time provide an easy way to access their data. Once I was able to connect using API the JSON data is easy to manipulate with in r and there are few libraries available for that.