DATA_607_Week9_Homework_Bikram

Assignment – Web APIs

The New York Times web site provides a rich set of APIs, as described here: https://developer.nytimes.com/apis.

Task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it into an R DataFrame.


Load the libraries
knitr::opts_chunk$set(echo = TRUE)
library(httr)
library(jsonlite)
library(dplyr)
library(listviewer)
library(kableExtra)
library(tidyverse)
library(purrr)
library(htmltools)


New York Times- Top(Technology) Stories API

Call the API
View the JSON data response attributes
tech_response <- GET("https://api.nytimes.com/svc/topstories/v2/technology.json?api-key=97KSRsEE7aPgeEDTu3tZT4sVYND9NofH")

raw_tech_json = fromJSON(rawToChar(tech_response$content))

names(raw_tech_json)
## [1] "status"       "copyright"    "section"      "last_updated" "num_results" 
## [6] "results"


Interactive View of the response

# Interactive JSON results viewer
listviewer::jsonedit(raw_tech_json$results, height = "400px", mode = "view")


Format date columns

publish_date <- as.POSIXct( substr(raw_tech_json$results$published_date,1,10)  , format = "%Y-%m-%d")
raw_tech_json$results$fmt_published_date <- format(publish_date, "%m/%d/%Y")


create_date <- as.POSIXct( substr(raw_tech_json$results$created_date,1,10)  , format = "%Y-%m-%d")
raw_tech_json$results$fmt_created_date <- format(create_date, "%m/%d/%Y")

Converting JSON data into R dataframe

# Get the Facet information by each story (row_n)
get_facet_data <- function(row_n) {
  purrr::pluck(raw_tech_json, "results", "des_facet", row_n)
}

# create the dataframe and tidy it up
tech_stories_df <- pluck(raw_tech_json, "results") %>%
  # convert to tibble
  as_tibble() %>%
  # select columns of interest
  select(title, abstract, fmt_published_date, fmt_created_date ) %>%
  mutate(data = map(row_number(), get_facet_data)) %>% 
  unnest_wider(data) %>%
  mutate(published_date =    )

names(tech_stories_df) <- c('Title', 'Story Abstract', 'Published Date', 'Created Date', 'Facet 1', 'Facet 2', 'Facet 3', 'Facet 4', 
                       'Facet 5', 'Facet 6', 'Facet 7', 'Facet 8', 'Facet 9', 'Facet 10', 'Facet 11', 'Facet 12', 'Facet 13' )


knitr::kable(tech_stories_df, format = "html")
Title Story Abstract Published Date Created Date Facet 1 Facet 2 Facet 3 Facet 4 Facet 5 Facet 6 Facet 7 Facet 8 Facet 9 Facet 10
In India, Facebook Grapples With an Amplified Version of Its Problems Internal documents show a struggle with misinformation, hate speech and celebrations of violence in the country, the company’s biggest market. 10/23/2021 10/23/2021 Social Media Rumors and Misinformation Whistle-Blowers Computers and the Internet Politics and Government Engineering and Engineers Artificial Intelligence Fringe Groups and Movements Hate Speech Muslims and Islam
Internal Alarm, Public Shrugs: Facebook’s Employees Dissect Its Election Role Company documents show that the social network’s employees repeatedly raised red flags about the spread of misinformation and conspiracies before and after the contested November vote. 10/22/2021 10/22/2021 Social Media Rumors and Misinformation Computers and the Internet Presidential Election of 2020 Storming of the US Capitol (Jan, 2021) Right-Wing Extremism and Alt-Right Voter Fraud (Election Fraud) QAnon NA NA
Eating Disorders and Social Media Prove Difficult to Untangle Social media platforms like TikTok and Instagram try to monitor for content related to the problem, but it is not always clear what to do about it. 10/22/2021 10/22/2021 Social Media Eating Disorders Youth Computers and the Internet Anorexia Nervosa Beauty (Concept) Weight NA NA NA
Russia Is Censoring the Internet, With Coercion and Black Boxes Quietly built over two years, the Kremlin’s censorship infrastructure gives it sweeping power to block sites. Many fear a new age of digital isolation. 10/22/2021 10/22/2021 Computers and the Internet Social Media Censorship Regulation and Deregulation of Industry Politics and Government NA NA NA NA NA
A Rare Win in the Cat-and-Mouse Game of Ransomware A team of private security sleuths, in their first public detailing of their efforts, discuss how they used cybercriminals’ mistakes to quietly help victims recover their data. 10/24/2021 10/24/2021 Computer Security Cyberattacks and Hackers Extortion and Blackmail NA NA NA NA NA NA NA
Key takeaways from Week 7 of the Elizabeth Holmes trial. Prosecutors focused on her alleged deceptions, trying to make the case that she intentionally misled Theranos’s investors and commercial partners. 10/22/2021 10/22/2021 Frauds and Swindling Securities and Commodities Violations Tests (Medical) Computers and the Internet NA NA NA NA NA NA
Google said it had successfully ‘slowed down’ European privacy rules, according to lawsuit. The new details came in an unredacted version of a lawsuit filed by Texas and 11 other states. 10/22/2021 10/22/2021 Computers and the Internet Privacy Regulation and Deregulation of Industry NA NA NA NA NA NA NA
Lyft Says 1,807 Sexual Assaults Occurred in Rides in 2019 The company’s first-ever safety report, which it committed two years ago to producing, also included four fatal assaults and 49 deaths in crashes. 10/21/2021 10/21/2021 Car Services and Livery Cabs Sex Crimes NA NA NA NA NA NA NA NA
Hackers lay claim to Donald Trump’s social app before its launch. Hackers tied to the group Anonymous defaced a private version of the former president’s app, Truth Social, posting images of defecating pigs and explicative-laced rants. 10/21/2021 10/21/2021 Computers and the Internet Cyberattacks and Hackers Social Media United States Politics and Government Cyberwarfare and Defense Computer Security NA NA NA NA
Google plans to lower the cut it takes in its app store to 15 percent. The move is a concession to regulators and lawmakers, who are challenging whether the company has overcharged developers. 10/21/2021 10/21/2021 Mobile Applications Android (Operating System) Prices (Fares, Fees and Rates) Computer and Video Games E-Commerce NA NA NA NA NA
Why the Cable Company Is Selling TVs It all comes down to having power over what Americans watch and how. 10/21/2021 10/21/2021 Television Sets and Media Devices Cable Television Video Recordings, Downloads and Streaming NA NA NA NA NA NA NA
Amazon Workers on Staten Island Aim for Union Vote The organizers say they will have enough signatures by Monday to file for an election with the National Labor Relations Board. The company is pushing back. 10/21/2021 10/21/2021 Organized Labor Labor and Jobs Strikes Workplace Hazards and Violations Wages and Salaries Collective Bargaining NA NA NA NA
OnlyFans May Be a Refuge for Nude Fine Art The Vienna Tourist Board has joined the adults-only site to display artworks that other social platforms have censored. 10/21/2021 10/21/2021 Art Museums Nudism and Nudity Censorship Social Media NA NA NA NA NA
Can We Have Healthy Conversations Online? Not all online spaces are terrible. Join Shira Ovide, a Times technology writer, for a virtual event on Nov. 18 exploring the forces behind internet communities that work. 10/20/2021 10/20/2021 Computers and the Internet Social Media internal-open-access NA NA NA NA NA NA NA
The Webb Telescope’s Latest Stumbling Block: Its Name The long-awaited successor to the Hubble Space Telescope is scheduled to launch in December. But the NASA official for whom it is named has been accused of homophobia. 10/20/2021 10/20/2021 Space and Astronomy Stars and Galaxies Hubble Space Telescope Names, Organizational Telescopes and Observatories James Webb Space Telescope your-feed-science NA NA NA
Netflix employees walk out to protest Dave Chappelle’s special. The protest involved dozens of employees and was one of the most visible signs of worker unrest in the history of the streaming service. 10/20/2021 10/20/2021 Web-Original Programming Comedy and Humor Television NA NA NA NA NA NA NA
The True Cost of Upgrading Your Phone Buying a $1,000 iPhone can be equivalent to giving up $17,000 in retirement savings or 2,500 cups of coffee. 10/20/2021 10/20/2021 Smartphones iPhone Prices (Fares, Fees and Rates) Content Type: Service Personal Finances NA NA NA NA NA
PayPal Is Said to Be in Talks to Buy Pinterest in $45 Billion Deal If completed, it would be the largest consumer internet deal of the past decade. 10/20/2021 10/20/2021 Mergers, Acquisitions and Divestitures Computers and the Internet NA NA NA NA NA NA NA NA
A Hearing Aid for Everyone Over-the-counter hearing aids have the potential to show government and tech companies at their best. 10/20/2021 10/20/2021 internal-sub-only Hearing Aids Ears and Hearing NA NA NA NA NA NA NA
Blacktag Has Big Plans A new streaming platform wants to be a destination for Black audiences around the world. 10/20/2021 10/20/2021 Web-Original Programming Mobile Applications Black People Video Recordings, Downloads and Streaming NA NA NA NA NA NA
AMC to Add Onscreen Captions at Some Locations The move was lauded by advocates for the deaf and the hard of hearing, but theater owners worry audiences don’t want captions. 10/20/2021 10/20/2021 Theaters (Buildings) Disabilities Movies Deafness NA NA NA NA NA NA
How to Talk to the World Through Free Translation Apps Open up Google Translate or Apple’s Translate and your mobile device turns into your personal language interpreter. 10/20/2021 10/20/2021 Translation and Interpreters Language and Languages Android (Operating System) iOS (Operating System) Mobile Applications Content Type: Service NA NA NA NA
A Eureka Moment, Recreated in Film The Museum of Fine Arts, Boston, is using modern technology to stir appreciation for ancient artistry — including an Athenian potter’s transformative “discovery” of red. 10/20/2021 10/20/2021 Art Museums Ceramics and Pottery Computers and the Internet Roman Civilization Greek Civilization NA NA NA NA
Mark Zuckerberg will be added to a Facebook privacy lawsuit. The District of Columbia case, which grew out of the Cambridge Analytica scandal, could expose the chief executive to financial and other penalties. 10/20/2021 10/20/2021 Suits and Litigation (Civil) Social Media Data-Mining and Database Marketing Consumer Protection Antitrust Laws and Competition Issues Privacy United States Politics and Government NA NA NA
Boeing Deepens Probe Into Astronaut Capsule Woes, Prompting More Delays The Starliner capsule for NASA crews is now unlikely to have another orbital flight test until the middle of next year. 10/19/2021 10/19/2021 Rocket Science and Propulsion Engines Private Spaceflight Space and Astronomy Engineering and Engineers NA NA NA NA NA
Brazilian Leader Accused of Crimes Against Humanity in Pandemic Response A report from lawmakers initially said Jair Bolsonaro should be charged with mass homicide and genocide, accusing him of letting Covid-19 spread in a failed bid for herd immunity. 10/19/2021 10/19/2021 Coronavirus (2019-nCoV) Legislatures and Parliaments Politics and Government Deaths (Fatalities) War Crimes, Genocide and Crimes Against Humanity NA NA NA NA NA
We Won’t Forget WeWork’s Meltdown Start-up implosions don’t matter to most people. But the feelings of unfairness affect us all. 10/19/2021 10/19/2021 Income Inequality Start-ups Computers and the Internet NA NA NA NA NA NA NA
Facebook will pay up to $14 million to settle claims it favored foreign workers. The Justice Department sued the company in December, arguing that Facebook had declined to “recruit, consider or hire” qualified United States workers for more than 2,000 positions. 10/19/2021 10/19/2021 Labor and Jobs Wages and Salaries Suits and Litigation (Civil) Foreign Workers Discrimination Social Media NA NA NA NA
YouTube Sued Over Animal Abuse Videos, Accused of Not Enforcing Ban An animal rights group said the site had ignored efforts to get clear violations taken down. YouTube said it had removed hundreds of thousands. 10/19/2021 10/19/2021 Video Recordings, Downloads and Streaming Animal Abuse, Rights and Welfare Suits and Litigation (Civil) NA NA NA NA NA NA NA