The New York Times web site provides a rich set of APIs, as described here: https://developer.nytimes.com/apis. We wiil start by signing up for an API key.
We will choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it into an R DataFrame.
#install.packages("httr")
#install.packages("jsonlite")
#install.packages("tidyverse") # For data manipulationlibrary(httr)
library(jsonlite)## Warning: package 'jsonlite' was built under R version 4.4.3library(tidyverse)## Warning: package 'tidyverse' was built under R version 4.4.3## Warning: package 'ggplot2' was built under R version 4.4.3## Warning: package 'dplyr' was built under R version 4.4.3## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.4 ✔ readr 2.1.5
## ✔ forcats 1.0.0 ✔ stringr 1.5.1
## ✔ ggplot2 3.5.1 ✔ tibble 3.2.1
## ✔ lubridate 1.9.4 ✔ tidyr 1.3.1
## ✔ purrr 1.0.4
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ purrr::flatten() masks jsonlite::flatten()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors#file.edit("~/.Renviron")api_key <- Sys.getenv("NYTIMES_API_KEY= RXrScA3v9v3wJ2sj4AGy6oAD0dKIaE7V
")# to get articles currently on the homepage.
# Define the API URL
url <- paste0("https://api.nytimes.com/svc/topstories/v2/home.json?api-key=RXrScA3v9v3wJ2sj4AGy6oAD0dKIaE7V", api_key)
# Fetch data from API
response <- GET(url)
# Convert response to JSON format
data <- content(response, as = "text", encoding = "UTF-8")
json_data <- fromJSON(data, flatten = TRUE) # Flatten nested lists
# Convert to Data Frame
nyt_df <- as.data.frame(json_data$results)
# View Data Frame structure
glimpse(nyt_df)## Rows: 26
## Columns: 19
## $ section <chr> "us", "world", "business", "business", "podcasts",…
## $ subsection <chr> "politics", "", "economy", "", "the-headlines", "m…
## $ title <chr> "White House Weighs Helping Farmers as Trump Escal…
## $ abstract <chr> "The Trump administration has discussed providing …
## $ url <chr> "https://www.nytimes.com/2025/03/31/us/politics/fa…
## $ uri <chr> "nyt://article/a78c89a3-064b-5df9-8bee-776654aee4f…
## $ byline <chr> "By Tony Romm", "By Damien Cave", "By Lydia DePill…
## $ item_type <chr> "Article", "Article", "Article", "Article", "Artic…
## $ updated_date <chr> "2025-03-31T06:27:05-04:00", "2025-03-31T06:44:03-…
## $ created_date <chr> "2025-03-31T05:02:32-04:00", "2025-03-31T00:01:16-…
## $ published_date <chr> "2025-03-31T05:02:32-04:00", "2025-03-31T00:01:16-…
## $ material_type_facet <chr> "", "", "", "", "", "", "", "", "", "", "", "", ""…
## $ kicker <chr> "", "News Analysis", "", "", "", "", "", "", "", "…
## $ des_facet <list> <"United States Politics and Government", "Federa…
## $ org_facet <list> "Agriculture Department", <"European Union", "Nor…
## $ per_facet <list> <"Rollins, Brooke (1972- )", "Trump, Donald J">, …
## $ geo_facet <list> "United States", <"United States", "Europe", "Mex…
## $ multimedia <list> [<data.frame[3 x 8]>], [<data.frame[3 x 8]>], [<d…
## $ short_url <chr> "", "", "", "", "", "", "", "", "", "", "", "", ""…url <- paste0('https://api.nytimes.com/svc/topstories/v2/science.json?api-key=RXrScA3v9v3wJ2sj4AGy6oAD0dKIaE7V', Sys.getenv("NYTIMES_API_KEY"))
url2 <- paste0('https://api.nytimes.com/svc/topstories/v2/health.json?api-key=RXrScA3v9v3wJ2sj4AGy6oAD0dKIaE7V', Sys.getenv("NYTIMES_API_KEY"))
science_data <- fromJSON(url)$results %>%
as.data.frame() %>%
add_column("web_section" = "Science", .before = "section")
health_data <- fromJSON(url2)$results %>%
as.data.frame() %>%
add_column("web_section" = "Health", .before = "section")head(science_data )## web_section section subsection
## 1 Science science
## 2 Science admin
## 3 Science science
## 4 Science health
## 5 Science health
## 6 Science health
## title
## 1 Sign Up for the Science Times Newsletter
## 2
## 3 A New Dinosaur Museum Rises From a Hole in the Ground in New Jersey
## 4 Drug Reduces Mysterious Particle Involved in Heart Attack Risk
## 5 Using a ‘Tiny Bit of Math’ May Improve Your Heart Health, Study Suggests
## 6 Top F.D.A. Vaccine Official Resigns, Citing Kennedy’s ‘Misinformation and Lies’
## abstract
## 1 Every week, we’ll bring you stories that capture the wonders of the human body, nature and the cosmos.
## 2
## 3 The museum hopes that after learning about the planet’s prehistoric past, people will do more to preserve Earth’s future.
## 4 The Eli Lilly drug caused a major drop in the blood levels of Lp(a), but further research is needed to show that it will prevent heart attacks and strokes.
## 5 Your average daily heart rate is a useful metric; so is your daily step count. Combining the two might be even better.
## 6 Dr. Peter Marks, a veteran of the agency, wrote that undermining confidence in vaccines is irresponsible and a danger to public health.
## url
## 1 null
## 2
## 3 https://www.nytimes.com/2025/03/31/science/dinosaur-fossil-museum-new-jersey.html
## 4 https://www.nytimes.com/2025/03/30/health/heart-attack-lpa-protein.html
## 5 https://www.nytimes.com/2025/03/29/health/heart-health-steps-smartwatch.html
## 6 https://www.nytimes.com/2025/03/28/health/fda-vaccines-rfk-jr-peter-marks.html
## uri
## 1 nyt://embeddedinteractive/daba9d03-29b8-5cfc-8cbd-74b8abf54a07
## 2 nyt://embeddedinteractive/6cb185ed-65ac-50a5-ace3-5bab81934245
## 3 nyt://article/cdc9fef4-fa42-5ffb-91ed-245f3e8e96cf
## 4 nyt://article/2bbb0f4b-e8f9-585d-a937-8c91f190a7b2
## 5 nyt://article/23380396-50f6-5c72-bfc6-52a3b8828441
## 6 nyt://article/346d46bf-4ec7-58a4-843c-da851c03fa73
## byline item_type
## 1 EmbeddedInteractive
## 2 EmbeddedInteractive
## 3 By Kenneth Chang Article
## 4 By Gina Kolata Article
## 5 By Matt Richtel Article
## 6 By Christina Jewett, Sheryl Gay Stolberg and Noah Weiland Article
## updated_date created_date published_date
## 1 2018-04-07T13:23:25-04:00 2016-02-05T18:18:53-05:00 2016-02-05T18:18:53-05:00
## 2 2015-07-20T13:45:03-04:00 2015-04-16T14:13:18-04:00 2015-04-16T14:13:18-04:00
## 3 2025-03-31T05:55:36-04:00 2025-03-31T05:02:48-04:00 2025-03-31T05:02:48-04:00
## 4 2025-03-30T22:55:38-04:00 2025-03-30T14:30:06-04:00 2025-03-30T14:30:06-04:00
## 5 2025-03-30T13:36:21-04:00 2025-03-29T09:30:04-04:00 2025-03-29T09:30:04-04:00
## 6 2025-03-29T21:45:33-04:00 2025-03-28T21:29:47-04:00 2025-03-28T21:29:47-04:00
## material_type_facet kicker
## 1
## 2
## 3
## 4
## 5
## 6
## des_facet
## 1
## 2
## 3 Paleontology, Fossils, Museums, Endangered and Extinct Species, Global Warming, Dinosaurs, Quarries, your-feed-science
## 4 Heart, Lipoproteins, Drugs (Pharmaceuticals), Clinical Trials, Stroke, Deaths (Fatalities), Research
## 5 Heart, Wearable Computing, Diabetes, Stroke, Exercise, Hypertension, Blood Pressure
## 6 Appointments and Executive Changes, Medicine and Health, Regulation and Deregulation of Industry, Vaccination and Immunization, Rumors and Misinformation, Measles, Coronavirus (2019-nCoV), Ethics and Official Misconduct, United States Politics and Government
## org_facet
## 1
## 2
## 3 Rowan University, Edelman Fossil Park and Museum
## 4 Eli Lilly and Company, New England Journal of Medicine
## 5 American Heart Assn, Cleveland Clinic
## 6 Food and Drug Administration
## per_facet geo_facet
## 1
## 2
## 3 Lacovara, Kenneth J Mantua Township (NJ)
## 4
## 5
## 6 Marks, Peter W, Kennedy, Robert F Jr, Trump, Donald J
## multimedia
## 1 https://static01.nyt.com/images/2016/02/06/science/sciencenewsletter2/sciencenewsletter2-superJumbo.jpg, https://static01.nyt.com/images/2016/02/06/science/sciencenewsletter2/sciencenewsletter2-mediumThreeByTwo440.jpg, https://static01.nyt.com/images/2016/02/06/science/sciencenewsletter2/sciencenewsletter2-thumbLarge.jpg, Super Jumbo, mediumThreeByTwo440, Large Thumbnail, 1534, 293, 150, 2048, 440, 150, image, image, image, photo, photo, photo, , , , NASA, via Associated Press, NASA, via Associated Press, NASA, via Associated Press
## 2 NULL
## 3 https://static01.nyt.com/images/2025/04/01/multimedia/31HS-sci-nj-dino-museum-06-cbhv/31HS-sci-nj-dino-museum-06-cbhv-superJumbo.jpg, https://static01.nyt.com/images/2025/04/01/multimedia/31HS-sci-nj-dino-museum-06-cbhv/31HS-sci-nj-dino-museum-06-cbhv-threeByTwoSmallAt2X.jpg, https://static01.nyt.com/images/2025/04/01/multimedia/31HS-sci-nj-dino-museum-06-cbhv/31HS-sci-nj-dino-museum-06-cbhv-thumbLarge.jpg, Super Jumbo, threeByTwoSmallAt2X, Large Thumbnail, 1365, 400, 150, 2048, 600, 150, image, image, image, photo, photo, photo, Ric Edelman, left, whose donation helped build the Edelman Fossil Park & Museum, looks up at a re-creation of a baby hadrosaur in one of the exhibits., Ric Edelman, left, whose donation helped build the Edelman Fossil Park & Museum, looks up at a re-creation of a baby hadrosaur in one of the exhibits., Ric Edelman, left, whose donation helped build the Edelman Fossil Park & Museum, looks up at a re-creation of a baby hadrosaur in one of the exhibits., Graham Dickie/The New York Times, Graham Dickie/The New York Times, Graham Dickie/The New York Times
## 4 https://static01.nyt.com/images/2025/03/30/multimedia/30HS-heart-1-fkbv/30HS-heart-1-fkbv-superJumbo.jpg, https://static01.nyt.com/images/2025/03/30/multimedia/30HS-heart-1-fkbv/30HS-heart-1-fkbv-threeByTwoSmallAt2X.jpg, https://static01.nyt.com/images/2025/03/30/multimedia/30HS-heart-1-fkbv/30HS-heart-1-fkbv-thumbLarge.jpg, Super Jumbo, threeByTwoSmallAt2X, Large Thumbnail, 1367, 400, 150, 2048, 600, 150, image, image, image, photo, photo, photo, Monte Wooden, 71, had a heart attack despite low risk on most measures. Doctors discovered he had a very high level of a tiny particle in his blood called Lp(a)., Monte Wooden, 71, had a heart attack despite low risk on most measures. Doctors discovered he had a very high level of a tiny particle in his blood called Lp(a)., Monte Wooden, 71, had a heart attack despite low risk on most measures. Doctors discovered he had a very high level of a tiny particle in his blood called Lp(a)., Max Whittaker for The New York Times, Max Whittaker for The New York Times, Max Whittaker for The New York Times
## 5 https://static01.nyt.com/images/2025/04/01/multimedia/29HS-SMARTWATCH-jgml/29HS-SMARTWATCH-jgml-superJumbo.jpg, https://static01.nyt.com/images/2025/04/01/multimedia/29HS-SMARTWATCH-jgml/29HS-SMARTWATCH-jgml-threeByTwoSmallAt2X.jpg, https://static01.nyt.com/images/2025/04/01/multimedia/29HS-SMARTWATCH-jgml/29HS-SMARTWATCH-jgml-thumbLarge.jpg, Super Jumbo, threeByTwoSmallAt2X, Large Thumbnail, 1365, 400, 150, 2048, 600, 150, image, image, image, photo, photo, photo, , , , Anya Broido for The New York Times, Anya Broido for The New York Times, Anya Broido for The New York Times
## 6 https://static01.nyt.com/images/2025/04/24/multimedia/28FDA-MARKS/28FDA-MARKS-superJumbo-v2.jpg, https://static01.nyt.com/images/2025/04/24/multimedia/28FDA-MARKS/28FDA-MARKS-threeByTwoSmallAt2X-v2.jpg, https://static01.nyt.com/images/2025/04/24/multimedia/28FDA-MARKS/28FDA-MARKS-thumbLarge.jpg, Super Jumbo, threeByTwoSmallAt2X, Large Thumbnail, 1366, 400, 150, 2048, 600, 150, image, image, image, photo, photo, photo, The Food and Drug Administration’s top vaccine official, Dr. Peter Marks, resigned Friday., The Food and Drug Administration’s top vaccine official, Dr. Peter Marks, resigned Friday., The Food and Drug Administration’s top vaccine official, Dr. Peter Marks, resigned Friday., Greg Nash, Greg Nash, Greg Nash
## short_url
## 1
## 2
## 3
## 4
## 5
## 6most_common_subsection <- nyt_df %>%
count(subsection) %>%
arrange(desc(n))
most_common_subsection %>%
knitr::kable(col.names = c("subsection Tag", "Articles counts"))| subsection Tag | Articles counts |
|---|---|
| 10 | |
| middleeast | 4 |
| asia | 3 |
| politics | 2 |
| africa | 1 |
| economy | 1 |
| europe | 1 |
| media | 1 |
| music | 1 |
| television | 1 |
| the-headlines | 1 |
# The descriptive tags
most_common_des <- nyt_df %>%
count(des_facet) %>%
arrange(desc(n)) %>%
filter(n >= 1)
most_common_des %>%
knitr::kable(col.names = c("subsection Tag", "Counts"))| subsection Tag | Counts |
|---|---|
| United States Politics and Government, Federal Aid (US) , Customs (Tariff) , Agriculture and Farming , United States Economy , Politics and Government , Taxation , International Trade and World Market | 1 |
| United States Politics and Government, International Relations , Defense and Military Forces , Customs (Tariff) , United States International Relations, Nuclear Weapons | 1 |
| United States Politics and Government , Deportation , Foreign Workers , Law and Legislation , Agriculture and Farming , Illegal Immigration , Labor and Jobs , Immigration and Emigration , Visas , Deferred Action for Childhood Arrivals, Building (Construction) , Day Laborers , Lobbying and Lobbyists | 1 |
| Standard & Poor’s 500-Stock Index , International Trade and World Market, Stocks and Bonds , Customs (Tariff) | 1 |
| Myanmar Earthquake (March 28, 2025), Politics and Government | 1 |
| Appointments and Executive Changes , Espionage and Intelligence Services, Politics and Government | 1 |
| Israel-Gaza War (2023- ) , War and Armed Conflicts , Defense and Military Forces | 1 |
| Israel-Gaza War (2023- ) , Defense and Military Forces, Evacuations and Evacuees | 1 |
| International Relations , Israel-Gaza War (2023- ) | 1 |
| Happiness , Saunas and Sweat Lodges, Forests and Forestry , Trees and Shrubs , Gardens and Gardening , Travel and Vacations , Local Food | 1 |
| Decisions and Verdicts , Embezzlement , Right-Wing Extremism and Alt-Right, Politics and Government , Legislatures and Parliaments , Fringe Groups and Movements , Law and Legislation , Elections | 1 |
| Earthquakes , Humanitarian Aid , United States International Relations, Foreign Aid , International Relations | 1 |
| Myanmar Earthquake (March 28, 2025), Buildings (Structures) , Accidents and Safety | 1 |
| Courts and the Judiciary , Elections, Courts and the Judiciary | 1 |
| United States Politics and Government, Conservatism (US Politics) | 1 |
| United States Politics and Government, Elections, House of Representatives , Politics and Government | 1 |
| International Trade and World Market , United States International Relations, International Relations | 1 |
| Electric and Hybrid Vehicles, Trucks and Trucking | 1 |
| Illegal Immigration , International Relations , Deportation , Terrorism , Refugees and Displaced Persons | 1 |
| United States Politics and Government , Presidential Election of 2024 , Signal Chat Leak (Trump Administration) | 1 |
| Russian Invasion of Ukraine (2022) , United States International Relations, United States Politics and Government, Foreign Aid , Defense and Military Forces | 1 |
| internal-storyline-no , United States Politics and Government, Federal Budget (US) | 1 |
| Demonstrations, Protests and Riots , Coups D’Etat and Attempted Coups D’Etat, Politics and Government | 1 |
| Deaths (Obituaries) , News and News Media , Television , United States Politics and Government, Radio , Peabody Awards | 1 |
| Television , Actors and Actresses , The Thorn Birds (TV Program), Shogun (1980) (TV Program) | 1 |
| Classical Music , Opera , Content Type: Personal Profile, Le Nozze di Figaro (Opera) | 1 |
# The descriptive tags in science_data
most_common_des <- science_data %>%
count(des_facet) %>%
arrange(desc(n)) %>%
filter(n >= 1)
most_common_des %>%
knitr::kable(col.names = c("subsection Tag", "Counts"))| subsection Tag | Counts |
|---|---|
| 2 | |
| Paleontology , Fossils , Museums , Endangered and Extinct Species, Global Warming , Dinosaurs , Quarries , your-feed-science | 1 |
| Heart , Lipoproteins , Drugs (Pharmaceuticals), Clinical Trials , Stroke , Deaths (Fatalities) , Research | 1 |
| Heart , Wearable Computing, Diabetes , Stroke , Exercise , Hypertension , Blood Pressure | 1 |
| Appointments and Executive Changes , Medicine and Health , Regulation and Deregulation of Industry, Vaccination and Immunization , Rumors and Misinformation , Measles , Coronavirus (2019-nCoV) , Ethics and Official Misconduct , United States Politics and Government | 1 |
| Fertilizer , Sewers and Sewage , PFAS (Per- and Polyfluoroalkyl Substances), Hazardous and Toxic Substances , Chemicals , Livestock , Environment , Pollution | 1 |
| Eclipses , Space and Astronomy , Sun , Moon , Earth , Content Type: Service | 1 |
| Global Warming , Gardens and Gardening, Flowers and Plants , Trees and Shrubs , Vegetables , Spring (Season) , Winter (Season) , Birds | 1 |
| Minorities , United States Politics and Government, Psychology and Psychologists , Diversity Initiatives , Colleges and Universities , Accreditation , Race and Ethnicity , Executive Orders and Memorandums , your-feed-science , your-feed-healthcare | 1 |
| Vaccination and Immunization | 1 |
| Abortion , Abortion Drugs , States (US) , Law and Legislation , Decisions and Verdicts , Suits and Litigation (Civil), Extradition , Telemedicine , Doctors , your-feed-healthcare | 1 |
| Global Warming , Greenhouse Gas Emissions , United States Politics and Government , Regulation and Deregulation of Industry, Air Pollution , Law and Legislation , Coal , Hazardous and Toxic Substances , Clean Air Act , Environment , Mercury (Metal) | 1 |
| Tuberculosis , Everything Is Tuberculosis: The History and Persistence of Our Deadliest Infection (Book), Books and Literature , Foreign Aid , Disease Rates , Poverty , Antibiotics , United States Politics and Government , Acquired Immune Deficiency Syndrome , Epidemics , Vaccination and Immunization , Bacteria , Drug Resistance (Microbial) , Drugs (Pharmaceuticals) , Content Type: Personal Profile | 1 |
| your-feed-science , Measles , Disease Rates , Epidemics , Vaccination and Immunization, Children and Childhood , Medicine and Health , States (US) | 1 |
| Coronavirus (2019-nCoV) , Epidemics , Disease Rates , Vaccination and Immunization , Research , United States Politics and Government, Drugs (Pharmaceuticals) | 1 |
| Federal-State Relations (US) , States (US) , Federal Aid (US) , United States Politics and Government, Medicine and Health , Coronavirus (2019-nCoV) , Disease Rates , your-feed-healthcare , your-feed-science | 1 |
| Foreign Aid , United States International Relations, United States Politics and Government, Developing Countries , Vaccination and Immunization , Malaria , Medicine and Health | 1 |
| Hunting and Trapping , Global Warming , Oceans and Seas , Dolphins and Porpoises , Economic Conditions and Trends | 1 |
| Area Planning and Renewal , Pedestrian Malls , Urban Areas , Parking , Sustainable Living , Real Estate and Housing (Residential), Transit Systems , Mixed-Use Developments | 1 |
| Geothermal Power, Heating , Hot Springs , Aquifers | 1 |
| Global Warming , Environment , Gardens and Gardening , Forests and Forestry , Trees and Shrubs , Conservation of Resources , Native Hawaiians , Alternative and Complementary Medicine | 1 |
| your-feed-science , Measles , Rumors and Misinformation , Vitamin A , Diet and Nutrition , Disease Rates , Alternative and Complementary Medicine , Dietary Supplements and Herbal Remedies, Vaccination and Immunization , Epidemics | 1 |
| Space and Astronomy, Private Spaceflight | 1 |
| Deserts , Flowers and Plants , Cable Cars and Trams, Snow and Snowstorms , Hikes and Hiking , Travel and Vacations, Forests and Forestry, Biodiversity , Mountaineering | 1 |
The goal of this assignment was to get JSON data from the New York Times API and transform it into an R data frame. We successfully performed the transformation.