DATA 607 - NY Times API Code Base

Retrieving and Analyzing JSON Data Using R

1. Introduction

This assignment uses the New York Times Most Popular API to retrieve article metadata in JSON format and analyze it in R.

2. Research Question

Which news sections appear most frequently among the most-viewed articles over the past seven days?

3. Load Packages

Install these packages once in the R console if necessary: install.packages(c("httr2", "dplyr", "ggplot2", "knitr", "tibble")).

library(httr2)
library(dplyr)
library(ggplot2)
library(knitr)
library(tibble)

4. Load API Key

Store the key in ~/.Renviron as NYT_API_KEY=your_actual_key, save the file, and restart RStudio. Never print or publish the key.

api_key <- Sys.getenv("NYT_API_KEY")
if (!nzchar(api_key)) {
  stop("NYT_API_KEY is missing. Add it to ~/.Renviron and restart RStudio.")
}
cat("API key found.\n")

API key found.

5. Request NY Times Data

api_url <- "https://api.nytimes.com/svc/mostpopular/v2/viewed/7.json"
response <- request(api_url) |>
  req_url_query(`api-key` = api_key) |>
  req_timeout(30) |>
  req_perform()
cat("HTTP status:", resp_status(response), "\n")

HTTP status: 200 

6. Extract JSON Data

nyt_json <- resp_body_json(response, simplifyVector = FALSE)
if (!identical(nyt_json$status, "OK")) {
  stop("API did not return status OK.")
}
if (length(nyt_json$results) == 0) {
  stop("API returned no articles.")
}
cat("Number of articles returned:", length(nyt_json$results), "\n")

Number of articles returned: 20 

print(names(nyt_json$results[[1]]))

 [1] "uri"            "url"            "id"             "asset_id"      
 [5] "source"         "published_date" "updated"        "section"       
 [9] "subsection"     "nytdsection"    "adx_keywords"   "column"        
[13] "byline"         "type"           "title"          "abstract"      
[17] "des_facet"      "org_facet"      "per_facet"      "geo_facet"     
[21] "media"          "eta_id"        

7. Convert JSON into a Data Frame

Extract a consistent set of article-level fields from the nested JSON response.

get_field <- function(article, field) {
  value <- article[[field]]
  if (is.null(value) || length(value) == 0) return(NA_character_)
  as.character(value[[1]])
}
fields <- c("id", "section", "subsection", "published_date", "updated", "byline", "type", "source")
nyt_articles <- bind_rows(lapply(nyt_json$results, function(article) {
  values <- lapply(fields, function(field) get_field(article, field))
  names(values) <- fields
  as_tibble(values)
}))
glimpse(nyt_articles)

Rows: 20
Columns: 8
$ id             <chr> "100000011253344", "100000011241938", "100000011214944"…
$ section        <chr> "U.S.", "U.S.", "U.S.", "U.S.", "New York", "U.S.", "U.…
$ subsection     <chr> "", "Elections", "Politics", "", "", "Elections", "Poli…
$ published_date <chr> "2026-11-01", "2026-11-03", "2026-10-28", "2026-10-28",…
$ updated        <chr> "2026-11-02 12:47:20", "2026-11-03 21:43:36", "2026-10-…
$ byline         <chr> "By Christina Morales, David Ovalle and Georgia Gee", "…
$ type           <chr> "Article", "Interactive", "Article", "Article", "Articl…
$ source         <chr> "New York Times", "New York Times", "New York Times", "…

8. Select Relevant Variables

nyt_clean <- nyt_articles |>
  select(all_of(fields))
glimpse(nyt_clean)

Rows: 20
Columns: 8
$ id             <chr> "100000011253344", "100000011241938", "100000011214944"…
$ section        <chr> "U.S.", "U.S.", "U.S.", "U.S.", "New York", "U.S.", "U.…
$ subsection     <chr> "", "Elections", "Politics", "", "", "Elections", "Poli…
$ published_date <chr> "2026-11-01", "2026-11-03", "2026-10-28", "2026-10-28",…
$ updated        <chr> "2026-11-02 12:47:20", "2026-11-03 21:43:36", "2026-10-…
$ byline         <chr> "By Christina Morales, David Ovalle and Georgia Gee", "…
$ type           <chr> "Article", "Interactive", "Article", "Article", "Articl…
$ source         <chr> "New York Times", "New York Times", "New York Times", "…

9. Clean the Data

duplicate_count <- sum(duplicated(nyt_clean$id))
cat("Duplicate article IDs:", duplicate_count, "\n")

Duplicate article IDs: 0 

nyt_clean <- nyt_clean |>
  distinct(id, .keep_all = TRUE) |>
  mutate(
    section = if_else(is.na(section) | trimws(section) == "", "Unknown", section),
    published_date = as.Date(published_date)
  )
missing_values <- data.frame(
  Variable = names(nyt_clean),
  Missing = colSums(is.na(nyt_clean))
)
kable(missing_values, caption = "Missing Values by Variable")

Missing Values by Variable

	Variable	Missing
id	id	0
section	section	0
subsection	subsection	0
published_date	published_date	0
updated	updated	0
byline	byline	0
type	type	0
source	source	0

10. Descriptive Statistics

total_articles <- nrow(nyt_clean)
total_sections <- n_distinct(nyt_clean$section)
cat("Total unique articles:", total_articles, "\n")

Total unique articles: 20 

cat("Unique sections:", total_sections, "\n")

Unique sections: 8 

section_summary <- nyt_clean |>
  count(section, name = "Article_Count", sort = TRUE) |>
  mutate(Percentage = round(100 * Article_Count / sum(Article_Count), 2))
kable(section_summary, caption = "Articles by News Section")

Articles by News Section

section	Article_Count	Percentage
U.S.	13	65
Arts	1	5
New York	1	5
Opinion	1	5
Polls	1	5
Style	1	5
Well	1	5
World	1	5

11. Visualization: Articles by Section

ggplot(section_summary, aes(x = reorder(section, Article_Count), y = Article_Count)) +
  geom_col(fill = "steelblue") +
  coord_flip() +
  labs(title = "Most Viewed Articles by News Section", subtitle = "API seven-day most-viewed ranking", x = "News Section", y = "Number of Articles") +
  theme_minimal()

12. Visualization: Publication Dates

date_summary <- nyt_clean |>
  filter(!is.na(published_date)) |>
  count(published_date, name = "Article_Count")
ggplot(date_summary, aes(x = published_date, y = Article_Count)) +
  geom_col(fill = "darkgreen") +
  labs(title = "Publication Dates of Popular Articles", x = "Publication Date", y = "Number of Articles") +
  theme_minimal()

13. Interpretation of Results

top_section <- section_summary |> slice_max(Article_Count, n = 1, with_ties = FALSE)
cat("The API returned **", total_articles, "** unique articles from **", total_sections, "** news sections.\n\n", sep = "")

The API returned 20 unique articles from 8 news sections.

cat("The most frequently represented section was **", top_section$section,
    "**, with **", top_section$Article_Count, "** articles (**", top_section$Percentage,
    "%** of the returned records).\n\n", sep = "")

The most frequently represented section was U.S., with 13 articles (65% of the returned records).

These findings describe the sections represented among the articles returned by the Most Popular API, not all articles published by the newspaper. The publication-date chart displays when the returned articles were originally published; these dates need not fall within the last seven days.

14. Reflection

This assignment helped me understand how APIs provide access to external datasets and how JSON information can be imported and transformed using R. I practiced API authentication, inspecting JSON structures, selecting variables, removing duplicates, examining missing values, and producing summary tables and visualizations. These techniques can also support educational research and analysis of publicly available datasets.

15. Conclusion

The NY Times API assignment demonstrates how to retrieve data from an online source, transform JSON into a structured data frame, and analyze the results using R. It also reinforces the importance of troubleshooting technical issues and protecting authentication credentials.

16. References

New York Times Developer Network. Most Popular API documentation.