Retrieving and Analyzing Article Metadata in R
This assignment uses the New York Times Most Popular API to retrieve article metadata in JSON format and analyze it in R. The research question is: Which news sections appear most frequently among the articles returned by the most-viewed endpoint for the previous seven days?
The packages httr2, dplyr, ggplot2, and knitr support the API request, data processing, visualization, and tables. The API response is parsed from JSON directly by httr2; a separate jsonlite call is unnecessary here.
packages <- c("httr2", "dplyr", "ggplot2", "knitr")
missing_packages <- packages[!vapply(packages, requireNamespace, logical(1), quietly = TRUE)]
if (length(missing_packages)) install.packages(missing_packages)
suppressPackageStartupMessages({
library(httr2)
library(dplyr)
library(ggplot2)
library(knitr)
})The key must be stored as NYT_API_KEY in the user’s .Renviron file. Do not paste the key into this document. If this check fails, run usethis::edit_r_environ(scope = "user") in the RStudio Console, add NYT_API_KEY=your_new_key, save, and restart R before rendering. A key previously shared publicly should be revoked and replaced.
api_key <- Sys.getenv("NYT_API_KEY")
if (!nzchar(api_key)) stop("NYT_API_KEY is missing. Configure .Renviron and restart R.")
url <- "https://api.nytimes.com/svc/mostpopular/v2/viewed/7.json"
response <- request(url) |>
req_url_query(`api-key` = api_key) |>
req_timeout(30) |>
req_perform()
cat("HTTP status:", resp_status(response), "\n")HTTP status: 200 nyt_json <- resp_body_json(response, simplifyVector = TRUE)
if (is.null(nyt_json$results) || nrow(nyt_json$results) == 0) {
stop("The API returned no article records.")
}
articles <- as_tibble(nyt_json$results)
cat("Records retrieved:", nrow(articles), "\n")Records retrieved: 20 Keep the variables needed for the analysis, remove duplicate IDs, label missing sections, and convert publication dates. The analysis uses metadata, not the full text of articles.
required <- c("id", "section", "published_date")
if (!all(required %in% names(articles))) {
stop("Required columns missing: ", paste(setdiff(required, names(articles)), collapse = ", "))
}
nyt_clean <- articles |>
select(any_of(c("id", "section", "subsection", "published_date", "updated", "byline", "type", "source"))) |>
distinct(id, .keep_all = TRUE) |>
mutate(
section = if_else(is.na(section) | trimws(section) == "", "Unknown", trimws(section)),
published_date = as.Date(published_date)
)
cat("Duplicates removed:", nrow(articles) - nrow(nyt_clean), "\n")Duplicates removed: 0 cat("Unique articles:", nrow(nyt_clean), "\n")Unique articles: 20 missing_values <- data.frame(
Variable = names(nyt_clean),
Missing = colSums(is.na(nyt_clean))
)
kable(missing_values, caption = "Missing Values by Variable")| Variable | Missing | |
|---|---|---|
| id | id | 0 |
| section | section | 0 |
| subsection | subsection | 0 |
| published_date | published_date | 0 |
| updated | updated | 0 |
| byline | byline | 0 |
| type | type | 0 |
| source | source | 0 |
Count articles by section and calculate each section’s share of the returned articles.
section_summary <- nyt_clean |>
count(section, name = "Article_Count", sort = TRUE) |>
mutate(Percentage = round(100 * Article_Count / sum(Article_Count), 1))
kable(section_summary, caption = "Most-Viewed Article Records by Section")| section | Article_Count | Percentage |
|---|---|---|
| U.S. | 9 | 45 |
| Opinion | 4 | 20 |
| World | 2 | 10 |
| Arts | 1 | 5 |
| New York | 1 | 5 |
| Polls | 1 | 5 |
| Science | 1 | 5 |
| Well | 1 | 5 |
ggplot(section_summary, aes(x = reorder(section, Article_Count), y = Article_Count)) +
geom_col(fill = "steelblue") +
coord_flip() +
labs(title = "Articles Returned by News Section", subtitle = "Most Popular API: 7-day viewed endpoint", x = "News section", y = "Number of articles") +
theme_minimal()
The chart below represents publication dates of articles in the API result, not the number of articles published by the newspaper on those dates.
date_summary <- nyt_clean |>
filter(!is.na(published_date)) |>
count(published_date, name = "Article_Count")
ggplot(date_summary, aes(x = published_date, y = Article_Count)) +
geom_col(fill = "darkgreen") +
labs(title = "Publication Dates of Retrieved Articles", x = "Publication date", y = "Number of retrieved articles") +
theme_minimal()
leader <- section_summary[1, ]
cat(sprintf("The API returned **%d unique articles** across **%d sections**. The most represented section was **%s**, with **%d articles (%.1f%%)** of the retrieved records.\n\n", nrow(nyt_clean), nrow(section_summary), leader$section, leader$Article_Count, leader$Percentage))The API returned 20 unique articles across 8 sections. The most represented section was U.S., with 9 articles (45.0%) of the retrieved records.
These results show only the sections included in the API response, not every published article or the total number of readers. Article counts by themselves do not show how many times each article was viewed. Since the API results can change, the tables and charts are based on the date when this document was created.
This assignment outlines a reproducible workflow for API-based analysis, including authentication, JSON retrieval, variable selection and cleaning, descriptive summary generation, and communication of findings using Quarto. The analysis determines which news sections are most frequently represented among the retrieved most-viewed articles, while acknowledging that the API results may not encompass all published news.
New York Times Developer Network. Most Popular API documentation.
Comments
This assignment showed me how APIs let us access external data for analysis and how to turn JSON results into an R data frame. Setting up API authentication was tricky, and it made me learn to pay close attention to the R environment and keep credentials safe. I also learned to check data structure, remove duplicates, handle missing values, summarize categories, and present results visually. As a teacher and scholar, I can apply these steps to public educational datasets and program evaluations, while protecting data privacy and being careful in how I interpret the results.