How the New York Times Covers Artificial Intelligence

Using the Article Search API to compare 2019 with 2026

Author

Aniss Sahraoui

Published

November 1, 2026

library(tidyverse)
library(httr2)
library(jsonlite)
library(knitr)

snapshot <- "https://raw.githubusercontent.com/AnissSahraoui/DATA607/main/Week9/data/"

accent   <- "#2a78d6"
accent_2 <- "#eb6834"
theme_nyt <- theme_minimal(base_size = 12) +
  theme(panel.grid.minor = element_blank(), plot.title.position = "plot")

1 The API and the Question

1.1 The endpoint

I used the Article Search API (https://api.nytimes.com/svc/search/v2/articlesearch.json). Given a search term and a date range, it returns article metadata — headline, byline, section, publication date, word count and keywords — ten articles at a time, along with meta.hits, the total number of matching articles. That total is the useful part: it is a complete count, not a sample.

1.2 The question

How has the Times’ coverage of artificial intelligence changed between 2019 and 2026 — both in volume, and in which sections of the paper carry it?

The idea is that a subject usually enters a newspaper through a specialist section and, if it becomes important enough, spreads into general news. If AI has followed that path, it should show up as a change in the mix of sections, not only in the number of articles.

I compare January–June 2019 against January–June 2026, the same six calendar months seven years apart, so seasonal differences do not affect the comparison.

2 The Request

2.1 Authentication

The key is never written into the code. It lives in my private ~/.Renviron file as NYT_API_KEY, outside the repository, and is read at run time:

api_key <- Sys.getenv("NYT_API_KEY")

# Report only whether a key was found, never the key itself.
tibble(key_found = nzchar(api_key), key_characters = nchar(api_key))

key_found	key_characters
TRUE	48

It is passed through req_url_query(), so httr2 handles the URL encoding. One consequence worth knowing: printing a request object would display the full URL, including the key, so this document never prints req — only the response status and the parsed body.

2.2 The parameters

# One request: all articles matching a topic in one calendar month.
search_month <- function(topic, year, month, page = 0) {
  start <- as.Date(sprintf("%d-%02d-01", year, month))
  end   <- seq(start, by = "1 month", length.out = 2)[2] - 1

  request("https://api.nytimes.com/svc/search/v2/articlesearch.json") |>
    req_url_query(
      q          = topic,                        # search term
      begin_date = format(start, "%Y%m%d"),      # inclusive start, YYYYMMDD
      end_date   = format(end,   "%Y%m%d"),      # inclusive end
      page       = page,                         # 0-indexed, 10 articles per page
      `api-key`  = api_key
    ) |>
    req_perform()
}

Parameter	Value used	Why
`q`	`artificial intelligence`	The topic being tracked
`begin_date` / `end_date`	One calendar month at a time	Gives an exact monthly count, and spreads the article sample evenly across each period
`page`	0 and 1	Ten articles per page, so two pages per month
`api-key`	from `Sys.getenv()`	Never hard coded

2.3 A live call

This chunk runs only when a key is available, so the document still renders for anyone without one. It prints the status and the hit count, and nothing that could leak the key.

resp <- request("https://api.nytimes.com/svc/search/v2/articlesearch.json") |>
  req_url_query(q = "artificial intelligence",
                begin_date = "20260601", end_date = "20260630",
                `api-key` = api_key) |>
  req_error(is_error = \(r) FALSE) |>
  req_perform()

parsed <- resp_body_json(resp)

tibble(
  status         = resp_status(resp),
  total_hits     = parsed$response$meta$hits,
  articles_on_page = length(parsed$response$docs)
)

status	total_hits	articles_on_page
200	490	10

Two practical notes from building this. The API is rate limited to roughly five requests per minute, so the collection script pauses 13 seconds between calls. And the facet_fields and fq parameters, which would have returned section counts directly, return nothing on this access tier — every fq filter I tried came back with zero hits — so the section breakdown is built from the returned articles instead.

3 From Nested JSON to a Tidy Data Frame

3.1 What the response looks like

The JSON is deeply nested. A single article holds the headline inside a headline object, the author inside a byline object, and a variable-length list of keyword objects:

doc <- parsed$response$docs[[1]]

tibble(
  field = c("headline", "byline", "keywords", "section_name"),
  shape = c(
    paste("object with:", paste(names(doc$headline), collapse = ", ")),
    paste("object with:", paste(names(doc$byline), collapse = ", ")),
    paste("list of", length(doc$keywords), "objects, each with name/value"),
    "plain string"
  )
)

field	shape
headline	object with: main, kicker, print_headline
byline	object with: original
keywords	list of 7 objects, each with name/value
section_name	plain string

3.2 The flattening code

Each nested piece is pulled into its own column. %||% substitutes NA when a field is absent, which matters because not every article has every field.

articles <- map_dfr(response$docs, \(d) tibble(
  headline         = d$headline$main    %||% NA_character_,  # unnest object
  byline           = d$byline$original  %||% NA_character_,  # unnest object
  section_name     = d$section_name     %||% NA_character_,
  subsection_name  = d$subsection_name  %||% NA_character_,
  news_desk        = d$news_desk        %||% NA_character_,
  type_of_material = d$type_of_material %||% NA_character_,
  pub_date         = d$pub_date         %||% NA_character_,
  word_count       = d$word_count       %||% NA_integer_,
  n_keywords       = length(d$keywords),                     # list -> count
  keywords         = paste(map_chr(d$keywords, \(k) k$value %||% NA_character_),
                           collapse = "; "),                 # list -> one string
  web_url          = d$web_url          %||% NA_character_
))

The full collection script is fetch_nyt.R.

3.3 The saved snapshot

Nobody without an API key can repeat the collection, and the live totals change daily as the Times publishes. The script therefore saves its results to CSV, and the analysis below reads those files from GitHub so the figures in this report stay fixed.

articles <- read_csv(paste0(snapshot, "nyt_ai_articles.csv"), show_col_types = FALSE)
monthly  <- read_csv(paste0(snapshot, "nyt_ai_monthly_totals.csv"), show_col_types = FALSE)

glimpse(articles)

Rows: 240
Columns: 13
$ headline         <chr> "Curbs on A.I. Exports? Silicon Valley Fears Losing I…
$ byline           <chr> "By Cade Metz", "By Steve Lohr", "By Dhruv Khullar", …
$ section_name     <chr> "Technology", "Technology", "Opinion", "Opinion", "Te…
$ subsection_name  <chr> NA, NA, NA, NA, NA, NA, NA, NA, "Art & Design", NA, N…
$ news_desk        <chr> "Business", "Business", "OpEd", "OpEd", "Business", "…
$ type_of_material <chr> "News", "News", "Op-Ed", "Op-Ed", "News", "News", "Ne…
$ pub_date         <dttm> 2019-01-01 18:37:45, 2019-01-20 17:12:40, 2019-01-31…
$ word_count       <dbl> 1048, 1103, 926, 1440, 1212, 1822, 1229, 1259, 559, 1…
$ n_keywords       <dbl> 7, 7, 4, 6, 6, 5, 7, 12, 8, 5, 12, 5, 5, 1, 9, 8, 12,…
$ keywords         <chr> "International Trade and World Market; Artificial Int…
$ web_url          <chr> "https://www.nytimes.com/2019/01/01/technology/artifi…
$ query_year       <dbl> 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019,…
$ query_month      <dbl> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,…

articles |>
  select(headline, section_name, type_of_material, pub_date, word_count, n_keywords) |>
  slice_head(n = 5)

headline	section_name	type_of_material	pub_date	word_count	n_keywords
Curbs on A.I. Exports? Silicon Valley Fears Losing Its Edge	Technology	News	2019-01-01 18:37:45	1048	7
How Do You Govern Machines That Can Learn? Policymakers Are Trying to Figure That Out	Technology	News	2019-01-20 17:12:40	1103	7
A.I. Could Worsen Health Disparities	Opinion	Op-Ed	2019-01-31 11:00:09	926	4
Warning! Everything Is Going Deep: ‘The Age of Surveillance Capitalism’	Opinion	Op-Ed	2019-01-30 00:28:13	1440	6
The Hidden Automation Agenda of the Davos Elite	Technology	News	2019-01-25 22:28:36	1212	6

4 Data-Cleaning Decisions

articles |>
  summarise(across(everything(), ~ sum(is.na(.)))) |>
  pivot_longer(everything(), names_to = "column", values_to = "missing") |>
  filter(missing > 0) |>
  kable()

column	missing
byline	13
subsection_name	184
keywords	3

Three decisions, each kept rather than silently patched:

subsection_name is missing for 184 of 240 articles. Most articles simply have no subsection, so the blank is meaningful. I left it as NA and did not use it in the analysis.
byline is missing for 13 articles, all from 2019. These are wire copy and briefing items that carry no author. I kept the rows, because the question is about sections rather than authors, and dropping them would quietly shrink the 2019 sample.
keywords is empty for 3 articles. The keyword list is flattened into one semicolon-separated string with a count alongside it, so an article with no keywords reads as NA with n_keywords of 0 rather than as an empty string.

articles |>
  summarise(
    rows              = n(),
    duplicate_urls    = sum(duplicated(web_url)),
    articles_per_year = n_distinct(query_year),
    sections          = n_distinct(section_name),
    date_parses       = sum(!is.na(as.Date(substr(pub_date, 1, 10))))
  ) |>
  kable()

rows	duplicate_urls	articles_per_year	sections	date_parses
240	0	2	24	240

No article appears twice, and every publication date parses.

The sampling decision, which matters most. The monthly totals are exact: they come from meta.hits, which counts every matching article. The section breakdown is a sample: 20 articles per month, 120 per period, drawn evenly across the six months so no single month dominates. The API returns articles in relevance order, so this is the 20 articles it ranked most relevant each month, not a random draw. I therefore treat the section percentages as indicative rather than exact, and lean on the shape of the differences rather than on individual percentage points.

5 Results

5.1 Volume

period_totals <- monthly |>
  group_by(year) |>
  summarise(articles = sum(total_hits), .groups = "drop")

period_totals |> kable(col.names = c("Period (Jan–Jun)", "Articles mentioning AI"))

Period (Jan–Jun)	Articles mentioning AI
2019	460
2026	2568

monthly |>
  mutate(month_name = factor(month.abb[month], levels = month.abb[1:6]),
         year = factor(year)) |>
  ggplot(aes(x = month_name, y = total_hits, fill = year)) +
  geom_col(position = position_dodge(width = 0.75), width = 0.65) +
  geom_text(aes(label = total_hits), position = position_dodge(width = 0.75),
            vjust = -0.4, size = 3.2) +
  scale_fill_manual(values = c("2019" = accent_2, "2026" = accent), name = NULL) +
  scale_y_continuous(expand = expansion(mult = c(0, 0.12))) +
  labs(title = "AI coverage is more than five times its 2019 level",
       x = NULL, y = "Articles") +
  theme_nyt +
  theme(legend.position = "top")

Figure 1: Articles mentioning artificial intelligence, by month. Counts are exact, taken from the API’s hit totals.

ratio <- period_totals$articles[period_totals$year == 2026] /
         period_totals$articles[period_totals$year == 2019]

The Times published 460 articles mentioning AI in the first half of 2019 and 2568 in the same months of 2026, a 5.6-fold increase. Every month of 2026 exceeds the whole of any month in 2019.

5.2 Which sections

section_mix <- articles |>
  count(query_year, section_name) |>
  group_by(query_year) |>
  mutate(share = 100 * n / sum(n)) |>
  ungroup()

comparison <- section_mix |>
  select(query_year, section_name, share) |>
  pivot_wider(names_from = query_year, values_from = share, names_prefix = "y") |>
  replace_na(list(y2019 = 0, y2026 = 0)) |>
  mutate(change = y2026 - y2019) |>
  arrange(desc(abs(change)))

comparison |>
  slice_head(n = 8) |>
  mutate(across(where(is.numeric), ~ round(.x, 1))) |>
  kable(col.names = c("Section", "2019 (%)", "2026 (%)", "Change (pp)"))

Section	2019 (%)	2026 (%)	Change (pp)
Opinion	20.8	6.7	-14.2
U.S.	3.3	13.3	10.0
New York	0.0	6.7	6.7
Fashion	4.2	0.0	-4.2
Podcasts	0.8	4.2	3.3
Business	22.5	20.8	-1.7
Health	1.7	0.0	-1.7
Arts	2.5	0.8	-1.7

section_mix |>
  group_by(section_name) |>
  filter(max(share) >= 3) |>
  ungroup() |>
  mutate(year = factor(query_year),
         section_name = fct_reorder(section_name, share, .fun = max)) |>
  ggplot(aes(x = share, y = section_name, fill = year)) +
  geom_col(position = position_dodge(width = 0.75), width = 0.65) +
  geom_text(aes(label = sprintf("%.1f%%", share)),
            position = position_dodge(width = 0.75), hjust = -0.15, size = 3) +
  scale_fill_manual(values = c("2019" = accent_2, "2026" = accent), name = NULL) +
  scale_x_continuous(expand = expansion(mult = c(0, 0.15))) +
  labs(title = "Technology still leads, but AI has moved into general news",
       x = "Share of sampled articles (%)", y = NULL) +
  theme_nyt +
  theme(legend.position = "top", panel.grid.major.y = element_blank())

Figure 2: Share of sampled AI articles by section. Only sections reaching 3% in either period are shown.

5.3 How concentrated is the coverage?

section_mix |>
  group_by(query_year) |>
  summarise(
    sections_used  = n_distinct(section_name),
    top_3_share    = round(sum(sort(share, decreasing = TRUE)[1:3]), 1),
    .groups = "drop"
  ) |>
  kable(col.names = c("Period", "Distinct sections", "Share in top 3 sections (%)"))

Period	Distinct sections	Share in top 3 sections (%)
2019	19	74.2
2026	19	64.2

articles |>
  mutate(commentary = type_of_material %in% c("Op-Ed", "Editorial", "Letter")) |>
  group_by(query_year) |>
  summarise(
    commentary_share = round(100 * mean(commentary), 1),
    median_words     = median(word_count),
    .groups = "drop"
  ) |>
  kable(col.names = c("Period", "Commentary (%)", "Median word count"))

Period	Commentary (%)	Median word count
2019	20.8	1100.5
2026	6.7	933.0

6 Conclusions

Volume has grown more than fivefold. AI went from 460 articles in the first half of 2019 to 2568 in the same months of 2026. This figure is exact rather than sampled, since it comes from the API’s own hit counts.
Technology remains the home section, and that has not changed. It accounts for about 30% of sampled AI articles in both periods, with Business around 21–23%. The subject did not migrate away from the technology desk; the rest of the paper joined in.
The clearest shift is from commentary to news. Opinion fell from 20.8% of sampled articles to 6.7%, while the U.S. section rose from 3.3% to 13.3% and New York went from nothing to 6.7%. Counting by article type rather than section says the same thing: commentary pieces fell from 20.8% to 6.7% of the sample.
Coverage has spread out. The top three sections held 74.2% of sampled AI articles in 2019 and 64.2% in 2026.
Read together, this looks like a subject becoming a beat. In 2019 AI was something the paper argued about, in Opinion and in Technology. By 2026 it is something reported on — in national news, in local New York coverage, and even in Real Estate and Theater. The articles are also slightly shorter, with the median falling from 1,100 to 933 words, which fits routine news reporting rather than occasional long essays.

What would strengthen this. The section percentages rest on 120 articles per period chosen by the API’s relevance ranking, not a random sample. With a higher access tier the facet_fields parameter would return exact section counts, and the same comparison could then be made on complete data rather than on a sample. The hit totals, which carry the headline finding, do not have this limitation.