Tidyverse Assignment | 2019 NYC Subway Rider Survey

Author

Shanice Smith

Published

November 15, 2026

Overview

Collected by the City Council (NYCC) and Speaker Corey Johnson in 2019, the 2019 NYC Subway Rider Survey data contains people’s experiences with our mass transit system. The CSV contains 10,704 responses and was downloaded for this assignment from NYC Open Data in November 2026. This vignette uses the survey responses to demonstrate common data tasks of reading CSVs, unpacking answers from a single cell, and reformatting timestamps into date-times.

Functions used:

Function Package What it does in this vignette
read_csv() readr Reads the survey CSV file into R
glimpse() dplyr Shows every column, its type, and its first values
filter() dplyr Keeps responses that answered the complaints question
mutate() dplyr Creates new columns
str_remove_all() stringr Removes the brackets and quotation marks around answers
separate_longer_delim() tidyr Gives each complaint its own row
count() dplyr Counts how often each value appears
parse_date_time() lubridate Converts text timestamps into date-times
wday(), hour() lubridate Extract the weekday and hour from a date-time

Setup

While the focus is tidyverse, we’ll also use knitr::kable() to display results as accessible HTML tables.

library(tidyverse)
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.2.1     ✔ readr     2.2.0
✔ forcats   1.0.1     ✔ stringr   1.6.0
✔ ggplot2   4.0.3     ✔ tibble    3.3.1
✔ lubridate 1.9.5     ✔ tidyr     1.3.2
✔ purrr     1.2.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

Loading the data

read_csv() comes from readr, reading the CSV file into R as a table. The function automatically guesses the column type from its content, so we manually define the data type for zip codes (labels) and subway line (mix of letters and numbers) to a character type. readr also turns the text NA into real missing values and reads true/false as a logical column automatically. glimpse comes from dplyr so we can see every column in a data frame, displaying as much data as possible.

url <- "https://raw.githubusercontent.com/shanicesmith98/data-607-assignments/refs/heads/main/data/Subway_Rider_Survey_(2019)_20261115.csv"

subway_survey <- read_csv(
  url,
  col_types = cols(
    zip_code = col_character(),
    subway_line_used_most_often = col_character()
  )
)

glimpse(subway_survey)
Rows: 10,704
Columns: 19
$ subway_line_used_most_often  <chr> "1", "E", "F", "7", "E", "1", "1", "1", "…
$ use_of_subway_frequency      <chr> "Once a week", "Weekdays, Mon - Fri", "On…
$ get_to_subway_via            <chr> "Bus", "Walk", "Bus", "Walk", "Bus", "Wal…
$ primary_use_of_subway        <chr> "Work", "Work", "Other", "Work", "School"…
$ average_length_subway_ride   <chr> "20 - 40 min", "40 - 60 min", "40 - 60 mi…
$ overall_satisfaction         <chr> "Neutral", "Not satisfied", "Very satisfi…
$ frequency_of_delays          <chr> "A few times a week", "A few times a week…
$ approximate_delay_duration   <chr> "10 - 20 min", "10 - 20 min", "20 - 45 mi…
$ alternative_transport        <chr> "Wait for next train", "Use another train…
$ frequency_of_rerouting       <chr> NA, "All the time", "Only at night", "Alm…
$ top_three_complaints         <chr> "['Delays', 'Crime / Safety', 'Station in…
$ most_common_reason_for_delay <chr> "Waiting for signal change", "Train traff…
$ is_subway_affordable         <lgl> TRUE, FALSE, TRUE, TRUE, TRUE, FALSE, FAL…
$ opinion_on_increased_fares   <chr> "Neutral", "Unaware", "Unaware", "Neutral…
$ submitted_at                 <chr> "01/07/2019 @ 07:21 AM", "01/07/2019 @ 07…
$ survey_stop_borough          <chr> "Queens", "Queens", "Queens", "Queens", "…
$ survey_stop_location         <chr> "Queens Plaza", "Queens Plaza", "Queens P…
$ zip_code                     <chr> "11367", NA, "11432", NA, "11436", "11102…
$ collection_method            <chr> "in person", "in person", "in person", "i…

Clean and count the complaints

The top_three_complaints column stores several answers in one cell, formatted like a list: ['Delays', 'High fares', 'Overcrowded trains']. Before we can count individual complaints, each one needs its own row. The pipeline below does this in four steps:

  1. filter() drops respondents who skipped the question.
  2. str_remove_all() deletes every bracket and quotation mark. The pattern "\\[|\\]|'" is a regular expression, a short code for describing text. The | means “or,” and the brackets need \\ in front because they have a special meaning in regular expressions.
  3. separate_longer_delim() splits each cell wherever it finds ", " (a comma followed by a space) and gives every complaint its own row.
  4. count() tallies each complaint and sorts the results from most to least common.

We then add a column showing what percentage of respondents listed each complaint. Also, despite the col name, some respondents only gave one or two complaints.

complaint_respondents <- subway_survey |>
  filter(!is.na(top_three_complaints)) |>
  nrow()


complaints <- subway_survey |>
  filter(!is.na(top_three_complaints)) |>
  mutate(complaint = str_remove_all(top_three_complaints, "\\[|\\]|'")) |>
  separate_longer_delim(complaint, delim = ", ") |>
  count(complaint, sort = TRUE) |>
  mutate(pct_of_respondents = round(100 * n / complaint_respondents, 1))

knitr::kable(
  complaints,
  col.names = c("Complaint", "Respondents who listed the complaint", "Percent of respondents"),
  format.args = list(big.mark = ","),
  caption = "Complaints listed by 2019 NYC Subway Rider Survey respondents"
)
Complaints listed by 2019 NYC Subway Rider Survey respondents
Complaint Respondents who listed the complaint Percent of respondents
Delays 7,255 72.0
Overcrowded trains 5,762 57.1
Frequent route changes 3,814 37.8
Station infrastructure 2,932 29.1
High fares 2,728 27.1
Customer service 2,360 23.4
Lack of handicap access 2,064 20.5
Crime / Safety 1,439 14.3
Old trains 961 9.5

72% of respondents listed delays as a complaint of their subway rider experience, with 57% listing overcrowded trains, and nearly 38% naming frequent route changes.

Parsing custom timestamps

The submitted_at column is stored as text like 01/07/2019 @ 07:21 AM, so R can’t sort it by time or pull out the hour. parse_date_time() from lubridate converts it into a real date-time.

The orders argument describes the layout: month, day, year, then a 12-hour clock hour (I), minutes (M), and AM/PM (p). lubridate skips the @ on its own. Setting tz = "America/New_York" records these as New York times; without it, lubridate labels them as UTC. Once the column is a real date-time, wday() extracts the day of the week and hour() extracts the hour on a 24-hour clock.

subway_survey <- subway_survey |>
  mutate(
    submitted = parse_date_time(submitted_at, orders = "mdy IMp", tz = "America/New_York"),
    weekday   = wday(submitted, label = TRUE, abbr = FALSE),
    hour      = hour(submitted)
  )

range(subway_survey$submitted)
[1] "2019-01-04 17:38:00 EST" "2019-01-13 07:00:00 EST"
knitr::kable(
  count(subway_survey, weekday),
  col.names = c("Day of the week", "Responses"),
  format.args = list(big.mark = ","),
  caption = "Survey responses by day of the week"
)
Survey responses by day of the week
Day of the week Responses
Sunday 29
Monday 1,470
Tuesday 3,161
Wednesday 2,817
Thursday 1,981
Friday 955
Saturday 291
knitr::kable(
  subway_survey |> count(hour, sort = TRUE) |> slice_head(n = 5),
  col.names = c("Hour (24-hour clock)", "Responses"),
  format.args = list(big.mark = ","),
  caption = "The five hours with the most survey responses"
)
The five hours with the most survey responses
Hour (24-hour clock) Responses
17 1,432
14 800
15 759
13 730
20 648

The survey ran only from January 4 - 13, 2019. Most responses arrived on weekdays, and the busiest hour was 17:00 (5 PM), which fits commuters answering on their way home.

References

Credit

Anthropic. (2026). Claude (Opus 5.5) [Large language model].