Homework Assignment: Analyzing NYC Flight Data

This homework assignment uses the flights dataset from the nycflights13 package, which contains real-world data on over 336,000 flights departing from New York City airports (JFK, LGA, EWR) in 2013. The dataset includes variables such as departure and arrival times (with date components), airline carrier (categorical), origin and destination airports (categorical), delays (with missing values for cancelled flights), distance, and more. It is sourced from the US Bureau of Transportation Statistics.

Objectives

This assignment reinforces the Week 4 topics:

  • Parsing and manipulating dates/times using lubridate.
  • Creating and analyzing time series with zoo.
  • Working with factors, inspecting levels, and recoding them.
  • Identifying and handling missing data (e.g., removal, imputation).

All questions (except the final reflection) require you to write and run R code to solve them. Submit your URL for your RPubs. Make sure to comment your code, along with key outputs (e.g., summaries, plots, or tables). Use the provided setup code to load the data.

Setup

Install and load the necessary packages if not already done:

#install.packages(c("nycflights13", "dplyr", "lubridate", "zoo", "forcats"))  # If needed
library(nycflights13)
library(dplyr)
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
library(lubridate)
## 
## Attaching package: 'lubridate'
## The following objects are masked from 'package:base':
## 
##     date, intersect, setdiff, union
library(zoo)
## 
## Attaching package: 'zoo'
## The following objects are masked from 'package:base':
## 
##     as.Date, as.Date.numeric
library(forcats)  # For factor recoding; base R alternatives are acceptable
data(flights)  # Load the dataset

Explore the data briefly with str(flights) and head(flights) to understand the structure. Note: Dates are in separate year, month, day columns; times are in dep_time and arr_time (as integers like 517 for 5:17 AM).

#Explore your data here
str(flights)
## tibble [336,776 × 19] (S3: tbl_df/tbl/data.frame)
##  $ year          : int [1:336776] 2013 2013 2013 2013 2013 2013 2013 2013 2013 2013 ...
##  $ month         : int [1:336776] 1 1 1 1 1 1 1 1 1 1 ...
##  $ day           : int [1:336776] 1 1 1 1 1 1 1 1 1 1 ...
##  $ dep_time      : int [1:336776] 517 533 542 544 554 554 555 557 557 558 ...
##  $ sched_dep_time: int [1:336776] 515 529 540 545 600 558 600 600 600 600 ...
##  $ dep_delay     : num [1:336776] 2 4 2 -1 -6 -4 -5 -3 -3 -2 ...
##  $ arr_time      : int [1:336776] 830 850 923 1004 812 740 913 709 838 753 ...
##  $ sched_arr_time: int [1:336776] 819 830 850 1022 837 728 854 723 846 745 ...
##  $ arr_delay     : num [1:336776] 11 20 33 -18 -25 12 19 -14 -8 8 ...
##  $ carrier       : chr [1:336776] "UA" "UA" "AA" "B6" ...
##  $ flight        : int [1:336776] 1545 1714 1141 725 461 1696 507 5708 79 301 ...
##  $ tailnum       : chr [1:336776] "N14228" "N24211" "N619AA" "N804JB" ...
##  $ origin        : chr [1:336776] "EWR" "LGA" "JFK" "JFK" ...
##  $ dest          : chr [1:336776] "IAH" "IAH" "MIA" "BQN" ...
##  $ air_time      : num [1:336776] 227 227 160 183 116 150 158 53 140 138 ...
##  $ distance      : num [1:336776] 1400 1416 1089 1576 762 ...
##  $ hour          : num [1:336776] 5 5 5 5 6 5 6 6 6 6 ...
##  $ minute        : num [1:336776] 15 29 40 45 0 58 0 0 0 0 ...
##  $ time_hour     : POSIXct[1:336776], format: "2013-01-01 05:00:00" "2013-01-01 05:00:00" ...
head(flights)
## # A tibble: 6 × 19
##    year month   day dep_time sched_dep_time dep_delay arr_time sched_arr_time
##   <int> <int> <int>    <int>          <int>     <dbl>    <int>          <int>
## 1  2013     1     1      517            515         2      830            819
## 2  2013     1     1      533            529         4      850            830
## 3  2013     1     1      542            540         2      923            850
## 4  2013     1     1      544            545        -1     1004           1022
## 5  2013     1     1      554            600        -6      812            837
## 6  2013     1     1      554            558        -4      740            728
## # ℹ 11 more variables: arr_delay <dbl>, carrier <chr>, flight <int>,
## #   tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>,
## #   hour <dbl>, minute <dbl>, time_hour <dttm>

Question 1: Creating Dates with lubridate

Create a column dep_datetime by combining year, month, day, and dep_time into a POSIXct datetime using lubridate. (Hint: Use make_datetime function to combine: year, month, day, for hour and min use division, e.g., hour = dep_time %/% 100, min = dep_time %% 100.)

Show the first 5 rows of flights with dep_datetime.

Output: First 5 rows showing year, month, day, dep_time, and dep_datetime.

flights_time <- flights |>
  mutate(dep_datetime = make_datetime(year, month, day, dep_time, hour = dep_time %/% 100, min = dep_time %% 100.))
# Verifying
head(flights_time$dep_datetime)
## [1] "2013-01-01 05:25:37 UTC" "2013-01-01 05:41:53 UTC"
## [3] "2013-01-01 05:51:02 UTC" "2013-01-01 05:53:04 UTC"
## [5] "2013-01-01 06:03:14 UTC" "2013-01-01 06:03:14 UTC"

Question 2: Simple Date Manipulations with lubridate

Using dep_datetime from Question 1, create a column weekday with the day of the week (e.g., “Mon”) using wday(dep_datetime, label = TRUE). Use table() to show how many flights occur on each weekday.

Output: The table of flight counts by weekday.

flights_week <- flights_time |>
  mutate(weekday = wday(dep_datetime, label = TRUE))
table(flights_week$weekday)
## 
##   Sun   Mon   Tue   Wed   Thu   Fri   Sat 
## 45526 49439 49321 48847 48625 48719 38044

Question 3: Time Series with zoo

Filter for flights from JFK (origin == “JFK”) and create a zoo time series of departure delays (dep_delay) by dep_datetime. Plot the time series (use plot()). (Hint: Use a subset to avoid memory issues, e.g., first 1000 JFK flights using `slice_head().)

Output: The time series plot.

JFK <- flights_time |>
  filter(origin == "JFK") |>
  arrange(dep_delay)  # Ensure time order

slice_head(JFK, n=1000)
## # A tibble: 1,000 × 20
##     year month   day dep_time sched_dep_time dep_delay arr_time sched_arr_time
##    <int> <int> <int>    <int>          <int>     <dbl>    <int>          <int>
##  1  2013    12     7     2040           2123       -43       40           2352
##  2  2013     3     2     1431           1455       -24     1601           1631
##  3  2013     5    18     1912           1935       -23     2126           2119
##  4  2013     2     6     1528           1550       -22     1750           1826
##  5  2013     9    14     2223           2245       -22     2327           2351
##  6  2013     3    16      909            930       -21     1038           1101
##  7  2013     5     1     1550           1610       -20     1704           1807
##  8  2013     5    23     1839           1859       -20     2210           2134
##  9  2013    12    10     1841           1900       -19     2028           2047
## 10  2013     3    12     1516           1535       -19     1829           1844
## # ℹ 990 more rows
## # ℹ 12 more variables: arr_delay <dbl>, carrier <chr>, flight <int>,
## #   tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>,
## #   hour <dbl>, minute <dbl>, time_hour <dttm>, dep_datetime <dttm>
for_delays <- zoo(JFK$dep_delay, JFK$dep_datetime)
## Warning in zoo(JFK$dep_delay, JFK$dep_datetime): some methods for "zoo" objects
## do not work if the index entries in 'order.by' are not unique
plot(for_delays, main = "Departure Delays for JFK Flights", ylab = "Departure Delays", xlab = "Date/Time")

Question 4: Working with Factors

Convert the origin column (airports: “JFK”, “LGA”, “EWR”) to a factor called origin_factor. Show the factor levels with levels() and create a frequency table with table(). Make a bar plot of flights by airport using barplot().

Output: The levels, frequency table, and bar plot.

LEVELS

flights_levels <- flights_time |>
  mutate(origin_factor = factor(origin))


levels(flights_levels$origin_factor)
## [1] "EWR" "JFK" "LGA"

FREQUENCY TABLE

table(flights_levels$origin_factor)
## 
##    EWR    JFK    LGA 
## 120835 111279 104662

BAR PLOT

barplot(table(flights_levels$origin_factor))

Question 5: Recoding Factors

Recode origin_factor from Question 4 into a new column origin_recoded with full names: “JFK” to “Kennedy”, “LGA” to “LaGuardia”, “EWR” to “Newark” using fct_recode() or base R. Create a bar plot of the recoded factor.

Output: The new levels and bar plot. NEW LEVELS

library(dplyr)

flights_recoded <- flights_levels |>
  mutate(
    origin_recoded = case_when(
      origin == "JFK" ~ "Kennedy",
      origin == "LGA"      ~ "LaGuardia",
      origin == "EWR"           ~ "Newark",
      TRUE ~ as.character(origin)  # keep as is, if not mentioned in code
    ),
    # Convert to factor with explicit order
    origin_recoded = factor(origin_recoded,
                            levels = c("Kennedy", "LaGuardia", "Newark"))
  )

BAR PLOT

barplot(table(flights_recoded$origin_recoded))

Question 6: Handling Missing Data

Count missing values in dep_delay and arr_delay using colSums(is.na(flights)). Impute missing dep_delay values with 0 (assuming no delay for cancelled flights) in a new column dep_delay_imputed. Create a frequency table of dep_delay_imputed for delays between -20 and 20 minutes (use filter() to subset).

Output: NA counts, and the frequency table for imputed delays.

colSums(is.na(flights)) # number of NAs per column
##           year          month            day       dep_time sched_dep_time 
##              0              0              0           8255              0 
##      dep_delay       arr_time sched_arr_time      arr_delay        carrier 
##           8255           8713              0           9430              0 
##         flight        tailnum         origin           dest       air_time 
##              0           2512              0              0           9430 
##       distance           hour         minute      time_hour 
##              0              0              0              0
flights_na <- flights |>
  mutate(dep_delay_imputed = ifelse(is.na(dep_delay), 
                                      0, 
                                      dep_delay))

Code below made with help from “Filter” R documentation: https://www.rdocumentation.org/packages/dplyr/versions/0.7.8/topics/filter As well as “between()” R documentation: https://www.rdocumentation.org/packages/dplyr/versions/0.7.8/topics/between

flights_filtered <- flights_na |>
  filter(between(dep_delay_imputed, -20, 20))

FREQUENCY TABLE

table(flights_filtered$dep_delay_imputed)
## 
##   -20   -19   -18   -17   -16   -15   -14   -13   -12   -11   -10    -9    -8 
##    37    19    81   110   162   408   498   901  1594  2727  5891  7875 11791 
##    -7    -6    -5    -4    -3    -2    -1     0     1     2     3     4     5 
## 16752 20701 24821 24619 24218 21516 18813 24769  8050  6233  5450  4807  4447 
##     6     7     8     9    10    11    12    13    14    15    16    17    18 
##  3789  3520  3381  3062  2859  2756  2494  2414  2256  2140  2085  1873  1749 
##    19    20 
##  1730  1704

Question 7: Reflection (No Coding)

Reflect on the assignment: What was easy or hard about working with flight dates or missing data? How might assuming zero delay for missing values (Question 6) affect conclusions about flight punctuality? What did you learn about NYC flights in 2013? (150-200 words)

For me the hardest part in working with missing data was replacing NAs. Mostly because I struggled with adjusting the code to replace it with 0 instead of the mean that we learned in class. If I was doing any further analysis utilizing other columns it would be even harder. Especially since I would have to make decisions regarding whether to replace NAs with zero, the max, the min, the mean, or to simply remove them from the data set. That would depend on what I am planning to do with the data set and whether it would have an effect on my conclusions. Working with the flight times was not too bad since the function “make_datetime” made interpreting the time easier. In Question 6, replacing all the NAs in the dep-delay column with zero would overestimate the punctuality of flights. All the conclusions regarding the timeliness of flights may be Overall, from the data set I learned most NYC flights in 2013 came out of Newark. Also that the most departure delays occurred in January, which makes sense since it is holiday season. People tend to travel with their family or visit warmer climates.