Homework Assignment: Analyzing NYC Flight Data

This homework assignment uses the flights dataset from the nycflights13 package, which contains real-world data on over 336,000 flights departing from New York City airports (JFK, LGA, EWR) in 2013. The dataset includes variables such as departure and arrival times (with date components), airline carrier (categorical), origin and destination airports (categorical), delays (with missing values for cancelled flights), distance, and more. It is sourced from the US Bureau of Transportation Statistics.

Objectives

  • Parsing and manipulating dates/times using lubridate.
  • Creating and analyzing time series with zoo.
  • Working with factors, inspecting levels, and recoding them.
  • Identifying and handling missing data.

Setup

#install.packages(c("nycflights13", "dplyr", "lubridate", "zoo", "forcats"))  # If needed
library(nycflights13)
library(dplyr)
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
library(lubridate)
## 
## Attaching package: 'lubridate'
## The following objects are masked from 'package:base':
## 
##     date, intersect, setdiff, union
library(zoo)
## 
## Attaching package: 'zoo'
## The following objects are masked from 'package:base':
## 
##     as.Date, as.Date.numeric
library(forcats)
data(flights)

Explore the data briefly with str(flights) and head(flights).

str(flights)
## tibble [336,776 × 19] (S3: tbl_df/tbl/data.frame)
##  $ year          : int [1:336776] 2013 2013 2013 2013 2013 2013 2013 2013 2013 2013 ...
##  $ month         : int [1:336776] 1 1 1 1 1 1 1 1 1 1 ...
##  $ day           : int [1:336776] 1 1 1 1 1 1 1 1 1 1 ...
##  $ dep_time      : int [1:336776] 517 533 542 544 554 554 555 557 557 558 ...
##  $ sched_dep_time: int [1:336776] 515 529 540 545 600 558 600 600 600 600 ...
##  $ dep_delay     : num [1:336776] 2 4 2 -1 -6 -4 -5 -3 -3 -2 ...
##  $ arr_time      : int [1:336776] 830 850 923 1004 812 740 913 709 838 753 ...
##  $ sched_arr_time: int [1:336776] 819 830 850 1022 837 728 854 723 846 745 ...
##  $ arr_delay     : num [1:336776] 11 20 33 -18 -25 12 19 -14 -8 8 ...
##  $ carrier       : chr [1:336776] "UA" "UA" "AA" "B6" ...
##  $ flight        : int [1:336776] 1545 1714 1141 725 461 1696 507 5708 79 301 ...
##  $ tailnum       : chr [1:336776] "N14228" "N24211" "N619AA" "N804JB" ...
##  $ origin        : chr [1:336776] "EWR" "LGA" "JFK" "JFK" ...
##  $ dest          : chr [1:336776] "IAH" "IAH" "MIA" "BQN" ...
##  $ air_time      : num [1:336776] 227 227 160 183 116 150 158 53 140 138 ...
##  $ distance      : num [1:336776] 1400 1416 1089 1576 762 ...
##  $ hour          : num [1:336776] 5 5 5 5 6 5 6 6 6 6 ...
##  $ minute        : num [1:336776] 15 29 40 45 0 58 0 0 0 0 ...
##  $ time_hour     : POSIXct[1:336776], format: "2013-01-01 05:00:00" "2013-01-01 05:00:00" ...
head(flights)
## # A tibble: 6 × 19
##    year month   day dep_time sched_dep_time dep_delay arr_time sched_arr_time
##   <int> <int> <int>    <int>          <int>     <dbl>    <int>          <int>
## 1  2013     1     1      517            515         2      830            819
## 2  2013     1     1      533            529         4      850            830
## 3  2013     1     1      542            540         2      923            850
## 4  2013     1     1      544            545        -1     1004           1022
## 5  2013     1     1      554            600        -6      812            837
## 6  2013     1     1      554            558        -4      740            728
## # ℹ 11 more variables: arr_delay <dbl>, carrier <chr>, flight <int>,
## #   tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>,
## #   hour <dbl>, minute <dbl>, time_hour <dttm>

Question 1: Creating Dates with lubridate

Create a column dep_datetime by combining year, month, day, and dep_time into a POSIXct datetime using lubridate.

flights <- flights |>
  mutate(
    dep_datetime = make_datetime(
      year,
      month,
      day,
      hour = dep_time %/% 100,
      min = dep_time %% 100
    )
  )

flights |>
  select(year, month, day, dep_time, dep_datetime) |>
  head(5)
## # A tibble: 5 × 5
##    year month   day dep_time dep_datetime       
##   <int> <int> <int>    <int> <dttm>             
## 1  2013     1     1      517 2013-01-01 05:17:00
## 2  2013     1     1      533 2013-01-01 05:33:00
## 3  2013     1     1      542 2013-01-01 05:42:00
## 4  2013     1     1      544 2013-01-01 05:44:00
## 5  2013     1     1      554 2013-01-01 05:54:00

Question 2: Simple Date Manipulations with lubridate

Create a column weekday and show how many flights occur on each weekday.

flights <- flights |>
  mutate(weekday = wday(dep_datetime, label = TRUE, abbr = TRUE))

table(flights$weekday)
## 
##   Sun   Mon   Tue   Wed   Thu   Fri   Sat 
## 45643 49468 49273 48858 48654 48703 37922

Question 3: Time Series with zoo

Filter for flights from JFK and create a zoo time series of departure delays using the first 1000 JFK flights.

jfk_flights <- flights |>
  filter(origin == "JFK", !is.na(dep_delay), !is.na(dep_datetime)) |>
  slice_head(n = 1000)

jfk_zoo <- zoo(jfk_flights$dep_delay, order.by = jfk_flights$dep_datetime)
## Warning in zoo(jfk_flights$dep_delay, order.by = jfk_flights$dep_datetime):
## some methods for "zoo" objects do not work if the index entries in 'order.by'
## are not unique
plot(jfk_zoo)

Question 4: Working with Factors

Convert origin to a factor, show the levels and frequency table, and create a bar plot.

flights <- flights |>
  mutate(origin_factor = factor(origin))

levels(flights$origin_factor)
## [1] "EWR" "JFK" "LGA"
table(flights$origin_factor)
## 
##    EWR    JFK    LGA 
## 120835 111279 104662
barplot(table(flights$origin_factor))

Question 5: Recoding Factors

Recode the airport abbreviations into full names and create a bar plot.

flights <- flights |>
  mutate(
    origin_recoded = fct_recode(
      origin_factor,
      Kennedy = "JFK",
      LaGuardia = "LGA",
      Newark = "EWR"
    )
  )

levels(flights$origin_recoded)
## [1] "Newark"    "Kennedy"   "LaGuardia"
barplot(table(flights$origin_recoded))

Question 6: Handling Missing Data

Count missing values in dep_delay and arr_delay. Then replace missing dep_delay values with 0 in a new column and create a frequency table for delays between -20 and 20 minutes.

colSums(is.na(flights[, c("dep_delay", "arr_delay")]))
## dep_delay arr_delay 
##      8255      9430
flights <- flights |>
  mutate(
    dep_delay_imputed = ifelse(is.na(dep_delay), 0, dep_delay)
  )

delay_table <- flights |>
  filter(dep_delay_imputed >= -20 & dep_delay_imputed <= 20)

table(delay_table$dep_delay_imputed)
## 
##   -20   -19   -18   -17   -16   -15   -14   -13   -12   -11   -10    -9    -8 
##    37    19    81   110   162   408   498   901  1594  2727  5891  7875 11791 
##    -7    -6    -5    -4    -3    -2    -1     0     1     2     3     4     5 
## 16752 20701 24821 24619 24218 21516 18813 24769  8050  6233  5450  4807  4447 
##     6     7     8     9    10    11    12    13    14    15    16    17    18 
##  3789  3520  3381  3062  2859  2756  2494  2414  2256  2140  2085  1873  1749 
##    19    20 
##  1730  1704

Question 7: Reflection

The easiest part for me was working with the factor variables because the airport codes were already simple categories. Converting them to factors and changing the names to Kennedy, LaGuardia, and Newark was easy to understand. The harder part was working with the flight dates because the year, month, day, and departure time were stored in separate columns. I had to combine them into one datetime column before I could use the weekday or make a time series.

For the missing data, replacing missing departure delays with zero makes the data easier to work with, but it could also make the flights look more on time than they really were. Some missing values may be from cancelled flights, so calling them zero delay is not always accurate. From this assignment, I learned that the NYC flight dataset has a lot of flights from three major airports and that departure delays can vary a lot. I also learned that cleaning dates and missing values is important before analyzing data.