In this lab we explore flights, specifically a random sample of domestic flights that departed from the three major New York City airports in 2013.
We will generate simple graphical and numerical summaries of data on these flights and explore delay times. You can find the instructions for this lab here.
# 1. load the library "tidyverse"
library(tidyverse)
# 2. use the read_csv file to read the dataset
nycflights <- read_csv("data/nycflights.csv")
glimpse(nycflights)## Rows: 32,735
## Columns: 16
## $ year <dbl> 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, …
## $ month <dbl> 6, 5, 12, 5, 7, 1, 12, 8, 9, 4, 6, 11, 4, 3, 10, 1, 2, 8, 10…
## $ day <dbl> 30, 7, 8, 14, 21, 1, 9, 13, 26, 30, 17, 22, 26, 25, 21, 23, …
## $ dep_time <dbl> 940, 1657, 859, 1841, 1102, 1817, 1259, 1920, 725, 1323, 940…
## $ dep_delay <dbl> 15, -3, -1, -4, -3, -3, 14, 85, -10, 62, 5, 5, -2, 115, -4, …
## $ arr_time <dbl> 1216, 2104, 1238, 2122, 1230, 2008, 1617, 2032, 1027, 1549, …
## $ arr_delay <dbl> -4, 10, 11, -34, -8, 3, 22, 71, -8, 60, -4, -2, 22, 91, -6, …
## $ carrier <chr> "VX", "DL", "DL", "DL", "9E", "AA", "WN", "B6", "AA", "EV", …
## $ tailnum <chr> "N626VA", "N3760C", "N712TW", "N914DL", "N823AY", "N3AXAA", …
## $ flight <dbl> 407, 329, 422, 2391, 3652, 353, 1428, 1407, 2279, 4162, 20, …
## $ origin <chr> "JFK", "JFK", "JFK", "JFK", "LGA", "LGA", "EWR", "JFK", "LGA…
## $ dest <chr> "LAX", "SJU", "LAX", "TPA", "ORF", "ORD", "HOU", "IAD", "MIA…
## $ air_time <dbl> 313, 216, 376, 135, 50, 138, 240, 48, 148, 110, 50, 161, 87,…
## $ distance <dbl> 2475, 1598, 2475, 1005, 296, 733, 1411, 228, 1096, 820, 264,…
## $ hour <dbl> 9, 16, 8, 18, 11, 18, 12, 19, 7, 13, 9, 13, 8, 20, 12, 20, 6…
## $ minute <dbl> 40, 57, 59, 41, 2, 17, 59, 20, 25, 23, 40, 20, 9, 54, 17, 24…
Question: Experiment with different binwidths to adjust your histogram in a way that will show its important features. You may also want to try using the + scale_x_log10() . What features are revealed now that were obscured in the original histogram? Note: When using the log scale, you may need to experiment with bin widths that are smaller than 1, such as binwidth=0.1 or even less!
# Write your code to create a histogram of delays
ggplot(data = nycflights, aes(x = dep_delay)) + geom_histogram(binwidth = 1)Answer: Almost all values are below 100, but the range goes beyond 1000 Thus, on the linear scale it was hard to distinguish individual values, while the logarithmic scale gives a better overview as it makes use of more of the diagram space.
The warnings are due to the log of 0 being negative infinity and for negative numbers being undefined, thus the 19936 flights that were without delays or early aren’t displayed here.
Question: Create a new data frame that includes flights headed to SFO in February, and save this data frame as sfo_feb_flights. How many flights meet these criteria? Try using the function nrow() inside of inline code in your answer, and knit your file to see that your text shows the answer correctly.
# Insert code for Exercise 2 here
sfo_feb_flights <- nycflights %>%
filter(dest == "SFO", month == 2)
save(sfo_feb_flights,file= "sfo_feb_flights.csv")Answer: There are 68 flights to SFO in February.
Question: Describe the distribution of the arrival delays of flights headed to SFO in February, using an appropriate histogram and summary statistics. Hint: The summary statistics you use should depend on the shape of the distribution.
# Insert code for Exercise 3 here
sfo_feb_flights %>%
summarise(min = min(dep_delay),max= max(dep_delay), median_dd = median(dep_delay), iqr_dd = IQR(dep_delay), n_flights = n())#Since it is heavily right skewed, median, IQR, min & max seemed better than mean and SD.
ggplot(data = sfo_feb_flights, aes(x="February flights", y=dep_delay)) + geom_boxplot()#What am I looking at?
ggplot(data = sfo_feb_flights, aes(x="February flights", y=dep_delay)) + geom_boxplot() + scale_y_log10()#that didnt work
ggplot(data = sfo_feb_flights, aes(x="February flights", y=dep_delay)) + geom_boxplot(outlier.shape = NA)#How to scale this more nicely?
ggplot(data = sfo_feb_flights, aes(x="February flights", y=dep_delay)) + geom_boxplot(outlier.shape = NA) + coord_cartesian(ylim=c(-10, 30))ggplot(data = sfo_feb_flights, aes(x = dep_delay)) + geom_histogram(binwidth = 0.1) +
scale_x_log10()Answer: To begin with the summary statistics overview and the linear histogram show the majority of flights are on time. Focussing on the logarithmic histogram you can see the majority of delayed flights being between 10 and a 100 minutes late.
Question: Calculate the median and interquartile
range for arr_delays of flights in in the
sfo_feb_flights data frame, grouped by carrier. Which
carrier has the most variable arrival delays?
# Insert code for the Exercise here
sfo_feb_flights %>%
group_by(carrier) %>%
summarise(median_dd = median(dep_delay), iqr_dd = IQR(dep_delay), n_flights = n())Answer: AA has both the highest median delay and the largest IQR, therefore being not only late, but also unpredictable.
Question: Create a list of origin
airports and their rate of on-time-departure. Then visualize the
distribution of on-time-departure rate across the three airports using a
segmented bar plot (see below). If you could select an airport based on
on time departure percentage, which NYC airport would you choose to fly
out of? Hint: For the segmented bar plot, will need to
map the aesthetic arguments as follows:
x = origin, fill = dep_type and a geom_bar()
layer. Create three plots, one with geom_bar() layer, one
with geom_bar(position = "fill") and the third with
geom_bar(position = "dodge"). Explain the difference
between the three results.
# Insert code for the Exercise here
nycflights <- nycflights %>%
mutate(dep_type = ifelse(dep_delay < 5, "on time", "delayed"))
#conditional coding
nycflights %>%
group_by(origin) %>%
summarise(on_time_dep_rate = sum(dep_type == "on time") / n()) %>%
arrange(desc(on_time_dep_rate))Answer: The first is a stacked bar graph showing the absolute number of on-time and delayed flights, the second displays the proportion, while the third is the paired bar graph shown in the instructions.
Question: Mutate the data frame so that it includes
a new variable that contains the average speed, avg_speed traveled by
the plane for each flight (in mph, or if you are brave, in km/h). Now
make a scatterplot of avg_speed vs. distance. Describe the relationship
between average speed and distance. Hint: Average speed can be
calculated as distance divided by number of hours of travel, and note
that air_time is given in minutes. You will need to use geom_point().
Think carefully which of the two variables is the predictor (on the
x-axis) and which is the outcome variable (on the
y-axis) and explain why you made this choice. Describe the
relationship between average speed and distance.
# Insert code for the Exercise here
nycflights <- nycflights %>%
mutate(avg_speed = (distance*1.6093) / (air_time/60))
ggplot(data = nycflights, aes(x=distance, y=avg_speed)) +
geom_point()Answer: There seems to be a asymptotic relationship with flights going less than a 1000 miles approaching a maximum avg_speed of around 700-800 km/h, and not much increasing past that point. This is likely due to the relative durations of takeoff and landing, thus lowering overall avg_speed on short-distance flights.
Question: Replicate the following plot and determine what is the cutoff point for the latest departure delay where you can still have a chance to arrive at your destination on time. Hint: The data frame plotted only contains flights from American Airlines, Delta Airlines, and United Airlines, and the points are colored by carrier. To determine the cut off point, try scaling the x-axis and the y-axis on the logarithmic scale. You can also filter the data, so that you plot only data where arr_delay <= 0.
# Insert code for the Exercise here
nycflights %>%
filter(carrier == "AA"|carrier =="DL"|carrier =="UA") %>%
ggplot( aes(x=dep_delay, y=arr_delay,color = carrier)) +
geom_point() nycflights %>%
filter(carrier == "AA"|carrier =="DL"|carrier =="UA") %>%
ggplot( aes(x=dep_delay, y=arr_delay,color = carrier)) +
geom_point() +
scale_x_log10() +
scale_y_log10() nycflights %>%
filter(carrier == "AA"|carrier =="DL"|carrier =="UA", arr_delay <=0) %>%
ggplot( aes(x=dep_delay, y=arr_delay,color = carrier)) +
geom_point() Answer: The graphs seem to indicate that beyond 25 minutes of departure delay, it is highly unlikely to arrive on time and beyond 50 it is almost impossible.