As usual, first, the libraries and the data are downloaded.
library(tidyverse)## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.2.1 ✔ readr 2.2.0
## ✔ forcats 1.0.1 ✔ stringr 1.6.0
## ✔ ggplot2 4.0.3 ✔ tibble 3.3.1
## ✔ lubridate 1.9.5 ✔ tidyr 1.3.2
## ✔ purrr 1.2.2
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errorslibrary(dplyr)
library(ggplot2)url<-"https://raw.githubusercontent.com/Enrique01234/607-Fall-2026/refs/heads/main/Delays2.csv"The raw data from GitHub are converted into a data frame.
Flights <- read_csv(file = url, show_col_types = FALSE, progress = FALSE)Flights## # A tibble: 4 × 7
## Airlines Time `Los Angeles` Phoenix `San Diego` `San Francisco` Seattle
## <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 Alaska On Time 497 221 212 503 1841
## 2 Alaska Delayed 62 12 20 102 305
## 3 AM West On Time 694 4840 383 320 201
## 4 AM West Delayed 117 415 65 129 61This data frame is not tidy as the variable, which is the number of flights, is distributed in rows (for the airline and whether the flight is classified as on time or delayed) and columns (cities).
The pivot_longer function is used to tidy the data frame. A new data frame (called Pivoted) is created. Although it may not be needed to see these two data frames, it helped me to double-check the operations worked.
Pivoted <- Flights|>
pivot_longer (!c(Airlines, Time), names_to = "City", values_to = "nF")
Pivoted## # A tibble: 20 × 4
## Airlines Time City nF
## <chr> <chr> <chr> <dbl>
## 1 Alaska On Time Los Angeles 497
## 2 Alaska On Time Phoenix 221
## 3 Alaska On Time San Diego 212
## 4 Alaska On Time San Francisco 503
## 5 Alaska On Time Seattle 1841
## 6 Alaska Delayed Los Angeles 62
## 7 Alaska Delayed Phoenix 12
## 8 Alaska Delayed San Diego 20
## 9 Alaska Delayed San Francisco 102
## 10 Alaska Delayed Seattle 305
## 11 AM West On Time Los Angeles 694
## 12 AM West On Time Phoenix 4840
## 13 AM West On Time San Diego 383
## 14 AM West On Time San Francisco 320
## 15 AM West On Time Seattle 201
## 16 AM West Delayed Los Angeles 117
## 17 AM West Delayed Phoenix 415
## 18 AM West Delayed San Diego 65
## 19 AM West Delayed San Francisco 129
## 20 AM West Delayed Seattle 61Initially, I estimated the different proportions in Excel. Although there is no need to create a new data frame, I wanted to make sure I kept my newly pivoted data frame intact (again, this step is not really needed, but as I was exploring different ways to group and select the observations, I wanted to be extra-cautious).
Pivoted2<- PivotedI started the calculations in R with the total flights by Airline.
Pivoted3<- Pivoted2|>
group_by(Airlines)|>
mutate(Total = sum(nF))Then, I calculated the total flights by Airline and city. These sums are needed to calculate the percentages.
Pivoted4 <- Pivoted3|>
group_by(Airlines, Time)|>
mutate(Total_by_subgroup = sum(nF))Once I had these new variables, I calculated the percentages for four subgroups of the data frame: Alaska flights On Time, Alaska flights Delayed, AM West flights On Time, and AM West flights Delayed.
These percentages are included in a new variable called “PropDelay_by_subgroup”. As both on time and delayed flights are included, it could have been called “PropDelay_or_ontime_by_subgroup” but it would have been too long. In order to be clear, this variable includes the proportion of delayed (and on time) flights for each airline. Thus, it has only two values per airline, which for each airline add up to 1.
Pivoted4 <- Pivoted4 |>
mutate(PropDelay_by_subgroup = Total_by_subgroup /Total)Afterwards, I calculated, for each airline the total flights per city.
Pivoted4 <- Pivoted4 |>
group_by(Airlines, City)|>
mutate(Flights_by_city = sum(nF))Then, in in one variable (“PropDelays_by_city_by_airline”), I calculated the percentage of on time and delayed flights per city for each airline
Pivoted4 <- Pivoted4 |>
mutate(PropDelays_by_city_by_airline = nF/Flights_by_city)This file, with all the calculated columns has been converted to a csv file. It has also been uploaded in GitHub
write.csv (Pivoted4,"Pivoted_4.csv")read.csv ("Pivoted_4.csv")## X Airlines Time City nF Total Total_by_subgroup
## 1 1 Alaska On Time Los Angeles 497 3775 3274
## 2 2 Alaska On Time Phoenix 221 3775 3274
## 3 3 Alaska On Time San Diego 212 3775 3274
## 4 4 Alaska On Time San Francisco 503 3775 3274
## 5 5 Alaska On Time Seattle 1841 3775 3274
## 6 6 Alaska Delayed Los Angeles 62 3775 501
## 7 7 Alaska Delayed Phoenix 12 3775 501
## 8 8 Alaska Delayed San Diego 20 3775 501
## 9 9 Alaska Delayed San Francisco 102 3775 501
## 10 10 Alaska Delayed Seattle 305 3775 501
## 11 11 AM West On Time Los Angeles 694 7225 6438
## 12 12 AM West On Time Phoenix 4840 7225 6438
## 13 13 AM West On Time San Diego 383 7225 6438
## 14 14 AM West On Time San Francisco 320 7225 6438
## 15 15 AM West On Time Seattle 201 7225 6438
## 16 16 AM West Delayed Los Angeles 117 7225 787
## 17 17 AM West Delayed Phoenix 415 7225 787
## 18 18 AM West Delayed San Diego 65 7225 787
## 19 19 AM West Delayed San Francisco 129 7225 787
## 20 20 AM West Delayed Seattle 61 7225 787
## PropDelay_by_subgroup Flights_by_city PropDelays_by_city_by_airline
## 1 0.8672848 559 0.88908766
## 2 0.8672848 233 0.94849785
## 3 0.8672848 232 0.91379310
## 4 0.8672848 605 0.83140496
## 5 0.8672848 2146 0.85787512
## 6 0.1327152 559 0.11091234
## 7 0.1327152 233 0.05150215
## 8 0.1327152 232 0.08620690
## 9 0.1327152 605 0.16859504
## 10 0.1327152 2146 0.14212488
## 11 0.8910727 811 0.85573366
## 12 0.8910727 5255 0.92102759
## 13 0.8910727 448 0.85491071
## 14 0.8910727 449 0.71269488
## 15 0.8910727 262 0.76717557
## 16 0.1089273 811 0.14426634
## 17 0.1089273 5255 0.07897241
## 18 0.1089273 448 0.14508929
## 19 0.1089273 449 0.28730512
## 20 0.1089273 262 0.23282443The results are shown in a table. The packcge to create tables needs to be downloaded.
install.packages("gt")## Error in `contrib.url()`:
## ! trying to use CRAN without setting a mirrorlibrary(gt)As I had difficulties properly selecting the columns for the tables, I created a new data frame (Pivoted 5). It includes only the columns neeed for the analysis.
Pivoted5<- Pivoted4|>
select(Airlines, Time, City, PropDelay_by_subgroup, PropDelays_by_city_by_airline)Pivoted5|>
gt()| Time | PropDelay_by_subgroup | PropDelays_by_city_by_airline |
|---|---|---|
| Alaska - Los Angeles | ||
| On Time | 0.8672848 | 0.88908766 |
| Delayed | 0.1327152 | 0.11091234 |
| Alaska - Phoenix | ||
| On Time | 0.8672848 | 0.94849785 |
| Delayed | 0.1327152 | 0.05150215 |
| Alaska - San Diego | ||
| On Time | 0.8672848 | 0.91379310 |
| Delayed | 0.1327152 | 0.08620690 |
| Alaska - San Francisco | ||
| On Time | 0.8672848 | 0.83140496 |
| Delayed | 0.1327152 | 0.16859504 |
| Alaska - Seattle | ||
| On Time | 0.8672848 | 0.85787512 |
| Delayed | 0.1327152 | 0.14212488 |
| AM West - Los Angeles | ||
| On Time | 0.8910727 | 0.85573366 |
| Delayed | 0.1089273 | 0.14426634 |
| AM West - Phoenix | ||
| On Time | 0.8910727 | 0.92102759 |
| Delayed | 0.1089273 | 0.07897241 |
| AM West - San Diego | ||
| On Time | 0.8910727 | 0.85491071 |
| Delayed | 0.1089273 | 0.14508929 |
| AM West - San Francisco | ||
| On Time | 0.8910727 | 0.71269488 |
| Delayed | 0.1089273 | 0.28730512 |
| AM West - Seattle | ||
| On Time | 0.8910727 | 0.76717557 |
| Delayed | 0.1089273 | 0.23282443 |
In the first column of the table (PropDelay_by_subgroup), it can be seen that the percentage of delayed flights for Alaska (13.3%) is higher than for AM West (10.1%). This percentage applies to all the flights for each airline.
However, in the second column (PropDelays_by_city_by_airline), the percentage of delayed flights is taken out of the flights per city. In this case, for all five cities, the percentage of delayed flights is lower for Alaska than for AM West. For instance, for San Francisco, Alaska flights were delayed 16.7% of the time while AM West flights were delayed 28.7% of the time.
This discrepancy is an example of Simpson’s paradox. In the work I do, for example measuring poverty across countries but also for sub-national units like provinces/states within countries, we use an expression which captures a similar effect: “Averages hide disparities”.