Using Point Estimation and Confidence Intervals on Airline Delay Data

Course: Exploring Data with R and Python
Presenter: Nicolette Fellows
Date: April 11, 2025

Airline delays are a frequent frustration for travelers. what is the probability for your flight to be delayed, really? Can we use data to estimate the likelihood of a delay?

In this lesson, we’ll use airline flight data to:

• ✅Understand point estimation using sample proportions
  
• ✅ Construct and interpret confidence intervals
• ✅ Apply these concepts to estimate real-world flight delay s
• ✅ Visualize findings using plots and interactive graphs

We’ll answer the questions:
How likely is it that a flight will be delayed, and how confident can we be in that estimate?

Found here: https://www.kaggle.com/datasets/jimschacko/airlines-dataset-to-predict-a-delay

Lets start by using flight records that include: - Airline - Departure and arrival airports - Day of the week - Scheduled departure time - Flight duration - Delay status (0 = on-time, 1 = delayed)

Below, we visualize the overall proportion of delayed vs. on-time flights.

A point estimate is the single best guess of an unknown population parameter based on sample data.

In our case, we’re estimating the proportion of flights that are delayed.

The formula for a sample proportion is:

\[ \hat{p} = \frac{x}{n} \]

Where:

• \(x\) = number of delayed flights
  
• \(n\) = total number of flights
  
• \(\hat{p}\) = sample proportion of delays (our point estimate)

We’ll calculate this value using the airline dataset on the next slide.

From our dataset, we compute the sample proportion of delayed flights:

A confidence interval gives a range of plausible values for a population parameter based on sample data.

For a sample proportion \(\hat{p}\), the 95% confidence interval is calculated as:

\[ \hat{p} \pm z \sqrt{ \frac{ \hat{p}(1 - \hat{p}) }{n} } \]

Where:

• \(\hat{p}\) = sample proportion (point estimate)
  
• \(n\) = sample size (number of observations)
  
• \(z\) = critical value from the normal distribution
  (1.96 for 95% confidence)

This interval tells us:
> “We are 95% confident that the true proportion of delayed flights lies within this range.”

We’ll compute this for our dataset next!

We’ll now calculate the 95% confidence interval for the true proportion of delayed flights:

# Sample proportion and size
p_hat <- mean(AirlinesXLS$Delay)
n <- nrow(AirlinesXLS)
z <- 1.96  # z* for 95% confidence

# Standard error
se <- sqrt(p_hat * (1 - p_hat) / n)

# Confidence interval
ci_lower <- p_hat - z * se
ci_upper <- p_hat + z * se

c(ci_lower, ci_upper)

## [1] 0.4441159 0.4467687

Below is a 3D scatter plot of a random sample of 1,000 flights, further broken down by top 5 Airlines for delays.
We explore how departure time and flight length relate to delay status.

✈️ In this lesson, we used real airline data to explore:

• ✅ Point estimation: Using sample data to estimate population proportions
• ✅ Confidence intervals: Quantifying uncertainty in our estimates
• ✅ Flight delay trends: Across time, flight duration, day of week, and airline
• ✅ Interactive visualizations: Using ggplot and plotly to bring data to life

Key takeaway: > Even simple stats like proportions and CIs can uncover powerful insights — especially when paired with clear, visual storytelling.