Project 2 - Air quality

Author

G Oliveira

DATA 110 - Project 2

Step 1 - Source

This data is from NYC Open Data, named “Air Quality and Health Impacts”

Direct URL: https://data.cityofnewyork.us/Environment/Air-Quality-and-Health-Impacts/c3uy-2p5r/about_data

  • The dataset has 12 variables and 19,827 rows containing 1 quantitative variable (Data Value, which holds the value for each indicator) and the rest categorical (Name, Measure, Geo Type Name, Geo Place Name, Time Period).

  • Variables I plan to use:

- Categorical: Geo Place Name (Community district and Borough)

- Quantitative: Year of the measurement (2009-2024), annual mean fine particles (PM2.5), in µg/m³, annual mean nitrogen dioxide, in ppb, summer mean nitrogen dioxide, in ppb, winter mean nitrogen dioxide, in ppb and summer mean ozone, in ppb.

Step 2 - Image

Air Quality. Image source: https://earth.org/air-pollution-in-nyc/, accessed November 18, 2026.

Step 3 - Topic introduction and dataset

I choose this topic because clean air is directly tied to human health and well being, yet for years, industrial growth prioritized production over environmental pollution. “Annually, ozone pollution in New York City causes around 400 premature deaths, 850 hospitalizations, and around 4,500 emergency visits for health issues such as asthma.” (Ronniya Varghese, 2022)

It is a government job to create strategy to reduce the impact of industries, and for me this theme is meaningful in order to create a healthy and green planet for my future.

Variable Meaning Type
district Community district (59 in NYC) Categorical
borough Borough, created from the first digit of the district code Categorical
year Year of the measurement (2009-2024) Quantitative
pm25 Annual mean fine particles (PM2.5), in µg/m³ (response) Quantitative
no2 Annual mean nitrogen dioxide, in ppb Quantitative
no2_summer Summer mean nitrogen dioxide, in ppb Quantitative
no2_winter Winter mean nitrogen dioxide, in ppb Quantitative
o3 Summer mean ozone, in ppb Quantitative

How the data was cleaned? The quantitive variables are written in lines, so I used filter() and select() to create a table for each indicator, then I used left_join() to marge the table with one row per district. After that I used mutate() and case_when() to create year and borough. The analysis table have 8 variables, 2 categorical and 6 quantitative.

Step 4 - Load the libraries and data

# load the libraries
library(tidyverse)
library(dplyr)
library(plotly)       # interactive graph
# load the dataset
aq <- read_csv("Air_Quality_and_Health_Impacts_20261109.csv")
Rows: 19827 Columns: 12
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (7): Name, Measure, Measure Info, Geo Type Name, Geo Place Name, Time Pe...
dbl (3): Unique ID, Geo Join ID, Data Value
num (1): Indicator ID
lgl (1): Message

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
# checking the data structure
glimpse(aq)
Rows: 19,827
Columns: 12
$ `Unique ID`      <dbl> 979194, 980097, 980098, 980099, 980100, 980101, 98010…
$ `Indicator ID`   <dbl> 1427, 1431, 1431, 1431, 1431, 1431, 1431, 1431, 1431,…
$ Name             <chr> "Fine particles (PM 2.5)", "Nitrogen dioxide (NO2)", …
$ Measure          <chr> "Winter mean", "Annual mean", "Annual mean", "Annual …
$ `Measure Info`   <chr> "mcg/m3", "ppb", "ppb", "ppb", "ppb", "ppb", "ppb", "…
$ `Geo Type Name`  <chr> "UHF34", "UHF34", "UHF34", "UHF34", "UHF34", "UHF34",…
$ `Geo Join ID`    <dbl> 104, 304, 303, 302, 301, 211, 210, 209, 208, 207, 206…
$ `Geo Place Name` <chr> "Pelham - Throgs Neck", "Upper West Side", "East Harl…
$ `Time Period`    <chr> "2024", "2024", "2024", "2024", "2024", "2024", "2024…
$ Start_Date       <chr> "01/01/2024", "01/01/2024", "01/01/2024", "01/01/2024…
$ `Data Value`     <dbl> 5.834572, 18.288730, 16.184849, 16.521823, 16.264552,…
$ Message          <lgl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, N…

There is one numeric column (Data Value), while the indicators are stored in rows

Step 5 - Cleaning the data

# Keep only the community districts
aq1 <- aq |> filter(`Geo Type Name` == "CD")

Transforming the rows into tables

# PM2.5 annual mean (the variable I want to predict)
t_pm25 <- aq1 |>
  filter(Name == "Fine particles (PM 2.5)", Measure == "Annual mean") |>
  select(`Geo Join ID`, `Geo Place Name`, `Time Period`, pm25 = `Data Value`)

# NO2 annual, summer, and winter means
t_no2 <- aq1 |>
  filter(Name == "Nitrogen dioxide (NO2)", Measure == "Annual mean") |>
  select(`Geo Join ID`, `Time Period`, no2 = `Data Value`)

t_no2_summer <- aq1 |>
  filter(Name == "Nitrogen dioxide (NO2)", Measure == "Summer mean") |>
  select(`Geo Join ID`, `Time Period`, no2_summer = `Data Value`)

t_no2_winter <- aq1 |>
  filter(Name == "Nitrogen dioxide (NO2)", Measure == "Winter mean") |>
  select(`Geo Join ID`, `Time Period`, no2_winter = `Data Value`)

# Ozone summer mean
t_o3 <- aq1 |>
  filter(Name == "Ozone (O3)", Measure == "Summer mean") |>
  select(`Geo Join ID`, `Time Period`, o3 = `Data Value`)

creating a new df

# marging the five tables by district and year
air <- t_pm25 |>
  left_join(t_no2, by = c("Geo Join ID", "Time Period")) |>
  left_join(t_no2_summer, by = c("Geo Join ID", "Time Period")) |>
  left_join(t_no2_winter, by = c("Geo Join ID", "Time Period")) |>
  left_join(t_o3, by = c("Geo Join ID", "Time Period")) |>
  mutate(year = as.numeric(`Time Period`),
         #The dataset has no borough column for districts, so I create it from the first digit of the district code
         borough = case_when(`Geo Join ID` < 200 ~ "Manhattan",
                    `Geo Join ID` < 300 ~ "Bronx",
                    `Geo Join ID` < 400 ~ "Brooklyn",
                    `Geo Join ID` < 500 ~ "Queens",
                    TRUE ~ "Staten Island"))|>
  rename(district = `Geo Place Name`) |>
  select(district, borough, year, pm25, no2, no2_summer, no2_winter, o3) |>
  # removing NAs only for the variables I use
  filter(!is.na(pm25) & !is.na(no2) & !is.na(no2_summer) &
           !is.na(no2_winter) & !is.na(o3))
head(air)
# A tibble: 6 × 8
  district                 borough  year  pm25   no2 no2_summer no2_winter    o3
  <chr>                    <chr>   <dbl> <dbl> <dbl>      <dbl>      <dbl> <dbl>
1 Throgs Neck and Co-op C… Bronx    2024  6.04  14.2       12.2       18.5  34.5
2 Parkchester and Soundvi… Bronx    2024  6.45  14.9       12.8       19.2  33.7
3 Riverdale and Fieldston… Bronx    2024  6.33  13.5       12.2       17.1  32.5
4 Kingsbridge Heights and… Bronx    2024  6.45  14.6       13.2       18.1  33.3
5 Belmont and East Tremon… Bronx    2024  6.63  15.3       13.6       19.1  33.2
6 Fordham and University … Bronx    2024  6.46  16.1       14.6       19.7  32.7

Step 6 - Exploring the data

#how many rows were left
air |> summarise(rows = n())
# A tibble: 1 × 1
   rows
  <int>
1   944

Trend over time

yearly <- air |>
  group_by(year) |>
  summarise(avg_pm25 = mean(pm25))

ggplot(yearly, aes(x = year, y = avg_pm25)) +
  geom_line() +
  geom_point() +
  labs(title = "Average PM2.5 over time", x = "Year",
       y = "Average PM2.5 (µg/m³)") +
  theme_minimal(base_size = 12)

This plot creates a trend over the years with the average PM2.5. The average for 2009 was 11µg/m³, while in 2024 is approximately 6.5µg/m³. That shows that over the time the levels decrease.

p1 <- ggplot(air, aes(x = no2, y = pm25)) +
  geom_point(alpha = 0.4) +
  geom_smooth(method = 'lm', formula = y ~ x) + # lm = linear model
  labs(title = "PM2.5 VERSUS NO2 IN NYC COMMUNITY DISTRICTS (2009-2024)",
       caption = "Source: NYC Open Data, Air Quality and Health Impacts",
       x = "Annual mean NO2 (ppb)",
       y = "Annual mean PM2.5 (µg/m³)") +
  theme_minimal(base_size = 12)
p1

The geom plot shows the relationship between annual mean of PM2.5 and NO2. There is a positive trend, meaning that as the NO2 increases, the PM2.5 also tends to increase.

air |>
  group_by(borough) |>
  summarise(avg_pm25 = mean(pm25)) |>
  ggplot(aes(x = borough, y = avg_pm25)) +
  geom_bar(stat = "identity", fill = "darkred") +
  labs(title = "Average PM2.5 by borough", x = "Borough",
       y = "Average PM2.5 (µg/m³)") +
  theme_minimal(base_size = 12)

The bar plot compares the Average PM2.5 by borough. Manhattan is the biggest bar in this plot, but the overall Average PM2.5 is higher than 7µg/m³.

Multiple linear regression

Checking correlation

cor_matrix <- air |>
  select(pm25, no2, no2_summer, no2_winter, o3, year) |>
  cor()
round(cor_matrix, 2)
            pm25   no2 no2_summer no2_winter    o3  year
pm25        1.00  0.89       0.87       0.83 -0.51 -0.81
no2         0.89  1.00       0.98       0.94 -0.66 -0.61
no2_summer  0.87  0.98       1.00       0.89 -0.68 -0.55
no2_winter  0.83  0.94       0.89       1.00 -0.68 -0.59
o3         -0.51 -0.66      -0.68      -0.68  1.00  0.27
year       -0.81 -0.61      -0.55      -0.59  0.27  1.00

The variable PM2 has a strong correlation between NO2 (0.89), NO2_summer(0.87) and NO2_winter (0.83) with a positive slope. Year (-0.81) also has a strong correlation, but with a negative slope, while O3(-0.51) has a moderate correlation.

The three NO2 predictors have the highest correlation. The collinearity happens because the variables are not independent of each other and explain the same thing for different periods.

Comparing the others predictors: NO2 and year (-0.55 to -0.61) have a negative slope, which happens the same with O3 and NO2 (-0.66 to -0.68) and for O3 and Year there is a little correlation (0.27)

Full model

The model has the equation: PM2.5 = 353.45+ 0.08(NO2)+ 0.15(NO2_SUMMER)+ 0.01(NO2_WINTER)+ 0.03(03)+ (-0.17(YEAR))

# Model 1: all predictors
fit1 <- lm(pm25 ~ no2 + no2_summer + no2_winter + o3 + year, data = air)
summary(fit1)

Call:
lm(formula = pm25 ~ no2 + no2_summer + no2_winter + o3 + year, 
    data = air)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.2451 -0.3820 -0.0440  0.3538  2.2883 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept) 353.453250  10.608075  33.319  < 2e-16 ***
no2           0.078457   0.036573   2.145   0.0322 *  
no2_summer    0.152436   0.025137   6.064 1.92e-09 ***
no2_winter    0.011496   0.014905   0.771   0.4407    
o3            0.038277   0.007727   4.954 8.64e-07 ***
year         -0.173910   0.005198 -33.454  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.5319 on 938 degrees of freedom
Multiple R-squared:  0.9166,    Adjusted R-squared:  0.9162 
F-statistic:  2063 on 5 and 938 DF,  p-value: < 2.2e-16

The predictor no2_winter had a large p-value, meaning that the predictor does not fit the model.

Removing the predictor with the largest p-value (above 0.05)

The model has the equation: PM2.5 = 352.35+ 0.10(NO2)+ 0.14(NO2_SUMMER)+ 0.04(03)+ (-0.17(YEAR))

fit2 <- lm(pm25 ~ no2 + no2_summer + o3 + year, data = air)
summary(fit2)

Call:
lm(formula = pm25 ~ no2 + no2_summer + o3 + year, data = air)

Residuals:
     Min       1Q   Median       3Q      Max 
-1.23423 -0.38135 -0.04404  0.35961  2.29813 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept) 352.348605  10.508676  33.529  < 2e-16 ***
no2           0.101409   0.021256   4.771 2.13e-06 ***
no2_summer    0.139612   0.018849   7.407 2.88e-13 ***
o3            0.035970   0.007123   5.050 5.32e-07 ***
year         -0.173309   0.005139 -33.726  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.5318 on 939 degrees of freedom
Multiple R-squared:  0.9166,    Adjusted R-squared:  0.9162 
F-statistic:  2579 on 4 and 939 DF,  p-value: < 2.2e-16

All the variables have a p-value with three asterisks, with suggests strong evidence that no2, no2_summer, o3 and year are related to pm25. Looking at the adjusted R-squared, 91.6% of the variation in pm25 is explained by the model. In other words, only about 8% of the variation is not explained by this model.

For each additional 1 ppb of NO2, the model predicts an increase of about 0.10 µg/m³ of PM2.5.

Trying a simpler model

The model has the equation: PM2.5 = 328.23+ 0.25(NO2)+ 0.03(03)+ (-0.16(YEAR))

fit3 <- lm(pm25 ~ no2 + o3 + year, data = air)
summary(fit3)

Call:
lm(formula = pm25 ~ no2 + o3 + year, data = air)

Residuals:
     Min       1Q   Median       3Q      Max 
-1.29427 -0.41543 -0.03907  0.35477  2.57316 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept) 328.229879  10.273718  31.949  < 2e-16 ***
no2           0.252769   0.006015  42.024  < 2e-16 ***
o3            0.030950   0.007291   4.245 2.41e-05 ***
year         -0.161661   0.005030 -32.137  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.5468 on 940 degrees of freedom
Multiple R-squared:  0.9117,    Adjusted R-squared:  0.9114 
F-statistic:  3235 on 3 and 940 DF,  p-value: < 2.2e-16

All the variables still have a p-value with three asterisks, but the adjusted R-squared of 91.1% did not suffer any improvement. We need to check if the simpler model is really the best.

# checking for improvement
anova(fit3, fit2)
Analysis of Variance Table

Model 1: pm25 ~ no2 + o3 + year
Model 2: pm25 ~ no2 + no2_summer + o3 + year
  Res.Df    RSS Df Sum of Sq      F    Pr(>F)    
1    940 281.09                                  
2    939 265.57  1    15.516 54.862 2.876e-13 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The p-value is small, so we choose the fit2. There is strong evidence that Nitrogen dioxide (NO2) measured in summer improves the model.

Step 8 - Final visualization

# Scatterplot with one color for each borough
p <- ggplot(air, aes(x = no2, y = pm25, color = borough,
                     text = paste("District:", district,
                                  "\nYear:", year,
                                  "\nNO2:", round(no2, 1), "ppb",
                                  "\nPM2.5:", round(pm25, 2), "µg/m³"))) +
  geom_point(size = 2, alpha = 0.6) +
  scale_color_brewer(name = "Borough", palette = "Set1") +
  labs(title = "Districts With Higher NO2 Levels \nAlso Have Higher PM2.5 (2009-2024)",
       caption = "Source: NYC Open Data, Air Quality and Health Impacts",
       x = "Annual mean nitrogen dioxide, NO2 (ppb)",
       y = "Annual mean fine particles, PM2.5 (µg/m³)") +
  theme_minimal(base_size = 12)

ggplotly(p, tooltip = "text")    # interactive version

Works cited:

Varghese, Ronniya. “Air Pollution in NYC: Causes, Effects, and Solutions.” Earth.Org, 15 Nov. 2022,https://earth.org/air-pollution-in-nyc/.