Code
library(tidyr)
library(dplyr)
library(ggplot2)Project 2 - Air Quality
Introduction
For this project, I will use the Air Quality dataset from the UCI Machine Learning Repository. The dataset contains air pollution measurements, temperature values, and sensor responses collected over time. The original dataset has 9,471 rows and 17 columns in wide format.
I will focus on carbon monoxide, nitrogen dioxide, temperature, and the sensor responses related to these two pollutants. My goal will be to transform the dataset into tidy format and examine how air pollutant concentrations change over time.
Original Dataset: https://archive.ics.uci.edu/dataset/360/air+quality
Planned Approach
First, I will load the original dataset from my GitHub repository into R. I will examine its structure and select seven variables for my analysis: measurement date, measurement time, temperature, carbon monoxide concentration, nitrogen dioxide concentration, carbon monoxide sensor response, and nitrogen dioxide sensor response. I will then rename the variables to make them easier to understand.
Next, I will use the pivot_longer() function from the tidyr package to transform the data from wide to long format. I will check for missing values and remove incomplete measurements. I will also identify and remove invalid values recorded as -200.
After cleaning the data, I will calculate summary statistics, including the mean, median, minimum, maximum, and standard deviation. Finally, I will calculate monthly average concentrations for carbon monoxide and nitrogen dioxide and use ggplot2 to visualize how they change over time.
Anticipated Challenges
One possible challenge will be handling missing and invalid values. The dataset uses -200 to represent missing measurements, so I will need to make sure these values do not affect my analysis results.
I will also need to convert the date and time columns into appropriate formats and make sure the measurements are correctly preserved during the wide-to-long transformation.
Code Base
Required Packages
I will use three R packages for this project. The tidyr package will be used to transform the dataset from wide to long format. The dplyr package will be used to select, rename, clean, and summarize the data. The ggplot2 package will be used to create visualizations of air quality measurements over time.
Data Source and Import
The dataset was obtained from the UCI Machine Learning Repository. It contains air pollution measurements, temperature values, and sensor responses collected over time.
I saved the original wide-format dataset as a CSV file in my public GitHub repository. I will import this file into R while preserving its original column names and structure.
Data Source: https://archive.ics.uci.edu/dataset/360/air+quality
Code
air_raw <- read.csv2(
"https://raw.githubusercontent.com/OzgeG01/DATA607-Assignment/refs/heads/main/Air_Quality.csv",
dec = ",",
check.names = FALSE
)
head(air_raw) Date Time CO(GT) PT08.S1(CO) NMHC(GT) C6H6(GT) PT08.S2(NMHC)
1 10.03.2004 18:00:00 2.6 1360 150 11.9 1046
2 10.03.2004 19:00:00 2.0 1292 112 9.4 955
3 10.03.2004 20:00:00 2.2 1402 88 9.0 939
4 10.03.2004 21:00:00 2.2 1376 80 9.2 948
5 10.03.2004 22:00:00 1.6 1272 51 6.5 836
6 10.03.2004 23:00:00 1.2 1197 38 4.7 750
NOx(GT) PT08.S3(NOx) NO2(GT) PT08.S4(NO2) PT08.S5(O3) T RH AH
1 166 1056 113 1692 1268 13.6 48.9 0.7578 NA NA
2 103 1174 92 1559 972 13.3 47.7 0.7255 NA NA
3 131 1140 114 1555 1074 11.9 54.0 0.7502 NA NA
4 172 1092 122 1584 1203 11.0 60.0 0.7867 NA NA
5 131 1205 116 1490 1110 11.2 59.6 0.7888 NA NA
6 89 1337 96 1393 949 11.2 59.2 0.7848 NA NAData Structure Before Tidying
Before tidying the dataset, I will examine its original structure using str(). This will help me identify the number of rows and columns, variable names, data types, and sample values. I will also check whether the dataset contains any empty or unnecessary columns before selecting the variables for analysis.
Code
str(air_raw)'data.frame': 9471 obs. of 17 variables:
$ Date : chr "10.03.2004" "10.03.2004" "10.03.2004" "10.03.2004" ...
$ Time : chr "18:00:00" "19:00:00" "20:00:00" "21:00:00" ...
$ CO(GT) : num 2.6 2 2.2 2.2 1.6 1.2 1.2 1 0.9 0.6 ...
$ PT08.S1(CO) : int 1360 1292 1402 1376 1272 1197 1185 1136 1094 1010 ...
$ NMHC(GT) : int 150 112 88 80 51 38 31 31 24 19 ...
$ C6H6(GT) : num 11.9 9.4 9 9.2 6.5 4.7 3.6 3.3 2.3 1.7 ...
$ PT08.S2(NMHC): int 1046 955 939 948 836 750 690 672 609 561 ...
$ NOx(GT) : int 166 103 131 172 131 89 62 62 45 -200 ...
$ PT08.S3(NOx) : int 1056 1174 1140 1092 1205 1337 1462 1453 1579 1705 ...
$ NO2(GT) : int 113 92 114 122 116 96 77 76 60 -200 ...
$ PT08.S4(NO2) : int 1692 1559 1555 1584 1490 1393 1333 1333 1276 1235 ...
$ PT08.S5(O3) : int 1268 972 1074 1203 1110 949 733 730 620 501 ...
$ T : num 13.6 13.3 11.9 11 11.2 11.2 11.3 10.7 10.7 10.3 ...
$ RH : num 48.9 47.7 54 60 59.6 59.2 56.8 60 59.7 60.2 ...
$ AH : num 0.758 0.726 0.75 0.787 0.789 ...
$ : logi NA NA NA NA NA NA ...
$ : logi NA NA NA NA NA NA ...The original dataset contains 9,471 observations and 17 variables. The variables include character, numeric, integer, and logical data types. The Date and Time columns are stored as character variables. Some measurements contain -200, which represents missing values in this dataset. The last two columns also appear to be empty. I will consider these issues during the data cleaning and transformation process.
Selecting Variables
The original dataset contains 17 variables, but I will select only the 7 variables relevant to my analysis. These include the measurement date and time, temperature, carbon monoxide concentration, nitrogen dioxide concentration, and their corresponding sensor responses. I will use select() to keep these variables without modifying the original dataset.
Code
air_selected <- air_raw %>%
select(
Date,
Time,
T,
`CO(GT)`,
`NO2(GT)`,
`PT08.S1(CO)`,
`PT08.S4(NO2)`
)
head(air_selected) Date Time T CO(GT) NO2(GT) PT08.S1(CO) PT08.S4(NO2)
1 10.03.2004 18:00:00 13.6 2.6 113 1360 1692
2 10.03.2004 19:00:00 13.3 2.0 92 1292 1559
3 10.03.2004 20:00:00 11.9 2.2 114 1402 1555
4 10.03.2004 21:00:00 11.0 2.2 122 1376 1584
5 10.03.2004 22:00:00 11.2 1.6 116 1272 1490
6 10.03.2004 23:00:00 11.2 1.2 96 1197 1393Renaming Variables
After selecting the 7 variables, I will use rename() to give five variables more descriptive names. These include temperature, carbon monoxide, nitrogen dioxide, and their corresponding sensor responses. I will keep the Date and Time column names unchanged. This will make the measurements easier to identify and understand during analysis.
Code
air_renamed <- air_selected %>%
rename(
Temperature = T,
CarbonMonoxide = `CO(GT)`,
NitrogenDioxide = `NO2(GT)`,
Sensor_CO = `PT08.S1(CO)`,
Sensor_NO2 = `PT08.S4(NO2)`
)
head(air_renamed) Date Time Temperature CarbonMonoxide NitrogenDioxide Sensor_CO
1 10.03.2004 18:00:00 13.6 2.6 113 1360
2 10.03.2004 19:00:00 13.3 2.0 92 1292
3 10.03.2004 20:00:00 11.9 2.2 114 1402
4 10.03.2004 21:00:00 11.0 2.2 122 1376
5 10.03.2004 22:00:00 11.2 1.6 116 1272
6 10.03.2004 23:00:00 11.2 1.2 96 1197
Sensor_NO2
1 1692
2 1559
3 1555
4 1584
5 1490
6 1393Transforming Wide Data to Long Format
I will use pivot_longer() to transform the carbon monoxide, nitrogen dioxide, and their corresponding sensor response columns from wide to long format. This will create two new columns: Variable, which identifies the measurement type, and Value, which contains the corresponding value. The Date, Time, and Temperature columns will remain unchanged.
Code
air_long <- air_renamed %>%
pivot_longer(
cols = c(
CarbonMonoxide,
NitrogenDioxide,
Sensor_CO,
Sensor_NO2
),
names_to = "Variable",
values_to = "Value"
)
head(air_long)# A tibble: 6 × 5
Date Time Temperature Variable Value
<chr> <chr> <dbl> <chr> <dbl>
1 10.03.2004 18:00:00 13.6 CarbonMonoxide 2.6
2 10.03.2004 18:00:00 13.6 NitrogenDioxide 113
3 10.03.2004 18:00:00 13.6 Sensor_CO 1360
4 10.03.2004 18:00:00 13.6 Sensor_NO2 1692
5 10.03.2004 19:00:00 13.3 CarbonMonoxide 2
6 10.03.2004 19:00:00 13.3 NitrogenDioxide 92 Missing Values
After transforming the dataset into long format, I will check for missing values. I will use is.na() to count the missing values for each measurement type. This will help me understand how many measurements are missing before cleaning the dataset.
Code
air_long %>%
group_by(Variable) %>%
summarise(
Missing_Count = sum(is.na(Value)),
.groups = "drop"
)# A tibble: 4 × 2
Variable Missing_Count
<chr> <int>
1 CarbonMonoxide 114
2 NitrogenDioxide 114
3 Sensor_CO 114
4 Sensor_NO2 114The results show that all four measurement types contain 114 missing values each, for a total of 456 missing values. These missing values need to be handled before performing the analysis. However, the dataset also uses -200 to represent missing measurements, so I will examine these values separately.
Removing Missing Values
After identifying the missing values, I will use drop_na() to remove rows where the measurement value is missing. I will save the cleaned data as air_clean and use it for the next cleaning steps. The original dataset will remain unchanged.
Code
air_clean <- air_long %>%
drop_na(Value)
head(air_clean)# A tibble: 6 × 5
Date Time Temperature Variable Value
<chr> <chr> <dbl> <chr> <dbl>
1 10.03.2004 18:00:00 13.6 CarbonMonoxide 2.6
2 10.03.2004 18:00:00 13.6 NitrogenDioxide 113
3 10.03.2004 18:00:00 13.6 Sensor_CO 1360
4 10.03.2004 18:00:00 13.6 Sensor_NO2 1692
5 10.03.2004 19:00:00 13.3 CarbonMonoxide 2
6 10.03.2004 19:00:00 13.3 NitrogenDioxide 92 Checking Date and Time Formats
Before continuing with data cleaning, I will examine the date and time variables. Both Date and Time were imported as character variables. I will check their formats to determine how they should be converted for the monthly air quality trend analysis.
Code
air_clean %>%
select(Date, Time) %>%
head()# A tibble: 6 × 2
Date Time
<chr> <chr>
1 10.03.2004 18:00:00
2 10.03.2004 18:00:00
3 10.03.2004 18:00:00
4 10.03.2004 18:00:00
5 10.03.2004 19:00:00
6 10.03.2004 19:00:00Converting Date and Time Formats
The measurement dates are stored in DD.MM.YYYY format, and the measurement times are stored in HH:MM format. Both variables were imported as character values. I will use as.Date() to convert the Date column into Date format and hms::as_hms() to convert the Time column into a time format. This will make the data suitable for further analysis of air quality measurements over time.
Code
air_clean <- air_clean %>%
mutate(
Date = as.Date(Date, format = "%d.%m.%Y"),
Time = hms::as_hms(Time)
)
head(air_clean)# A tibble: 6 × 5
Date Time Temperature Variable Value
<date> <time> <dbl> <chr> <dbl>
1 2004-03-10 18:00 13.6 CarbonMonoxide 2.6
2 2004-03-10 18:00 13.6 NitrogenDioxide 113
3 2004-03-10 18:00 13.6 Sensor_CO 1360
4 2004-03-10 18:00 13.6 Sensor_NO2 1692
5 2004-03-10 19:00 13.3 CarbonMonoxide 2
6 2004-03-10 19:00 13.3 NitrogenDioxide 92 Checking Invalid Values (-200)
After converting the date and time variables, I will check for invalid measurements recorded as -200. According to the UCI Air Quality dataset documentation, -200 represents missing measurements. I will count these values for each measurement type before removing them from the analysis.
Code
air_clean %>%
group_by(Variable) %>%
summarise(
Invalid_Count = sum(Value == -200, na.rm = TRUE),
.groups = "drop"
)# A tibble: 4 × 2
Variable Invalid_Count
<chr> <int>
1 CarbonMonoxide 1683
2 NitrogenDioxide 1642
3 Sensor_CO 366
4 Sensor_NO2 366The results show that all four measurement types contain invalid values recorded as -200. Carbon monoxide has the highest number of invalid values (1,683), followed by nitrogen dioxide (1,642). Both sensor response variables contain 366 invalid values each. In total, I identified 4,057 invalid measurement values. I will exclude these values from the analysis because -200 represents missing measurements rather than actual air quality readings.
Handling Invalid Values
After identifying the invalid values, I will remove rows where the measurement value is -200. I will not replace these values with zero because this would affect the summary statistics and trend analysis. I will keep the original dataset unchanged and use the cleaned data for further analysis.
Code
air_tidy <- air_clean %>%
filter(Value != -200)
head(air_tidy)# A tibble: 6 × 5
Date Time Temperature Variable Value
<date> <time> <dbl> <chr> <dbl>
1 2004-03-10 18:00 13.6 CarbonMonoxide 2.6
2 2004-03-10 18:00 13.6 NitrogenDioxide 113
3 2004-03-10 18:00 13.6 Sensor_CO 1360
4 2004-03-10 18:00 13.6 Sensor_NO2 1692
5 2004-03-10 19:00 13.3 CarbonMonoxide 2
6 2004-03-10 19:00 13.3 NitrogenDioxide 92 Summary Statistics
After cleaning the dataset, I will calculate summary statistics for carbon monoxide, nitrogen dioxide, and their corresponding sensor responses. I will use the tidy dataset to calculate the number of observations, mean, median, minimum, maximum, and standard deviation for each measurement type. These statistics will help me understand the distribution and variability of air quality measurements.
Code
air_summary <- air_tidy %>%
group_by(Variable) %>%
summarise(
Count = n(),
Mean = round(mean(Value), 2),
Median = round(median(Value), 2),
Min = round(min(Value), 2),
Max = round(max(Value), 2),
SD = round(sd(Value), 2),
.groups = "drop"
)
air_summary# A tibble: 4 × 7
Variable Count Mean Median Min Max SD
<chr> <int> <dbl> <dbl> <dbl> <dbl> <dbl>
1 CarbonMonoxide 7674 2.15 1.8 0.1 11.9 1.45
2 NitrogenDioxide 7715 113. 109 2 340 48.4
3 Sensor_CO 8991 1100. 1063 647 2040 217.
4 Sensor_NO2 8991 1456. 1463 551 2775 346. The summary statistics show that carbon monoxide has 7,674 valid observations, with a mean of 2.15 and a median of 1.8. Nitrogen dioxide has 7,715 valid observations, with a mean of 113.09 and a median of 109. The CO and NO2 sensors each have 8,991 valid observations. The sensor measurements have different scales from the pollutant concentrations, so their values should not be compared directly. The differences in observation counts also show that some measurements were missing or invalid.
Monthly Air Pollution Trends
To examine changes in air pollution over time, I will calculate the monthly average concentrations of carbon monoxide and nitrogen dioxide. I will use the cleaned dataset and group the observations by month and pollutant type. This will help me identify how pollution levels change over time.
Code
air_monthly <- air_tidy %>%
filter(Variable %in% c("CarbonMonoxide", "NitrogenDioxide")) %>%
mutate(Month = format(Date, "%Y-%m")) %>%
group_by(Month, Variable) %>%
summarise(
Mean = round(mean(Value), 2),
Count = n(),
.groups = "drop"
)
air_monthly# A tibble: 28 × 4
Month Variable Mean Count
<chr> <chr> <dbl> <int>
1 2004-03 CarbonMonoxide 2.29 492
2 2004-03 NitrogenDioxide 103. 488
3 2004-04 CarbonMonoxide 2.3 495
4 2004-04 NitrogenDioxide 96.9 486
5 2004-05 CarbonMonoxide 1.94 592
6 2004-05 NitrogenDioxide 93.2 598
7 2004-06 CarbonMonoxide 1.9 644
8 2004-06 NitrogenDioxide 94.1 648
9 2004-07 CarbonMonoxide 1.78 558
10 2004-07 NitrogenDioxide 104. 711
# ℹ 18 more rowsMonthly Air Pollution Trends Visualization
I will create line charts to visualize the monthly average concentrations of carbon monoxide and nitrogen dioxide. I will use separate panels because the two pollutants have different measurement units and scales. These charts will help me identify changes and possible seasonal patterns in air pollution levels.
Code
ggplot(air_monthly,
aes(x = Month, y = Mean, group = Variable)) +
geom_line(color = "steelblue", linewidth = 1) +
geom_point(color = "steelblue", size = 2) +
facet_wrap(~Variable, scales = "free_y",
labeller = as_labeller(c(
CarbonMonoxide = "Carbon Monoxide (mg/m3)",
NitrogenDioxide = "Nitrogen Dioxide (ug/m3)"
))) +
labs(
title = "Monthly Average Air Pollution Levels",
x = "Month",
y = "Average Concentration"
) +
theme_minimal() +
theme(
axis.text.x = element_text(angle = 45, hjust = 1)
)
The line charts show that carbon monoxide and nitrogen dioxide concentrations changed over time. Carbon monoxide levels decreased during the summer of 2004, reached higher values between October and December 2004, and then decreased again in early 2005. Nitrogen dioxide showed a different pattern, with its lowest monthly average in August 2004 and its highest in February 2005. Overall, both pollutants showed monthly fluctuations, but their patterns were not identical. These results suggest possible seasonal differences in air pollution levels.
Conclusion
In this project, I analyzed the UCI Air Quality dataset to examine changes in carbon monoxide (CO) and nitrogen dioxide (NO2) concentrations over time. I selected relevant variables, transformed the data from wide to long format, and handled missing and invalid measurement values.
The summary statistics showed that the average CO concentration was 2.15 mg/m3, while the average NO2 concentration was 113.09 ug/m3. The number of valid observations differed between variables because some measurements were missing or invalid.
The monthly analysis showed that CO concentrations were relatively high between October and December 2004, while NO2 reached its highest monthly average in February 2005. Both pollutants showed changes over time, but their patterns were different.
Overall, this analysis helped me understand how data cleaning and transformation can support air quality analysis. The results suggest possible seasonal patterns, but additional data and analysis would be needed to determine the factors behind these changes.
Citation
I used ChatGPT (GPT-5.6 Luna), developed by OpenAI, to help me understand some of the functions I used in this assignment and to check my work.
OpenAI. (2026). ChatGPT (GPT-5.6 Luna) [Large language model]. [https://chatgpt.com/ Accessed October 9, 2026].
Video Explainer
Link: