Exploration of Air Quality Dataset

This data set has daily air quality measurements in New York from May to September 1973 over a period of 5 months.

Variable	Type	Unit	Description	Detail
Ozone	numeric	parts per billion	mean Ozone concentration	Roosevelt island
Solar.R	numeric		Solar radiaion	central park
Wind	numeric	miles per hour	average wind speed	LaGuardia airport
Temp	numeric	Fahrenheit	maximum daily temperature	LaGuardia airport
Month	numeric		Month of observation	5 values
Day	numeric		day of month	153 days total

Necessary packages

require(ggplot2)
require(moments)

Let’s make a copy of the dataset for the following analysis.

aq <- data.frame(airquality)

Names of columns

colnames(aq)

[1] "Ozone"   "Solar.R" "Wind"    "Temp"    "Month"   "Day"    

Let’s look at some of the rows:

head(aq)

Several entries are not available. Let’s summarize:

colSums(is.na(aq))

  Ozone Solar.R    Wind    Temp   Month     Day 
     37       7       0       0       0       0 

Day should be a factor variable

aq$Day <- factor(aq$Day, levels=c(1:31), ordered=TRUE)

Month should be a factor variable

aq$Month <- factor(aq$Month, levels=5:9, labels=month.abb[5:9], ordered=TRUE)

Summary statistiscs for all the variables:

summary(aq)

     Ozone           Solar.R           Wind             Temp      
 Min.   :  1.00   Min.   :  7.0   Min.   : 1.700   Min.   :56.00  
 1st Qu.: 18.00   1st Qu.:115.8   1st Qu.: 7.400   1st Qu.:72.00  
 Median : 31.50   Median :205.0   Median : 9.700   Median :79.00  
 Mean   : 42.13   Mean   :185.9   Mean   : 9.958   Mean   :77.88  
 3rd Qu.: 63.25   3rd Qu.:258.8   3rd Qu.:11.500   3rd Qu.:85.00  
 Max.   :168.00   Max.   :334.0   Max.   :20.700   Max.   :97.00  
 NA's   :37       NA's   :7                                       
 Month         Day     
 May:31   1      :  5  
 Jun:30   2      :  5  
 Jul:31   3      :  5  
 Aug:31   4      :  5  
 Sep:30   5      :  5  
          6      :  5  
          (Other):123  

1 Variables

1.1 Ozone

summary(aq$Ozone)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
   1.00   18.00   31.50   42.13   63.25  168.00      37 

Histogram

ggplot(data=aq) + geom_histogram(mapping=aes(Ozone))

The histogram appears assymetric.

We can check the normalcy of the distribution by measuring its skewness and kurtosis

skewness(aq$Ozone, na.rm = T)

[1] 1.225681

The data is skewed heavily towards right.

Let’s also measure kurtosis:

kurtosis(aq$Ozone, na.rm=T)

[1] 4.184071

This is leptokurtic.

Let’s apply a transformation to see if the histogram becomes more symmetric:

aq$Ozone.sqrt <- sqrt(aq$Ozone)
ggplot(data=aq) + geom_histogram(mapping=aes(Ozone.sqrt))

We have stored the square root of Ozone concentration values in the aq dataframe itself as a new column. This is indeed more symmetric and looking like normal distribution.

Let’s verify the skewness:

skewness(aq$Ozone.sqrt, na.rm=T)

[1] 0.5143488

Also the kurtosis:

kurtosis(aq$Ozone.sqrt, na.rm=T)

[1] 2.561186

There is indeed improvement in skewness.

Seasonal variation in data:

qplot(Month, Ozone, data=aq, geom="boxplot", color=Month)

1.2 Solar.R

summary(aq$Solar.R)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
    7.0   115.8   205.0   185.9   258.8   334.0       7 

Histogram

ggplot(data=aq) + geom_histogram(mapping=aes(Solar.R))

Seasonal variation in data:

qplot(Month, Solar.R, data=aq, geom="boxplot", color=Month)

1.3 Wind

summary(aq$Wind)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  1.700   7.400   9.700   9.958  11.500  20.700 

Histogram

ggplot(data=aq) + geom_histogram(mapping=aes(Wind))

Let’s verify the normalcy of the distribution

skewness(aq$Wind)

[1] 0.3443985

kurtosis(aq$Wind)

[1] 3.068849

This is very close to a normal distribution.

Seasonal variation in data:

qplot(Month, Wind, data=aq, geom="boxplot", color=Month)

1.4 Temp

summary(aq$Temp)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  56.00   72.00   79.00   77.88   85.00   97.00 

Seasonal variation in data:

qplot(Month, Temp, data=aq, geom="boxplot", color=Month)

1.5 Month

table(aq$Month)

May Jun Jul Aug Sep 
 31  30  31  31  30 

This just reflects the number of days in each month from May to September.

1.6 Day

table(aq$Day)

 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 
 5  5  5  5  5  5  5  5  5  5  5  5  5  5  5  5  5  5  5  5  5  5  5  5 
25 26 27 28 29 30 31 
 5  5  5  5  5  5  3 

As expected, day 31 appears only 3 times.

2 Correlations

Let us find out the Pearson correlation coefficients between the different variables in the dataset:

# Only numerical variables can be used for correlation
columns <- c('Ozone.sqrt', 'Solar.R', 'Wind', 'Temp')
# We identify non-na rows as rows with na values cause problems 
rows <- rowSums(is.na(aq)) == 0
# Compute correlation of variables and round to 2 digits
round(cor(aq[rows, columns]), 2)

           Ozone.sqrt Solar.R  Wind  Temp
Ozone.sqrt       1.00    0.40 -0.61  0.75
Solar.R          0.40    1.00 -0.13  0.29
Wind            -0.61   -0.13  1.00 -0.50
Temp             0.75    0.29 -0.50  1.00

• Solar Radiation is moderately correlated with ozone concentration.
• wind and ozone seem strongly correlated.
• Temperature and ozone are strongly correlated.
• Very weak correlation between wind and solar radiation.
• Temperature and solar radition are weakly correlated.
• Temperature and wind are moderately correlated.

Let us verify these correlations using scatter plots.

2.1 Very weak correlation:

qplot(Solar.R, Wind, data=aq, geom="point", color=Month)

2.2 Weak correlation:

qplot(Temp, Solar.R, data=aq, geom="point", color=Month)

2.3 Moderate correlation:

qplot(Solar.R, Ozone.sqrt, data=aq, color=Month, geom="point")

qplot(Temp, Wind, data=aq, color=Month, geom="point")

2.4 Strong correlation

qplot(Wind, Ozone.sqrt, data=aq, color=Month, geom='point')

qplot(Temp, Ozone.sqrt, data=aq, color=Month, geom='point')