R Notebook

https://github.com/rinckd/math-creativity/blob/0b33fdfb200e7a2524a154e97b9749bd215471c9/quiz_1.R https://github.com/SkillSmart/Coursera_SoftwareDevelopment-in-R/blob/2643b77a18b9fa8b05d1043e7e71be5c8debbd51/RProgrammingEnv/Week4/Quiz.Rmd https://github.com/yufree/democode/blob/0f307fc61e7b61d99a61ba37082378d008055589/tidyverse/hw.Rmd

Mastering R Class 1: Final Quiz

library(readr)
library(dplyr)
library(tidyr)
library(readxl)

Introduction

The goal of this assignment is to take datasets that are either messy or simply not tidy and to make them tidy datasets. The objective is to gain some familiarity with the functions for reading in data into R and calculating basic summary statistics on the data. In particular, we will make use of the following packages: dplyr, tidyr, readr, and readxl.

Before staring the quiz you will need to download the data for the quiz, which can be found in the file quiz_data.zip. The zip archive file contains two files:

daily_SPEC_2014.csv.bz2: a compressed CSV file containing daily measurements of particulate matter chemical constituents in the United States for the year 2014. Note that you should NOT have to decompress this file. The data are measured at a network of federal, state, and local monitors and assembled by the EPA. In this dataset, the “Arithmetic Mean” column provides the level of the indicated chemical constituent and the “Parameter.Name” column provides the name of the chemical constituent. The combination of a “State Code”, a “County Code”, and a “Site Num”, uniquely identifies a monitoring site (the location of which is provided by the “Latitude” and “Longitude” columns).

if(!file.exists("./data/pollution.csv")){
pollution <- read_csv("data/daily_SPEC_2014.csv.bz2")
write_csv(pollution, "./data/pollution.csv")
}
pollution <- read_csv("./data/pollution.csv")

aqs_sites.xlsx: An excel spreadsheet containing metadata about each of the monitoring sites in the United States where pollution measurements are made. In particular, the “Land Use” and “Location Setting” variables contain information about what kinds of areas the monitors are located in (i.e. “residential” vs. “forest”). Once the data have been downloaded to your working directory, you can begin the quiz assignment. For this assignment, you may want to review Sections 1.2 through 1.5 of Mastering Software Development in R.

1.Question

Use the readr package to read the daily_SPEC_2014.csv.bz2 data file in to R. This file contains daily levels of fine particulate matter (PM2.5) chemical constituents across the United States. The data are measured at a network of federal, state, and local monitors and assembled by the EPA.

In this dataset, the “Sample.Value” (see bolded words a couple of paragraphs before) column provides the level of the indicated chemical constituent and the “Parameter.Name” column provides the name of the chemical constituent. The combination of a “State.Code”, a “County.Code”, and a “Site.Num”, uniquely identifies a monitoring site (the location of which is provided by the “Latitude” and “Longitude” columns).

For all of the questions below, you can ignore the missing values in the dataset, so when taking averages, just remove the missing values before taking the average (i.e. you can use na.rm = TRUE in the mean() function)

What is average Sample.Value for “Bromine PM2.5 LC” in the state of Wisconsin in this dataset?

Q1 <- pollution %>% filter(`Parameter Name`== "Bromine PM2.5 LC", `State Name` == "Wisconsin")
Q1 <- pollution %>% summarise(avg = mean(`Arithmetic Mean`, na.rm=TRUE))
Q1

2.Question

Calculate the average of each chemical constituent across all states, monitoring sites and all time points.

Which constituent Parameter.Name has the highest average level?

Q2 <-  pollution %>% 
  group_by(`Parameter Name`,`State Name`, `Site Num`, `Date Local`) %>%
        summarise(avg = mean(`Arithmetic Mean`)) %>%
        arrange(desc(avg))
Q2

Question 3

Which monitoring site has the highest average level of “Sulfate PM2.5 LC” across all time?

Indicate the state code, county code, and site number.

Q3_1 <-  pollution %>% filter(`Parameter Name` == "Sulfate PM2.5 LC") %>%
        group_by(`Site Num`,`County Code`,`State Code`) %>%
        summarise(avg = mean(`Arithmetic Mean`)) %>%
        arrange(desc(avg))
Q3_1

Find the mean concentration for all pollutants over time in one City

Q3_2 <- pollution %>% filter(`City Name` == 'Charlotte') %>% summarise(avg = mean(`Arithmetic Mean`))
Q3_2

Question 4

What is the absolute difference in the average levels of “EC PM2.5 LC TOR” between the states California and Arizona, across all time and all monitoring sites?

# Calculating the Pollutant Means for California, Arizona
Q4 <- pollution %>% filter(`State Name` %in% c("California", "Arizona"), `Parameter Name`=="EC PM2.5 LC TOR") %>% 
  group_by(`State Name`) %>% 
  summarise(avg_concentration = mean(`Arithmetic Mean`))
# Calculating the absolute difference
Q4 <- abs(Q4[1,2]-Q4[2,2])
Q4

Question 5

What is the median level of “OC PM2.5 LC TOR” in the western United States, across all time? Define western as any monitoring location that has a Longitude LESS THAN -100.

Q5 <- pollution %>% filter(Longitude < -100, `Parameter Name` == "OC PM2.5 LC TOR") %>% 
  summarise(med_level = median(`Arithmetic Mean`))
Q5

Question 6

Use the readxl package to read the file aqs_sites.xlsx into R (you may need to install the package first). This file contains metadata about each of the monitoring sites in the EPA’s monitoring system. In particular, the “Land Use” and “Location Setting” variables contain information about what kinds of areas the monitors are located in (i.e. “residential” vs. “forest”).

sites <- read_excel("./data/aqs_sites.xlsx")

How many monitoring sites are labelled as both RESIDENTIAL for “Land Use” and SUBURBAN for “Location Setting”?

Q6 <- sites %>% filter(`Land Use` == "RESIDENTIAL", `Location Setting` =="SUBURBAN") %>% summarise(n())
Q6

Question 7

What is the median level of “EC PM2.5 LC TOR” amongst monitoring sites that are labelled as both “RESIDENTIAL” and “SUBURBAN” in the eastern U.S., where eastern is defined as Longitude greater than or equal to -100?

In order to join the tables we have to set the codes the same

# Check if the identifiers are the same
names(pollution)[1:5] == names(sites)[1:5]

[1]  TRUE  TRUE FALSE FALSE FALSE

str(pollution[1:5])

Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   2098160 obs. of  5 variables:
 $ State Code    : chr  "01" "01" "01" "01" ...
 $ County Code   : chr  "003" "003" "003" "003" ...
 $ Site Num      : chr  "0010" "0010" "0010" "0010" ...
 $ Parameter Code: int  68105 68105 68105 68105 68105 68105 68105 68105 68105 68105 ...
 $ POC           : int  1 1 1 1 1 1 1 1 1 1 ...

str(sites[1:5])

Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   20239 obs. of  5 variables:
 $ State Code : num  1 1 1 1 1 1 1 1 1 1 ...
 $ County Code: num  1 1 1 3 3 3 3 5 5 7 ...
 $ Site Number: num  1 2 3 1 2 3 10 1 2 1 ...
 $ Latitude   : num  32.4 32.4 32.3 0 30.6 ...
 $ Longitude  : num  -86.5 -86.4 -86.8 0 -87.7 ...

Need to change a few variables in the pollution data

pollution$`State Code` <- as.numeric(pollution$`State Code`)
pollution$`County Code` <- as.numeric(pollution$`County Code`)
pollution$`Site Num` <- as.numeric(pollution$`Site Num`)

Now you can join the two DFs on (State, County, SiteId)

joined_tbl <- inner_join(sites, pollution)

Joining, by = c("State Code", "County Code", "Latitude", "Longitude", "Datum", "Local Site Name", "Address", "State Name", "County Name", "City Name", "CBSA Name")

summary(joined_tbl$`Arithmetic Mean`)

    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
 -99.000    0.001    0.196  138.900   19.900 5453.000 

Q7 <- joined_tbl %>% 
  group_by(`State Code`, `County Code`,`Parameter Name`) %>% 
  summarise(avg_concentration = mean(`Arithmetic Mean`)) %>% 
  arrange(desc(avg_concentration))

Calculating the median level

Q7 <- joined_tbl %>% group_by(`State Code`, `County Code`, `Site Number`, `Parameter Name`) %>%
     filter(`Location Setting`== "SUBURBAN", `Land Use`=="RESIDENTIAL", 
            `Parameter Name`== "EC PM2.5 LC TOR", Longitude >= -100) %>%
     summarise(avg_concentration = mean(`Arithmetic Mean`))
Q7

Question 8

Amongst monitoring sites that are labeled as COMMERCIAL for “Land Use”, which month of the year has the highest average levels of “Sulfate PM2.5 LC”?

library(lubridate)

package ‘lubridate’ was built under R version 3.2.5
Attaching package: ‘lubridate’

The following object is masked from ‘package:base’:

    date

Q8 <- joined_tbl %>% filter(`Land Use`=="COMMERCIAL", `Parameter Name`=="Sulfate PM2.5 LC") %>%
     group_by(month(`Date Local`)) %>% 
     summarise(avg_concentration = mean(`Arithmetic Mean`)) %>%
     arrange(desc(avg_concentration))
Q8

Question 9:

Take a look at the data for the monitoring site identified by State Code 6, County Code 65, and Site Number 8001 (this monitor is in California). At this monitor, for how many days is the sum of “Sulfate PM2.5 LC” and “Total Nitrate PM2.5 LC” greater than 10?

For each of the chemical constituents, there will be some dates that have multiple Sample.Value’s at this monitoring site. When there are multiple values on a given date, take the average of the constituent values for that date.

Q9 <- joined_tbl %>% 
  filter(`State Code`==6, `County Code`==65, `Site Number`==8001, `Parameter Name` %in% c("Total Nitrate PM2.5 LC", "Sulfate PM2.5 LC")) %>% 
  group_by(`Site Number`, month = month(`Date Local`),`Parameter Name`, day = day(`Date Local`) ) %>% 
  summarise(avg_value = sum(mean(`Arithmetic Mean`))) %>% 
  filter(avg_value > 10) %>% 
  arrange(month,day) 
Q9

Question 10:

Which monitoring site in the dataset has the highest correlation between “Sulfate PM2.5 LC” and “Total Nitrate PM2.5 LC” across all dates? Identify the monitoring site by it’s State, County, and Site Number code.

For each of the chemical constituents, there will be some dates that have multiple Sample.Value’s at a monitoring site. When there are multiple values on a given date, take the average of the constituent values for that date.

Correlations between to variables can be computed with the cor() function.

Q10ANS <- mapply(Q10Function, Q10, numSites)

Error in filter_impl(.data, dots) : incorrect length (97), expecting: 1