Reproducible Research PA2
Charlie Zuo
Monday, February 16, 2015
U.S. NOAA Storm Data Analysis
Synopsis
Severe weather conditions can cause both public health and economic problems for communities and municipalities across the U.S..In this research project, we are aiming to analyze the storm data from U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database, which tracks characteristics of major storms and weather events. We will look to address two questions using this data: 1. Across the U.S., which type of events (as indicated in the EVTYPE variable) are most harmful with respect to population health? 2. Across the United States, which types of events have the greatest economic consequences?
Data Processing
This section will process the dataset and read it into environment to prepare for analysis
setwd("C:/Users/Charlie/Desktop/Data Science/Reproducible Research")
library(plyr)
library(reshape2)
library(R.utils)
#bunzip2("repdata-data-StormData.csv.bz2", "stormData.csv", remove=FALSE)
stormData <- read.csv("stormData.csv", sep = ",")
head(stormData)## STATE__ BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE
## 1 1 4/18/1950 0:00:00 0130 CST 97 MOBILE AL
## 2 1 4/18/1950 0:00:00 0145 CST 3 BALDWIN AL
## 3 1 2/20/1951 0:00:00 1600 CST 57 FAYETTE AL
## 4 1 6/8/1951 0:00:00 0900 CST 89 MADISON AL
## 5 1 11/15/1951 0:00:00 1500 CST 43 CULLMAN AL
## 6 1 11/15/1951 0:00:00 2000 CST 77 LAUDERDALE AL
## EVTYPE BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END
## 1 TORNADO 0 0
## 2 TORNADO 0 0
## 3 TORNADO 0 0
## 4 TORNADO 0 0
## 5 TORNADO 0 0
## 6 TORNADO 0 0
## COUNTYENDN END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES
## 1 NA 0 14.0 100 3 0 0
## 2 NA 0 2.0 150 2 0 0
## 3 NA 0 0.1 123 2 0 0
## 4 NA 0 0.0 100 2 0 0
## 5 NA 0 0.0 150 2 0 0
## 6 NA 0 1.5 177 2 0 0
## INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES
## 1 15 25.0 K 0
## 2 0 2.5 K 0
## 3 2 25.0 K 0
## 4 2 2.5 K 0
## 5 2 2.5 K 0
## 6 6 2.5 K 0
## LATITUDE LONGITUDE LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1 3040 8812 3051 8806 1
## 2 3042 8755 0 0 2
## 3 3340 8742 0 0 3
## 4 3458 8626 0 0 4
## 5 3412 8642 0 0 5
## 6 3450 8748 0 0 6Exploratory Analysis
We will look at the data first to see what variables are in the dataset, and in particular for the fields that will be used to answer our two questions.
str(stormData)## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
## $ BGN_TIME : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
## $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
## $ STATE : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ EVTYPE : Factor w/ 985 levels " HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : Factor w/ 35 levels ""," N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_LOCATI: Factor w/ 54429 levels "","- 1 N Albion",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_DATE : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_TIME : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_LOCATI: Factor w/ 34506 levels "","- .5 NNW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ WFO : Factor w/ 542 levels ""," CI","$AC",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ ZONENAMES : Factor w/ 25112 levels ""," "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : Factor w/ 436774 levels "","-2 at Deer Park\n",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...We found that the following columns are of interest to us when trying to answer these questions: EVTYPE, FATALITIES, INJURIES, PROPDMG, PROPDMGEXP, CROPDMG, CROPDMGEXP. Thus, we will take this subset of data and look at it more closely.
col <- c("EVTYPE", "FATALITIES", "INJURIES", "PROPDMG", "PROPDMGEXP", "CROPDMG", "CROPDMGEXP")
storm2 <- stormData[,col]Weather Event Impact on Public Health
We will take a look at the type of weather event that has had major impact on public health by looking at EVTYPE, FATALITIES, INJURIES variables in the dataset.
fatal <- aggregate(FATALITIES ~ EVTYPE, data = storm2, FUN = sum)
injury <- aggregate(INJURIES ~ EVTYPE, data = storm2, FUN = sum)
# rank the top 10 weather events in terms of fatalities and injuries
topfatal <- fatal[order(-fatal$FATALITIES), ][1:10, ]
topinjury <- injury[order(-injury$INJURIES), ][1:10, ]
par(mfrow = c(1, 2), mar = c(12, 4, 3, 2), mgp = c(3, 1, 0), cex = 0.9)
barplot(topfatal$FATALITIES, las = 3, names.arg = topfatal$EVTYPE, main = "Weather Events With The Top 10 Highest Fatalities", ylab = "number of fatalities", col = "gray")
barplot(topinjury$INJURIES, las = 3, names.arg = topinjury$EVTYPE, main = "Weather Events With the Top 10 Highest Injuries", ylab = "number of injuries", col = "blue")
Weather Event Impact on Economy
Impact on Properties
We will prepare the PROPDMG, PROPDMGEXP data for our analysis on weather events’ impact that causes property damages.
unique(storm2$PROPDMGEXP)## [1] K M B m + 0 5 6 ? 4 2 3 h 7 H - 1 8
## Levels: - ? + 0 1 2 3 4 5 6 7 8 B h H K m M# convert data in PROPDMGEXP to quantitative values
storm2$PROPEXP[storm2$PROPDMGEXP == "K"] <- 1000
storm2$PROPEXP[storm2$PROPDMGEXP == "M"] <- 1000000
storm2$PROPEXP[storm2$PROPDMGEXP == ""] <- 1
storm2$PROPEXP[storm2$PROPDMGEXP == "B"] <- 1000000000
storm2$PROPEXP[storm2$PROPDMGEXP == "m"] <- 1000000
storm2$PROPEXP[storm2$PROPDMGEXP == "0"] <- 1
storm2$PROPEXP[storm2$PROPDMGEXP == "5"] <- 100000
storm2$PROPEXP[storm2$PROPDMGEXP == "6"] <- 1000000
storm2$PROPEXP[storm2$PROPDMGEXP == "4"] <- 10000
storm2$PROPEXP[storm2$PROPDMGEXP == "2"] <- 100
storm2$PROPEXP[storm2$PROPDMGEXP == "3"] <- 1000
storm2$PROPEXP[storm2$PROPDMGEXP == "h"] <- 100
storm2$PROPEXP[storm2$PROPDMGEXP == "7"] <- 10000000
storm2$PROPEXP[storm2$PROPDMGEXP == "H"] <- 100
storm2$PROPEXP[storm2$PROPDMGEXP == "1"] <- 10
storm2$PROPEXP[storm2$PROPDMGEXP == "8"] <- 100000000
storm2$PROPEXP[storm2$PROPDMGEXP == "+"] <- 0
storm2$PROPEXP[storm2$PROPDMGEXP == "-"] <- 0
storm2$PROPEXP[storm2$PROPDMGEXP == "?"] <- 0
# Computing the amount of total property damage for each weather event
storm2$PROPDMGVAL <- storm2$PROPDMG * storm2$PROPEXPImpact on Crops
unique(storm2$CROPDMGEXP)## [1] M K m B ? 0 k 2
## Levels: ? 0 2 B k K m M# convert data in CROPDMGEXP to quantitative values
storm2$CROPEXP[storm2$CROPDMGEXP == "M"] <- 1000000
storm2$CROPEXP[storm2$CROPDMGEXP == "m"] <- 1000000
storm2$CROPEXP[storm2$CROPDMGEXP == "K"] <- 1000
storm2$CROPEXP[storm2$CROPDMGEXP == "k"] <- 1000
storm2$CROPEXP[storm2$CROPDMGEXP == "B"] <- 1000000000
storm2$CROPEXP[storm2$CROPDMGEXP == "0"] <- 1
storm2$CROPEXP[storm2$CROPDMGEXP == "2"] <- 100
storm2$CROPEXP[storm2$CROPDMGEXP == "?"] <- 0
# Compute total damage to crops
storm2$CROPDMGVAL <- storm2$CROPDMG * storm2$CROPEXPNow, we quantify and visualize the total amount damage in $ to crops and properties
propdmg <- aggregate(PROPDMGVAL ~ EVTYPE, data = storm2, FUN = sum)
cropdmg <- aggregate(CROPDMGVAL ~ EVTYPE, data = storm2, FUN = sum)
# ranking top 10 events with highest amount in property damage
toppropdmg <- propdmg[order(-propdmg$PROPDMGVAL), ][1:10, ]
# ranking top 10 events with highest amount in crop damage
topcropdmg <- cropdmg[order(-cropdmg$CROPDMGVAL), ][1:10, ]
par(mfrow = c(1, 2), mar = c(12, 4, 3, 2), mgp = c(3, 1, 0), cex = 0.8)
barplot(toppropdmg$PROPDMGVAL/(10^9), las = 3, names.arg = toppropdmg$EVTYPE, main = "Top 10 Events with Highest Amount of Damages to Properties", ylab = "Total Cost of damages in billions", col = "lightblue1")
barplot(topcropdmg$CROPDMGVAL/(10^9), las = 3, names.arg = topcropdmg$EVTYPE, main = "Top 10 Events With Highst Amount Damages to Crops", ylab = "Total Cost of damages in billions", col = "lightblue4")
Results
From the research above, we can conclude the following weather events that are most harmful (cause the most damages) to public health are:
Tornado, Excessive Heat, Flood
Also derived from our research, we can conclude that the following weather events have the greatest economic consequences:
Flood, Drought, Tornado and Hurricanes