Reproducible Research : Assignment_2
Samarkande
January 19, 2016
Data Analysis of Strong Weather Events in United States
Synopsis
The basic goal of this project is to explore the U.S. National Oceanic and Atmospheric Administration NOAA Storm Database (with the following National Weather Service documentation) from 1950 to November 2011 and address the following questions:
- Across the United States, which types of events (as indicated in the
EVTYPEvariable) are most harmful with respect to population health? - Across the United States, which types of events have the greatest economic consequences?
The present analysis evaluates effects of each weather events on the population health (fatalities and injuries) and economical consequences (property and crop damages) across the database’s range of years and when these events promtly occur (rates).
Population health: We found that tornados, excessive heats and floods are the three first responsibles of human fatalities. However, excessive heats, avalanche and strong water currents are the most fatality when they occur. Tornados are mainly responsibles of injuries. With these tornados, excessive heats and hurricanes make the highest injury rates.
Economical consequences: The floods, hurricane, strong water currents, tornados and ice are mainly the causality of economical damages, especially into the property class. However, hurricanes, water currents and droughts lead to the strongest rates. In opposite of the two first, droughts affect strongly the crop class.
Data Processing
Required libraries for this study:
suppressMessages(require(R.utils))## Warning: package 'R.utils' was built under R version 3.2.3## Warning: package 'R.oo' was built under R version 3.2.3## Warning: package 'R.methodsS3' was built under R version 3.2.3suppressMessages(require(dplyr))## Warning: package 'dplyr' was built under R version 3.2.2suppressMessages(require(tidyr))## Warning: package 'tidyr' was built under R version 3.2.2suppressMessages(require(ggplot2))This data set is downloaded and saved in the R working directory:
# create a data and figures repos
if(!file.exists('./data')){dir.create('./data')}
if(!file.exists('./figures')){dir.create('./figures')}
# Load the data set
if(!file.exists('./data/stormData.csv')){
fileUrl <- 'http://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2'
download.file(fileUrl,destfile='./data/stormData.csv.bz2',mode = 'wb')
bunzip2('data/stormData.csv.bz2')
}
# Read data
storm <- read.csv('./data/stormData.csv')
# Structure of the data set
str(storm)## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
## $ BGN_TIME : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
## $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
## $ STATE : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ EVTYPE : Factor w/ 985 levels " HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : Factor w/ 35 levels ""," N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_LOCATI: Factor w/ 54429 levels "","- 1 N Albion",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_DATE : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_TIME : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_LOCATI: Factor w/ 34506 levels "","- .5 NNW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ WFO : Factor w/ 542 levels ""," CI","$AC",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ ZONENAMES : Factor w/ 25112 levels ""," "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : Factor w/ 436774 levels "","-2 at Deer Park\n",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...We can reduce the size of the data set by considering essential variables for the present study. These variables are:
EVTYPE: Type of weather eventsFATALITIES: Number of fatalitiesINJURIES: Number of injuriesPROPDMG: Property damages (in USD)CROPDMG: Crop damages (in USD)PROPDMGEXP: Multiple according to property damagesCROPDMGEXP: Multiple according to crop damages
# Select our variables
mydata<- select(storm,EVTYPE,FATALITIES,INJURIES,PROPDMG,PROPDMGEXP,CROPDMG,CROPDMGEXP)levels(mydata$CROPDMGEXP)## [1] "" "?" "0" "2" "B" "k" "K" "m" "M"levels(mydata$PROPDMGEXP)## [1] "" "-" "?" "+" "0" "1" "2" "3" "4" "5" "6" "7" "8" "B" "h" "H" "K"
## [18] "m" "M"From the above, we assign numerical multiples and the corresponding property/crop damage costs (in million of USD) according to weather events:
# Assign dollars expenses units
mydata_2<-mydata %>%
mutate(PROPDMGEXP=
ifelse(grepl("b",PROPDMGEXP,ignore.case=T),10^9,
ifelse(grepl("m",PROPDMGEXP,ignore.case=T),10^6,
ifelse(grepl("k",PROPDMGEXP,ignore.case=T),10^3,
ifelse(grepl("h",PROPDMGEXP,ignore.case=T),10^2,
ifelse(grepl("0",PROPDMGEXP),10^0,
ifelse(grepl("1",PROPDMGEXP),10^1,
ifelse(grepl("2",PROPDMGEXP),10^2,
ifelse(grepl("3",PROPDMGEXP),10^3,
ifelse(grepl("4",PROPDMGEXP),10^4,
ifelse(grepl("5",PROPDMGEXP),10^5,
ifelse(grepl("6",PROPDMGEXP),10^6,
ifelse(grepl("7",PROPDMGEXP),10^7,
ifelse(grepl("8",PROPDMGEXP),10^8,
NA))))))))))))))%>%
mutate(CROPDMGEXP=
ifelse(grepl("b",CROPDMGEXP,ignore.case=T),10^9,
ifelse(grepl("m",CROPDMGEXP,ignore.case=T),10^6,
ifelse(grepl("k",CROPDMGEXP,ignore.case=T),10^3,
ifelse(grepl("h",CROPDMGEXP,ignore.case=T),10^2,
ifelse(grepl("0",CROPDMGEXP),10^0,
ifelse(grepl("1",CROPDMGEXP),10^1,
ifelse(grepl("2",CROPDMGEXP),10^2,
ifelse(grepl("3",CROPDMGEXP),10^3,
ifelse(grepl("4",CROPDMGEXP),10^4,
ifelse(grepl("5",CROPDMGEXP),10^5,
ifelse(grepl("6",CROPDMGEXP),10^6,
ifelse(grepl("7",CROPDMGEXP),10^7,
ifelse(grepl("8",CROPDMGEXP),10^8,
NA))))))))))))))%>%
mutate(USD_Property_Damage=PROPDMGEXP*PROPDMG/10^6, # Property Damage costs (in million of dollars unit)
USD_Crop_Damage=CROPDMGEXP*CROPDMG/10^6) # Crop Damage costs (in million of dollars unit)head(levels(mydata_2$EVTYPE),50)## [1] " HIGH SURF ADVISORY" " COASTAL FLOOD"
## [3] " FLASH FLOOD" " LIGHTNING"
## [5] " TSTM WIND" " TSTM WIND (G45)"
## [7] " WATERSPOUT" " WIND"
## [9] "?" "ABNORMAL WARMTH"
## [11] "ABNORMALLY DRY" "ABNORMALLY WET"
## [13] "ACCUMULATED SNOWFALL" "AGRICULTURAL FREEZE"
## [15] "APACHE COUNTY" "ASTRONOMICAL HIGH TIDE"
## [17] "ASTRONOMICAL LOW TIDE" "AVALANCE"
## [19] "AVALANCHE" "BEACH EROSIN"
## [21] "Beach Erosion" "BEACH EROSION"
## [23] "BEACH EROSION/COASTAL FLOOD" "BEACH FLOOD"
## [25] "BELOW NORMAL PRECIPITATION" "BITTER WIND CHILL"
## [27] "BITTER WIND CHILL TEMPERATURES" "Black Ice"
## [29] "BLACK ICE" "BLIZZARD"
## [31] "BLIZZARD AND EXTREME WIND CHIL" "BLIZZARD AND HEAVY SNOW"
## [33] "Blizzard Summary" "BLIZZARD WEATHER"
## [35] "BLIZZARD/FREEZING RAIN" "BLIZZARD/HEAVY SNOW"
## [37] "BLIZZARD/HIGH WIND" "BLIZZARD/WINTER STORM"
## [39] "BLOW-OUT TIDE" "BLOW-OUT TIDES"
## [41] "BLOWING DUST" "blowing snow"
## [43] "Blowing Snow" "BLOWING SNOW"
## [45] "BLOWING SNOW- EXTREME WIND CHI" "BLOWING SNOW & EXTREME WIND CH"
## [47] "BLOWING SNOW/EXTREME WIND CHIL" "BREAKUP FLOODING"
## [49] "BRUSH FIRE" "BRUSH FIRES"From the above, we propose to gather weather events into global events as follows:
# Assign weather events
mydata_3 <- mydata_2 %>%
mutate(Type_Events =
ifelse(grepl('AVALAN', ignore.case = T, EVTYPE), 'Avalanche',
ifelse(grepl('COLD|COOL|FREEZE|FROST|HYPOTH|LOW TEMP', ignore.case = T, EVTYPE), 'Cold',
ifelse(grepl('CURRENT|SEAS|SURF|SURGE|SWELL|TIDE|TSUNAMI|WAVE', ignore.case = T, EVTYPE), 'Current',
ifelse(grepl('DUST', ignore.case = T, EVTYPE), 'Dust Storm',
ifelse(grepl('DROUGHT|DRY', ignore.case = T, EVTYPE), 'Drought',
ifelse(grepl('FLOOD|WATER', ignore.case = T, EVTYPE), 'Flood',
ifelse(grepl('FIRE', ignore.case = T, EVTYPE), 'Fire',
ifelse(grepl('FOG', ignore.case = T, EVTYPE), 'Fog',
ifelse(grepl('HEAT|HIGH TEMP|WARM', ignore.case = T, EVTYPE), 'Heat',
ifelse(grepl('HURRICANE|TROPICAL|TYPHOON', ignore.case = T, EVTYPE), 'Hurricane',
ifelse(grepl('FREEZING|HAIL|ICE|ICY|GLAZE', ignore.case = T, EVTYPE), 'Ice',
ifelse(grepl('LIGHTNING', ignore.case = T, EVTYPE), 'Lightning',
ifelse(grepl('PRECIP|RAIN|SHOWER|WET', ignore.case = T, EVTYPE), 'Rain',
ifelse(grepl('SLIDE', ignore.case = T, EVTYPE), 'Land/Mudslide',
ifelse(grepl('BLIZZARD|SNOW|SLEET|WINT', ignore.case = T, EVTYPE), 'Snow',
ifelse(grepl('FUNNEL|TORNADO|WATERSPOUT|WATER SPOUT|WAYTERSPOUT', ignore.case = T, EVTYPE), 'Tornado',
ifelse(grepl('MICROBURST|WIND', ignore.case = T, EVTYPE), 'T.S. Wind',
ifelse(grepl('VOLCAN', ignore.case = T, EVTYPE), 'Volcano',
'Others'))))))))))))))))))) %>%
rename(Number_Fatalities=FATALITIES,Number_Injuries=INJURIES) %>% # Rename variables
select(Type_Events,Number_Fatalities,Number_Injuries,USD_Property_Damage,USD_Crop_Damage) # Select our final variables
# Assign factor to type of events
mydata_3$Type_Events<-factor(mydata_3$Type_Events)
head(mydata_3)## Type_Events Number_Fatalities Number_Injuries USD_Property_Damage
## 1 Tornado 0 15 0.0250
## 2 Tornado 0 0 0.0025
## 3 Tornado 0 2 0.0250
## 4 Tornado 0 2 0.0025
## 5 Tornado 0 2 0.0025
## 6 Tornado 0 6 0.0025
## USD_Crop_Damage
## 1 NA
## 2 NA
## 3 NA
## 4 NA
## 5 NA
## 6 NAsummary(mydata_3)## Type_Events Number_Fatalities Number_Injuries
## T.S. Wind:362130 Min. : 0.0000 Min. : 0.0000
## Ice :293019 1st Qu.: 0.0000 1st Qu.: 0.0000
## Flood : 86598 Median : 0.0000 Median : 0.0000
## Tornado : 67668 Mean : 0.0168 Mean : 0.1557
## Snow : 40042 3rd Qu.: 0.0000 3rd Qu.: 0.0000
## Lightning: 15775 Max. :583.0000 Max. :1700.0000
## (Other) : 37065
## USD_Property_Damage USD_Crop_Damage
## Min. : 0 Min. :0e+00
## 1st Qu.: 0 1st Qu.:0e+00
## Median : 0 Median :0e+00
## Mean : 1 Mean :2e-01
## 3rd Qu.: 0 3rd Qu.:0e+00
## Max. :115000 Max. :5e+03
## NA's :465948 NA's :618420Results
Effects on human health
- Fatalities study
We analyse fatalities according to the entire range of years (Sum_Fatalities) and to weather events (Rate_Fatalities):
# Group by events
by_event<-mydata_3%>%group_by(Type_Events)
# Fatalities resume
Fatalities_Res<-by_event %>%
summarize(Sum_Fatalities=sum(Number_Fatalities),Frequency_Events=n(),
Rate_Fatalities=Sum_Fatalities/Frequency_Events) %>%
gather(Sum_Rate_Fatalities,Values,Sum_Fatalities,Rate_Fatalities) %>%
select(Type_Events,Sum_Rate_Fatalities,Values)
str(Fatalities_Res)## Classes 'tbl_df', 'tbl' and 'data.frame': 38 obs. of 3 variables:
## $ Type_Events : Factor w/ 19 levels "Avalanche","Cold",..: 1 2 3 4 5 6 7 8 9 10 ...
## $ Sum_Rate_Fatalities: Factor w/ 2 levels "Sum_Fatalities",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ Values : num 225 468 1041 5 24 ...# Plot fatalities concerns
png('./figures/unnamed-chunk-9.png',width=800,height=600)
ggplot(data = Fatalities_Res,aes(x = reorder(Type_Events,-Values) ,y = Values))+
geom_bar(stat='identity',aes(fill=Type_Events))+
facet_grid(Sum_Rate_Fatalities~.,scale = 'free_y')+
theme_bw()+guides(fill=F)+
xlab('Type of events')+ggtitle('Fatalities Study')+
theme(axis.text= element_text(size=13,face='bold'),
axis.text.x= element_text(angle = 90,vjust = 0.3),axis.text.y= element_text(vjust = 0.3),
axis.title= element_text(size=15,face='bold'),
axis.title.x= element_text(vjust=0.3),axis.title.y= element_text(vjust=0.3),
title=element_text(size=15,face='bold',vjust = 2))
dev.off()## png
## 2
Observations: from the above plot, we can see that tornados, excessive heats and floods are the three first responsibles of fatalities across the database’s range. However, excessive heats, avalanche and strong water currents makes the highest fatality rates, i.e. these weather events humanly cost a lot when they occur.
- Injuries study
We analyse injuries according to the entire range of years (Sum_Injuries) and to weather events (Rate_Injuries):
# Injuries resume
Injuries_Res<-by_event %>%
summarize(Sum_Injuries=sum(Number_Injuries),Frequency_Events=n(),
Rate_Injuries=Sum_Injuries/Frequency_Events) %>%
gather(Sum_Rate_Injuries,Values,Sum_Injuries,Rate_Injuries) %>%
select(Type_Events,Sum_Rate_Injuries,Values)
str(Injuries_Res)## Classes 'tbl_df', 'tbl' and 'data.frame': 38 obs. of 3 variables:
## $ Type_Events : Factor w/ 19 levels "Avalanche","Cold",..: 1 2 3 4 5 6 7 8 9 10 ...
## $ Sum_Rate_Injuries: Factor w/ 2 levels "Sum_Injuries",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ Values : num 171 323 1395 33 483 ...# Plot injuries concerns
png('./figures/unnamed-chunk-10.png',width=800,height=600)
ggplot(data = Injuries_Res,aes(x = reorder(Type_Events,-Values) ,y = Values))+
geom_bar(stat='identity',aes(fill=Type_Events))+
facet_grid(Sum_Rate_Injuries~.,scale = 'free_y')+
theme_bw()+guides(fill=F)+
xlab('Type of events')+ggtitle('Injuries Study')+
theme(axis.text= element_text(size=13,face='bold'),
axis.text.x= element_text(angle = 90,vjust = 0.3),axis.text.y= element_text(vjust = 0.3),
axis.title= element_text(size=15,face='bold'),
axis.title.x= element_text(vjust=0.3),axis.title.y= element_text(vjust=0.3),
title=element_text(size=15,face='bold',vjust = 2))
dev.off()## png
## 2
Observations: from the above plot, we can see that tornados, are mainly the responsible of injuries across the dataset’s range of years. However, excessive heats, tornados and hurricanes lead to highest injury rates, i.e. these events humanly make cost damages when they promptly happen.
Effects on economy
- Property and Crop Damages
We omit missing data and we analyse damage costs on properties and crops according to the entire range of years (Sum_DMG) and to weather events (Rate_DMG):
# Group by events according to properties (without missing data)
by_event_prop<-mydata_3 %>%
filter(USD_Property_Damage=!is.na(USD_Property_Damage)) %>%
group_by(Type_Events)
# Property expenses resume
USD_Prop_Res<-by_event_prop %>%
summarize(Frequency_Events=n(),
Sum_DMG=sum(USD_Property_Damage),Rate_DMG=Sum_DMG/Frequency_Events) %>%
gather(Sum_Rate,Values,Sum_DMG,Rate_DMG) %>%
mutate(Type='Properties') %>%
select(-Frequency_Events)
# Group by events according to crops (without missing data)
by_event_crop<-mydata_3 %>%
filter(USD_Crop_Damage=!is.na(USD_Crop_Damage)) %>%
group_by(Type_Events)
# Crop expenses resume
USD_Crop_Res<-by_event_crop %>%
summarize(Frequency_Events=n(),
Sum_DMG=sum(USD_Crop_Damage),Rate_DMG=Sum_DMG/Frequency_Events)%>%
gather(Sum_Rate,Values,Sum_DMG,Rate_DMG) %>%
mutate(Type='Crops') %>%
select(-Frequency_Events)
USD_Res<-rbind(USD_Prop_Res,USD_Crop_Res)
USD_Res$Type<-factor(USD_Res$Type)
# Plot damage expenses
png('./figures/unnamed-chunk-11.png',width=800,height=600)
ggplot(data = USD_Res,aes(x = reorder(Type_Events,-Values) ,y = Values))+
geom_bar(stat='identity',position = 'dodge',aes(fill=Type))+
facet_grid(Sum_Rate~.,scale = 'free_y')+
theme_bw()+
xlab('Type of events')+ylab('Damages (Million of Dollars)')+ggtitle('Damages Study')+
theme(axis.text= element_text(size=13,face='bold'),
axis.text.x= element_text(angle = 90,vjust = 0.3),axis.text.y= element_text(vjust = 0.3),
axis.title= element_text(size=15,face='bold'),
axis.title.x= element_text(vjust=0.3),axis.title.y= element_text(vjust=1),
title=element_text(size=15,face='bold',vjust = 2))
dev.off()## png
## 2
Observations: The above plot shows that floods, hurricane, strong water currents, tornados and ice make mainly the economical damages across the database’s range of years. These damages are especially found into the property class. However, hurricanes, water currents and droughts make strong damage costs when they happen. In opposite of the two first, droughts lead especially to the economical crop damages.
Conclusions
We have analysed effects of weather events on the human health (fatalities and injuries) and economy (property and crop damages). These effects are found in the entire range of years (1950-2011) and for each weather events when they occur (damage rates). The present study has shown that:
Fatalities: Tornados, excessive heats and floods are the three first responsibles of fatalities. However, excessive heats, avalanche and strong water currents make the highest fatality rates.
Injuries: Tornados are mainly responsibles of injuries. However, excessive heats, tornados and hurricanes make the highest injury rates.
Economical damages: Floods, hurricane, strong water currents, tornados and ice are mainly responsibles of economical damages, especially into the property class. However, hurricanes, water currents and droughts make the strongest rates. In opposite of the two first, droughts make strongly crop damages.
This project is reproducible and was done with the following environment:
# Software environment
sessionInfo()## R version 3.2.0 (2015-04-16)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 8 x64 (build 9200)
##
## locale:
## [1] LC_COLLATE=English_United States.1252
## [2] LC_CTYPE=English_United States.1252
## [3] LC_MONETARY=English_United States.1252
## [4] LC_NUMERIC=C
## [5] LC_TIME=English_United States.1252
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] ggplot2_1.0.1 tidyr_0.3.1 dplyr_0.4.3 R.utils_2.2.0
## [5] R.oo_1.19.0 R.methodsS3_1.7.0
##
## loaded via a namespace (and not attached):
## [1] Rcpp_0.12.2 knitr_1.11 magrittr_1.5 MASS_7.3-45
## [5] munsell_0.4.2 colorspace_1.2-6 R6_2.1.1 stringr_1.0.0
## [9] plyr_1.8.3 tools_3.2.0 parallel_3.2.0 grid_3.2.0
## [13] gtable_0.1.2 DBI_0.3.1 htmltools_0.2.6 lazyeval_0.1.10
## [17] yaml_2.1.13 assertthat_0.1 digest_0.6.8 reshape2_1.4.1
## [21] formatR_1.2.1 evaluate_0.8 rmarkdown_0.9 labeling_0.3
## [25] stringi_1.0-1 scales_0.3.0 proto_0.3-10