The analysis of the severe weather events impact in USA (1995-2011)
Synopsis
This project involves exploring the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. This database tracks characteristics of major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage.
We’ve narrowed our data down to fewer variables. After performing the necessary transformations, we aggregate the reduced data set by event type, in order to figure out the top 10 of events which are most harmful to population health, and those with the greatest economic consequences.
Data processing
For the sake of reproducibility, we use R to download the data file and then read it, the output will be stored in a variable called “storm.data”
download.file("https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2","D:/RWSPACE/project52/stormdata.bz2")
storm.data <- read.csv("D:/RWSPACE/project52/stormdata.bz2")Having a look into our data (dimension, variables, classes, …)
dim(storm.data)## [1] 902297 37head(storm.data)## STATE__ BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE
## 1 1 4/18/1950 0:00:00 0130 CST 97 MOBILE AL
## 2 1 4/18/1950 0:00:00 0145 CST 3 BALDWIN AL
## 3 1 2/20/1951 0:00:00 1600 CST 57 FAYETTE AL
## 4 1 6/8/1951 0:00:00 0900 CST 89 MADISON AL
## 5 1 11/15/1951 0:00:00 1500 CST 43 CULLMAN AL
## 6 1 11/15/1951 0:00:00 2000 CST 77 LAUDERDALE AL
## EVTYPE BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END
## 1 TORNADO 0 0
## 2 TORNADO 0 0
## 3 TORNADO 0 0
## 4 TORNADO 0 0
## 5 TORNADO 0 0
## 6 TORNADO 0 0
## COUNTYENDN END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES
## 1 NA 0 14.0 100 3 0 0
## 2 NA 0 2.0 150 2 0 0
## 3 NA 0 0.1 123 2 0 0
## 4 NA 0 0.0 100 2 0 0
## 5 NA 0 0.0 150 2 0 0
## 6 NA 0 1.5 177 2 0 0
## INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES
## 1 15 25.0 K 0
## 2 0 2.5 K 0
## 3 2 25.0 K 0
## 4 2 2.5 K 0
## 5 2 2.5 K 0
## 6 6 2.5 K 0
## LATITUDE LONGITUDE LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1 3040 8812 3051 8806 1
## 2 3042 8755 0 0 2
## 3 3340 8742 0 0 3
## 4 3458 8626 0 0 4
## 5 3412 8642 0 0 5
## 6 3450 8748 0 0 6str(storm.data)## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
## $ BGN_TIME : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
## $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
## $ STATE : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ EVTYPE : Factor w/ 985 levels " HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : Factor w/ 35 levels ""," N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_LOCATI: Factor w/ 54429 levels "","- 1 N Albion",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_DATE : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_TIME : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_LOCATI: Factor w/ 34506 levels "","- .5 NNW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ WFO : Factor w/ 542 levels ""," CI","$AC",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ ZONENAMES : Factor w/ 25112 levels ""," "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : Factor w/ 436781 levels "","-2 at Deer Park\n",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...Our analysis will be focused in human fatalities/injuries and enconomic damages. Hence, we will narrow our variables down to relevant ones. The new data set is called “storm”
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionstorm <- select(storm.data,EVTYPE,FATALITIES,INJURIES,PROPDMG,PROPDMGEXP,CROPDMG,CROPDMGEXP)
head(storm)## EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## 1 TORNADO 0 15 25.0 K 0
## 2 TORNADO 0 0 2.5 K 0
## 3 TORNADO 0 2 25.0 K 0
## 4 TORNADO 0 2 2.5 K 0
## 5 TORNADO 0 2 2.5 K 0
## 6 TORNADO 0 6 2.5 K 0The fatalities and injuries are represented by numerical data classes, and need no more processing before the reporting stage. Whereas we have to consider the exponent factor for both property and crop damage.
unique(storm$PROPDMGEXP)## [1] K M B m + 0 5 6 ? 4 2 3 h 7 H - 1 8
## Levels: - ? + 0 1 2 3 4 5 6 7 8 B h H K m Munique(storm$CROPDMGEXP)## [1] M K m B ? 0 k 2
## Levels: ? 0 2 B k K m MIn order to have less exponent levels, we transform all of its values to Capital letters :
storm$PROPDMGEXP <- toupper(storm$PROPDMGEXP)
unique(storm$PROPDMGEXP)## [1] "K" "M" "" "B" "+" "0" "5" "6" "?" "4" "2" "3" "H" "7" "-" "1" "8"storm$CROPDMGEXP <- toupper(storm$CROPDMGEXP)
unique(storm$CROPDMGEXP)## [1] "" "M" "K" "B" "?" "0" "2"We notice that there the exponents are represented either by numbers or letters. We assume that the special characters (+,?,-), Zeros and blanks mean that no transformation is needed.
Writing a function of transformation
exp.trans <- function(column){
column[column == "H"]<- 100
column[column == "K"]<- 1000
column[column == "M"]<- 1000000
column[column == "B"]<- 1000000000
column[column == "1"]<- 10
column[column == "2"]<- 100
column[column == "3"]<- 1000
column[column == "4"]<- 10000
column[column == "5"]<- 100000
column[column == "6"]<- 1000000
column[column == "7"]<- 10000000
column[column == "8"]<- 100000000
column[column == "0"]<- 1
column[column == "+"]<- 1
column[column == "-"]<- 1
column[column == "?"]<- 1
column[column == ""] <- 1
return(column)
}We will use the function written above to transform exponent columns and calculate the value of property/crop damage and store it in new columns called “PROPDMGVAL” and “CROPDMGVAL”
#Property damage
storm$PROPDMGEXP <- exp.trans(storm$PROPDMGEXP)
mode(storm$PROPDMGEXP) <- "numeric"
storm <- mutate(storm,PROPDMGVAL = PROPDMGEXP * PROPDMG)
#Crop damage
storm$CROPDMGEXP <- exp.trans(storm$CROPDMGEXP)
mode(storm$CROPDMGEXP) <- "numeric"
storm <- mutate(storm,CROPDMGVAL = CROPDMGEXP * CROPDMG)In order to have the economic consequences of weather events, we sum the property and crop damages
storm <- mutate(storm,ECODMG = CROPDMGVAL + PROPDMGVAL)Now and as a last step of reporting stage preparations, we will aggregate our data by event type, for each item (Fatalities/Injuries and Property/Crop damage)
fatal.sum <- aggregate(FATALITIES ~ EVTYPE,storm,sum)
injury.sum <- aggregate(INJURIES ~ EVTYPE,storm,sum)
ecodmg.sum <- aggregate(ECODMG ~ EVTYPE,storm,sum)As we are interested in severe weather events, we will sort our data to filter the top 10 for each category (Fatalities/Injuries and Economic impacts)
top.fatal <- fatal.sum[order(-fatal.sum$FATALITIES),][1:10,]
top.injury <- injury.sum[order(-injury.sum$INJURIES),][1:10,]
top.ecodmg <- ecodmg.sum[order(-ecodmg.sum$ECODMG),][1:10,]Results
Most harmful events to population health
The analysis aim to address the question of severe harmful weather events to population health. We choose to report separately the events leading to deaths and to injuries.
library(ggplot2)## Warning: package 'ggplot2' was built under R version 3.2.3ggplot(data = top.fatal, aes(x=reorder(top.fatal$EVTYPE,-top.fatal$FATALITIES), y=top.fatal$FATALITIES)) + geom_bar(fill="red", stat = "identity") + theme(axis.text.x = element_text(angle = 90, hjust = 1)) + ylab("Number of deaths") + xlab("Events") + ggtitle("Total Number of deaths in USA due to severe weather events 1995-2011")
ggplot(data = top.injury, aes(x=reorder(top.injury$EVTYPE,-top.injury$INJURIES), y=top.injury$INJURIES)) + geom_bar(fill="orange", stat = "identity") + theme(axis.text.x = element_text(angle = 90, hjust = 1)) + ylab("Number of injuries") + xlab("Weather events") + ggtitle("Total Number of injuries in USA due to severe weather events 1995-2011")
Events types having the greatest economic damages
ggplot(data = top.ecodmg, aes(x=reorder(top.ecodmg$EVTYPE,-top.ecodmg$ECODMG), y=top.ecodmg$ECODMG/(10^9))) + geom_bar(fill="black", stat = "identity") + theme(axis.text.x = element_text(angle = 90, hjust = 1)) + ylab("Damages cost ($ billions)") + xlab("Weather events") + ggtitle("Total damages cost of weather events 1995-2011")
Conclusion
We can conclude that the Tornado was responsible for most of the deaths and injuries in USA, caused by severe weather events between 1995 and 2011 (5633 deaths and 91346 injuries). Excessive heat and Thunderstorm wind came in the 2nd position as the most events causing fatalities and injuries respectively.
Economically speaking, the floods have the greatest impact, causing over than $150 billion in damages. Tornado is ranking third and hence could be considered the most dangerous weather event experienced by USA between 1995 and 2011