The Impact of Weather Events on the Health and Economy of the United States - Ranking of the Top Events
DT
09 August 2016
Synopsis:
Storms and other severe weather events can cause huge population injury, casualty, loss of property and other economic assets over the entire country. Quantifying and categorizing the outocome of those events help to prioritize and reduce the inherent risks, and minimize the degree of damage. This report will explore the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. The NOAA storm database is the largest database that includes storm and weather events in the United States, including features about date, location, fatalities, injuries, and structural damage from the year period (in this study) 1950 to end of November 2011. By using this database, we address the following questions:
- Which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?
- Which types of events have the greatest economic consequences?
Data Processing:
The file can be downloaded here (you will need bzip to unzip the file) or loaded directly into R by:
fileUrl <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
download.file(fileUrl, destfile="./StormData.bz2")
rawdata <- read.csv("./StormData.bz2")
#This step can take some minutes, depending on Computer setup.The file is downloaded to the local directory and loaded into R (read.csv()) by creating the object rawdata. We can take an exploratory look at the data:
str(rawdata)## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
## $ BGN_TIME : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
## $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
## $ STATE : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ EVTYPE : Factor w/ 985 levels " HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : Factor w/ 35 levels ""," N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_LOCATI: Factor w/ 54429 levels "","- 1 N Albion",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_DATE : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_TIME : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_LOCATI: Factor w/ 34506 levels "","- .5 NNW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ WFO : Factor w/ 542 levels ""," CI","$AC",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ ZONENAMES : Factor w/ 25112 levels ""," "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : Factor w/ 436781 levels "","-2 at Deer Park\n",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...The data has 902297 observables and 37 variables. There are a large number of variables and we can find the details about their acronynms in NOAA documentation. We can then verify some of these variables as being directly related to the Health and Economic aspect of the storm weather events:
- EVTYPE (i.e. Event type)
- FATALITIES
- INJURIES
- PROPDMG (i.e. Property damage)
- PROPDMGEXP
- CROPDMG (i.e. Crop damage)
- CROPDMGEXP
Since we are interested in the relationship between the type of events and Health/Economy, it is rational to subset the original data by:
# Some assumptions: You have installed ggplot2, reshape2 and dplyr packages. If not, please run:
# install.packages('ggplot2'); install.packages('reshape2'); install.packages('dplyr');
#
library(dplyr)
dataSubset <- select(rawdata,EVTYPE,FATALITIES,INJURIES,PROPDMG,PROPDMGEXP,CROPDMG,CROPDMGEXP)
head(dataSubset,3)## EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## 1 TORNADO 0 15 25.0 K 0
## 2 TORNADO 0 0 2.5 K 0
## 3 TORNADO 0 2 25.0 K 0sum(is.na(dataSubset))## [1] 0No NAs values are found in the dataSubset. The variables FATALITIES, INJURIES, PROPDMG and CROPDMG are numerical. The variables for Property and Crop damage are the best estimates expressed in dollar amounts. From the data structure, we also observe that there are 985 levels for EVTYPE, 19 and 9 levels for PROPDMGEXP and CROPDMGEXP, respectively. From NOAA documentation PROPDMGEXP and CROPDMGEXP are characters used to signify magnitude include “K” for thousands, “M” for millions, and “B” for billions. So, the maximum value for the level in those variables would be 7, i.e., “” (empty field), “K” and “k”, “M” and “m”, “B” and “b”. The function table() tabulates the elements and the frequency that they appear for each of those variables:
table(dataSubset$PROPDMGEXP)##
## - ? + 0 1 2 3 4 5 6
## 465934 1 8 5 216 25 13 4 4 28 4
## 7 8 B h H K m M
## 5 1 40 1 6 424665 7 11330table(dataSubset$CROPDMGEXP)##
## ? 0 2 B k K m M
## 618413 7 19 1 9 21 281832 1 1994We note that the elements “”, “K”, “M”, and “B” represent the majority of data entries for PROPDMGEXP and CROPDMGEXP. Since the variables PROPDMG and CROPDMG are the numerical, we will redefine their values considering the magnitude defined in PROPDMGEXP and CROPDMGEXP, respectively by converting all in “B” (billions unit). Then,if the element value in PROPDMGEXP or CROPDMGEXP is “B” or “b”, we divide the corresponding value in PROPDMG or CROPDMG by 1. If the element value is “M” or “m” we divide by 10\(^3\), for “K” or “k” divide by 10\(^6\), and for “” (empty field) and “0” by 10\(^9\). For the any other element values in PROPDMGEXP and CROPDMGEXP we assume that they have the same contribution of “” (empty field) and “0”, that is, we divide the related values in PROPDMG or CROPDMG by 10\(^9\). Furthermore, new variables are created to store the results: PropertyDamage and CropDamage with “Billions” US$ as the unit. This process can be made by using the following code:
billions <- c("B","b")
millions <- c("M","m")
thousands <- c("K","k")
allMagnitude <- c(thousands,millions,billions)
dataSubset$PropertyDamage = with(dataSubset,
(PROPDMGEXP %in% billions)*PROPDMG +
(PROPDMGEXP %in% millions)*PROPDMG/1e3 +
(PROPDMGEXP %in% thousands)*PROPDMG/1e6 +
as.numeric(!(PROPDMGEXP %in% allMagnitude))*(PROPDMG/1e9)) The last line in the above code keeps the values in situ (magnitude of “one”). Also any other “symbol” aside from billions, millions and thousands is assumed to have magnitude “one”, this includes “”. Each term of the above code in the function with() represents a converted vector in “billions” units. The same is applied to the CropDamage:
dataSubset$CropDamage = with(dataSubset,
(CROPDMGEXP %in% billions)*CROPDMG +
(CROPDMGEXP %in% millions)*CROPDMG/1e3 +
(CROPDMGEXP %in% thousands)*CROPDMG/1e6 +
as.numeric(!(CROPDMGEXP %in% allMagnitude))*(CROPDMG/1e9))
head(dataSubset,3)## EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## 1 TORNADO 0 15 25.0 K 0
## 2 TORNADO 0 0 2.5 K 0
## 3 TORNADO 0 2 25.0 K 0
## PropertyDamage CropDamage
## 1 2.5e-05 0
## 2 2.5e-06 0
## 3 2.5e-05 0Two new variables HealthDamage and CapitalAssetDamageare created. HealthDamage is the sum of FATALITIES and INJURIES, and CapitalAssetDamage is the sum of PropertyDamage and CropDamage. The dataSubset is transformed by aggregating EVTYPE and summing the corresponding observable values for the each of the variables. The variables PROPDMG, CROPDMG, PROPDMGEXP and CROPDMGEXPare excluded as described in the following code:
dataSubset = mutate(dataSubset,HealthDamage = FATALITIES + INJURIES, CapitalAssetDamage=PropertyDamage+CropDamage )
dataSubset = aggregate(data=dataSubset, cbind(FATALITIES,INJURIES,HealthDamage,PropertyDamage,CropDamage,CapitalAssetDamage) ~ EVTYPE, sum)
dim(dataSubset)## [1] 985 7head(dataSubset,3)## EVTYPE FATALITIES INJURIES HealthDamage PropertyDamage
## 1 HIGH SURF ADVISORY 0 0 0 2e-04
## 2 COASTAL FLOOD 0 0 0 0e+00
## 3 FLASH FLOOD 0 0 0 5e-05
## CropDamage CapitalAssetDamage
## 1 0 2e-04
## 2 0 0e+00
## 3 0 5e-05EVTYPE is now aggregated into 985 observables.
Results:
Part A
After processing the data, we can obtain the total amount of the variables during the recorded period between the year 1950 to the end of November 2011:
totalSum <- round(colSums(dataSubset[,-1]),1)
totalSum[1:3]## FATALITIES INJURIES HealthDamage
## 15145 140528 155673We can subset the data into the events responsible for 90% of the total Fatalities and Injuries (HealthDamage), and rank them in decreasing order. But first, a function topRank is created:
topRank <- function(df,name,threshold){
temp <- df[order(df[,name],decreasing = TRUE),]
temp[which(cumsum(temp[,name])/sum(temp[,name]) < threshold),]}Then, EVTYPE is ranking according to HealthDamage
rank <- topRank(dataSubset,"HealthDamage",0.9)
rownames(rank)<-c()
tempHealthDamage <- rank
tempHealthDamage[,1:4]## EVTYPE FATALITIES INJURIES HealthDamage
## 1 TORNADO 5633 91346 96979
## 2 EXCESSIVE HEAT 1903 6525 8428
## 3 TSTM WIND 504 6957 7461
## 4 FLOOD 470 6789 7259
## 5 LIGHTNING 816 5230 6046
## 6 HEAT 937 2100 3037
## 7 FLASH FLOOD 978 1777 2755
## 8 ICE STORM 89 1975 2064
## 9 THUNDERSTORM WIND 133 1488 1621
## 10 WINTER STORM 206 1321 1527
## 11 HIGH WIND 248 1137 1385
## 12 HAIL 15 1361 1376To ensure that this ranking includes the top FATALITIES and INJURIES, we can also obtain the ranks of each one, separately:
rank <- topRank(dataSubset,"FATALITIES",0.9)
rownames(rank)<-c()
tempFatalities <- rank
tempFatalities[,1:2]## EVTYPE FATALITIES
## 1 TORNADO 5633
## 2 EXCESSIVE HEAT 1903
## 3 FLASH FLOOD 978
## 4 HEAT 937
## 5 LIGHTNING 816
## 6 TSTM WIND 504
## 7 FLOOD 470
## 8 RIP CURRENT 368
## 9 HIGH WIND 248
## 10 AVALANCHE 224
## 11 WINTER STORM 206
## 12 RIP CURRENTS 204
## 13 HEAT WAVE 172
## 14 EXTREME COLD 160
## 15 THUNDERSTORM WIND 133
## 16 HEAVY SNOW 127
## 17 EXTREME COLD/WIND CHILL 125
## 18 STRONG WIND 103
## 19 BLIZZARD 101
## 20 HIGH SURF 101
## 21 HEAVY RAIN 98rank <- topRank(dataSubset,"INJURIES",0.9)
rownames(rank)<-c()
tempInjuries <- rank
tempInjuries[,c(1,3)]## EVTYPE INJURIES
## 1 TORNADO 91346
## 2 TSTM WIND 6957
## 3 FLOOD 6789
## 4 EXCESSIVE HEAT 6525
## 5 LIGHTNING 5230
## 6 HEAT 2100
## 7 ICE STORM 1975
## 8 FLASH FLOOD 1777
## 9 THUNDERSTORM WIND 1488
## 10 HAIL 1361There are 21 and 10 different events responsible for 90% of the total incidents for FATALITIES and INJURIES, respectively.
We can now plot the Event type versus FATALITIES and INJURIES:
library(reshape2)
#change the order of factor levels of EVTYPE to align it with the previous rank.
tempHealthDamage$EVTYPE <- factor(tempHealthDamage$EVTYPE, levels = tempHealthDamage$EVTYPE[order(tempHealthDamage$HealthDamage)])
#reshape the data in order to factorize in the plot
tempHealthDamage <- melt(tempHealthDamage,id=c("EVTYPE","HealthDamage","PropertyDamage","CropDamage","CapitalAssetDamage"))library(ggplot2)
ggplot(tempHealthDamage, aes(EVTYPE,value))+
geom_bar(stat="identity",aes(fill = factor(variable)),position = "dodge")+ coord_flip()+
labs(fill = "")+ xlab("Event Type")+ ylab("Counts")+
scale_fill_brewer(palette="Set1")+
ggtitle("Fatalities and Injuries caused by Weather Events in the USA during years: 1950 - 2011")+
scale_y_log10(breaks = scales::trans_breaks("log10", function(x) 10^x),labels = scales::trans_format("log10", scales::math_format(10^.x)))
Figure 1 - Bar Plot (in Log scale) of types of events that are most harmful to population health. Red color for Fatalities and Blue color for Injuries
Without any doubt, TORNADO is the most dangerous natural event that causes the largest number of deaths and injuries. It is responsible for:
#Fatality, Injury, and Total Health Damage Percentage from the Total, respectively per each variable
round(100*tempFatalities[which(tempFatalities$EVTYPE=="TORNADO"),-1]/totalSum,1)[1:3]## FATALITIES INJURIES HealthDamage
## 1 37.2 65 62.337.2% and 65% of the total number of fatalities and injurie during this time period.
EXESSIVE HEAT is the second most fatal and TSTM WIND (ThunderStorm Wind) the second most significant event to cause injuries, followed closely by FLOOD.
Part B
The cost of weather events in the US economy is measured by the impact on Properties and Crops, that is, the cost of the damage caused:
totalSum[4:6]## PropertyDamage CropDamage CapitalAssetDamage
## 427.3 49.1 476.4From the above, we observe that the total cost of damage on Properties is estimated to be 9 folds higher than the total cost of damage to Crops.
We can rank the 90% of total damage costs (CapitalAssetDamage) caused by the weather events as follows:
# Variable values in Billions US$
rank <- topRank(dataSubset,"CapitalAssetDamage",0.9)
rownames(rank)<-c()
tempCapitalAssetDamage <- rank
tempCapitalAssetDamage[,c(1,5:7)]## EVTYPE PropertyDamage CropDamage CapitalAssetDamage
## 1 FLOOD 144.657710 5.6619684 150.319678
## 2 HURRICANE/TYPHOON 69.305840 2.6078728 71.913713
## 3 TORNADO 56.937161 0.4149533 57.352114
## 4 STORM SURGE 43.323536 0.0000050 43.323541
## 5 HAIL 15.732267 3.0259545 18.758222
## 6 FLASH FLOOD 16.140812 1.4213171 17.562129
## 7 DROUGHT 1.046106 13.9725660 15.018672
## 8 HURRICANE 11.868319 2.7419100 14.610229
## 9 RIVER FLOOD 5.118945 5.0294590 10.148404
## 10 ICE STORM 3.944928 5.0221135 8.967041
## 11 TROPICAL STORM 7.703891 0.6783460 8.382237
## 12 WINTER STORM 6.688497 0.0269440 6.715441To ensure that this ranking includes the top PropertyDamage and CropDamage, we can also obtain the ranks of each one, separately:
# Variable values in Billions US$
rank <- topRank(dataSubset,"PropertyDamage",0.9)
rownames(rank)<-c()
tempPropertyDamage <- rank
tempPropertyDamage[,c("EVTYPE","PropertyDamage")]## EVTYPE PropertyDamage
## 1 FLOOD 144.657710
## 2 HURRICANE/TYPHOON 69.305840
## 3 TORNADO 56.937161
## 4 STORM SURGE 43.323536
## 5 FLASH FLOOD 16.140812
## 6 HAIL 15.732267
## 7 HURRICANE 11.868319
## 8 TROPICAL STORM 7.703891
## 9 WINTER STORM 6.688497
## 10 HIGH WIND 5.270046
## 11 RIVER FLOOD 5.118945# Variable values in Billions US$
rank <- topRank(dataSubset,"CropDamage",0.9)
rownames(rank)<-c()
tempCropDamage <- rank
tempCropDamage[,c("EVTYPE","CropDamage")]## EVTYPE CropDamage
## 1 DROUGHT 13.9725660
## 2 FLOOD 5.6619684
## 3 RIVER FLOOD 5.0294590
## 4 ICE STORM 5.0221135
## 5 HAIL 3.0259545
## 6 HURRICANE 2.7419100
## 7 HURRICANE/TYPHOON 2.6078728
## 8 FLASH FLOOD 1.4213171
## 9 EXTREME COLD 1.2929730
## 10 FROST/FREEZE 1.0940860
## 11 HEAVY RAIN 0.7333998
## 12 TROPICAL STORM 0.6783460
## 13 HIGH WIND 0.6385713The following plot is ranked according to the above results, and it shows the impact of the weather events on the loss of Properties and Crops in Billions of US$ value:
#change the order of factor levels of EVTYPE to align it with the previous rank.
tempCapitalAssetDamage$EVTYPE <- factor(tempCapitalAssetDamage$EVTYPE, levels = tempCapitalAssetDamage$EVTYPE[order(tempCapitalAssetDamage$CapitalAssetDamage)])
#reshape the data in order to factorize in the plot
tempCapitalAssetDamage1 <- melt(tempCapitalAssetDamage,id=c("EVTYPE","FATALITIES","INJURIES", "HealthDamage","CapitalAssetDamage"))ggplot(tempCapitalAssetDamage1, aes(EVTYPE,value))+
geom_bar(stat="identity",aes(fill = factor(variable)))+ coord_flip() +
labs(fill = "")+ xlab("Event Type")+ ylab("Billion US$")+
scale_fill_brewer(palette="Set1")+
ggtitle("Properties and Crop losses by Weather Events in the USA during years: 1950 - 2011") 
Figure 2 - Bar Plot of Economic Loss (in Billion US$) by Weather Events. Red color for Property Damages and Blue color for Crop Damages
#Damage or Loss Percentage from the Total, respectively per each variable
round(100*tempCapitalAssetDamage[which(tempCapitalAssetDamage$EVTYPE=="FLOOD"),-1]/totalSum,1)[4:6]## PropertyDamage CropDamage CapitalAssetDamage
## 1 33.9 11.5 31.6FLOOD is by large the most economically devastating event that causes the largest loss tn the US economy because of its extension of destruction to Properties. By itself, it accounts for 33.9% of total loss. HURRICANE/TYPHOON and TORNADO are the second and third most devasting events to Properties, with FLOOD having caused ca. 271 Billions US$ in Property damage, or 63.4% of total Property damage during years 1950-2011.
By constrast, DROUGHT event is the most economically damaging for Crops, figuring as the top one for all the events that cause damage to Crops. It is responsible for 28.5% of total Crops loss. FLOOD, and RIVER FLOOD are the second and third most damaging events to Crops. The top three events have caused a loss of approximately 8.7 Billions US$, or 17.7% of total Crop loss during the years 1950-2011.
TORNADO has a small impact on Crops (0.8% of total loss to Crops, or 0.41 Billions US$), but as we observed above, it is the event causing the most fatalities and it also figures as the third most damaging to Property loss accounting for 13.3%
Conclusions:
From the NOAA database that covers weather events during the recording period from year 1950 to end November 2011, we have processed and derived results that answer the following requested questions:
Which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?
The top 3 most Fatal weather events are:
tempFatalities[1:3,1:2]## EVTYPE FATALITIES
## 1 TORNADO 5633
## 2 EXCESSIVE HEAT 1903
## 3 FLASH FLOOD 978The top 3 most significant weather events causing Injuries are:
tempInjuries[1:3,c(1,3)]## EVTYPE INJURIES
## 1 TORNADO 91346
## 2 TSTM WIND 6957
## 3 FLOOD 6789Which types of events have the greatest economic consequences? The top 3 most costly (in Billions US$) weather events are:
tempPropertyDamage[1:3,c(1,5)]## EVTYPE PropertyDamage
## 1 FLOOD 144.65771
## 2 HURRICANE/TYPHOON 69.30584
## 3 TORNADO 56.93716The top 3 most damaging (in Billions US$) to Crops weather events are:
tempCropDamage[1:3,c(1,6)]## EVTYPE CropDamage
## 1 DROUGHT 13.972566
## 2 FLOOD 5.661968
## 3 RIVER FLOOD 5.029459