The most severe types of weather events in the US (2000 to 2014)
Synopsis
This research identifies the weather events that caused the most harm to human health and the most economic damages in the US since the beginning of the year 2000 through November of 2011.
I found that tornadoes, excessive heat, lightning, and thunderstorms were the events causing the most injuries and fatalities. I also found that floods, hurricanes/typhoons, storm surges, and tornadoes caused the most economic damage as measured by damages to crops and property.
To complete this analysis, I used the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database, covering events since 1950. In order to perform my analysis, I extracted the data for the years under analysis, chose the variables measuring harm and economic damage, and recoded the databases categorization of the weather events to make that categorization more consistent.
Data Processing
I donwloaded the data from the course website and read it into a data frame:
fileUrl <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
download.file(fileUrl, destfile = "database.csv.bz2")
dat <- read.csv("database.csv.bz2", na.strings = c("", "NA"))
str(dat)## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : chr "4/18/1950 0:00:00" "4/18/1950 0:00:00" "2/20/1951 0:00:00" "6/8/1951 0:00:00" ...
## $ BGN_TIME : chr "0130" "0145" "1600" "0900" ...
## $ TIME_ZONE : chr "CST" "CST" "CST" "CST" ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: chr "MOBILE" "BALDWIN" "FAYETTE" "MADISON" ...
## $ STATE : chr "AL" "AL" "AL" "AL" ...
## $ EVTYPE : chr "TORNADO" "TORNADO" "TORNADO" "TORNADO" ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : chr NA NA NA NA ...
## $ BGN_LOCATI: chr NA NA NA NA ...
## $ END_DATE : chr NA NA NA NA ...
## $ END_TIME : chr NA NA NA NA ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : chr NA NA NA NA ...
## $ END_LOCATI: chr NA NA NA NA ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: chr "K" "K" "K" "K" ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: chr NA NA NA NA ...
## $ WFO : chr NA NA NA NA ...
## $ STATEOFFIC: chr NA NA NA NA ...
## $ ZONENAMES : chr NA NA NA NA ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : chr NA NA NA NA ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...I then cleaned up the variable names by making them lower case and removing "_" signs.
names(dat) <- tolower(names(dat))
names(dat) <- gsub("_", "", names(dat))The variables indicating the beginning (bgndate) and end (enddate) dates of each event are formatted as characters, and contain uninformative time data - a string “0:00:00” is placed at the end of every entry. I thus modified these two data variables by removing the uninformative string and formatting the result as a date.
#remove uninformative 0:00:00 string from bgndate and enddate
dat$bgndate <- gsub(" 0:00:00", "", dat$bgndate)
dat$enddate <- gsub(" 0:00:00", "", dat$enddate)
#convert to date format
dat$bgndate <- as.Date(dat$bgndate, "%m/%d/%Y")
dat$enddate <- as.Date(dat$enddate, "%m/%d/%Y")In keeping with the principles of tidy data, I deleted the first column, containing a variable named “state” which took on numerical values, and the fifth column, containing a variable named “county” which also took on numerical values. The reason why I deleted these columns is that the database contains other columns, respectively titled “state” and “countyname” that presented the same information in a more descriptive way (with full county names and abbreviated state names rather than numerical codes).
dat <- dat[, -c(1, 5)]To further clean the database, I ran the code below to find which variables contained actual information:
apply(dat, 2, function(x) length(unique(x)))This allowed me to idenfity the variables that had one unique value (countyend and countyendn). Since these variables always display the same value, they are uniformative, so I deleted them:
#loads tidyverse package
library(tidyverse)dat <- dat %>% select(!c(countyend, countyendn))I then created a new dataframe with the variables that might be relevant to this project:
subdat <- dat %>% select(bgndate, countyname, state, evtype, fatalities:cropdmgexp)The resulting dataframe is tidy-ish. There are inconsistencies in how several variables are coded, but fixing this would take an amount of time that I think is unreasonable for the assignment at hand. I thus treated the resulting dataframe as the basis from which I will subset the analytic data.
The goal of the project is to find the events that cause the greatest harm to human health and those with the greatest economic consequences. Since data far into the past are more incomplete,as stated in the assignment prompt, and they are also more unevenly coded (typos or inconsistent notation are frequent), it is reasonable to conjecture that old data might introduce bias in our analysis, in particular if some events were sistematically less likely to be coded correctly.
I thus decided to restrict attention in my analysis to events occuring since 01/01/2000. This is an arbitrary threshold, but without an amount of work and knowledge beyond the scope of this assignment it is impossible to arrive at an appropriate threshold. The code below create the subseted data:
andat <- subset(subdat, bgndate > as.Date("1999-12-31"))I measure the harm caused by an event to a population’s health as the sum of the variables “fatalities” and “injuries:”
andat$harm <- andat$fatalities + andat$injuriesTwo variables are relevant to the calculation of the economic impact of a weather event: “propdmg” and “cropdmg.” The values that these variables take are expressed in different units depending on the weather event. These units are expressed in the variables “propdmgexp” and “cropdmgexp.” I ran the function unique to determine the units in which the variables are expressed:
unique(andat$propdmgexp)## [1] "K" NA "M" "B" "0"unique(andat$cropdmgexp)## [1] "K" NA "M" "B"After reading the documentation of the database, I determined that K corresponds to thousands, M to millions, and B to Billions. 0 and NA mean that there is no record of damage. The different scales in which damages are expressed makes it necessary to calculate new variables that use uniform scales. I will do this using a for loop with a if test. Since a value of NA does not return TRUE or FALSE in an if test, I must first convert the NAS in columns “propdmgexp” and “cropdmgexp:”
#replace NAs using tools from the tidyr package
andat$propdmgexp <- andat$propdmgexp %>% replace_na(0)
andat$cropdmgexp <- andat$cropdmgexp %>% replace_na(0)I can now run a for loop with if conditions to create a new variable called “property” that calculates the property damages in thousands of dollars. When the data are not available, I input the damage as 0.
for (i in 1:nrow(andat)) {
if (andat$propdmgexp[i] == "K") {
andat$property[i] <- andat$propdmg[i]
} else if (andat$propdmgexp[i] =="M") {
andat$property[i] <- andat$propdmg[i]*1000
} else if (andat$propdmgexp[i] =="B") {
andat$property[i] <- andat$propdmg[i]*1000000
} else(andat$property[i] <- 0)
}According to the same principle, I also calculated a variable called “crop” to measure the damage to crops in thousands of dollars:
for (i in 1:nrow(andat)) {
if (andat$cropdmgexp[i] == "K") {
andat$crop[i] <- andat$cropdmg[i]
} else if (andat$cropdmgexp[i] =="M") {
andat$crop[i] <- andat$cropdmg[i]*1000
} else if (andat$cropdmgexp[i] =="B") {
andat$crop[i] <- andat$cropdmg[i]*1000000
} else(andat$crop[i] <- 0)
}I measure the economic impact of an event as the sum of the monetary value of property and crop damage:
andat$econ <- andat$property + andat$cropThe data frame that results from these computations contains the analytic data of this assignment. The data contain 196 different types of weather events:
length(unique(andat$evtype))## [1] 196After inspection of the variable’s values, I found that some events of the same nature were coded differently. I thus recoded them below:
#remove triple blank space at start of value
andat$evtype <- gsub("^ ", "", andat$evtype)
#remove single blank space at start of value
andat$evtype <- gsub("^ ", "", andat$evtype)
#uniformize coastal flooding
andat$evtype <- gsub("COASTAL FLOODING", "COASTAL FLOOD", andat$evtype)
#uniformize lake effect
andat$evtype <- gsub("LAKE-", "LAKE ", andat$evtype)
#uniformize mudslide
andat$evtype <- gsub("MUD SLIDE", "MUDSLIDE", andat$evtype)
#uniformize NON-TSTM WIND
andat$evtype <- gsub("NON TSTM WIND", "NON-TSTM WIND", andat$evtype)
#uniformize rip current
andat$evtype <- gsub("RIP CURRENTS", "RIP CURRENT", andat$evtype)
#uniformize strong wind
andat$evtype <- gsub("STRONG WINDS", "STRONG WIND", andat$evtype)
#uniformize winter weather mix
andat$evtype <- gsub("WINTER WEATHER/MIX", "WINTER WEATHER MIX", andat$evtype)
#uniformize thunderstorm wind
andat$evtype <- gsub("TSTM WIND", "THUNDERSTORM WIND", andat$evtype)This reduces the number of unique events to 182:
length(unique(andat$evtype))## [1] 182Results
To measure harm to health (hth) of each type of event, I calculated the sum of the “harm” variable, which is itself the sum of injuries and fatalities, across the various events. To measure damage to the economy (dte), did the same for the econ variable. I stored the results in the data frame “con,” for consequences.
con <- andat %>% group_by(evtype) %>% summarize(hth = sum(harm), dte = sum(econ))## `summarise()` ungrouping output (override with `.groups` argument)I want to find the events that account for 90% of the harm to health and of the economic damage. To that end, and for harm to health, I ran the following code:
# creates a a frame with events and harm to health ordered by amount of harm (fatalities + injuries)
hthorder <- con[order(con$hth, decreasing = TRUE),1:2]
# adds a column with cumulative share
hthorder$cum <- cumsum(hthorder$hth)/sum(hthorder$hth)
#display the events that account for up to 90% of the harm
hthorder[hthorder$cum < 0.9, ]## # A tibble: 14 x 3
## evtype hth cum
## <chr> <dbl> <dbl>
## 1 TORNADO 16406 0.399
## 2 EXCESSIVE HEAT 4721 0.514
## 3 LIGHTNING 3459 0.598
## 4 THUNDERSTORM WIND 3399 0.680
## 5 HEAT 1453 0.716
## 6 FLASH FLOOD 1412 0.750
## 7 HURRICANE/TYPHOON 1339 0.783
## 8 WILDFIRE 986 0.807
## 9 RIP CURRENT 845 0.827
## 10 HIGH WIND 808 0.847
## 11 FLOOD 581 0.861
## 12 HAIL 545 0.874
## 13 WINTER STORM 540 0.887
## 14 WINTER WEATHER 376 0.896The graph below depicts the total impact of these events:
ghth <- ggplot(hthorder[hthorder$cum < 0.9, ], aes(x = reorder(evtype, - hth), y = hth))
ghth + theme(axis.text.x = element_text(angle = 90)) + geom_col(fill = "steelblue") + labs(title = "Harm to human health from extreme weather events in the US (2000-2011)", subtitle = "Fatalities and injuries from event types accounting for 90% of all fatalities and injuries", x = "weather event", y = "fatalies + injuries")
As the graph shows, in the period of analysis, tornadoes were the most dangerous event to human health, followed by excessive heat, lightning, and thunderstorms.
I performed a similar analysis to find the events that create the most economic damage:
# creates a a frame with events and economic damage ordered by damage (crop + property)
dteorder <- con[order(con$dte, decreasing = TRUE),c(1,3)]
# adds a column with cumulative share
dteorder$cum <- cumsum(dteorder$dte)/sum(dteorder$dte)
#display the events that account for up to 90% of the harm
dteorder[dteorder$cum < 0.9, ]## # A tibble: 8 x 3
## evtype dte cum
## <chr> <dbl> <dbl>
## 1 FLOOD 138913008. 0.392
## 2 HURRICANE/TYPHOON 71913713. 0.595
## 3 STORM SURGE 43170935 0.717
## 4 TORNADO 19695553. 0.772
## 5 HAIL 13773372. 0.811
## 6 FLASH FLOOD 12781544. 0.847
## 7 DROUGHT 9982231 0.875
## 8 TROPICAL STORM 7607242. 0.897This information is showecased in the graphed below:
gdte <- ggplot(dteorder[dteorder$cum < 0.9, ], aes(x = reorder(evtype, - dte), y = dte))
gdte + theme(axis.text.x = element_text(angle = 90)) + geom_col(fill = "indianred") + labs(title = "Damage to the economy from extreme weather events in the US (2000-2011)", subtitle = "Damage to crops and property from event types accounting for 90% of all damage", x = "weather event", y = "Damage in thousands of $US")
As the graph shows, floods, hurricanes/typhoons, storm surges, and tornadoes created the most economic damage in the period under analysis.