Analysis of Storm Data between 1995 and 2011
Synopsis
In this report we explore and analyze data from the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. We seek to answer two main questions:
-Across the United States, which types of events are most harmful with respect to population health?
-Across the United States, which types of events have the greatest economic consequences?
We found that the top 5 most harmful strom events with respect to deaths and injuries are tornado, heat related, flood, lightning, and TSTM wind. As for the most costly with respect to property and cost damage, the top 5 most cosly strom events are flood, TSTM wind, tornado, hail, and thunderstorm wind.
Data Processing
Here we load in the data and create a new variable “BGN_YEAR” which describes the year in which the storm event began. This will be used later to both filter out data and show trends over time.
storm_df <- read.csv("data/repdata-data-StormData.csv")
library(lubridate)
storm_df$BGN_DATE <- as.Date(storm_df$BGN_DATE,"%m/%d/%Y")
storm_df$BGN_YEAR <- year(storm_df$BGN_DATE)After a closer look at the unique event type names, we can see that there are many that are very similar, somtimes only differing by case or spelling. Below, some events are grouped into larger logical groupings for the rest of the analysis.
heat_related <- grepl("HEAT",
storm_df$EVTYPE,
ignore.case = TRUE)
storm_df$EVTYPE_clean <- ifelse(heat_related,"HEAT RELATED",
as.character(storm_df$EVTYPE))
blizzard <- grepl("BLIZZARD",
storm_df$EVTYPE_clean,
ignore.case = TRUE)
storm_df$EVTYPE_clean <- ifelse(blizzard,"BLIZZARD",
as.character(storm_df$EVTYPE_clean))
flood <- grepl("FLOOD",
storm_df$EVTYPE_clean,
ignore.case = TRUE)
storm_df$EVTYPE_clean <- ifelse(flood,"FLOOD",
as.character(storm_df$EVTYPE_clean))
tornado <- grepl("tornado",
storm_df$EVTYPE_clean,
ignore.case = TRUE)
storm_df$EVTYPE_clean <- ifelse(tornado,"TORNADO",
as.character(storm_df$EVTYPE_clean))
hail <- grepl("hail",
storm_df$EVTYPE_clean,
ignore.case = TRUE)
storm_df$EVTYPE_clean <- ifelse(hail,"HAIL",
as.character(storm_df$EVTYPE_clean))
cold <- grepl("cold",
storm_df$EVTYPE_clean,
ignore.case = TRUE)
storm_df$EVTYPE_clean <- ifelse(cold,"COLD RELATED",
as.character(storm_df$EVTYPE_clean))
rip_cur <- grepl("RIP CURRENTS",
storm_df$EVTYPE_clean,
ignore.case = TRUE)
storm_df$EVTYPE_clean <- ifelse(rip_cur,"RIP CURRENT",
as.character(storm_df$EVTYPE_clean))
storm_df$EVTYPE_clean <- factor(storm_df$EVTYPE_clean)Results
Most Harmful Stroms
First we will take a look at which storm events were most harmful to population health. To do this we focus on fatalities and injuries of an event. We also create a variable to track “total harm” as the sum of fatalities and injuries for each event type.
library(dplyr)##
## Attaching package: 'dplyr'
##
## The following objects are masked from 'package:stats':
##
## filter, lag
##
## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(ggplot2)# top single year deaths
most_deaths <- storm_df[which.max(storm_df$FATALITIES),]
most_injuries <- storm_df[which.max(storm_df$INJURIES),]A few intersting things to note are that the single event that resulted in the most deaths was from a heat wave in Northeast Illinois. The heat wave lasted from Wednesday July 12 through Sunday July 16 1995, and there were a total of 583 deaths as a result. The most ever injuries was the result of a tornado in Wichita, TX. In the 1979 tornado, 1700 people were reported with injuries.
Below we aggregate the data by event type to look at some summary statistics about the events. Since there were very few events recorded prior to 1995 we start our analysis at that point in time.
# avg number of harm per event
harm_by_event <- storm_df %>%
filter(BGN_YEAR > 1994) %>%
mutate(combined_harm = FATALITIES + INJURIES) %>%
group_by(EVTYPE_clean) %>%
summarise(total_deaths = sum(FATALITIES),
death_rate = mean(FATALITIES),
total_injuries = sum(INJURIES),
injury_rate = mean(INJURIES),
total_harm = sum(combined_harm),
harm_rate = mean(combined_harm),
event_count = n()) %>%
filter(total_harm > 0) %>%
arrange(desc(total_harm))
print(harm_by_event)## Source: local data frame [142 x 8]
##
## EVTYPE_clean total_deaths death_rate total_injuries injury_rate
## 1 TORNADO 1545 0.063390 21787 0.89390
## 2 HEAT RELATED 3087 1.186395 9103 3.49846
## 3 FLOOD 1386 0.017494 8519 0.10753
## 4 LIGHTNING 729 0.051050 4631 0.32430
## 5 TSTM WIND 241 0.001869 3630 0.02816
## 6 THUNDERSTORM WIND 131 0.001603 1426 0.01744
## 7 WINTER STORM 195 0.017147 1298 0.11414
## 8 HURRICANE/TYPHOON 64 0.727273 1275 14.48864
## 9 HIGH WIND 241 0.012077 1093 0.05477
## 10 RIP CURRENT 569 0.742820 524 0.68407
## .. ... ... ... ... ...
## Variables not shown: total_harm (dbl), harm_rate (dbl), event_count (int)It is interesting to note that there are some events where the total number of deaths or injuries is not as high relitive to others, but the death rate/injury rate for these events can be very high. This occurs in the storms that occur less often but have a big impact.
harm_pareto <- harm_by_event %>%
select(EVTYPE_clean, total_harm) %>%
mutate(harm_percent = total_harm/sum(total_harm)) %>%
filter(rank(desc(total_harm)) <= 5) %>%
arrange(desc(total_harm)) %>%
mutate(cum_freq = cumsum(harm_percent))
print(harm_pareto)## Source: local data frame [5 x 4]
##
## EVTYPE_clean total_harm harm_percent cum_freq
## 1 TORNADO 23332 0.32103 0.3210
## 2 HEAT RELATED 12190 0.16773 0.4888
## 3 FLOOD 9905 0.13629 0.6250
## 4 LIGHTNING 5360 0.07375 0.6988
## 5 TSTM WIND 3871 0.05326 0.7521Here we see that the top 5 storm events account for over 75% of the total deaths and injuries. The top event, tornado, accounts for ~32% by itself!
top_five_names <- harm_pareto$EVTYPE_clean
harm_pareto$EVTYPE_clean <- as.character(harm_pareto$EVTYPE_clean)
harm_pareto$EVTYPE_clean <- factor(harm_pareto$EVTYPE_clean,
levels = top_five_names)
harm_pareto_plot <- ggplot(harm_pareto) +
geom_bar(aes(x = EVTYPE_clean,
y = total_harm),
stat = "identity") +
labs(title = "Top Harmful Storm Events",
x = "Event Type",
y = "Total Harm (Fatalities + Injuries)")
print(harm_pareto_plot)
# yearly trend
recent_years_trend <- storm_df %>%
mutate(harm = FATALITIES + INJURIES) %>%
group_by(BGN_YEAR, EVTYPE_clean) %>%
summarise(total_harm = sum(harm)) %>%
filter(BGN_YEAR > 1994,
EVTYPE_clean %in% top_five_names)
harm_trend_plot <- ggplot(recent_years_trend) +
geom_line(aes(x = BGN_YEAR,
y = total_harm,
color = EVTYPE_clean),
size = 1) +
labs(title = "Yearly Event Harm since 1995",
x = "Year",
y = "Total Harm (Fatalities + Injuries)")
print(harm_trend_plot)
From the time series plot above we see that there were some massive floods in 1998 and 2011 was a big year for tornados. Also, 1995, 1999 and 2006 were unusually high years for heat related deaths and injuries. The other events seem to be relatively stable over the past years.
Most Costly Storms
For the next part of this analysis we will be focusing on the varibles that describe both property and crop damage caused by each storm event recorded in the data base.
The table below can be used to explore which events were the most costly with respect to total combinded cost, property cost, or crop cost.
cost_by_event <- storm_df %>%
filter(BGN_YEAR > 1994) %>%
mutate(combined_cost = PROPDMG + CROPDMG) %>%
group_by(EVTYPE_clean) %>%
summarise(total_prodmg = sum(PROPDMG),
mean_propdmg = mean(PROPDMG),
total_cropdmg = sum(CROPDMG),
mean_cropdmg = mean(CROPDMG),
total_cost = sum(combined_cost),
mean_cost = mean(combined_cost),
event_count = n()) %>%
arrange(desc(total_cost))
print(cost_by_event)## Source: local data frame [622 x 8]
##
## EVTYPE_clean total_prodmg mean_propdmg total_cropdmg mean_cropdmg
## 1 FLOOD 2164010 27.314 326259 4.1181
## 2 TSTM WIND 1333344 10.342 109143 0.8466
## 3 TORNADO 1241027 50.918 91558 3.7565
## 4 HAIL 607419 2.791 521476 2.3963
## 5 THUNDERSTORM WIND 862965 10.557 66702 0.8160
## 6 LIGHTNING 513832 35.983 2275 0.1593
## 7 HIGH WIND 315550 15.812 17268 0.8653
## 8 WINTER STORM 127120 11.178 1964 0.1727
## 9 THUNDERSTORM WINDS 107470 10.703 3500 0.3485
## 10 HEAVY SNOW 93350 6.346 1661 0.1129
## .. ... ... ... ... ...
## Variables not shown: total_cost (dbl), mean_cost (dbl), event_count (int)cost_pareto <- cost_by_event %>%
select(EVTYPE_clean, total_cost) %>%
mutate(cost_percent = total_cost/sum(total_cost)) %>%
filter(rank(desc(total_cost)) <= 5) %>%
arrange(desc(total_cost)) %>%
mutate(cum_freq = cumsum(cost_percent))
top_five_names_cost <- cost_pareto$EVTYPE_clean
print(cost_pareto)## Source: local data frame [5 x 4]
##
## EVTYPE_clean total_cost cost_percent cum_freq
## 1 FLOOD 2490269 0.2707 0.2707
## 2 TSTM WIND 1442486 0.1568 0.4276
## 3 TORNADO 1332586 0.1449 0.5725
## 4 HAIL 1128895 0.1227 0.6952
## 5 THUNDERSTORM WIND 929667 0.1011 0.7963We can see that the top 5 events with respect to total combined cost from property damage and crop damage account for almost 80% of the total costs from damages. Here the most costly event, floods, account for 27%.
# yearly trend
recent_years_trend_cost <- storm_df %>%
mutate(cost = PROPDMG + CROPDMG) %>%
group_by(BGN_YEAR, EVTYPE_clean) %>%
summarise(total_cost = sum(cost)) %>%
filter(BGN_YEAR > 1994,
EVTYPE_clean %in% top_five_names_cost)
cost_trend_plot <- ggplot(recent_years_trend_cost) +
geom_line(aes(x = BGN_YEAR,
y = total_cost,
color = EVTYPE_clean),
size = 1) +
labs(title = "Yearly Event Cost since 1995",
x = "Year",
y = "Total Cost (Property Damage + Crop Damage)")
print(cost_trend_plot)
It appears that damages from thunderstorms and wind were relatively low up until 2006 where they spike pretty high for the past several years.