Course Project 2: Economic and Health Impact of Severe Weather in the United States, 1950-2011
Synopsis
This analysis explores the U.S. National Oceanic and Atmospheric Administration (NOAA) Storm Database to identify which types of severe weather events are most harmful to population health and which have the greatest economic consequences. The raw data cover 1950 through November 2011, though event recording is far more complete in later years. After loading the raw data, we retain only the variables relevant to health and economic impact, consolidate the highly inconsistent event-type labels into a smaller set of standardized categories, and convert the property/crop damage magnitude-and-exponent columns into single dollar values. We then rank event types by total fatalities plus injuries to answer the population-health question, and by total property plus crop damage to answer the economic-consequences question. Across the full period, tornadoes are associated with by far the largest numbers of fatalities and injuries, while floods and hurricanes/typhoons account for the largest economic losses. These results are intended to help inform resource-prioritization decisions for severe weather preparedness.
Data Processing
Loading the raw data
The raw data are provided as a bzip2-compressed CSV file. We read it
directly using data.table::fread(), which can decompress
and parse the file without a separate manual decompression step.
cache=TRUE to optimize performance.
library(data.table)## Warning: package 'data.table' was built under R version 4.5.3storm_data <- fread("repdata_data_StormData.csv.bz2")
dim(storm_data)## [1] 902297 37Selecting relevant variables
The raw data contain 37 columns, most of which (location codes,
remarks, narrative text, latitude/longitude, etc.) are not needed to
answer either of our two questions. Population health impact is captured
by FATALITIES and INJURIES; economic impact is
captured by PROPDMG/PROPDMGEXP (property
damage) and CROPDMG/CROPDMGEXP (crop damage).
We also keep BGN_DATE to examine the time coverage of the
data, and EVTYPE to group by event type.
cols <- c("BGN_DATE", "EVTYPE", "FATALITIES", "INJURIES",
"PROPDMG", "PROPDMGEXP", "CROPDMG", "CROPDMGEXP")
storm_subset <- storm_data[, ..cols]
head(storm_subset)## BGN_DATE EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG
## <char> <char> <num> <num> <num> <char> <num>
## 1: 4/18/1950 0:00:00 TORNADO 0 15 25.0 K 0
## 2: 4/18/1950 0:00:00 TORNADO 0 0 2.5 K 0
## 3: 2/20/1951 0:00:00 TORNADO 0 2 25.0 K 0
## 4: 6/8/1951 0:00:00 TORNADO 0 2 2.5 K 0
## 5: 11/15/1951 0:00:00 TORNADO 0 2 2.5 K 0
## 6: 11/15/1951 0:00:00 TORNADO 0 6 2.5 K 0
## CROPDMGEXP
## <char>
## 1:
## 2:
## 3:
## 4:
## 5:
## 6:Examining data completeness over time
The database documentation notes that earlier years have far fewer recorded events, likely due to incomplete record-keeping rather than fewer actual events. We confirm this pattern before proceeding.
storm_subset[, year := as.numeric(format(
as.Date(BGN_DATE, format = "%m/%d/%Y"), "%Y"))]
year_counts <- storm_subset[, .N, by = year][order(year)]
head(year_counts, 10)## year N
## <num> <int>
## 1: 1950 223
## 2: 1951 269
## 3: 1952 272
## 4: 1953 492
## 5: 1954 609
## 6: 1955 1413
## 7: 1956 1703
## 8: 1957 2184
## 9: 1958 2213
## 10: 1959 1813tail(year_counts, 10)## year N
## <num> <int>
## 1: 2002 36293
## 2: 2003 39752
## 3: 2004 39363
## 4: 2005 39184
## 5: 2006 44034
## 6: 2007 43289
## 7: 2008 55663
## 8: 2009 45817
## 9: 2010 48161
## 10: 2011 62174Given this clear increase in recorded events after the 1990s (consistent with improved record-keeping rather than an actual increase in storm frequency), we retain the full period for our totals but note this limitation explicitly. Raw totals for early decades are likely undercounts relative to more recent, more completely recorded years.
Standardizing event types (EVTYPE)
The raw EVTYPE field contains an unwieldy number of
near-duplicate labels (inconsistent capitalization, abbreviations, minor
spelling variants, and overlapping categories, e.g. “TSTM WIND”
vs. “THUNDERSTORM WIND” vs. “THUNDERSTORM WINDS”). Left uncleaned, this
would artificially fragment the counts for what are really the same
event type. We first normalize case and whitespace, then consolidate the
most common overlapping labels into a smaller set of standardized
categories using pattern matching. This cleanup focuses on the
highest-frequency categories, since these dominate the totals that
answer our two questions; rare/idiosyncratic labels are left as-is or
grouped into “OTHER”.
storm_subset[, evtype_clean := trimws(toupper(EVTYPE))]
length(unique(storm_subset$EVTYPE))## [1] 985length(unique(storm_subset$evtype_clean))## [1] 890storm_subset[grepl("TSTM|THUNDERSTORM", evtype_clean),
evtype_clean := "THUNDERSTORM WIND"]
storm_subset[grepl("TORNADO", evtype_clean),
evtype_clean := "TORNADO"]
storm_subset[grepl("FLASH FLOOD", evtype_clean),
evtype_clean := "FLASH FLOOD"]
storm_subset[grepl("^FLOOD|RIVER FLOOD|COASTAL FLOOD", evtype_clean),
evtype_clean := "FLOOD"]
storm_subset[grepl("HURRICANE|TYPHOON", evtype_clean),
evtype_clean := "HURRICANE/TYPHOON"]
storm_subset[grepl("^HEAT|EXCESSIVE HEAT", evtype_clean),
evtype_clean := "HEAT"]
storm_subset[grepl("^COLD|EXTREME COLD|WIND CHILL", evtype_clean),
evtype_clean := "COLD/WIND CHILL"]
storm_subset[grepl("LIGHTNING", evtype_clean),
evtype_clean := "LIGHTNING"]
storm_subset[grepl("^HAIL", evtype_clean),
evtype_clean := "HAIL"]
storm_subset[grepl("WINTER STORM", evtype_clean),
evtype_clean := "WINTER STORM"]
storm_subset[grepl("HIGH WIND", evtype_clean),
evtype_clean := "HIGH WIND"]
storm_subset[grepl("STORM SURGE", evtype_clean),
evtype_clean := "STORM SURGE/TIDE"]
length(unique(storm_subset$evtype_clean))## [1] 599Converting damage values to dollar amounts
PROPDMG/CROPDMG give a numeric magnitude,
while PROPDMGEXP/CROPDMGEXP give a separate
character code for its order of magnitude (“K” = thousand, “M” =
million, “B” = billion). These exponent columns also contain a long tail
of inconsistent/ambiguous codes (blank, digits, symbols). We first
examine their distribution, then map the standard codes to numeric
multipliers and treat all other, rare codes as a multiplier of 1 (i.e.,
the raw magnitude, unscaled), since they account for a small share of
records and no documented standard interpretation exists for them.
table(storm_subset$PROPDMGEXP, useNA = "ifany")##
## - ? + 0 1 2 3 4 5 6
## 465934 1 8 5 216 25 13 4 4 28 4
## 7 8 B h H K m M
## 5 1 40 1 6 424665 7 11330table(storm_subset$CROPDMGEXP, useNA = "ifany")##
## ? 0 2 B k K m M
## 618413 7 19 1 9 21 281832 1 1994exp_to_multiplier <- function(exp_code) {
exp_code <- toupper(trimws(exp_code))
mult <- rep(1, length(exp_code))
mult[exp_code == "K"] <- 1e3
mult[exp_code == "M"] <- 1e6
mult[exp_code == "B"] <- 1e9
mult
}
storm_subset[, prop_damage_dollars := PROPDMG * exp_to_multiplier(PROPDMGEXP)]
storm_subset[, crop_damage_dollars := CROPDMG * exp_to_multiplier(CROPDMGEXP)]
storm_subset[, total_damage_dollars := prop_damage_dollars + crop_damage_dollars]Aggregating by event type
Finally, we aggregate fatalities, injuries, and total economic damage by standardized event type, which is what the Results section presents.
health_by_event <- storm_subset[, .(
total_fatalities = sum(FATALITIES, na.rm = TRUE),
total_injuries = sum(INJURIES, na.rm = TRUE)
), by = evtype_clean]
health_by_event[, total_harm := total_fatalities + total_injuries]
health_by_event <- health_by_event[order(-total_harm)]
economic_by_event <- storm_subset[, .(
total_damage = sum(total_damage_dollars, na.rm = TRUE)
), by = evtype_clean]
economic_by_event <- economic_by_event[order(-total_damage)]Results
Which event types are most harmful to population health?
The figure below shows the top 10 event types by combined fatalities and injuries across the full period of record.
top_health <- head(health_by_event, 10)
par(mar = c(8, 5, 4, 2))
bp <- barplot(top_health$total_harm,
names.arg = top_health$evtype_clean,
las = 2,
col = "firebrick",
main = "Top 10 Event Types by Total Health Impact",
ylab = "Total Fatalities + Injuries")
Figure 1. Total combined fatalities and injuries (1950–2011) for the 10 most harmful event types, after consolidating near-duplicate event-type labels. Tornadoes are responsible for substantially more combined casualties than any other event type, followed by events such as excessive heat and thunderstorm wind.
top_health[, .(evtype_clean, total_fatalities, total_injuries, total_harm)]## evtype_clean total_fatalities total_injuries total_harm
## <char> <num> <num> <num>
## 1: TORNADO 5636 91407 97043
## 2: HEAT 3040 9019 12059
## 3: THUNDERSTORM WIND 754 9544 10298
## 4: FLOOD 488 6801 7289
## 5: LIGHTNING 817 5231 6048
## 6: FLASH FLOOD 1035 1802 2837
## 7: ICE STORM 89 1975 2064
## 8: HIGH WIND 296 1508 1804
## 9: WINTER STORM 217 1353 1570
## 10: HURRICANE/TYPHOON 135 1333 1468Which event types have the greatest economic consequences?
The figure below shows the top 10 event types by combined property and crop damage (in billions of dollars) across the full period of record.
top_economic <- head(economic_by_event, 10)
top_economic[, total_damage_billions := total_damage / 1e9]
par(mar = c(8, 5, 4, 2))
barplot(top_economic$total_damage_billions,
names.arg = top_economic$evtype_clean,
las = 2,
col = "steelblue",
main = "Top 10 Event Types by Total Economic Damage",
ylab = "Total Damage (Billions of USD)")
Figure 2. Total combined property and crop damage (1950–2011), in billions of USD, for the 10 event types with the greatest economic consequences. Flooding and hurricane/typhoon events account for the largest shares of total economic loss, with storm surge/tide and tornado events also contributing substantially.
top_economic[, .(evtype_clean, total_damage_billions)]## evtype_clean total_damage_billions
## <char> <num>
## 1: FLOOD 161.277554
## 2: HURRICANE/TYPHOON 90.872528
## 3: TORNADO 57.408060
## 4: STORM SURGE/TIDE 47.965579
## 5: HAIL 19.000564
## 6: FLASH FLOOD 18.438605
## 7: DROUGHT 15.018672
## 8: THUNDERSTORM WIND 13.850329
## 9: ICE STORM 8.967041
## 10: TROPICAL STORM 8.382237Conclusion
Tornadoes are the event type most harmful to population health by a wide margin, while flooding and hurricane/typhoon events are responsible for the greatest economic losses. Because event recording became substantially more complete after the 1990s, raw totals over the full 1950–2011 period likely understate the relative contribution of earlier decades.