Exploring the NOAA Storm Database
Anne Strader
5/26/2020
Synopsis
Storms and other severe weather events can cause both public health and economic problems for communities and municipalities. Many severe events can result in fatalities, injuries, and property damage, and preventing such outcomes to the extent possible is a key concern.
This project involves exploring the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. This database tracks characteristics of major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage.
The following questions are investigated in this analysis:
- Across the United States, which types of events (as indicated in the EVTYPE) variable are most harmful with respect to population health?
- Across the United States, which types of events have the greatest economic consequences?
Load Necessary R Libraries
library(dplyr)## Warning: package 'dplyr' was built under R version 3.6.2##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(ggplot2)## Warning: package 'ggplot2' was built under R version 3.6.3Data Processing
The NOAA storm dataset for this project is available here, in the form of a .csv file compressed using the bzip2 algorithm.
First, the data are downloaded:
# define the URL where the source data are stored
sourceURL <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
# define the source data zip file name
sourceDataZipFile <- "StormData.csv.bz2"
# check if dataset has already been downloaded in working directory, and download it if not
if (!file.exists(sourceDataZipFile)) {
download.file(sourceURL, sourceDataZipFile)
}Read in the data and examine the basic structure of the dataset:
# read the dataset
stormData <- read.csv(bzfile("StormData.csv.bz2"))
# summarize the dataset
str(stormData)## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
## $ BGN_TIME : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
## $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
## $ STATE : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ EVTYPE : Factor w/ 985 levels " HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : Factor w/ 35 levels ""," N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_LOCATI: Factor w/ 54429 levels "","- 1 N Albion",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_DATE : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_TIME : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_LOCATI: Factor w/ 34506 levels "","- .5 NNW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ WFO : Factor w/ 542 levels ""," CI","$AC",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ ZONENAMES : Factor w/ 25112 levels ""," "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : Factor w/ 436781 levels "","-2 at Deer Park\n",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...head(stormData)## STATE__ BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE EVTYPE
## 1 1 4/18/1950 0:00:00 0130 CST 97 MOBILE AL TORNADO
## 2 1 4/18/1950 0:00:00 0145 CST 3 BALDWIN AL TORNADO
## 3 1 2/20/1951 0:00:00 1600 CST 57 FAYETTE AL TORNADO
## 4 1 6/8/1951 0:00:00 0900 CST 89 MADISON AL TORNADO
## 5 1 11/15/1951 0:00:00 1500 CST 43 CULLMAN AL TORNADO
## 6 1 11/15/1951 0:00:00 2000 CST 77 LAUDERDALE AL TORNADO
## BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END COUNTYENDN
## 1 0 0 NA
## 2 0 0 NA
## 3 0 0 NA
## 4 0 0 NA
## 5 0 0 NA
## 6 0 0 NA
## END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES INJURIES PROPDMG
## 1 0 14.0 100 3 0 0 15 25.0
## 2 0 2.0 150 2 0 0 0 2.5
## 3 0 0.1 123 2 0 0 2 25.0
## 4 0 0.0 100 2 0 0 2 2.5
## 5 0 0.0 150 2 0 0 2 2.5
## 6 0 1.5 177 2 0 0 6 2.5
## PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES LATITUDE LONGITUDE
## 1 K 0 3040 8812
## 2 K 0 3042 8755
## 3 K 0 3340 8742
## 4 K 0 3458 8626
## 5 K 0 3412 8642
## 6 K 0 3450 8748
## LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1 3051 8806 1
## 2 0 0 2
## 3 0 0 3
## 4 0 0 4
## 5 0 0 5
## 6 0 0 6The dataset contains 37 variables and 902,297 observations. Information such as event time, event location, event type is included, as well as information about injuries/fatalities and property damage.
To answer the questions mentioned in the synopsis, we need information about:
- the event type
- EVTYPE (factor, type of event)
- damage to population health (injuries, fatalities, etc.)
- FATALITIES (numeric, number of fatalities)
- INJURIES (numeric, number of injuries)
- damage to property
- PROPDMG (numeric, monetary property damage amount, to be multiplied by an exponent specified in the variable “PROPDMGEXP”)
- PROPDMGEXP (K = thousands, M = millions, B = billions)
- CROPDMG (numeric, monetary crop damage amount, to be multiplied by an exponent specified in the variable “CROPDMGEXP”)
- CROPDMGEXP (K = thousands, M = millions, B = billions)
The dataset is then filtered, keeping only the relevant variables:
# create a vector of the relevant variable names
stormDataRelVars <- c("EVTYPE", "FATALITIES", "INJURIES", "PROPDMG", "PROPDMGEXP", "CROPDMG", "CROPDMGEXP")
# subset the dataset so that it only contains relevant variables
stormData <- stormData[ , stormDataRelVars]
head(stormData)## EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## 1 TORNADO 0 15 25.0 K 0
## 2 TORNADO 0 0 2.5 K 0
## 3 TORNADO 0 2 25.0 K 0
## 4 TORNADO 0 2 2.5 K 0
## 5 TORNADO 0 2 2.5 K 0
## 6 TORNADO 0 6 2.5 K 0Exponents are missing for crop damage in the first few rows of the dataset. These observations all have $0 crop damage, but let’s check the unique values for both exponents anyway:
# unique property damage exponents
unique(stormData$PROPDMGEXP)## [1] K M B m + 0 5 6 ? 4 2 3 h 7 H - 1 8
## Levels: - ? + 0 1 2 3 4 5 6 7 8 B h H K m M# unique crop damage exponents
unique(stormData$CROPDMGEXP)## [1] M K m B ? 0 k 2
## Levels: ? 0 2 B k K m MThe damage exponents (values of PROPDMGEXP and CROPDMGEXP) are defined as follows, based on the NOAA documentation and exponent guide:
- H, h = hundreds
- K, k = thousands
- M, m = millions
- B, b = billions
- (+) = 1
- (-) = 0
- (?) = 0
- blank/empty character = 0
- 0…8 = 10
Next, multipliers are defined for each exponent and the monetary damage values PROPDMG and CROPDMG are updated:
# define and sort unique exponent values
exponents <- sort(unique(as.character(stormData$PROPDMGEXP)))
# define multipliers corresponding to exponent values
dmgMultipliers <- c(0, 0, 0, 1, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10^9, 100, 100, 1000, 10^6, 10^6)
# define column for property damage multiplier
stormData$PROPDMGMULTI <- dmgMultipliers[match(stormData$PROPDMGEXP, exponents)]
# define column for crop damage multiplier
stormData$CROPDMGMULTI <- dmgMultipliers[match(stormData$CROPDMGEXP, exponents)]
# update property damage costs
stormData$PROPDMG <- stormData$PROPDMG * stormData$PROPDMGMULTI
# update crop damage costs
stormData$CROPDMG <- stormData$CROPDMG * stormData$CROPDMGMULTI
# remove exponent and multiplier columns in dataset
stormData <- stormData[!names(stormData) %in% c("PROPDMGEXP", "PROPDMGMULTI", "CROPDMGEXP", "CROPDMGMULTI")]
# look at the dataset
head(stormData)## EVTYPE FATALITIES INJURIES PROPDMG CROPDMG
## 1 TORNADO 0 15 25000 0
## 2 TORNADO 0 0 2500 0
## 3 TORNADO 0 2 25000 0
## 4 TORNADO 0 2 2500 0
## 5 TORNADO 0 2 2500 0
## 6 TORNADO 0 6 2500 0Data Processing for Question 1: Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?
Create a data frame that contains the total number of fatalities per event type and sort by descending total number of fatalities:
deathByEVTYPE <- stormData %>% select(EVTYPE, FATALITIES) %>% group_by(EVTYPE) %>% summarize(totalFatalities = sum(FATALITIES)) %>% arrange(-totalFatalities)
head(deathByEVTYPE)## # A tibble: 6 x 2
## EVTYPE totalFatalities
## <fct> <dbl>
## 1 TORNADO 5633
## 2 EXCESSIVE HEAT 1903
## 3 FLASH FLOOD 978
## 4 HEAT 937
## 5 LIGHTNING 816
## 6 TSTM WIND 504Create a data frame that contains the total number of injuries per event type and sort by descending total number of injuries:
injuryByEVTYPE <- stormData %>% select(EVTYPE, INJURIES) %>% group_by(EVTYPE) %>% summarize(totalInjuries = sum(INJURIES)) %>% arrange(-totalInjuries)
head(injuryByEVTYPE)## # A tibble: 6 x 2
## EVTYPE totalInjuries
## <fct> <dbl>
## 1 TORNADO 91346
## 2 TSTM WIND 6957
## 3 FLOOD 6789
## 4 EXCESSIVE HEAT 6525
## 5 LIGHTNING 5230
## 6 HEAT 2100Data Processing for Question 2: Across the United States, which types of events have the greatest economic consequences?
Create a column in the stormData data frame with total monetary damage:
stormData$totalDMG <- stormData$PROPDMG + stormData$CROPDMG
head(stormData)## EVTYPE FATALITIES INJURIES PROPDMG CROPDMG totalDMG
## 1 TORNADO 0 15 25000 0 25000
## 2 TORNADO 0 0 2500 0 2500
## 3 TORNADO 0 2 25000 0 25000
## 4 TORNADO 0 2 2500 0 2500
## 5 TORNADO 0 2 2500 0 2500
## 6 TORNADO 0 6 2500 0 2500Create a data frame that contains the total monetary damage per event type and sort by descending amount of total monetary damage:
damageByEVTYPE <- stormData %>% select(EVTYPE, totalDMG) %>% group_by(EVTYPE) %>% summarize(dmgByEvent = sum(totalDMG)) %>% arrange(-dmgByEvent)
head(damageByEVTYPE)## # A tibble: 6 x 2
## EVTYPE dmgByEvent
## <fct> <dbl>
## 1 FLOOD 150319678250
## 2 HURRICANE/TYPHOON 71913712800
## 3 TORNADO 57352117607
## 4 STORM SURGE 43323541000
## 5 FLASH FLOOD 17562132111
## 6 DROUGHT 15018672000Results
Question 1: Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?
Plot the seven event types with the greatest number of fatalities:
# initiate plot
g <- ggplot(deathByEVTYPE[1:7, ], aes(x=reorder(EVTYPE, -totalFatalities), y=totalFatalities))
# create a bar plot of total numbers of fatalities by event type
g <- g + geom_bar(stat="identity", aes(fill=EVTYPE))
# add a title and relabel axes
g <- g + labs(x="event type", y = "total number of fatalities") + ggtitle("Weather Events with the Highest Numbers of Fatalities")
# center title and adjust event type labels on x-axis
g <- g + theme(plot.title=element_text(hjust=0.5), axis.text.x = element_text(angle=90, hjust=1, vjust=0.5))
# display plot
g
Tornados cause by far the most fatalities, but there are also substantial contributions from excessive heat and flash flooding.
Plot the seven event types with the greatest number of injuries:
# initiate plot
g <- ggplot(injuryByEVTYPE[1:7, ], aes(x=reorder(EVTYPE, -totalInjuries), y=totalInjuries))
# create a bar plot of total numbers of fatalities by event type
g <- g + geom_bar(stat="identity", aes(fill=EVTYPE))
# add a title and relabel axes
g <- g + labs(x="event type", y = "total number of injuries") + ggtitle("Weather Events with the Highest Numbers of Injuries")
# center title and adjust event type labels on x-axis
g <- g + theme(plot.title=element_text(hjust=0.5), axis.text.x = element_text(angle=90, hjust=1, vjust=0.5))
# display plot
g
As well as causing the most fatalities, tornados cause several times as many injuries as other event types.
Question 2: Across the United States, which types of events have the greatest economic consequences?
Plot the seven event types with the greatest total monetary damage (property + crop):
# initiate plot
g <- ggplot(damageByEVTYPE[1:7, ], aes(x=reorder(EVTYPE, -dmgByEvent), y=dmgByEvent))
# create a bar plot of total numbers of fatalities by event type
g <- g + geom_bar(stat="identity", aes(fill=EVTYPE))
# add a title and relabel axes
g <- g + labs(x="event type", y = "total monetary damage (USD)") + ggtitle("Weather Events with the Highest Economic Damage (Property + Crop)")
# center title and adjust event type labels on x-axis
g <- g + theme(plot.title=element_text(hjust=0.5), axis.text.x = element_text(angle=90, hjust=1, vjust=0.5))
# display plot
g
Flooding causes the greatest amount of economic damage, followed by the hurricane/typhoon category and tornados.