The NOAA Storm Data is analysed to assess the impact on human health and the economic impact of different weather events in the United States. For the analysis a specific version of the NOAA Storm Data from 2011 was used. The analysis focused on the time interval from 1993 to 2011, and the event types were as defined by NOAA in 2007. To assess the impact on human health the overall number of fatalities and injuries resulting from the different event types was calculated. The highest overall number of fatalities were reported for the event type “Excessive Heat”, and then highest number of injuries was reported for the event type “Tornado”. The economic impact was determined by calculating the overall cost resulting from property damage and crop damage. The highest cost from property damage was reported for the event type “Hurricane (Typhoon)”, while the highest cost from crop damage was reported for event type “Drought”.
The following software was used for the analysis:
R version 4.2.2 (2022-10-31 ucrt)
RStudio 2022.12.0+353 “Elsbeth Geranium” Release (7d165dcfc1b6d300eb247738db2c7076234f6ef0, 2022-12-03) for Windows
Set knitr options to not show warnings and messages to make the document less verbose.
knitr::opts_chunk$set(echo = TRUE, warning = FALSE, message = FALSE)
The .bz2 file is unzipped with bunzi2 from the R.utils package. Data processing will be done with dplyr, tidyr and lubridate, therefore the tidyverse package is loaded. Event-types are matched with amatch() from the stringdist package. Plots are done with lattice package, and the panel.needle() function is from the caret package.
library(R.utils)
library(tidyverse)
library(stringdist)
library(lattice)
library(caret)
Data are downloaded using link provided in instructions. The code checks whether the data file in the sub-directory data1 exists, and if not, downloads and unzips file in data directory.
main_dir <- getwd()
sub_dir <- "data1"
## check if "data" sub-directory exists, if not download and unzip files in data folder
if (!file.exists(file.path(paste(main_dir, sub_dir,"StormData.csv", sep ="/")))){
url <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
temp <- tempfile()
dir.create(paste(main_dir, sub_dir, sep ="/"))
download.file(url, temp)
bunzip2(filename = temp, destname = file.path(paste(main_dir, sub_dir,"StormData.csv", sep ="/")), remove = FALSE, skip = TRUE)
unlink(c(temp))
rm(temp)
} else {print("data available")}
## [1] "data available"
Data are downloaded and in the data1 sub-directory.
The data are read-in with standard read.csv.
a <- Sys.time()
StormData <- read.csv("./data1/StormData.csv", header = TRUE)
b <- Sys.time()
b - a
## Time difference of 21.50598 secs
The data set is quite large with 902297 rows and 37 variables, it was therefore cached to reduce run-time during report creation.
The official Event Names are taken from the “NATIONAL WEATHER SERVICE INSTRUCTION 10-1605 AUGUST 17, 2007”. The Event Names were manually copied from the document, saved as the text file EVENTTABLE.txt and read into R. To ensure reproducibility data loaded into R are printed in this document.
EVTAB <- read.table("./EVENTTABLE.txt", header = TRUE)
EVTAB
## EVENT
## 1 Astronomical Low Tide
## 2 Avalanche
## 3 Blizzard
## 4 Coastal Flood
## 5 Cold/Wind Chill
## 6 Debris Flow
## 7 Dense Fog
## 8 Dense Smoke
## 9 Drought
## 10 Dust Devil
## 11 Dust Storm
## 12 Excessive Heat
## 13 Extreme Cold/Wind Chill
## 14 Flash Flood
## 15 Flood
## 16 Frost/Freeze
## 17 Funnel Cloud
## 18 Freezing Fog
## 19 Hail
## 20 Heat
## 21 Heavy Rain
## 22 Heavy Snow
## 23 High Surf
## 24 High Wind
## 25 Hurricane (Typhoon)
## 26 Ice Storm
## 27 Lake-Effect Snow
## 28 Lakeshore Flood
## 29 Lightning
## 30 Marine Hail
## 31 Marine High Wind
## 32 Marine Strong Wind
## 33 Marine Thunderstorm Wind
## 34 Rip Current
## 35 Seiche
## 36 Sleet
## 37 Storm Surge/Tide
## 38 Strong Wind
## 39 Thunderstorm Wind
## 40 Tornado
## 41 Tropical Depression
## 42 Tropical Storm
## 43 Tsunami
## 44 Volcanic Ash
## 45 Waterspout
## 46 Wildfire
## 47 Winter Storm
## 48 Winter Weather
EVTAB$EVENT <- str_to_upper(EVTAB$EVENT)
No systematic evaluation for outliers or data validity is performed. However, in the PROPDMG field a very high value is observed for the entry with REFNUM = “605943”. The REMARKS field was inspected, and based on the several millions $ damage from a flood in Napa valley reported there, the entry in the field PROPDMGEXP was changed from “B” to “M”. It is very likely an erroneous data-entry.
StormData$PROPDMGEXP[StormData$REFNUM == "605943"] <- "M"
The approach is to compare the overall fatalities, injuries, crop damage, and property damage in a given time-span (1993 to 2011) for the different event types.
Data are pre-processed with the following steps:
With these pre-processing steps the data are in the required shape for further analysis.
a <- Sys.time()
StormData$BGN_DATE <- gsub(' 0:00:00','', StormData$BGN_DATE)
StormData$BGN_DATE <- mdy(StormData$BGN_DATE)
SD.Processed <- StormData |>
select(BGN_DATE, STATE, EVTYPE, F, MAG, FATALITIES, INJURIES, PROPDMG, PROPDMGEXP, CROPDMG, CROPDMGEXP, REMARKS, REFNUM) |>
filter(BGN_DATE > "1993-01-01" & (FATALITIES > 0 | INJURIES > 0 | PROPDMG > 0 | CROPDMG > 0)) |>
mutate(PROPDMGEXP = case_match(PROPDMGEXP,
c("", "+", "-", "?") ~ "1",
as.character(c(1:9)) ~ "10",
c("H", "h") ~ "100",
c("K", "k") ~ "1000",
c("M", "m") ~ "1000000",
c("B", "b") ~ "1000000000",
.default = PROPDMGEXP)) |>
mutate(CROPDMGEXP = case_match(CROPDMGEXP,
c("", "+", "-", "?") ~ "1",
as.character(c(1:9)) ~ "10",
c("H", "h") ~ "100",
c("K", "k") ~ "1000",
c("M", "m") ~ "1000000",
c("B", "b") ~ "1000000000",
.default = CROPDMGEXP)) |>
mutate_at(c("PROPDMGEXP", "CROPDMGEXP"), as.numeric) |>
mutate(PROPERTY = PROPDMG*PROPDMGEXP/1000000000, CROP = CROPDMG*CROPDMGEXP/1000000000) |>
mutate(EVTYPE = EVTAB$EVENT[amatch(EVTYPE, EVTAB$EVENT, maxDist=30)]) |>
pivot_longer(cols = c("FATALITIES", "INJURIES"),
names_to = "CASETYPE",
values_to = "CASES") |>
pivot_longer(cols = c("PROPERTY", "CROP"),
names_to = "DMGTYPE",
values_to = "DMG")
b <- Sys.time()
b-a
## Time difference of 5.388555 secs
For further analysis, separate data.frames for fatalities (SD.Fatalities) and financial impact (SD.DMG) are created by summarizing the fatalities and injuries, and crop and property damage, respectively.
SD.Fatalities <- SD.Processed |>
group_by(EVTYPE, CASETYPE) |>
summarise(sum = sum(CASES), n = n())
SD.DMG <- SD.Processed |>
group_by(EVTYPE, DMGTYPE) |>
summarise(sum = sum(DMG), n = n())
xyplot(as.factor(EVTYPE) ~ sum | CASETYPE,
data = SD.Fatalities,
layout = c(2,1),
main = "Impact of weather events on human health",
xlab = "Total number of cases (1993 - 2011)",
ylab = "Event (NWS Event Name)",
scales=list(x = "free", y=list(rot=0)),
panel = function(...){
panel.needle(...)
})
SD.Fatalities |> as.tibble() |> rmarkdown::paged_table()
The plot and the table show the impact of different weather events on human health are measured by overall number of fatalities and injuries in the time interval from 1993 to 2011. The weather events with the highest impact on human health are the following:
It is noteworthy that the event type “HURRICANE (TYPHOON)” is only associated with 144 fatalities, despite the fact that for instance Hurricane Katrina in 2005 resulted in 1392 direct and indirect casualties in what was described as the single most devastating natural disaster in the United States (Knabb, Rhome, and Brown 2023). The number of injuries associated to the event type “HURRICANE (TYPHOON)” is 2558, which seems also comparatively low for this event type considering cases such as Hurricane Katrina.
To check whether the fatalities resulting from Hurricane Katrina were recorded under a different event type than “HURRICANE (TYPHOON)” the REMARKS field of the original data set is checked whether it contains the string “Katrina”. Fatalities of these records are summarized.
Kat.N.F <- length(StormData$FATALITIES[grepl("Katrina", StormData$REMARKS)])
Kat.N.F
## [1] 296
Kat.S.F <- sum(StormData$FATALITIES[grepl("Katrina", StormData$REMARKS)])
Kat.S.F
## [1] 31
There are 296 entries with the string “Katrina” in their REMARKS field. These entries record overall 31 fatalities. This number of fatalities is lower than the overall number of fatalities attributed to Hurricane Katrina. This is indicating that the fatalities related to Hurricane Katrina are not recorded in the current version of the Storm Data set.
xyplot(as.factor(EVTYPE) ~ sum | DMGTYPE,
data = SD.DMG,
layout = c(2,1),
main = "Economic impact of weather events",
xlab = "Total economic impact (1993 - 2011) [billion $]",
ylab = "Event (NWS Event Name)",
scales=list(x = "free", y=list(rot=0)),
panel = function(...){
panel.needle(...)
})
SD.DMG |> as.tibble() |> rmarkdown::paged_table()
SD.Processed$REFNUM[SD.Processed$EVTYPE == "STORM SURGE/TIDE" & SD.Processed$DMG == max(SD.Processed$DMG)]
## [1] 577616 577616
SD.Processed$DMG[SD.Processed$REFNUM == "577616"]
## [1] 31.3 0.0 31.3 0.0
The plot and the table above show the economic impact of different weather events as measured by overall cost for crop damage and property damage in the time interval of 1993 to 2011. The weather event types with the highest impact are the following:
It is noteworthy that the property damage for one particular event can be distributed over several event types. For instance, there is 31.3 billion $ in property damage recorded under event type “STORM SURGE/TIDE” (REFNUM : 577616) that can be attributed to Hurricane Katrina based on the inspection of the REMARKS field.
The simple strategy of summarizing fatalities, injuries, crop damage, and property damage over the time period of 1993 to 2011 is used to identify those event types that have the largest human health impact and economic impact. The event types “EXCESSIVE HEAT” and “TORNADO” are identified as having the highest impact on human health in terms of fatalities and injuries, respectively. The highest economic impact is seen for the event types “DROUGHT” and “HURRICANE (TYPHOON)” for crop damage and property damage, respectively. Closer inspection of the Storm Data data set indicated that record of very significant events (e.g. Hurricane Katrina) may be incomplete. Further research with updated versions of the Storm Data data set are needed to verify the conclusions of the report.