Most Harmfull for Health & Highest Economic Consequences

By Oziel Hinojosa

2024-07-05

Synopsis

In this report we aim to identify which weather events cause the highest consequences monetarily and bodily injury. There was no hypothesis established prior to the research. Raw data from NOAA’s Storm Data database was obtained.

The analysis identified that Tornados cause the highest human health consequences (fatalities plus injuries) amongst all weather events reported to NOAA. Floods have the highest economic consequences. The two plots below clearly show this. There were 96,979 reported bodily injuries from tornados and over 150 billion in property damages from flooding.

Data Processing

The analysis started by reviewing NOAA’s National Weather Service Instruction 10-1605. By reviewing this document it is up to individuals to provide weather event information and to follow the data entry guidelines and coding conventions specified in the document. This highlighted the need to check for data quality errors along with the standard check NAs.

As shown in the code below. The CSV file is downloaded from NOAAs site. As of July 1st, 2024 there are 902,297 observations and 37 variables. Hence the code aims to provide subsets of the data. From section 2.7 of the document alphabetical characters are used to signify magnitude and include “K” for thousands, “M” for millions, and “B” for billions. Upon reviewing the columns PROPDMGEXP and CROPDMGEXP two things were apparent. First data entry or data quality issues. So assumptions were made for characters that did not represent a dollar figure. Second, a conversation from alphabetical codes is needed in order to calculate a dollar amount.

Hence I created to mapping tables in order to insert dollar amounts for property damage calculation. I used inner joins to remove non-matching records if they existed in order to avoid NAs and create the first subset for later aggregation and further grouping into human damage and property damage. Once the dollar amounts were available per event, the data was split and summarized. Along with only selecting the top ten events for plotting purposes - i.e. readibility.

## load the packages
library(dplyr)

## 
## Attaching package: 'dplyr'

## The following objects are masked from 'package:stats':
## 
##     filter, lag

## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

library(ggplot2)
library(scales)

## Download the data set
url <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
download.file(url, destfile = "StormData.bz2", method = "curl")

# load the data into a data frame
dfStormData <- read.csv("StormData.bz2", header = TRUE)

print("Structure of data frame.")

## [1] "Structure of data frame."

str(dfStormData)

## 'data.frame':    902297 obs. of  37 variables:
##  $ STATE__   : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ BGN_DATE  : chr  "4/18/1950 0:00:00" "4/18/1950 0:00:00" "2/20/1951 0:00:00" "6/8/1951 0:00:00" ...
##  $ BGN_TIME  : chr  "0130" "0145" "1600" "0900" ...
##  $ TIME_ZONE : chr  "CST" "CST" "CST" "CST" ...
##  $ COUNTY    : num  97 3 57 89 43 77 9 123 125 57 ...
##  $ COUNTYNAME: chr  "MOBILE" "BALDWIN" "FAYETTE" "MADISON" ...
##  $ STATE     : chr  "AL" "AL" "AL" "AL" ...
##  $ EVTYPE    : chr  "TORNADO" "TORNADO" "TORNADO" "TORNADO" ...
##  $ BGN_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ BGN_AZI   : chr  "" "" "" "" ...
##  $ BGN_LOCATI: chr  "" "" "" "" ...
##  $ END_DATE  : chr  "" "" "" "" ...
##  $ END_TIME  : chr  "" "" "" "" ...
##  $ COUNTY_END: num  0 0 0 0 0 0 0 0 0 0 ...
##  $ COUNTYENDN: logi  NA NA NA NA NA NA ...
##  $ END_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ END_AZI   : chr  "" "" "" "" ...
##  $ END_LOCATI: chr  "" "" "" "" ...
##  $ LENGTH    : num  14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
##  $ WIDTH     : num  100 150 123 100 150 177 33 33 100 100 ...
##  $ F         : int  3 2 2 2 2 2 2 1 3 3 ...
##  $ MAG       : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ FATALITIES: num  0 0 0 0 0 0 0 0 1 0 ...
##  $ INJURIES  : num  15 0 2 2 2 6 1 0 14 0 ...
##  $ PROPDMG   : num  25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
##  $ PROPDMGEXP: chr  "K" "K" "K" "K" ...
##  $ CROPDMG   : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CROPDMGEXP: chr  "" "" "" "" ...
##  $ WFO       : chr  "" "" "" "" ...
##  $ STATEOFFIC: chr  "" "" "" "" ...
##  $ ZONENAMES : chr  "" "" "" "" ...
##  $ LATITUDE  : num  3040 3042 3340 3458 3412 ...
##  $ LONGITUDE : num  8812 8755 8742 8626 8642 ...
##  $ LATITUDE_E: num  3051 0 0 0 0 ...
##  $ LONGITUDE_: num  8806 0 0 0 0 ...
##  $ REMARKS   : chr  "" "" "" "" ...
##  $ REFNUM    : num  1 2 3 4 5 6 7 8 9 10 ...

#### From the Storm Data documentation provided through this project 2 course material
#### I can deduce that:
#### -- There are 48 event types. All started being captured starting on Jan 1996
#### -- Prior to that Tornado from 1950-1955. 
#### -- Then Tornado, Thunderstorm Wind, & Hail from Jan 1955 - Jan 1996
#### -- Cost is estimated based on APPENDIX B – Property Damage Estimates
#### -- There are direct and indirect reports of fatalities and injuries
#### -- Based on the assignment the following columns are required. 
####    --- EVTYPE (event type), FATALITIES, INJURIES, 
####    --- PROPDMG (Flood-Related Damage), PROPDMGEXP, CROPDMG, & CROPDMGEXP

#### -- DMG = damage. EXP =  magnitude. Estimates should be rounded to 
#### three significant digits, followed by an alphabetical character signifying the #### magnitude of the number, i.e., 1.55B

#### NOOA Database column information at
#### https://www.ncei.noaa.gov/pub/data/swdi/stormevents/csvfiles/Storm-Data-Bulk-csv-Format.pdf

#### We will need to convert EXP to their numeric representation. Check DQ and
#### arbitrarily assign a value. 
#### Property = "K" "M" ""  "B" "m" "+" "0" "5" "6" "?" "4" "2" "3" "h" "7" "H" "-" "1" "8"
#### Crop = ""  "M" "K" "m" "B" "?" "0" "k" "2"


#### Keep only the sum damage columns, sum fatalities and injuries and remove NAs
dfSubset <- dfStormData %>% select(EVTYPE,FATALITIES, INJURIES, PROPDMG, 
                                   PROPDMGEXP, CROPDMG, CROPDMGEXP)
rm(dfStormData)
              
dictPropOld <- c("K", "M", "", "B", "m", "+", "0", "5", "6", "?", "4", "2", "3", "h", "7", "H", "-", "1", "8")
dictPropNew <- c(10^3, 10^6, 1, 10^9, 10^6, 0, 1, 10^5, 10^6, 0, 10^4, 100, 10^3, 100, 10^7, 10^2, 0, 10, 10^8)
dfPropDmg <- tibble(PROPDMGEXP = dictPropOld, propDmgVal = dictPropNew)

dictCorpOld <- c("", "M", "K", "m", "B", "?", "0", "k", "2")
dictCorpNew <- c(1, 10^6, 10^3, 10^6, 10^9, 0, 1, 10^3, 100)
dfCropDmg <- tibble(CROPDMGEXP = dictCorpOld, cropDmgVal = dictCorpNew)

dfSubset <- dfSubset %>% select(EVTYPE,FATALITIES, INJURIES, PROPDMG, 
                                      PROPDMGEXP, CROPDMG, CROPDMGEXP) 

finalDF <- inner_join(dfSubset,dfPropDmg,by = join_by("PROPDMGEXP"))
finalDF <- inner_join(finalDF, dfCropDmg, by = join_by("CROPDMGEXP"))
finalDF <- finalDF %>% mutate(finalDF, totalDmg = (cropDmgVal*CROPDMG)+(propDmgVal*PROPDMG))
finalDF <- finalDF %>% mutate(totalHumanDmg = FATALITIES + INJURIES)


rm(dfSubset)
str(finalDF)

## 'data.frame':    902297 obs. of  11 variables:
##  $ EVTYPE       : chr  "TORNADO" "TORNADO" "TORNADO" "TORNADO" ...
##  $ FATALITIES   : num  0 0 0 0 0 0 0 0 1 0 ...
##  $ INJURIES     : num  15 0 2 2 2 6 1 0 14 0 ...
##  $ PROPDMG      : num  25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
##  $ PROPDMGEXP   : chr  "K" "K" "K" "K" ...
##  $ CROPDMG      : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CROPDMGEXP   : chr  "" "" "" "" ...
##  $ propDmgVal   : num  1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 ...
##  $ cropDmgVal   : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ totalDmg     : num  25000 2500 25000 2500 2500 2500 2500 2500 25000 25000 ...
##  $ totalHumanDmg: num  15 0 2 2 2 6 1 0 15 0 ...

### Now that the data is cleaned and the observations we need have been created 
### let us focus on plotting.
### Let us create two dataframews. One for top 10 damage and the second on most harmful
dfHumanDmg <-  finalDF %>% group_by(EVTYPE) %>% 
  summarise(total = sum(totalHumanDmg)) %>% 
  select(EVTYPE, total) %>% arrange(desc(total)) %>% slice(1:10)

dfPropDmg <-  finalDF %>% group_by(EVTYPE) %>% 
  summarise(total = sum(totalDmg)) %>% 
  select(EVTYPE, total) %>% arrange(desc(total)) %>% slice(1:10)

Results

As show in the plots. Tornados are at the top in human fatalities plus injuries. And floods cause the most property damage. In retrospect one could have established a hypothesis that due to high wind speeds, flying debris, and unpredictability Tornados would be at the top. And water damage from water during a flooding event could have also be a hypothesis.

#Let's plot

ggplot(dfHumanDmg, aes(x = reorder(EVTYPE, total), y = total)) +
  geom_bar(stat = "identity", fill = "#FFDB6D", color = "#C4961A") +
  labs(x = "Event Type", y = "# of Fatalities + Injuries", title = "Top 10 Fatalities+Injuries by Weather Event") +
  coord_flip()  +
  theme_classic()