1. Synopsis

The US National Oceanic and Atmospheric Administration (NOAA) has established a database recording storm events and the impact on population and infrastructures since 1950 in the United States. We can use this database to determine which events are the most harmful on population or have the greatest economic impact. Since the data collection methods have changed over the time, we have focused our analysis from January 2001 to end of 2011. We first evaluate the evolution of impact on population and economy consequences over the time by focusing on 6 variables (“injuries”, “fatalities”, “CROPDMG” and “CROPDMGEXP”, “PROPDMG” and “PROPDMGEXP”). To assess which events were the most harmful on population health, we calculated the total number of persons injured or killed since 2001 for each of the 48 events. We processed the same way to evaluate the economical consequences on properties and crops.

2. Data processing

We download the file using the link provided in the instructions page:

if(!file.exists("StormData.csv.bz2")){
        download.file("https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2", "StormData.csv.bz2")}
data <- read.csv("StormData.csv.bz2")

## The following libraries are loaded
library(dplyr)
library(stringr)
library(stringdist)
library(lubridate)
library(ggplot2)
library(reshape2)
library(gridExtra)
library(forcats)

We first evaluate the content of the dataset:

str(data)
## 'data.frame':    902297 obs. of  37 variables:
##  $ STATE__   : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ BGN_DATE  : chr  "4/18/1950 0:00:00" "4/18/1950 0:00:00" "2/20/1951 0:00:00" "6/8/1951 0:00:00" ...
##  $ BGN_TIME  : chr  "0130" "0145" "1600" "0900" ...
##  $ TIME_ZONE : chr  "CST" "CST" "CST" "CST" ...
##  $ COUNTY    : num  97 3 57 89 43 77 9 123 125 57 ...
##  $ COUNTYNAME: chr  "MOBILE" "BALDWIN" "FAYETTE" "MADISON" ...
##  $ STATE     : chr  "AL" "AL" "AL" "AL" ...
##  $ EVTYPE    : chr  "TORNADO" "TORNADO" "TORNADO" "TORNADO" ...
##  $ BGN_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ BGN_AZI   : chr  "" "" "" "" ...
##  $ BGN_LOCATI: chr  "" "" "" "" ...
##  $ END_DATE  : chr  "" "" "" "" ...
##  $ END_TIME  : chr  "" "" "" "" ...
##  $ COUNTY_END: num  0 0 0 0 0 0 0 0 0 0 ...
##  $ COUNTYENDN: logi  NA NA NA NA NA NA ...
##  $ END_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ END_AZI   : chr  "" "" "" "" ...
##  $ END_LOCATI: chr  "" "" "" "" ...
##  $ LENGTH    : num  14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
##  $ WIDTH     : num  100 150 123 100 150 177 33 33 100 100 ...
##  $ F         : int  3 2 2 2 2 2 2 1 3 3 ...
##  $ MAG       : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ FATALITIES: num  0 0 0 0 0 0 0 0 1 0 ...
##  $ INJURIES  : num  15 0 2 2 2 6 1 0 14 0 ...
##  $ PROPDMG   : num  25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
##  $ PROPDMGEXP: chr  "K" "K" "K" "K" ...
##  $ CROPDMG   : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CROPDMGEXP: chr  "" "" "" "" ...
##  $ WFO       : chr  "" "" "" "" ...
##  $ STATEOFFIC: chr  "" "" "" "" ...
##  $ ZONENAMES : chr  "" "" "" "" ...
##  $ LATITUDE  : num  3040 3042 3340 3458 3412 ...
##  $ LONGITUDE : num  8812 8755 8742 8626 8642 ...
##  $ LATITUDE_E: num  3051 0 0 0 0 ...
##  $ LONGITUDE_: num  8806 0 0 0 0 ...
##  $ REMARKS   : chr  "" "" "" "" ...
##  $ REFNUM    : num  1 2 3 4 5 6 7 8 9 10 ...

We will adopt a strategy in 3 parts to clean the dataset:
1. Select the columns and rows of interest
2. Clean the “EVTYPE” column
3. Handle the exponent values for crops and properties

2.1. Select the columns and rows of interest

For the purpose of the study, we only need a couple of variables: STATE (not particularly needed but nice to have), DATE (to select rows based on date), EVTYPE (the type of events), FATALITIES, INJURIES (fatalities and injuries to respond to the question about the harmfulness of the event), PROPDMG, PROPDMGEXP, CROPDMG, CROPDMGEXP (these additional variables to measure the damages on properties and crops).

data[c("DATE", "TIME")] <- str_split_fixed(data$BGN_DATE, " ", 2) # to split time and date into 2 columns
data_tidy <- data %>% # to select only certain column and create a new dataset
        select(STATE, DATE, EVTYPE, FATALITIES, INJURIES, PROPDMG, PROPDMGEXP, CROPDMG, CROPDMGEXP)
data_tidy$DATE <- mdy(data_tidy$DATE)

We got a new dataset called “data_tidy” with 9 variables only and data since 1950, when the first events are recorded. We can now check the number of events measured.

str(unique(data_tidy$EVTYPE))
##  chr [1:985] "TORNADO" "TSTM WIND" "HAIL" "FREEZING RAIN" "SNOW" ...

We get a large list of events. According to the NOAA website https://www.ncdc.noaa.gov/stormevents/details.jsp all events were only recorded from 1996, but standardization of the events did not happen before 2000 when a drop-down selector for Event Type on the data entry interface was included https://www.ncdc.noaa.gov/stormevents/details.jsp?type=collection. Hence, it will be more advised to filter the events prior to 2000. We can start our selection from January 2001 to make sure this standardization has been properly implemented.

data_tidy <- with(data_tidy, data_tidy[(DATE >= "2001-01-01"), ]) # select data from January 2001
unique(data_tidy$EVTYPE) # to check the number of unique events recorded
##   [1] "TSTM WIND"                      "HAIL"                          
##   [3] "TORNADO"                        "FLASH FLOOD"                   
##   [5] "HIGH WIND"                      "FLOOD"                         
##   [7] "WILD/FOREST FIRE"               "WINTER STORM"                  
##   [9] "EXCESSIVE HEAT"                 "LIGHTNING"                     
##  [11] "FUNNEL CLOUD"                   "HEAVY SNOW"                    
##  [13] "EXTREME COLD"                   "WATERSPOUT"                    
##  [15] "COASTAL FLOODING"               "TROPICAL STORM"                
##  [17] "TSTM WIND/HAIL"                 "HEAVY RAIN"                    
##  [19] "RIP CURRENTS"                   "HEAVY SURF"                    
##  [21] "AVALANCHE"                      "EXTREME WINDCHILL"             
##  [23] "BLIZZARD"                       "ICE STORM"                     
##  [25] "VOLCANIC ASH"                   "LATE SEASON SNOW"              
##  [27] "FOG"                            "URBAN/SML STREAM FLD"          
##  [29] "DUST STORM"                     "WIND DAMAGE"                   
##  [31] "DRY"                            "FROST"                         
##  [33] "STORM SURGE"                    "SNOW"                          
##  [35] "WIND"                           "FREEZE"                        
##  [37] "WINTER WEATHER"                 "GUSTY LAKE WIND"               
##  [39] "ABNORMALLY DRY"                 "SMALL HAIL"                    
##  [41] "WINTER WEATHER MIX"             "THUNDERSTORM"                  
##  [43] "DUST DEVIL"                     "RED FLAG CRITERIA"             
##  [45] "MUD SLIDE"                      "THUNDERSTORMS"                 
##  [47] "RAIN"                           "WND"                           
##  [49] "DRY MICROBURST"                 "FREEZING RAIN"                 
##  [51] "STRONG WIND"                    "RECORD HEAT"                   
##  [53] "RECORD COLD"                    "BLACK ICE"                     
##  [55] "WINTRY MIX"                     "CSTL FLOODING/EROSION"         
##  [57] "UNSEASONABLY DRY"               "UNSEASONABLY HOT"              
##  [59] "UNSEASONABLY COOL"              "UNSEASONABLY WARM"             
##  [61] "DROUGHT"                        "SMOKE"                         
##  [63] " WATERSPOUT"                    "HIGH SURF"                     
##  [65] "SNOW ADVISORY"                  "WIND ADVISORY"                 
##  [67] "HURRICANE"                      "LIGHT SNOW"                    
##  [69] "EXTREMELY WET"                  "SNOW/SLEET"                    
##  [71] "UNUSUALLY LATE SNOW"            "UNSEASONABLY COLD"             
##  [73] "VERY DRY"                       "RECORD LOW RAINFALL"           
##  [75] "ROGUE WAVE"                     "SNOW/FREEZING RAIN"            
##  [77] "SNOW/BLOWING SNOW"              "SNOWMELT FLOODING"             
##  [79] "GRADIENT WIND"                  "LANDSLIDE"                     
##  [81] "RECORD WARMTH"                  "PROLONG COLD"                  
##  [83] "MONTHLY PRECIPITATION"          "COLD"                          
##  [85] "PROLONG WARMTH"                 "STRONG WINDS"                  
##  [87] "MODERATE SNOWFALL"              "MIXED PRECIPITATION"           
##  [89] "EXTREME WINDCHILL TEMPERATURES" "ACCUMULATED SNOWFALL"          
##  [91] "FALLING SNOW/ICE"               "DRY CONDITIONS"                
##  [93] "DUST DEVEL"                     "NON-TSTM WIND"                 
##  [95] "NON TSTM WIND"                  "MUDSLIDE"                      
##  [97] "OTHER"                          "LAKE EFFECT SNOW"              
##  [99] "DRY SPELL"                      "BRUSH FIRE"                    
## [101] "GUSTY WINDS"                    "GUSTY THUNDERSTORM WINDS"      
## [103] "BEACH EROSION"                  "ROUGH SEAS"                    
## [105] "LIGHT FREEZING RAIN"            "DENSE FOG"                     
## [107] "PATCHY ICE"                     "SNOW SHOWERS"                  
## [109] "HEAVY RAIN EFFECTS"             "HEAT"                          
## [111] "BLOWING DUST"                   "SNOW DROUGHT"                  
## [113] "GUSTY WIND"                     "FUNNEL CLOUDS"                 
## [115] "SEVERE THUNDERSTORMS"           "EXCESSIVE HEAT/DROUGHT"        
## [117] "HIGH WATER"                     "NORTHERN LIGHTS"               
## [119] "MARINE TSTM WIND"               "   HIGH SURF ADVISORY"         
## [121] "HIGH SURF ADVISORY"             "HIGH SEAS"                     
## [123] "WIND CHILL"                     "HAZARDOUS SURF"                
## [125] "WILDFIRE"                       "FROST/FREEZE"                  
## [127] "WINTER WEATHER/MIX"             "ASTRONOMICAL HIGH TIDE"        
## [129] "COLD WEATHER"                   "HARD FREEZE"                   
## [131] "WHIRLWIND"                      "VERY WARM"                     
## [133] "ABNORMALLY WET"                 "UNUSUALLY COLD"                
## [135] "COASTAL FLOOD"                  "TORNADO DEBRIS"                
## [137] "EXTREME COLD/WIND CHILL"        "ICE ON ROAD"                   
## [139] "EARLY SNOWFALL"                 "FREEZING DRIZZLE"              
## [141] "FIRST SNOW"                     "SNOW SQUALLS"                  
## [143] "SEICHE"                         "ICY ROADS"                     
## [145] "ICE/SNOW"                       "DROWNING"                      
## [147] "GUSTY THUNDERSTORM WIND"        "MARINE HAIL"                   
## [149] "HIGH SURF ADVISORIES"           "RIP CURRENT"                   
## [151] "HURRICANE/TYPHOON"              "HEAVY SURF/HIGH SURF"          
## [153] "TROPICAL DEPRESSION"            "SLEET STORM"                   
## [155] "COLD WIND CHILL TEMPERATURES"   "STORM SURGE/TIDE"              
## [157] "FREEZING FOG"                   "COLD/WIND CHILL"               
## [159] "LAKE-EFFECT SNOW"               "MARINE HIGH WIND"              
## [161] "THUNDERSTORM WIND"              "TSUNAMI"                       
## [163] "SLEET"                          "DENSE SMOKE"                   
## [165] "LAKESHORE FLOOD"                "MARINE THUNDERSTORM WIND"      
## [167] "MARINE STRONG WIND"             "ASTRONOMICAL LOW TIDE"         
## [169] "VOLCANIC ASHFALL"

We can see that more than 48 events were recorded. So we need to clean the dataset and rename when necessary the events to narrow done to the 48 events.

2.2. Clean the “EVTYPE” column

To get the list of the 48 events, we generate a list based on the dataset from October 2006 when the NWS switched from Paradox to Windows SQL Server.

events <- c(unique(with(data_tidy, data_tidy[(DATE >= "2006-11-01"),3])))
print(events)
##  [1] "BLIZZARD"                 "HIGH WIND"               
##  [3] "HEAVY SNOW"               "THUNDERSTORM WIND"       
##  [5] "TORNADO"                  "FLASH FLOOD"             
##  [7] "HAIL"                     "FUNNEL CLOUD"            
##  [9] "DENSE FOG"                "WINTER WEATHER"          
## [11] "FLOOD"                    "WINTER STORM"            
## [13] "STRONG WIND"              "HEAVY RAIN"              
## [15] "TSUNAMI"                  "WILDFIRE"                
## [17] "FROST/FREEZE"             "HEAT"                    
## [19] "DUST STORM"               "HIGH SURF"               
## [21] "LIGHTNING"                "DROUGHT"                 
## [23] "LANDSLIDE"                "AVALANCHE"               
## [25] "COASTAL FLOOD"            "EXTREME COLD/WIND CHILL" 
## [27] "RIP CURRENT"              "SLEET"                   
## [29] "LAKE-EFFECT SNOW"         "ICE STORM"               
## [31] "FREEZING FOG"             "SEICHE"                  
## [33] "COLD/WIND CHILL"          "WATERSPOUT"              
## [35] "MARINE THUNDERSTORM WIND" "MARINE HAIL"             
## [37] "MARINE STRONG WIND"       "DUST DEVIL"              
## [39] "ASTRONOMICAL LOW TIDE"    "STORM SURGE/TIDE"        
## [41] "EXCESSIVE HEAT"           "DENSE SMOKE"             
## [43] "TROPICAL STORM"           "TROPICAL DEPRESSION"     
## [45] "HURRICANE"                "LAKESHORE FLOOD"         
## [47] "MARINE HIGH WIND"         "VOLCANIC ASHFALL"

We can split the “data_tidy” dataset in two, between the data matching the 48 events and the one where events are recorded differently. We will call this dataset “data_toclean”.

data_toclean <- data_tidy %>% filter(!EVTYPE %in% events) # create a new dataset with the data to clean
data_tidy <- data_tidy %>% filter(EVTYPE %in% events) # remove the data that need cleaning

Among the 488,692 initial observations, we have 83,968 observations to clean. Let’s have a look at the events to clean:

unique(data_toclean$EVTYPE)
##   [1] "TSTM WIND"                      "WILD/FOREST FIRE"              
##   [3] "EXTREME COLD"                   "COASTAL FLOODING"              
##   [5] "TSTM WIND/HAIL"                 "RIP CURRENTS"                  
##   [7] "HEAVY SURF"                     "EXTREME WINDCHILL"             
##   [9] "VOLCANIC ASH"                   "LATE SEASON SNOW"              
##  [11] "FOG"                            "URBAN/SML STREAM FLD"          
##  [13] "WIND DAMAGE"                    "DRY"                           
##  [15] "FROST"                          "STORM SURGE"                   
##  [17] "SNOW"                           "WIND"                          
##  [19] "FREEZE"                         "GUSTY LAKE WIND"               
##  [21] "ABNORMALLY DRY"                 "SMALL HAIL"                    
##  [23] "WINTER WEATHER MIX"             "THUNDERSTORM"                  
##  [25] "RED FLAG CRITERIA"              "MUD SLIDE"                     
##  [27] "THUNDERSTORMS"                  "RAIN"                          
##  [29] "WND"                            "DRY MICROBURST"                
##  [31] "FREEZING RAIN"                  "RECORD HEAT"                   
##  [33] "RECORD COLD"                    "BLACK ICE"                     
##  [35] "WINTRY MIX"                     "CSTL FLOODING/EROSION"         
##  [37] "UNSEASONABLY DRY"               "UNSEASONABLY HOT"              
##  [39] "UNSEASONABLY COOL"              "UNSEASONABLY WARM"             
##  [41] "SMOKE"                          " WATERSPOUT"                   
##  [43] "SNOW ADVISORY"                  "WIND ADVISORY"                 
##  [45] "LIGHT SNOW"                     "EXTREMELY WET"                 
##  [47] "SNOW/SLEET"                     "UNUSUALLY LATE SNOW"           
##  [49] "UNSEASONABLY COLD"              "VERY DRY"                      
##  [51] "RECORD LOW RAINFALL"            "ROGUE WAVE"                    
##  [53] "SNOW/FREEZING RAIN"             "SNOW/BLOWING SNOW"             
##  [55] "SNOWMELT FLOODING"              "GRADIENT WIND"                 
##  [57] "RECORD WARMTH"                  "PROLONG COLD"                  
##  [59] "MONTHLY PRECIPITATION"          "COLD"                          
##  [61] "PROLONG WARMTH"                 "STRONG WINDS"                  
##  [63] "MODERATE SNOWFALL"              "MIXED PRECIPITATION"           
##  [65] "EXTREME WINDCHILL TEMPERATURES" "ACCUMULATED SNOWFALL"          
##  [67] "FALLING SNOW/ICE"               "DRY CONDITIONS"                
##  [69] "DUST DEVEL"                     "NON-TSTM WIND"                 
##  [71] "NON TSTM WIND"                  "MUDSLIDE"                      
##  [73] "OTHER"                          "LAKE EFFECT SNOW"              
##  [75] "DRY SPELL"                      "BRUSH FIRE"                    
##  [77] "GUSTY WINDS"                    "GUSTY THUNDERSTORM WINDS"      
##  [79] "BEACH EROSION"                  "ROUGH SEAS"                    
##  [81] "LIGHT FREEZING RAIN"            "PATCHY ICE"                    
##  [83] "SNOW SHOWERS"                   "HEAVY RAIN EFFECTS"            
##  [85] "BLOWING DUST"                   "SNOW DROUGHT"                  
##  [87] "GUSTY WIND"                     "FUNNEL CLOUDS"                 
##  [89] "SEVERE THUNDERSTORMS"           "EXCESSIVE HEAT/DROUGHT"        
##  [91] "HIGH WATER"                     "NORTHERN LIGHTS"               
##  [93] "MARINE TSTM WIND"               "   HIGH SURF ADVISORY"         
##  [95] "HIGH SURF ADVISORY"             "HIGH SEAS"                     
##  [97] "WIND CHILL"                     "HAZARDOUS SURF"                
##  [99] "WINTER WEATHER/MIX"             "ASTRONOMICAL HIGH TIDE"        
## [101] "COLD WEATHER"                   "HARD FREEZE"                   
## [103] "WHIRLWIND"                      "VERY WARM"                     
## [105] "ABNORMALLY WET"                 "UNUSUALLY COLD"                
## [107] "TORNADO DEBRIS"                 "ICE ON ROAD"                   
## [109] "EARLY SNOWFALL"                 "FREEZING DRIZZLE"              
## [111] "FIRST SNOW"                     "SNOW SQUALLS"                  
## [113] "ICY ROADS"                      "ICE/SNOW"                      
## [115] "DROWNING"                       "GUSTY THUNDERSTORM WIND"       
## [117] "HIGH SURF ADVISORIES"           "HURRICANE/TYPHOON"             
## [119] "HEAVY SURF/HIGH SURF"           "SLEET STORM"                   
## [121] "COLD WIND CHILL TEMPERATURES"

We can see that 80 events are recorded differently from the standardized events. The next steps will be to replace these events by the standardized names. For this, we will apply a subjective judgment on defining the type of event. “OTHER” could not be classified and was left as it is.

data_toclean$EVTYPE[grepl("SURF",data_toclean$EVTYPE, ignore.case = TRUE)]<-"HIGH SURF"
data_toclean$EVTYPE[grepl("^TSTM|THUNDERSTORM",data_toclean$EVTYPE, ignore.case = TRUE)]<-"THUNDERSTORM WIND"
data_toclean$EVTYPE[grepl("WND|^WIND|^GUSTY|NON TSTM WIND|NON-TSTM WIND|GRADIENT WIND",data_toclean$EVTYPE, ignore.case = TRUE)]<-"HIGH WIND"
data_toclean$EVTYPE[grepl("SNOW",data_toclean$EVTYPE, ignore.case = TRUE)]<-"HIGH SNOW"
data_toclean$EVTYPE[grepl("HAIL",data_toclean$EVTYPE, ignore.case = TRUE)]<-"HAIL"
data_toclean$EVTYPE[grepl("COASTAL FLOOD",data_toclean$EVTYPE, ignore.case = TRUE)]<-"COASTAL FLOOD"
data_toclean$EVTYPE[grepl("freez|winter we",data_toclean$EVTYPE, ignore.case = TRUE)]<-"WINTER WEATHER"
data_toclean$EVTYPE[grepl("tornado|WHIRLWIND",data_toclean$EVTYPE, ignore.case = TRUE)]<-"TORNADO"
data_toclean$EVTYPE[grepl("fog",data_toclean$EVTYPE, ignore.case = TRUE)]<-"DENSE FOG"
data_toclean$EVTYPE[grepl("frost|ICE ON ROAD|BLACK ICE|ICY ROADS|UNSEASONABLY COOL|PATCHY ICE",data_toclean$EVTYPE, ignore.case = TRUE)]<-"FROST/FREEZE"
data_toclean$EVTYPE[grepl("fire",data_toclean$EVTYPE, ignore.case = TRUE)]<-"WILDFIRE"
data_toclean$EVTYPE[grepl("cold",data_toclean$EVTYPE, ignore.case = TRUE)]<-"EXTREME COLD/WIND CHILL"
data_toclean$EVTYPE[grepl("current|DROWNING",data_toclean$EVTYPE, ignore.case = TRUE)]<-"RIP CURRENT"
data_toclean$EVTYPE[grepl("^storm",data_toclean$EVTYPE, ignore.case = TRUE)]<-"STORM SURGE/TIDE"
data_toclean$EVTYPE[grepl("sleet",data_toclean$EVTYPE, ignore.case = TRUE)]<-"SLEET"
data_toclean$EVTYPE[grepl("marine tstm",data_toclean$EVTYPE, ignore.case = TRUE)]<-"MARINE THUNDERSTORM WIND"
data_toclean$EVTYPE[grepl("astronimical",data_toclean$EVTYPE, ignore.case = TRUE)]<-"ASTRONOMICAL LOW TIDE"
data_toclean$EVTYPE[grepl("smoke",data_toclean$EVTYPE, ignore.case = TRUE)]<-"DENSE SMOKE"
data_toclean$EVTYPE[grepl("HURRICANE",data_toclean$EVTYPE, ignore.case = TRUE)]<-"HURRICANE"
data_toclean$EVTYPE[grepl("marine",data_toclean$EVTYPE, ignore.case = TRUE)]<- "MARINE THUNDERSTORM WIND"
data_toclean$EVTYPE[grepl("volc",data_toclean$EVTYPE, ignore.case = TRUE)]<- "VOLCANIC ASHFALL"
data_toclean$EVTYPE[grepl("dry|RECORD LOW RAINFALL",data_toclean$EVTYPE, ignore.case = TRUE)]<- "DROUGHT"
data_toclean$EVTYPE[grepl("URBAN/SML STREAM FLD|ABNORMALLY WET|MONTHLY PRECIPITATION|MIXED PRECIPITATION",data_toclean$EVTYPE, ignore.case = TRUE)]<- "HEAVY RAIN"
data_toclean$EVTYPE[grepl("High snow",data_toclean$EVTYPE, ignore.case = TRUE)]<- "HEAVY SNOW"
data_toclean$EVTYPE[grepl("UNSEASONABLY WARM|VERY WARM|WARMTH",data_toclean$EVTYPE, ignore.case = TRUE)]<- "HEAT"
data_toclean$EVTYPE[grepl("WINTRY MIX|NORTHERN LIGHTS",data_toclean$EVTYPE, ignore.case = TRUE)]<- "WINTER WEATHER"
data_toclean$EVTYPE[grepl("EXTREME WINDCHILL",data_toclean$EVTYPE, ignore.case = TRUE)]<- "EXTREME COLD/WIND CHILL"
data_toclean$EVTYPE[grepl("MUDSLIDE|MUD SLIDE|BEACH EROSION",data_toclean$EVTYPE, ignore.case = TRUE)]<- "LANDSLIDE"
data_toclean$EVTYPE[grepl("STRONG WINDS",data_toclean$EVTYPE, ignore.case = TRUE)]<- "STRONG WIND"
data_toclean$EVTYPE[grepl("ROUGH SEAS|HIGH SEAS|ASTRONOMICAL HIGH TIDE|HIGH WATER|ROGUE WAVE",data_toclean$EVTYPE, ignore.case = TRUE)]<- "STORM SURGE/TIDE"
data_toclean$EVTYPE[grepl("RECORD HEAT|EXCESSIVE HEAT/DROUGHT",data_toclean$EVTYPE, ignore.case = TRUE)]<- "EXCESSIVE HEAT"
data_toclean$EVTYPE[grepl("UNSEASONABLY HOT",data_toclean$EVTYPE, ignore.case = TRUE)]<- "HEAT"
data_toclean$EVTYPE[grepl(" WATERSPOUT",data_toclean$EVTYPE, ignore.case = TRUE)]<- "WATERSPOUT"
data_toclean$EVTYPE[grepl("FUNNEL CLOUDS",data_toclean$EVTYPE, ignore.case = TRUE)]<- "FUNNEL CLOUD"
data_toclean$EVTYPE[grepl("CSTL FLOODING/EROSION",data_toclean$EVTYPE, ignore.case = TRUE)]<- "COASTAL FLOOD"
data_toclean$EVTYPE[grepl("RAIN|HEAVY RAIN EFFECTS|EXTREMELY WET",data_toclean$EVTYPE, ignore.case = TRUE)]<- "HEAVY RAIN"
data_toclean$EVTYPE[grepl("RED FLAG CRITERIA",data_toclean$EVTYPE, ignore.case = TRUE)]<- "WILDFIRE"
data_toclean$EVTYPE[grepl("BLOWING DUST|DUST DEVEL",data_toclean$EVTYPE, ignore.case = TRUE)]<- "DUST STORM"

Finally, we will re-combine the two dataset into one “data_tidy”

data_tidy <- rbind(data_tidy, data_toclean)
unique(data_tidy$EVTYPE)
##  [1] "HAIL"                     "TORNADO"                 
##  [3] "FLASH FLOOD"              "HIGH WIND"               
##  [5] "FLOOD"                    "WINTER STORM"            
##  [7] "EXCESSIVE HEAT"           "LIGHTNING"               
##  [9] "FUNNEL CLOUD"             "HEAVY SNOW"              
## [11] "WATERSPOUT"               "TROPICAL STORM"          
## [13] "HEAVY RAIN"               "AVALANCHE"               
## [15] "BLIZZARD"                 "ICE STORM"               
## [17] "DUST STORM"               "WINTER WEATHER"          
## [19] "DUST DEVIL"               "STRONG WIND"             
## [21] "DROUGHT"                  "HIGH SURF"               
## [23] "HURRICANE"                "LANDSLIDE"               
## [25] "DENSE FOG"                "HEAT"                    
## [27] "WILDFIRE"                 "FROST/FREEZE"            
## [29] "COASTAL FLOOD"            "EXTREME COLD/WIND CHILL" 
## [31] "SEICHE"                   "MARINE HAIL"             
## [33] "RIP CURRENT"              "TROPICAL DEPRESSION"     
## [35] "STORM SURGE/TIDE"         "FREEZING FOG"            
## [37] "COLD/WIND CHILL"          "LAKE-EFFECT SNOW"        
## [39] "MARINE HIGH WIND"         "THUNDERSTORM WIND"       
## [41] "TSUNAMI"                  "SLEET"                   
## [43] "DENSE SMOKE"              "LAKESHORE FLOOD"         
## [45] "MARINE THUNDERSTORM WIND" "MARINE STRONG WIND"      
## [47] "ASTRONOMICAL LOW TIDE"    "VOLCANIC ASHFALL"        
## [49] "OTHER"

2.3. Handle the exponent values for crops and properties

The codebook gives information related to the definition of character in the propdmgexp and cropdmgexp columns https://github.com/lgreski/datasciencectacontent/blob/master/markdown/repdata-stormAnalysisCodebook.md

We first check the type of characters present in these columns

unique(data_tidy$PROPDMGEXP)
## [1] ""  "K" "M" "B" "0"
unique(data_tidy$CROPDMGEXP)
## [1] ""  "K" "M" "B"

Next, we multiply the values from propdmg or cropdmg variables whenever a “K”, “M”, or “B” is present.

# to handle the exponent values for properties
data_tidy <- data_tidy %>% mutate(PROPDMG = case_when(
        PROPDMGEXP == "K" ~ PROPDMG*1000,
        PROPDMGEXP == "M" ~ PROPDMG*1000000,
        PROPDMGEXP == "B" ~ PROPDMG*1000000000,
        TRUE ~ PROPDMG*1
))

# to handle the exponent values for properties
data_tidy <- data_tidy %>% mutate(CROPDMG = case_when(
        CROPDMGEXP == "K" ~ CROPDMG*1000,
        CROPDMGEXP == "M" ~ CROPDMG*1000000,
        CROPDMGEXP == "B" ~ CROPDMG*1000000000,
        TRUE ~ CROPDMG*1
))

Finally, we can remove the cropdmgexp and propdmgexp variables from the table

data_tidy <- data_tidy %>%
        select(-CROPDMGEXP, -PROPDMGEXP)

For the rest of the analysis, only the year in the DATE column will be informative. We are now changing the format to display the year only:

data_tidy$DATE <- year(data_tidy$DATE) # keep only year

3.Results

3.1. Evolution of the impact on population and economy since 2001

In order to evaluate which events have the most impact on population or economically, we first summarize the database by year and type of events (Table 1).

table1_melt <- melt(data_tidy, id = c("DATE", "EVTYPE"), measure.vars = c("FATALITIES", "INJURIES", "PROPDMG", "CROPDMG")) 
table1 <- dcast(table1_melt, DATE+EVTYPE~variable, fun.aggregate = sum)
table1$TOTAL.POPULATION <-  table1$FATALITIES + table1$INJURIES
table1$TOTAL.DAMAGE <- table1$PROPDMG + table1$CROPDMG
table1 <- table1 %>% select(DATE, EVTYPE, TOTAL.POPULATION, FATALITIES, INJURIES, TOTAL.DAMAGE, PROPDMG, CROPDMG)
knitr::kable(table1, caption = "Table 1: Summary of events and impact on population and economic since 2001")
Table 1: Summary of events and impact on population and economic since 2001
DATE EVTYPE TOTAL.POPULATION FATALITIES INJURIES TOTAL.DAMAGE PROPDMG CROPDMG
2001 AVALANCHE 24 18 6 44000 44000 0
2001 BLIZZARD 9 4 5 8145000 8145000 0
2001 COASTAL FLOOD 1 1 0 16993000 16993000 0
2001 DENSE FOG 74 7 67 1283000 1283000 0
2001 DENSE SMOKE 0 0 0 0 0 0
2001 DROUGHT 1 1 0 1274041500 169500 1273872000
2001 DUST DEVIL 1 0 1 3000 3000 0
2001 DUST STORM 5 0 5 180000 180000 0
2001 EXCESSIVE HEAT 610 165 445 0 0 0
2001 EXTREME COLD/WIND CHILL 4 4 0 84330000 0 84330000
2001 FLASH FLOOD 297 34 263 657393500 633851000 23542500
2001 FLOOD 21 13 8 604171000 584771000 19400000
2001 FROST/FREEZE 0 0 0 0 0 0
2001 FUNNEL CLOUD 0 0 0 0 0 0
2001 HAIL 32 0 32 2638727740 2368341440 270386300
2001 HEAT 1 1 0 0 0 0
2001 HEAVY RAIN 12 5 7 44263500 22662500 21601000
2001 HEAVY SNOW 23 3 20 65774000 65774000 0
2001 HIGH SURF 23 3 20 622500 622500 0
2001 HIGH WIND 104 13 91 69041570 66481570 2560000
2001 HURRICANE 1 1 0 1100000 650000 450000
2001 ICE STORM 12 5 7 10032000 10032000 0
2001 LANDSLIDE 0 0 0 1325000 1325000 0
2001 LIGHTNING 416 44 372 45610600 43602800 2007800
2001 MARINE THUNDERSTORM WIND 0 0 0 0 0 0
2001 OTHER 4 0 4 0 0 0
2001 RIP CURRENT 125 49 76 60000 60000 0
2001 STORM SURGE/TIDE 8 5 3 1460000 1460000 0
2001 STRONG WIND 9 1 8 322000 322000 0
2001 THUNDERSTORM WIND 359 17 342 379785660 318727660 61058000
2001 TORNADO 783 40 743 637497400 630086900 7410500
2001 TROPICAL STORM 29 23 6 5188110000 5185910000 2200000
2001 VOLCANIC ASHFALL 0 0 0 500000 500000 0
2001 WATERSPOUT 0 0 0 0 0 0
2001 WILDFIRE 53 5 48 45172800 45172800 0
2001 WINTER STORM 84 7 77 19868000 19808000 60000
2001 WINTER WEATHER 65 0 65 47915000 65000 47850000
2002 AVALANCHE 29 18 11 31500 31500 0
2002 BLIZZARD 11 4 7 185000 185000 0
2002 COASTAL FLOOD 0 0 0 100000 100000 0
2002 DENSE FOG 168 22 146 3975000 3975000 0
2002 DENSE SMOKE 0 0 0 0 0 0
2002 DROUGHT 7 1 6 737718000 83000 737635000
2002 DUST DEVIL 1 0 1 13300 13300 0
2002 DUST STORM 47 2 45 427000 427000 0
2002 EXCESSIVE HEAT 545 167 378 0 0 0
2002 EXTREME COLD/WIND CHILL 11 11 0 70663000 303000 70360000
2002 FLASH FLOOD 103 32 71 267968500 258248500 9720000
2002 FLOOD 15 10 5 467419300 394639800 72779500
2002 FROST/FREEZE 1 1 0 0 0 0
2002 FUNNEL CLOUD 0 0 0 0 0 0
2002 HAIL 125 0 125 479113590 325268840 153844750
2002 HEAT 0 0 0 0 0 0
2002 HEAVY RAIN 15 5 10 4628200 3566200 1062000
2002 HEAVY SNOW 41 2 39 11337400 11337400 0
2002 HIGH SURF 12 5 7 42000 42000 0
2002 HIGH WIND 152 26 126 116385900 82835900 33550000
2002 HURRICANE 321 2 319 1012255010 744255010 268000000
2002 ICE STORM 49 4 45 700860000 700860000 0
2002 LANDSLIDE 0 0 0 0 0 0
2002 LIGHTNING 307 51 256 43556600 43537600 19000
2002 MARINE HAIL 0 0 0 0 0 0
2002 MARINE THUNDERSTORM WIND 0 0 0 158000 158000 0
2002 RIP CURRENT 104 44 60 100000 100000 0
2002 SEICHE 0 0 0 35000 35000 0
2002 STORM SURGE/TIDE 7 6 1 10558000 10558000 0
2002 STRONG WIND 5 2 3 172000 172000 0
2002 THUNDERSTORM WIND 298 17 281 344768750 290076500 54692250
2002 TORNADO 1023 55 968 802060210 801261200 799010
2002 TROPICAL STORM 5 1 4 193204000 187304000 5900000
2002 VOLCANIC ASHFALL 0 0 0 0 0 0
2002 WATERSPOUT 1 0 1 0 0 0
2002 WILDFIRE 139 1 138 202833630 200827000 2006630
2002 WINTER STORM 108 9 99 40621700 40621700 0
2002 WINTER WEATHER 3 0 3 60000 60000 0
2003 AVALANCHE 17 10 7 15000 15000 0
2003 BLIZZARD 4 0 4 62135000 62135000 0
2003 COASTAL FLOOD 0 0 0 0 0 0
2003 DENSE FOG 48 5 43 1675000 1675000 0
2003 DROUGHT 0 0 0 1217630000 645150000 572480000
2003 DUST DEVIL 4 1 3 130000 130000 0
2003 DUST STORM 93 2 91 284000 284000 0
2003 EXCESSIVE HEAT 210 36 174 2000 0 2000
2003 EXTREME COLD/WIND CHILL 34 20 14 17910000 860000 17050000
2003 FLASH FLOOD 130 69 61 2142321000 2124036500 18284500
2003 FLOOD 23 18 5 558468900 418678900 139790000
2003 FROST/FREEZE 0 0 0 11120000 0 11120000
2003 FUNNEL CLOUD 0 0 0 0 0 0
2003 HAIL 121 0 121 615969520 505952470 110017050
2003 HEAVY RAIN 44 6 38 78697000 8884000 69813000
2003 HEAVY SNOW 15 2 13 64458080 64458080 0
2003 HIGH SURF 21 13 8 8180000 8180000 0
2003 HIGH WIND 142 15 127 198615520 157610520 41005000
2003 HURRICANE 8 6 2 1017140000 987770000 29370000
2003 ICE STORM 37 6 31 278166000 269616000 8550000
2003 LANDSLIDE 31 15 16 5591000 5591000 0
2003 LIGHTNING 282 44 238 25771750 25728750 43000
2003 MARINE HAIL 0 0 0 0 0 0
2003 MARINE THUNDERSTORM WIND 4 2 2 3215000 3215000 0
2003 RIP CURRENT 88 41 47 0 0 0
2003 SLEET 0 0 0 0 0 0
2003 STORM SURGE/TIDE 1 0 1 61892000 61892000 0
2003 STRONG WIND 38 9 29 9153600 9153600 0
2003 THUNDERSTORM WIND 241 17 224 422409120 355886320 66522800
2003 TORNADO 1141 54 1087 1281558270 1265650770 15907500
2003 TROPICAL DEPRESSION 0 0 0 50000 50000 0
2003 TROPICAL STORM 238 8 230 849038000 837088000 11950000
2003 VOLCANIC ASHFALL 0 0 0 0 0 0
2003 WATERSPOUT 0 0 0 0 0 0
2003 WILDFIRE 275 24 251 2362777830 2331612330 31165500
2003 WINTER STORM 68 16 52 102714000 102714000 0
2003 WINTER WEATHER 16 4 12 531000 531000 0
2004 AVALANCHE 18 12 6 90000 90000 0
2004 BLIZZARD 5 1 4 242000 242000 0
2004 DENSE FOG 46 1 45 1590000 1590000 0
2004 DROUGHT 0 0 0 1190000 0 1190000
2004 DUST DEVIL 5 0 5 8800 8800 0
2004 DUST STORM 11 0 11 80000 80000 0
2004 EXCESSIVE HEAT 80 6 74 0 0 0
2004 EXTREME COLD/WIND CHILL 29 27 2 650000 650000 0
2004 FLASH FLOOD 87 58 29 868884250 856662250 12222000
2004 FLOOD 123 24 99 1173603400 844435500 329167900
2004 FROST/FREEZE 0 0 0 0 0 0
2004 FUNNEL CLOUD 0 0 0 0 0 0
2004 HAIL 4 0 4 535992480 463019380 72973100
2004 HEAVY RAIN 13 4 9 4069000 3619000 450000
2004 HEAVY SNOW 142 3 139 25812400 25612400 200000
2004 HIGH SURF 53 24 29 305000 305000 0
2004 HIGH WIND 62 13 49 3516832850 3224745850 292087000
2004 HURRICANE 862 23 839 18922255800 18356435000 565820800
2004 ICE STORM 10 0 10 46222000 46222000 0
2004 LANDSLIDE 15 4 11 32289000 32289000 0
2004 LIGHTNING 312 33 279 26047800 26006000 41800
2004 MARINE HAIL 0 0 0 0 0 0
2004 MARINE THUNDERSTORM WIND 6 6 0 243000 243000 0
2004 RIP CURRENT 55 32 23 0 0 0
2004 SEICHE 0 0 0 0 0 0
2004 SLEET 0 0 0 0 0 0
2004 STORM SURGE/TIDE 0 0 0 45730000 45730000 0
2004 STRONG WIND 32 13 19 135494140 87044140 48450000
2004 THUNDERSTORM WIND 259 16 243 182297150 166653400 15643750
2004 TORNADO 431 35 396 548718950 536587450 12131500
2004 TROPICAL DEPRESSION 0 0 0 335000 335000 0
2004 TROPICAL STORM 12 11 1 601100000 499600000 101500000
2004 VOLCANIC ASHFALL 0 0 0 0 0 0
2004 WATERSPOUT 0 0 0 2000 2000 0
2004 WILDFIRE 63 0 63 17232700 16932700 300000
2004 WINTER STORM 25 9 16 109454000 109454000 0
2004 WINTER WEATHER 36 15 21 2005000 2005000 0
2005 AVALANCHE 27 11 16 200000 200000 0
2005 BLIZZARD 3 1 2 15840000 15840000 0
2005 COASTAL FLOOD 1 0 1 450000 450000 0
2005 COLD/WIND CHILL 0 0 0 0 0 0
2005 DENSE FOG 12 0 12 642000 642000 0
2005 DROUGHT 0 0 0 1388494000 77414000 1311080000
2005 DUST DEVIL 0 0 0 30800 30800 0
2005 DUST STORM 32 0 32 70000 70000 0
2005 EXCESSIVE HEAT 456 158 298 3000000 3000000 0
2005 EXTREME COLD/WIND CHILL 25 24 1 100000 100000 0
2005 FLASH FLOOD 50 28 22 328520700 293188200 35332500
2005 FLOOD 31 15 16 1313378090 1244483090 68895000
2005 FREEZING FOG 0 0 0 0 0 0
2005 FROST/FREEZE 0 0 0 139225000 0 139225000
2005 FUNNEL CLOUD 0 0 0 15000 15000 0
2005 HAIL 9 1 8 536592230 480469230 56123000
2005 HEAVY RAIN 47 8 39 492808500 264197500 228611000
2005 HEAVY SNOW 18 7 11 19188000 19078000 110000
2005 HIGH SURF 16 5 11 1400000 1400000 0
2005 HIGH WIND 16 2 14 65581500 44251500 21330000
2005 HURRICANE 152 34 118 51799317000 49786635000 2012682000
2005 ICE STORM 6 2 4 146084500 146049500 35000
2005 LAKE-EFFECT SNOW 0 0 0 80000 80000 0
2005 LANDSLIDE 22 11 11 91515000 91515000 0
2005 LIGHTNING 347 38 309 52849100 52434100 415000
2005 MARINE HAIL 0 0 0 0 0 0
2005 MARINE HIGH WIND 0 0 0 21000 21000 0
2005 MARINE THUNDERSTORM WIND 1 1 0 305000 305000 0
2005 RIP CURRENT 44 35 9 0 0 0
2005 SEICHE 0 0 0 25000 25000 0
2005 SLEET 0 0 0 0 0 0
2005 STORM SURGE/TIDE 1 0 1 43059400000 43059400000 0
2005 STRONG WIND 34 5 29 14862650 14257650 605000
2005 THUNDERSTORM WIND 200 15 185 410655100 398724100 11931000
2005 TORNADO 575 38 537 503938400 421833500 82104900
2005 TROPICAL DEPRESSION 0 0 0 50000 50000 0
2005 TROPICAL STORM 11 0 11 286627000 220127000 66500000
2005 VOLCANIC ASHFALL 0 0 0 0 0 0
2005 WATERSPOUT 0 0 0 0 0 0
2005 WILDFIRE 88 6 82 40877400 40654500 222900
2005 WINTER STORM 34 15 19 109573500 109573500 0
2005 WINTER WEATHER 45 9 36 3277000 3277000 0
2006 AVALANCHE 19 13 6 0 0 0
2006 BLIZZARD 1 1 0 4685000 4685000 0
2006 COASTAL FLOOD 0 0 0 51824000 51824000 0
2006 COLD/WIND CHILL 2 2 0 0 0 0
2006 DENSE FOG 43 3 40 650000 650000 0
2006 DENSE SMOKE 0 0 0 0 0 0
2006 DROUGHT 4 0 4 2498130000 122305000 2375825000
2006 DUST DEVIL 1 0 1 37000 37000 0
2006 DUST STORM 24 2 22 2940000 690000 2250000
2006 EXCESSIVE HEAT 1198 205 993 492570000 170000 492400000
2006 EXTREME COLD/WIND CHILL 13 8 5 0 0 0
2006 FLASH FLOOD 77 59 18 2161608130 2136595130 25013000
2006 FLOOD 24 19 5 116611270440 116516105440 95165000
2006 FREEZING FOG 0 0 0 0 0 0
2006 FROST/FREEZE 0 0 0 11940000 0 11940000
2006 FUNNEL CLOUD 0 0 0 0 0 0
2006 HAIL 18 0 18 1433507000 1300570000 132937000
2006 HEAT 567 47 520 86500 10000 76500
2006 HEAVY RAIN 5 1 4 102128010 52383010 49745000
2006 HEAVY SNOW 2 0 2 8136750 8136750 0
2006 HIGH SURF 22 13 9 360000 360000 0
2006 HIGH WIND 76 19 57 138173100 123144100 15029000
2006 ICE STORM 6 6 0 22002000 22002000 0
2006 LAKE-EFFECT SNOW 0 0 0 23328000 23328000 0
2006 LAKESHORE FLOOD 0 0 0 10000 10000 0
2006 LANDSLIDE 5 3 2 78889500 58889500 20000000
2006 LIGHTNING 290 45 245 63772280 63758280 14000
2006 MARINE HAIL 0 0 0 0 0 0
2006 MARINE HIGH WIND 0 0 0 136000 136000 0
2006 MARINE STRONG WIND 2 2 0 16000 16000 0
2006 MARINE THUNDERSTORM WIND 7 1 6 1501000 1501000 0
2006 RIP CURRENT 50 26 24 0 0 0
2006 SEICHE 0 0 0 10000 10000 0
2006 SLEET 0 0 0 0 0 0
2006 STORM SURGE/TIDE 0 0 0 705000 705000 0
2006 STRONG WIND 42 17 25 17820000 17698500 121500
2006 THUNDERSTORM WIND 254 13 241 438881080 407825780 31055300
2006 TORNADO 1059 67 992 758960000 752273000 6687000
2006 TROPICAL STORM 1 0 1 44945200 1655200 43290000
2006 TSUNAMI 0 0 0 9203000 9203000 0
2006 VOLCANIC ASHFALL 0 0 0 0 0 0
2006 WATERSPOUT 0 0 0 25000 25000 0
2006 WILDFIRE 107 15 92 411797400 179107000 232690400
2006 WINTER STORM 23 9 14 79398000 79398000 0
2006 WINTER WEATHER 25 3 22 2227500 2227500 0
2007 ASTRONOMICAL LOW TIDE 0 0 0 120000 120000 0
2007 AVALANCHE 30 18 12 2000 2000 0
2007 BLIZZARD 0 0 0 279000 279000 0
2007 COASTAL FLOOD 0 0 0 77470060 77470060 0
2007 COLD/WIND CHILL 14 4 10 591000 91000 500000
2007 DENSE FOG 3 0 3 440000 440000 0
2007 DENSE SMOKE 0 0 0 100000 100000 0
2007 DROUGHT 0 0 0 374677000 825000 373852000
2007 DUST DEVIL 0 0 0 0 0 0
2007 DUST STORM 4 0 4 780000 780000 0
2007 EXCESSIVE HEAT 558 22 536 0 0 0
2007 EXTREME COLD/WIND CHILL 24 23 1 3000 3000 0
2007 FLASH FLOOD 121 70 51 1277552900 1227564900 49988000
2007 FLOOD 25 17 8 1030828500 512321500 518507000
2007 FREEZING FOG 0 0 0 0 0 0
2007 FROST/FREEZE 0 0 0 379699000 640000 379059000
2007 FUNNEL CLOUD 0 0 0 0 0 0
2007 HAIL 25 0 25 530995250 375726250 155269000
2007 HEAT 52 22 30 0 0 0
2007 HEAVY RAIN 10 2 8 80472030 78570030 1902000
2007 HEAVY SNOW 0 0 0 1168000 1168000 0
2007 HIGH SURF 10 10 0 7355000 7355000 0
2007 HIGH WIND 31 3 28 197744500 197695500 49000
2007 HURRICANE 13 1 12 37700000 37700000 0
2007 ICE STORM 10 2 8 474170000 474170000 0
2007 LAKE-EFFECT SNOW 0 0 0 4871000 4871000 0
2007 LAKESHORE FLOOD 0 0 0 30000 30000 0
2007 LANDSLIDE 6 3 3 50952500 50952500 0
2007 LIGHTNING 184 45 139 82058800 81996800 62000
2007 MARINE HAIL 0 0 0 0 0 0
2007 MARINE HIGH WIND 0 0 0 0 0 0
2007 MARINE STRONG WIND 18 3 15 3000 3000 0
2007 MARINE THUNDERSTORM WIND 4 2 2 102000 102000 0
2007 RIP CURRENT 60 46 14 0 0 0
2007 SLEET 0 0 0 0 0 0
2007 STORM SURGE/TIDE 5 0 5 4000000 4000000 0
2007 STRONG WIND 28 7 21 3673820 3538820 135000
2007 THUNDERSTORM WIND 270 16 254 406490450 218065450 188425000
2007 TORNADO 740 81 659 1408022650 1400756650 7266000
2007 TROPICAL DEPRESSION 0 0 0 0 0 0
2007 TROPICAL STORM 0 0 0 350000 345000 5000
2007 VOLCANIC ASHFALL 0 0 0 0 0 0
2007 WATERSPOUT 0 0 0 70000 70000 0
2007 WILDFIRE 202 17 185 1020603700 1004493700 16110000
2007 WINTER STORM 5 3 2 24035000 24012000 23000
2007 WINTER WEATHER 160 4 156 2676000 2676000 0
2008 ASTRONOMICAL LOW TIDE 0 0 0 200000 200000 0
2008 AVALANCHE 39 25 14 1126000 1126000 0
2008 BLIZZARD 2 2 0 521000 521000 0
2008 COASTAL FLOOD 2 2 0 978000 978000 0
2008 COLD/WIND CHILL 31 31 0 0 0 0
2008 DENSE FOG 0 0 0 430000 430000 0
2008 DENSE SMOKE 0 0 0 0 0 0
2008 DROUGHT 0 0 0 645000 100000 545000
2008 DUST DEVIL 15 1 14 60530 60530 0
2008 DUST STORM 0 0 0 0 0 0
2008 EXCESSIVE HEAT 146 15 131 0 0 0
2008 EXTREME COLD/WIND CHILL 8 8 0 0 0 0
2008 FLASH FLOOD 85 55 30 1559310660 1147531660 411779000
2008 FLOOD 38 22 16 3115787650 2037495650 1078292000
2008 FREEZING FOG 0 0 0 0 0 0
2008 FROST/FREEZE 0 0 0 120132000 40000 120092000
2008 FUNNEL CLOUD 0 0 0 25000 25000 0
2008 HAIL 14 2 12 606047300 427831300 178216000
2008 HEAT 35 27 8 0 0 0
2008 HEAVY RAIN 18 2 16 12402500 7902500 4500000
2008 HEAVY SNOW 22 4 18 5538000 5538000 0
2008 HIGH SURF 64 6 58 9824000 9824000 0
2008 HIGH WIND 67 11 56 833856470 786725470 47131000
2008 HURRICANE 1 1 0 2589380000 2419380000 170000000
2008 ICE STORM 0 0 0 87469000 87469000 0
2008 LAKE-EFFECT SNOW 0 0 0 2911000 2911000 0
2008 LANDSLIDE 0 0 0 2421000 2421000 0
2008 LIGHTNING 232 25 207 58976660 58873660 103000
2008 MARINE HAIL 0 0 0 4000 4000 0
2008 MARINE HIGH WIND 2 1 1 110000 110000 0
2008 MARINE STRONG WIND 4 4 0 26100 26100 0
2008 MARINE THUNDERSTORM WIND 8 2 6 162000 162000 0
2008 RIP CURRENT 65 44 21 0 0 0
2008 SEICHE 0 0 0 75000 75000 0
2008 SLEET 0 0 0 0 0 0
2008 STORM SURGE/TIDE 11 11 0 4595885000 4595035000 850000
2008 STRONG WIND 43 17 26 8762540 8713540 49000
2008 THUNDERSTORM WIND 278 25 253 1185744580 1159133580 26611000
2008 TORNADO 1819 129 1690 1724441840 1706231840 18210000
2008 TROPICAL DEPRESSION 0 0 0 1300000 1300000 0
2008 TROPICAL STORM 15 2 13 261973750 110568750 151405000
2008 VOLCANIC ASHFALL 0 0 0 0 0 0
2008 WATERSPOUT 0 0 0 10200 10200 0
2008 WILDFIRE 35 3 32 201948100 199938100 2010000
2008 WINTER STORM 4 0 4 783718000 783718000 0
2008 WINTER WEATHER 88 11 77 5973200 5973200 0
2009 ASTRONOMICAL LOW TIDE 0 0 0 0 0 0
2009 AVALANCHE 10 7 3 1001000 1001000 0
2009 BLIZZARD 4 3 1 24061000 24061000 0
2009 COASTAL FLOOD 0 0 0 53169500 53169500 0
2009 COLD/WIND CHILL 11 11 0 0 0 0
2009 DENSE FOG 0 0 0 755000 755000 0
2009 DENSE SMOKE 0 0 0 0 0 0
2009 DROUGHT 0 0 0 13988000 50000 13938000
2009 DUST DEVIL 3 0 3 226500 226500 0
2009 DUST STORM 3 3 0 1410000 660000 750000
2009 EXCESSIVE HEAT 19 7 12 0 0 0
2009 EXTREME COLD/WIND CHILL 17 17 0 0 0 0
2009 FLASH FLOOD 50 33 17 451537970 442489970 9048000
2009 FLOOD 33 23 10 637512660 607782660 29730000
2009 FREEZING FOG 0 0 0 2000000 2000000 0
2009 FROST/FREEZE 0 0 0 65085000 85000 65000000
2009 FUNNEL CLOUD 0 0 0 0 0 0
2009 HAIL 70 0 70 1790994550 1440000550 350994000
2009 HEAT 40 26 14 1510000 1510000 0
2009 HEAVY RAIN 15 6 9 5811000 5651000 160000
2009 HEAVY SNOW 2 2 0 18952000 18952000 0
2009 HIGH SURF 44 15 29 16455000 16455000 0
2009 HIGH WIND 26 4 22 147056150 147026150 30000
2009 HURRICANE 1 0 1 30000 20000 10000
2009 ICE STORM 6 3 3 205154000 205154000 0
2009 LAKE-EFFECT SNOW 0 0 0 1657000 1657000 0
2009 LANDSLIDE 9 0 9 43885000 43885000 0
2009 LIGHTNING 235 34 201 43868350 43857350 11000
2009 MARINE HAIL 0 0 0 0 0 0
2009 MARINE HIGH WIND 0 0 0 0 0 0
2009 MARINE STRONG WIND 3 1 2 20000 20000 0
2009 MARINE THUNDERSTORM WIND 4 2 2 51500 51500 0
2009 RIP CURRENT 59 39 20 1000 1000 0
2009 SEICHE 0 0 0 15000 15000 0
2009 SLEET 0 0 0 0 0 0
2009 STORM SURGE/TIDE 0 0 0 118000 118000 0
2009 STRONG WIND 34 10 24 5972000 5970000 2000
2009 THUNDERSTORM WIND 208 20 188 1430389250 1397967250 32422000
2009 TORNADO 418 21 397 584292100 565814100 18478000
2009 TROPICAL DEPRESSION 0 0 0 2000 2000 0
2009 TROPICAL STORM 1 1 0 221000 221000 0
2009 TSUNAMI 161 32 129 81020000 81000000 20000
2009 WATERSPOUT 0 0 0 10000 10000 0
2009 WILDFIRE 70 2 68 93901600 92774600 1127000
2009 WINTER STORM 35 1 34 21362000 20862000 500000
2009 WINTER WEATHER 96 10 86 5929000 5929000 0
2010 ASTRONOMICAL LOW TIDE 0 0 0 0 0 0
2010 AVALANCHE 42 22 20 201800 201800 0
2010 BLIZZARD 0 0 0 1138000 1138000 0
2010 COASTAL FLOOD 0 0 0 5655000 5655000 0
2010 COLD/WIND CHILL 27 26 1 1929000 1829000 100000
2010 DENSE FOG 0 0 0 975000 975000 0
2010 DENSE SMOKE 0 0 0 0 0 0
2010 DROUGHT 0 0 0 5807000 0 5807000
2010 DUST DEVIL 0 0 0 61100 61100 0
2010 DUST STORM 0 0 0 140000 140000 0
2010 EXCESSIVE HEAT 102 39 63 90000 90000 0
2010 EXTREME COLD/WIND CHILL 1 1 0 0 0 0
2010 FLASH FLOOD 255 67 188 991720900 839040900 152680000
2010 FLOOD 168 41 127 4198737540 3094028540 1104709000
2010 FREEZING FOG 0 0 0 182000 182000 0
2010 FROST/FREEZE 0 0 0 357415000 3175000 354240000
2010 FUNNEL CLOUD 0 0 0 0 0 0
2010 HAIL 42 0 42 3506089950 3404109950 101980000
2010 HEAT 83 44 39 100000 0 100000
2010 HEAVY RAIN 7 0 7 9208500 5107500 4101000
2010 HEAVY SNOW 20 2 18 26015800 26015800 0
2010 HIGH SURF 37 17 20 49111500 49111500 0
2010 HIGH WIND 24 7 17 27727920 27659920 68000
2010 HURRICANE 0 0 0 0 0 0
2010 ICE STORM 1 0 1 5337800 5337800 0
2010 LAKE-EFFECT SNOW 0 0 0 6415000 6415000 0
2010 LAKESHORE FLOOD 0 0 0 0 0 0
2010 LANDSLIDE 0 0 0 17399000 17399000 0
2010 LIGHTNING 211 29 182 71600700 71148700 452000
2010 MARINE HAIL 0 0 0 0 0 0
2010 MARINE HIGH WIND 0 0 0 20010 20010 0
2010 MARINE STRONG WIND 2 2 0 1630 1630 0
2010 MARINE THUNDERSTORM WIND 2 0 2 11100 11100 0
2010 RIP CURRENT 94 49 45 0 0 0
2010 SEICHE 0 0 0 0 0 0
2010 SLEET 0 0 0 0 0 0
2010 STORM SURGE/TIDE 0 0 0 795000 795000 0
2010 STRONG WIND 36 8 28 11491570 10959570 532000
2010 THUNDERSTORM WIND 338 15 323 227213000 216401000 10812000
2010 TORNADO 744 45 699 1164565450 1136920450 27645000
2010 TROPICAL STORM 0 0 0 17719400 12709400 5010000
2010 TSUNAMI 0 0 0 305000 305000 0
2010 WATERSPOUT 0 0 0 1000 1000 0
2010 WILDFIRE 26 2 24 246938670 244888670 2050000
2010 WINTER STORM 10 6 4 61948000 61948000 0
2010 WINTER WEATHER 8 3 5 17705300 2705300 15000000
2011 ASTRONOMICAL LOW TIDE 0 0 0 0 0 0
2011 AVALANCHE 17 9 8 55000 55000 0
2011 BLIZZARD 2 2 0 2742000 2742000 0
2011 COASTAL FLOOD 2 1 1 27274000 27274000 0
2011 COLD/WIND CHILL 22 21 1 70000 70000 0
2011 DENSE FOG 0 0 0 162000 162000 0
2011 DENSE SMOKE 0 0 0 0 0 0
2011 DROUGHT 0 0 0 31388000 114000 31274000
2011 DUST DEVIL 8 0 8 33000 33000 0
2011 DUST STORM 4 0 4 848000 848000 0
2011 EXCESSIVE HEAT 174 36 138 1143200 1143200 0
2011 EXTREME COLD/WIND CHILL 3 2 1 7035000 7035000 0
2011 FLASH FLOOD 98 68 30 1472491700 1384044700 88447000
2011 FLOOD 68 58 10 7872549450 7717677450 154872000
2011 FREEZING FOG 0 0 0 0 0 0
2011 FROST/FREEZE 0 0 0 18950000 5540000 13410000
2011 FUNNEL CLOUD 0 0 0 40100 40100 0
2011 HAIL 31 0 31 533663550 451329550 82334000
2011 HEAT 674 63 611 0 0 0
2011 HEAVY RAIN 2 1 1 32504000 11791000 20713000
2011 HEAVY SNOW 0 0 0 16145300 16125300 20000
2011 HIGH SURF 22 11 11 222000 222000 0
2011 HIGH WIND 15 4 11 86244000 41951000 44293000
2011 HURRICANE 0 0 0 21000000 10500000 10500000
2011 ICE STORM 0 0 0 7917500 7837500 80000
2011 LAKE-EFFECT SNOW 0 0 0 853000 853000 0
2011 LAKESHORE FLOOD 0 0 0 7500000 7500000 0
2011 LANDSLIDE 1 1 0 21153000 21136000 17000
2011 LIGHTNING 220 26 194 47090920 46978920 112000
2011 MARINE HAIL 0 0 0 0 0 0
2011 MARINE HIGH WIND 0 0 0 1010000 1010000 0
2011 MARINE STRONG WIND 7 2 5 351600 351600 0
2011 MARINE THUNDERSTORM WIND 17 3 14 158800 108800 50000
2011 RIP CURRENT 56 29 27 0 0 0
2011 SEICHE 0 0 0 0 0 0
2011 SLEET 0 0 0 0 0 0
2011 STORM SURGE/TIDE 0 0 0 40695000 40695000 0
2011 STRONG WIND 43 10 33 31604130 16545130 15059000
2011 THUNDERSTORM WIND 427 54 373 521723410 381891410 139832000
2011 TORNADO 6750 587 6163 9850961700 9819600700 31361000
2011 TROPICAL STORM 5 4 1 163243200 138742200 24501000
2011 TSUNAMI 1 1 0 53554000 53554000 0
2011 WATERSPOUT 0 0 0 5110000 5110000 0
2011 WILDFIRE 122 6 116 658115400 648318400 9797000
2011 WINTER STORM 1 1 0 18227000 18157000 70000
2011 WINTER WEATHER 2 2 0 1895000 1895000 0

Legend: EVTYPE - type of events, PROPDMG - damages to properties in $, CROPDMG - damages to crops in $, TOTAL.POPULATION - pool of fatalities and injuries, TOTAL.DAMAGE - pool of damages to properties and crops in $.

Table 1 shows that the weather events did not have an impact every year. Some events had an impact only for a year or two. Hence, an average per year would not be representative of the impactness of the events. We will then summarize the total number of fatalities, injuries, damage to properties or damage to crop from 2001 to 2011 in Table 2.

table2 <- dcast(table1_melt, DATE~variable, fun.aggregate = sum)
table2$TOTAL.POPULATION <-  table2$FATALITIES + table2$INJURIES
table2$TOTAL.DAMAGE <- table2$PROPDMG + table2$CROPDMG
table2 <- table2 %>% select(DATE, TOTAL.POPULATION, FATALITIES, INJURIES, TOTAL.DAMAGE, PROPDMG, CROPDMG)
knitr::kable(table2, caption = "Table 2: Summary of impact on population and economy over the time")
Table 2: Summary of impact on population and economy over the time
DATE TOTAL.POPULATION FATALITIES INJURIES TOTAL.DAMAGE PROPDMG CROPDMG
2001 3190 469 2721 11843771770 10027043670 1816728100
2002 3653 498 3155 5511250590 4100882450 1410368140
2003 3374 443 2931 11397618590 10254548240 1143070350
2004 2796 370 2426 26798776720 25346598870 1452177850
2005 2303 469 1834 100824993470 96789791170 4035202300
2006 3967 599 3368 125471672890 121937434190 3534238700
2007 2612 421 2191 7480086160 5788934160 1691152000
2008 3191 488 2703 17778176080 15568383080 2209793000
2009 1687 333 1354 5749424130 5227204130 522220000
2010 2280 425 1855 11031773640 9246487640 1785286000
2011 8794 1002 7792 21555723960 20888981960 666742000

Interestingly, the total impact on population which includes the number of people injured and killed, independently of the type of events, was relatively stable over the time but surged in 2011 as presented in Figure 1. In contrast, the highest impact on properties and crops were observed in 2005 and 2006. These results suggest that the economical impact and harmfulness on population are not correlated.

Figure 1. Evolution of impact on population and economy over the time

Figure 1. Evolution of impact on population and economy over the time

3.2 Tornadoes are the most harmful event on population health

We have observed the evolution of the impact of population and economy over the time. Next, we will investigate which type of events is the most harmful to population health, all years combined since 2001.

Figure 2. Types of events impacting population health

Figure 2. Types of events impacting population health

The figure 2 represents all event types recorded since 2001 and their impact on population health, including injuries, fatalities or the pool number of injuries and fatalities. The bar graph is ordered by impact. Tornadoes have the most impact on population, both in terms of fatalities and injuries. The second most impact event is the excessive heat, although markedly lower on injuries compared to tornado. In third position, flash flood led to the highest number of fatalities while thunderstorm winds had more impact on injuries.

These results show that tornadoes are the most harmful event on population health.

3.3 Flood has the most impact on properties and droughts on crops

Similar to the impact on population health, we next evaluate the impact of the events on properties and crops damages.

Figure 3. Types of events impacting properties and crops

Figure 3. Types of events impacting properties and crops

The Figure 3 represents all event types recorded since 2001 and their impact on economy, including the cost for property damages, crop damage or the combined cost on properties and crops. The bar graph is ordered by impact. The highest costs are reported for properties. Flood is the event which has the most impact on properties and second highest on crops. Hurricanes have the second highest impact on properties and third highest impact on crops. In contrast, droughts have the most impact on crops but are only in 13th position on properties.

These results show differences between impact on properties and crops with flood being the most impactful on properties while drought had more damages on crops.

4. Conclusions

By analyzing the Storm database from the NOAA we can evaluate the evolution of the weather events over the time and their impact on population health and economy. Interestingly, peak of impact on population and economy was not observed the same years. The peak impact on population was observed in 2011 while on economy, the most expensive damages were observed in 2006. Tornadoes have injured and killed the highest number of person. Interestingly, tornadoes did not have the highest impact on economy. Flood led to the highest cost on properties and droughts on the crops.