Title

Across the US, what type of weather events is the most damaging? Let’s find out.

synopsis

This document will attempt to find out what type of weather Events
across the US, 1.) have the most impact on ‘population health’ and 2.) have the most impact on ‘economic value’ We will be using data from National Climatic Data Center Storm Events (or more specifically, this version[47 Mb])
According to data from 1996 - 2011 across the US, we have found that HURRICANE cause the most property damage while DROUGHT cause the most damage to Crops. For population health, EXCESSIVE HEAT is number 1 leading cause of Fatalities among weather events and TORNADO caused the most injuries.

Data Processing

Load the data into variable dat

#https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2
#load data
# place repdata_data_StormData.csv.bz2 in working directory before running code
dat <- read.csv("repdata_data_StormData.csv.bz2")

EDA and Data Cleaning

We will be subsetting our data in the following steps:

  1. only include date after "1996-01-01": from documentation, we learn that complete Event Types are only available after 1996. Before that year the data is heavily bias towards Tornados Events. We will be analyzing only the data from after year 1996.

  2. Since our main question are about damage and people affected from storm events, we will be keeping only the following columns: "BGN_DATE","EVTYPE", "FATALITIES", "INJURIES", "PROPDMG" "PROPDMGEXP", "CROPDMG", "CROPDMGEXP", "REMARKS", "REFNUM"

  3. Keep only rows with data on damage or people afflicted. Some rows have 0 values in damage (PROPDMG, CROPDMG) and people affected (FATALITIES,INJURIES)

require(dplyr) ## for dplyr::select
## Loading required package: dplyr
## Warning: package 'dplyr' was built under R version 4.2.3
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
dat$BGN_DATE<- as.Date(dat$BGN_DATE, format = "%m/%d/%Y")

# date after 1996-01-01
dat <- dat[dat$BGN_DATE >= "1996-01-01",]

# keep certain columns
dat <- select(dat, c("BGN_DATE","EVTYPE", "FATALITIES", "INJURIES", "PROPDMG", "PROPDMGEXP", "CROPDMG", "CROPDMGEXP", "REMARKS", "REFNUM"))

#keep rows where damage > 0 or people affected > 0
dat <- dat[dat$CROPDMG > 0 | dat$PROPDMG > 0| dat$FATALITIES > 0 | dat$INJURIES >0,]
summary(dat)
##     BGN_DATE             EVTYPE            FATALITIES           INJURIES       
##  Min.   :1996-01-01   Length:201318      Min.   :  0.00000   Min.   :   0.000  
##  1st Qu.:2000-06-21   Class :character   1st Qu.:  0.00000   1st Qu.:   0.000  
##  Median :2005-04-22   Mode  :character   Median :  0.00000   Median :   0.000  
##  Mean   :2004-09-25                      Mean   :  0.04337   Mean   :   0.288  
##  3rd Qu.:2009-02-11                      3rd Qu.:  0.00000   3rd Qu.:   0.000  
##  Max.   :2011-11-30                      Max.   :158.00000   Max.   :1150.000  
##     PROPDMG         PROPDMGEXP           CROPDMG        CROPDMGEXP       
##  Min.   :   0.00   Length:201318      Min.   :  0.00   Length:201318     
##  1st Qu.:   2.00   Class :character   1st Qu.:  0.00   Class :character  
##  Median :   6.00   Mode  :character   Median :  0.00   Mode  :character  
##  Mean   :  37.96                      Mean   :  5.97                     
##  3rd Qu.:  25.00                      3rd Qu.:  0.00                     
##  Max.   :5000.00                      Max.   :990.00                     
##    REMARKS              REFNUM      
##  Length:201318      Min.   :248768  
##  Class :character   1st Qu.:394793  
##  Mode  :character   Median :570208  
##                     Mean   :573211  
##                     3rd Qu.:749106  
##                     Max.   :902260

columns of interest: "BGN_DATE","EVTYPE", "FATALITIES", "INJURIES", "PROPDMG" "PROPDMGEXP", "CROPDMG", "CROPDMGEXP", "REMARKS", "REFNUM"does not seems to contain any NA or errors at first glance.

Data Cleaning and transformation

we will need the information in EVTYPE columns for Events Type. lets explore that.

tmp<-sort(table(dat$EVTYPE),decreasing =TRUE)
dim(tmp) ## 222 Events type
## [1] 222
head(tmp); sum(head(tmp)) ## top 6 accounted for 170K of total 200K 
## 
##         TSTM WIND THUNDERSTORM WIND              HAIL       FLASH FLOOD 
##             61775             43097             22679             19011 
##           TORNADO         LIGHTNING 
##             12366             11152
## [1] 170080
set.seed(555)
sample(names(tmp),20)
##  [1] "LAKESHORE FLOOD"          "Beach Erosion"           
##  [3] "COASTAL FLOODING/EROSION" "Damaging Freeze"         
##  [5] "Freeze"                   "Snow"                    
##  [7] "DOWNBURST"                "NON-TSTM WIND"           
##  [9] "Extended Cold"            "URBAN/SML STREAM FLD"    
## [11] "HIGH WATER"               "Light snow"              
## [13] "HEAVY SEAS"               "Wind Damage"             
## [15] "Tidal Flooding"           "Mudslide"                
## [17] "COLD AND SNOW"            "UNSEASONAL RAIN"         
## [19] "Gusty wind/rain"          "HIGH WINDS"

Column EVTYPE contains mixing upper case and lower case, mispelling and non standard events type. We will deal with those by converting them to upper case, then use regular-expression to match them with Event Types in NWSI 10-1605. If no match is found, the EVTYPE will be assigned as "OTHER"

dat$EVTYPE <- toupper(dat$EVTYPE)

# Event name from documentation.
EventName <- c("Astronomical Low Tide",
                         "Avalanche",
                         "Blizzard",
                         "Coastal Flood",
                         "Cold/Wind Chill",
                         "Debris Flow",
                         "Dense Fog",
                         "Dense Smoke",
                         "Drought",
                         "Dust Devil",
                         "Dust Storm",
                         "Excessive Heat",
                         "Extreme Cold/Wind Chill",
                         "Flash Flood",
                         "Flood",
                         "Frost/Freeze",
                         "Funnel Cloud",
                         "Freezing Fog",
                         "Hail",
                         "Heat",
                         "Heavy Rain",
                         "Heavy Snow",
                         "High Surf",
                         "High Wind",
                         "Hurricane (Typhoon)",
                         "Ice Storm",
                         "Lake-Effect Snow",
                         "Lakeshore Flood",
                         "Lightning",
                         "Marine Hail",
                         "Marine High Wind",
                         "Marine Strong Wind",
                         "Marine Thunderstorm Wind",
                         "Rip Current",
                         "Seiche",
                         "Sleet",
                         "Storm Surge/Tide",
                         "Strong Wind",
                         "Thunderstorm Wind",
                         "Tornado",
                         "Tropical Depression",
                         "Tropical Storm",
                         "Tsunami",
                         "Volcanic Ash",
                         "Waterspout",
                         "Wildfire",
                         "Winter Storm",
                         "Winter Weather")
EventName <- toupper(EventName)

# regex for pattern matching
EventNamePattern <- c("ASTRONIMICAL LOW TIDE",
                         "AVALANCHE",
                         "BLIZZARD",
                         "(^COASTAL+ *FLOOD)|EROSION",
                         "(COLD|WIND)+ *CHILL$",
                         "DEBRIS FLOW|LANDSLIDE|SLIDE",
                         "FOG",
                         "DENSE SMOKE",
                         "DROUGHT",
                         "^DUST+.*DEVIL$",
                         "DUST",
                         "(EXCESSIVE|RECORD)+.*HEAT$|^HEAT WAVE$",
                         "CHILL|COLD",
                         "FLASH",
                         "FLOOD|FLD",
                         "FROST|FREEZE|^IC.*ROADS?",
                         "FUNNEL",
                         "^FREEZING+.*FOG",
                         "HAIL",
                         "HEAT|WARM",
                         "RAIN",
                         "SNOW",
                         "SURF|WAVE|SEAS$",
                      "HIGH WIND|STRONG WIND|^WINDS?$|^WIND DAMAGE$|^GUSTY WIND",
                         "HURRICANE|TYPHOON",
                         "ICE STORM",
                         "^LAKE+.*SNOW",
                         "LAKESHORE FLOOD",
                         "LIGHTNING",
                         "^MARINE HAIL",
                         "^MARINE HIGH WIND",
                         "^MARINE STRONG WIND",
                         "^MARINE THUNDERSTORM WIND|^MARINE TSTM WIND",
                         "^RIP CURRENT",
                         "SEICHE",
                         "SLEET",
                         "SURGE",
                         "^STRONG WIND",
                         "THUNDERSTORM|^ ?TSTM WIND|BURST|^WHIRLWIND$",
                         "TORNAD",
                         "TROPICAL DEPRESSION",
                         "TROPICAL STORM",
                         "^TSUNAMI",
                         "VOLCA",
                         "WATER",
                         "FIRE",
                         "INTER STORM",
                         "WINT")

## function for cleaning EVTYPE
cleanEVTYPE <- function(vchr){
    EventID <- integer(length(vchr))
    matched <- logical(length(vchr))
    
    ## step 1/3, try exact match
    for(ev in 1:length(EventName)){
        EventID[vchr == EventName[ev]] <- ev
    }
    matched <- EventID > 0
    
    ## step 2/3, try pattern matching
    for(i in 1:length(EventNamePattern)){
        grepEventID <- grep(EventNamePattern[i], vchr[!matched], perl = TRUE)
        EventID[!matched][grepEventID] <- i
        matched[!matched][grepEventID] <- TRUE
    }
    
    ## step 3/3, assign unmatched with 'Other' and return
    cleaned <- character(length(vchr))
    cleaned[EventID > 0] <- EventName[EventID[EventID > 0]]
    cleaned[EventID == 0] <- "OTHER"
    return(cleaned)
}

## clean col EVTYPE and convert to factor
dat$EVENT <- as.factor(cleanEVTYPE(dat$EVTYPE))

# check mapping result
tmp <- unique(cbind(new = as.character(dat$EVENT), original = dat$EVTYPE))
tmp[order(tmp[,"new"]),]; paste("OTHER:",sum(dat$EVENT == "OTHER"),", TOTAL:",dim(dat)[1] )
##        new                        original                   
##   [1,] "ASTRONOMICAL LOW TIDE"    "ASTRONOMICAL LOW TIDE"    
##   [2,] "AVALANCHE"                "AVALANCHE"                
##   [3,] "BLIZZARD"                 "BLIZZARD"                 
##   [4,] "COASTAL FLOOD"            "EROSION/CSTL FLOOD"       
##   [5,] "COASTAL FLOOD"            "BEACH EROSION"            
##   [6,] "COASTAL FLOOD"            "COASTAL FLOODING"         
##   [7,] "COASTAL FLOOD"            "COASTAL FLOOD"            
##   [8,] "COASTAL FLOOD"            "COASTAL FLOODING/EROSION" 
##   [9,] "COASTAL FLOOD"            "COASTAL EROSION"          
##  [10,] "COASTAL FLOOD"            "COASTAL  FLOODING/EROSION"
##  [11,] "COLD/WIND CHILL"          "EXTREME WINDCHILL"        
##  [12,] "COLD/WIND CHILL"          "COLD/WIND CHILL"          
##  [13,] "DEBRIS FLOW"              "MUDSLIDES"                
##  [14,] "DEBRIS FLOW"              "MUDSLIDE"                 
##  [15,] "DEBRIS FLOW"              "MUD SLIDE"                
##  [16,] "DEBRIS FLOW"              "LANDSLIDES"               
##  [17,] "DEBRIS FLOW"              "ROCK SLIDE"               
##  [18,] "DEBRIS FLOW"              "LANDSLIDE"                
##  [19,] "DENSE FOG"                "FOG"                      
##  [20,] "DENSE FOG"                "DENSE FOG"                
##  [21,] "DENSE SMOKE"              "DENSE SMOKE"              
##  [22,] "DROUGHT"                  "DROUGHT"                  
##  [23,] "DUST DEVIL"               "DUST DEVIL"               
##  [24,] "DUST STORM"               "DUST STORM"               
##  [25,] "DUST STORM"               "BLOWING DUST"             
##  [26,] "EXCESSIVE HEAT"           "EXCESSIVE HEAT"           
##  [27,] "EXCESSIVE HEAT"           "HEAT WAVE"                
##  [28,] "EXCESSIVE HEAT"           "RECORD HEAT"              
##  [29,] "EXTREME COLD/WIND CHILL"  "EXTREME COLD"             
##  [30,] "EXTREME COLD/WIND CHILL"  "UNSEASONABLE COLD"        
##  [31,] "EXTREME COLD/WIND CHILL"  "EXTENDED COLD"            
##  [32,] "EXTREME COLD/WIND CHILL"  "COLD"                     
##  [33,] "EXTREME COLD/WIND CHILL"  "COLD TEMPERATURE"         
##  [34,] "EXTREME COLD/WIND CHILL"  "COLD AND SNOW"            
##  [35,] "EXTREME COLD/WIND CHILL"  "UNSEASONABLY COLD"        
##  [36,] "EXTREME COLD/WIND CHILL"  "COLD WEATHER"             
##  [37,] "EXTREME COLD/WIND CHILL"  "EXTREME COLD/WIND CHILL"  
##  [38,] "FLASH FLOOD"              "FLASH FLOOD"              
##  [39,] "FLASH FLOOD"              "FLOOD/FLASH/FLOOD"        
##  [40,] "FLASH FLOOD"              "FLASH FLOOD/FLOOD"        
##  [41,] "FLASH FLOOD"              " FLASH FLOOD"             
##  [42,] "FLOOD"                    "FLOOD"                    
##  [43,] "FLOOD"                    "ICE JAM FLOOD (MINOR"     
##  [44,] "FLOOD"                    "URBAN/SML STREAM FLD"     
##  [45,] "FLOOD"                    "RIVER FLOODING"           
##  [46,] "FLOOD"                    "TIDAL FLOODING"           
##  [47,] "FLOOD"                    "RIVER FLOOD"              
##  [48,] "FREEZING FOG"             "FREEZING FOG"             
##  [49,] "FROST/FREEZE"             "FREEZE"                   
##  [50,] "FROST/FREEZE"             "DAMAGING FREEZE"          
##  [51,] "FROST/FREEZE"             "EARLY FROST"              
##  [52,] "FROST/FREEZE"             "HARD FREEZE"              
##  [53,] "FROST/FREEZE"             "FROST/FREEZE"             
##  [54,] "FROST/FREEZE"             "AGRICULTURAL FREEZE"      
##  [55,] "FROST/FREEZE"             "ICY ROADS"                
##  [56,] "FROST/FREEZE"             "FROST"                    
##  [57,] "FROST/FREEZE"             "ICE ROADS"                
##  [58,] "FROST/FREEZE"             "ICE ON ROAD"              
##  [59,] "FUNNEL CLOUD"             "FUNNEL CLOUD"             
##  [60,] "HAIL"                     "HAIL"                     
##  [61,] "HAIL"                     "TSTM WIND/HAIL"           
##  [62,] "HAIL"                     "SMALL HAIL"               
##  [63,] "HAIL"                     "GUSTY WIND/HAIL"          
##  [64,] "HEAT"                     "HEAT"                     
##  [65,] "HEAT"                     "UNSEASONABLY WARM"        
##  [66,] "HEAT"                     "WARM WEATHER"             
##  [67,] "HEAVY RAIN"               "FREEZING RAIN"            
##  [68,] "HEAVY RAIN"               "HEAVY RAIN"               
##  [69,] "HEAVY RAIN"               "HEAVY RAIN/HIGH SURF"     
##  [70,] "HEAVY RAIN"               "TORRENTIAL RAINFALL"      
##  [71,] "HEAVY RAIN"               "GUSTY WIND/RAIN"          
##  [72,] "HEAVY RAIN"               "GUSTY WIND/HVY RAIN"      
##  [73,] "HEAVY RAIN"               "RAIN/SNOW"                
##  [74,] "HEAVY RAIN"               "UNSEASONAL RAIN"          
##  [75,] "HEAVY RAIN"               "LIGHT FREEZING RAIN"      
##  [76,] "HEAVY RAIN"               "RAIN"                     
##  [77,] "HEAVY SNOW"               "HEAVY SNOW"               
##  [78,] "HEAVY SNOW"               "HEAVY SNOW SHOWER"        
##  [79,] "HEAVY SNOW"               "LIGHT SNOW"               
##  [80,] "HEAVY SNOW"               "SNOW"                     
##  [81,] "HEAVY SNOW"               "SNOW SQUALLS"             
##  [82,] "HEAVY SNOW"               "LIGHT SNOWFALL"           
##  [83,] "HEAVY SNOW"               "SNOW AND ICE"             
##  [84,] "HEAVY SNOW"               "SNOW SQUALL"              
##  [85,] "HEAVY SNOW"               "LAKE EFFECT SNOW"         
##  [86,] "HEAVY SNOW"               "BLOWING SNOW"             
##  [87,] "HEAVY SNOW"               "EXCESSIVE SNOW"           
##  [88,] "HEAVY SNOW"               "LATE SEASON SNOW"         
##  [89,] "HEAVY SNOW"               "FALLING SNOW/ICE"         
##  [90,] "HIGH SURF"                "ROUGH SURF"               
##  [91,] "HIGH SURF"                "HEAVY SURF"               
##  [92,] "HIGH SURF"                "HIGH SURF"                
##  [93,] "HIGH SURF"                "HEAVY SURF AND WIND"      
##  [94,] "HIGH SURF"                "HEAVY SEAS"               
##  [95,] "HIGH SURF"                "WIND AND WAVE"            
##  [96,] "HIGH SURF"                "HIGH SEAS"                
##  [97,] "HIGH SURF"                "ROUGH SEAS"               
##  [98,] "HIGH SURF"                "ROGUE WAVE"               
##  [99,] "HIGH SURF"                "   HIGH SURF ADVISORY"    
## [100,] "HIGH SURF"                "HAZARDOUS SURF"           
## [101,] "HIGH SURF"                "HEAVY SURF/HIGH SURF"     
## [102,] "HIGH WIND"                "HIGH WIND"                
## [103,] "HIGH WIND"                "WINDS"                    
## [104,] "HIGH WIND"                "STRONG WINDS"             
## [105,] "HIGH WIND"                "WIND DAMAGE"              
## [106,] "HIGH WIND"                "WIND"                     
## [107,] "HIGH WIND"                "GUSTY WINDS"              
## [108,] "HIGH WIND"                "GUSTY WIND"               
## [109,] "HIGH WIND"                "HIGH WINDS"               
## [110,] "HIGH WIND"                "HIGH WIND (G40)"          
## [111,] "HURRICANE (TYPHOON)"      "HURRICANE"                
## [112,] "HURRICANE (TYPHOON)"      "HURRICANE EDOUARD"        
## [113,] "HURRICANE (TYPHOON)"      "TYPHOON"                  
## [114,] "HURRICANE (TYPHOON)"      "HURRICANE/TYPHOON"        
## [115,] "ICE STORM"                "ICE STORM"                
## [116,] "LAKE-EFFECT SNOW"         "LAKE-EFFECT SNOW"         
## [117,] "LAKESHORE FLOOD"          "LAKESHORE FLOOD"          
## [118,] "LIGHTNING"                "LIGHTNING"                
## [119,] "LIGHTNING"                "TSTM WIND AND LIGHTNING"  
## [120,] "MARINE HAIL"              "MARINE HAIL"              
## [121,] "MARINE HIGH WIND"         "MARINE HIGH WIND"         
## [122,] "MARINE STRONG WIND"       "MARINE STRONG WIND"       
## [123,] "MARINE THUNDERSTORM WIND" "MARINE TSTM WIND"         
## [124,] "MARINE THUNDERSTORM WIND" "MARINE THUNDERSTORM WIND" 
## [125,] "OTHER"                    "OTHER"                    
## [126,] "OTHER"                    "MARINE ACCIDENT"          
## [127,] "OTHER"                    "COASTAL STORM"            
## [128,] "OTHER"                    "LANDSLUMP"                
## [129,] "OTHER"                    "GLAZE"                    
## [130,] "OTHER"                    "MIXED PRECIP"             
## [131,] "OTHER"                    "FREEZING SPRAY"           
## [132,] "OTHER"                    "FREEZING DRIZZLE"         
## [133,] "OTHER"                    "HYPOTHERMIA/EXPOSURE"     
## [134,] "OTHER"                    "MIXED PRECIPITATION"      
## [135,] "OTHER"                    "BLACK ICE"                
## [136,] "OTHER"                    "COASTALSTORM"             
## [137,] "OTHER"                    "DAM BREAK"                
## [138,] "OTHER"                    "GRADIENT WIND"            
## [139,] "OTHER"                    "HIGH SWELLS"              
## [140,] "OTHER"                    "HYPERTHERMIA/EXPOSURE"    
## [141,] "OTHER"                    "LANDSPOUT"                
## [142,] "OTHER"                    "NON-SEVERE WIND DAMAGE"   
## [143,] "OTHER"                    "NON-TSTM WIND"            
## [144,] "OTHER"                    "NON TSTM WIND"            
## [145,] "OTHER"                    "DROWNING"                 
## [146,] "OTHER"                    "ASTRONOMICAL HIGH TIDE"   
## [147,] "RIP CURRENT"              "RIP CURRENTS"             
## [148,] "RIP CURRENT"              "RIP CURRENT"              
## [149,] "SEICHE"                   "SEICHE"                   
## [150,] "STORM SURGE/TIDE"         "STORM SURGE"              
## [151,] "STORM SURGE/TIDE"         "STORM SURGE/TIDE"         
## [152,] "STRONG WIND"              "STRONG WIND"              
## [153,] "THUNDERSTORM WIND"        "TSTM WIND"                
## [154,] "THUNDERSTORM WIND"        "DRY MICROBURST"           
## [155,] "THUNDERSTORM WIND"        "WHIRLWIND"                
## [156,] "THUNDERSTORM WIND"        "DOWNBURST"                
## [157,] "THUNDERSTORM WIND"        "MICROBURST"               
## [158,] "THUNDERSTORM WIND"        "TSTM WIND (G45)"          
## [159,] "THUNDERSTORM WIND"        "TSTM WIND 40"             
## [160,] "THUNDERSTORM WIND"        "TSTM WIND 45"             
## [161,] "THUNDERSTORM WIND"        "TSTM WIND (41)"           
## [162,] "THUNDERSTORM WIND"        "TSTM WIND (G40)"          
## [163,] "THUNDERSTORM WIND"        "THUNDERSTORM"             
## [164,] "THUNDERSTORM WIND"        "WET MICROBURST"           
## [165,] "THUNDERSTORM WIND"        " TSTM WIND (G45)"         
## [166,] "THUNDERSTORM WIND"        "TSTM WIND  (G45)"         
## [167,] "THUNDERSTORM WIND"        "TSTM WIND (G35)"          
## [168,] "THUNDERSTORM WIND"        " TSTM WIND"               
## [169,] "THUNDERSTORM WIND"        "TSTM WIND G45"            
## [170,] "THUNDERSTORM WIND"        "THUNDERSTORM WIND (G40)"  
## [171,] "THUNDERSTORM WIND"        "THUNDERSTORM WIND"        
## [172,] "TORNADO"                  "TORNADO"                  
## [173,] "TROPICAL DEPRESSION"      "TROPICAL DEPRESSION"      
## [174,] "TROPICAL STORM"           "TROPICAL STORM"           
## [175,] "TSUNAMI"                  "TSUNAMI"                  
## [176,] "VOLCANIC ASH"             "VOLCANIC ASH"             
## [177,] "WATERSPOUT"               "WATERSPOUT"               
## [178,] "WATERSPOUT"               "HIGH WATER"               
## [179,] "WILDFIRE"                 "WILD/FOREST FIRE"         
## [180,] "WILDFIRE"                 "BRUSH FIRE"               
## [181,] "WILDFIRE"                 "WILDFIRE"                 
## [182,] "WINTER STORM"             "WINTER STORM"             
## [183,] "WINTER WEATHER"           "WINTRY MIX"               
## [184,] "WINTER WEATHER"           "WINTER WEATHER"           
## [185,] "WINTER WEATHER"           "WINTER WEATHER MIX"       
## [186,] "WINTER WEATHER"           "WINTER WEATHER/MIX"
## [1] "OTHER: 119 , TOTAL: 201318"

We have now clean data in EVTYPE and put it in new column called EVENT as factors. small percentage of EVTYPE get categorize as OTHER (119 / 201318). We could confirm the effect size of the EVENTS == "OTHER" later, But this should be good enough for our analyses.

Let’s look at top 10 most common Events

tmp<-sort(table(dat$EVENT),decreasing =TRUE)
dim(tmp) ## 222 Events type
## [1] 48
head(tmp,10); paste("TOP 10:",sum(head(tmp,10)), ", TOTAL:",dim(dat)[1])
## 
## THUNDERSTORM WIND              HAIL       FLASH FLOOD           TORNADO 
##            105013             23132             19014             12366 
##         LIGHTNING             FLOOD         HIGH WIND       STRONG WIND 
##             11153             10306              5562              3369 
##      WINTER STORM        HEAVY SNOW 
##              1460              1230
## [1] "TOP 10: 192605 , TOTAL: 201318"

from NWSI 10-1605, we learned that damage are stored in a specific columns format, PROPDMG and CROPDMG contains the significant digits, PROPDMGEXP and CROPDMGEXP stored the magnitude of the number. “K” for Thousand, “M” for Million and “B” for Billion. We will also create new columns call DMGTOTAL for Total Damage and PPLAFFECTED for number of people affected by each weather event

require(dplyr)
cropDamageMultiplier <- case_when(
    dat$CROPDMGEXP == '' ~ 0,
    dat$CROPDMGEXP %in% c('k','K') ~ 1e3,
    dat$CROPDMGEXP %in% c('m','M') ~ 1e6,
    dat$CROPDMGEXP %in% c('b','B') ~ 1e9,
    .default = 0
)
propDamageMultiplier <- case_when(
    dat$PROPDMGEXP == '' ~ 0,
    dat$PROPDMGEXP %in% c('k','K') ~ 1e3,
    dat$PROPDMGEXP %in% c('m','M') ~ 1e6,
    dat$PROPDMGEXP %in% c('b','B') ~ 1e9,
    .default = 0
)

dat$CROPDMG <- dat$CROPDMG * cropDamageMultiplier
dat$PROPDMG <- dat$PROPDMG * propDamageMultiplier
dat$DMGTOTAL <- dat$CROPDMG + dat$PROPDMG
dat$PPLAFFECTED <- dat$FATALITIES + dat$INJURIES

There is a known errorneous data point at REFNUM == 605943, refering to Napa River Flood in 2005/2006, Napa, CA. Renato P. dos Santos, 2016 gave some comments on many error in the data set. This data point incorrectly record damage as 115 Billion USD. we will remove this data point.

## remove napa river flood REFNUM 605943
dat <- dat[dat$REFNUM != 605943,]

Result

Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?

plot number of People Affected by Weather Events

require(tidyr) ## for tidyr::pivot_wider, pivot_longer
## Loading required package: tidyr
## Warning: package 'tidyr' was built under R version 4.2.3
ppl <- dat %>% 
    group_by(EVENT) %>% 
    summarize(#People = sum(PPLAFFECTED), 
              INJURIES = sum(INJURIES), 
              FATALITIES = sum(FATALITIES)
              ) %>% 
    arrange(desc(INJURIES + FATALITIES))
tmp <- ppl %>% 
    pivot_longer(!EVENT, names_to = "type", values_to="People") %>% 
    pivot_wider(names_from = EVENT, values_from = People)

plt <- barplot(as.matrix(tmp[,-c(1)][,1:10])+1, ## +1 for log scale, #[,1:10] for top 10
        horiz = T,
        beside = T,
        log = "x",
        axes = T,
        xlab = "Number of People",
        las = 1, ## rotates text label
        cex.names = 0.5,
        border = T,
        main = "Number of People affected by each Weather Events"
)
abline(v = c(1000,5000), col = "darkred", lty = 2, lwd = 3)
legend("topright",
       legend = tmp[["type"]],
       fill = gray.colors(2),
       angle = 90)

top3injuries <- ppl[order(ppl$INJURIES,decreasing = T)[1:3],]
top3fatalities <- ppl[order(ppl$FATALITIES,decreasing = T)[1:3],]
totInjuries <- sum(ppl$INJURIES)
totFatalities <- sum(ppl$FATALITIES)

"TOP 3 INJURIES"; top3injuries; paste("Top 3 Injuries accounted for",
      round(sum(top3injuries$INJURIES/totInjuries)*100,1),
      "% of Total recorded Injuries")
## [1] "TOP 3 INJURIES"
## # A tibble: 3 × 3
##   EVENT          INJURIES FATALITIES
##   <fct>             <dbl>      <dbl>
## 1 TORNADO           20667       1511
## 2 FLOOD              6839        444
## 3 EXCESSIVE HEAT     6461       1799
## [1] "Top 3 Injuries accounted for 58.6 % of Total recorded Injuries"
"TOP 3 FATALITIES"; top3fatalities; paste("Top 3 Fatalities accounted for",
      round(sum(top3fatalities$FATALITIES/totFatalities)*100,1),
      "% of Total recorded Fatalities")
## [1] "TOP 3 FATALITIES"
## # A tibble: 3 × 3
##   EVENT          INJURIES FATALITIES
##   <fct>             <dbl>      <dbl>
## 1 EXCESSIVE HEAT     6461       1799
## 2 TORNADO           20667       1511
## 3 FLASH FLOOD        1674        887
## [1] "Top 3 Fatalities accounted for 48.1 % of Total recorded Fatalities"

\(TORNADO\) Events caused the most injuries at \(2.0667\times 10^{4}\) people, while \(EXCESSIVE HEAT\) cause the most fatalities at \(1799\).

Across the United States, which types of events have the greatest economic consequences?

require(tidyr) ## for tidyr::pivot_wider, pivot_longer
econ <- dat %>% 
    group_by(EVENT) %>% 
    summarize(PROPDMG = sum(PROPDMG)/1e6, 
              CROPDMG = sum(CROPDMG)/1e6
              ) %>% 
    arrange(desc(PROPDMG + CROPDMG))
tmp <- econ %>% 
    pivot_longer(!EVENT, names_to = "type", values_to="DAMAGE") %>% 
    pivot_wider(names_from = EVENT, values_from = DAMAGE)

plt <- barplot(as.matrix(tmp[,-c(1)][,1:10])+1, ## +1 for log scale, #[,1:10] for top 10
        horiz = T,
        #beside = T,
        #log = "x",
        axes = T,
        xlab = "DAMAGE [Million USD]",
        las = 1, ## rotates text label
        cex.names = 0.5,
        border = T,
        main = "Monetary Damage from Weather Events"
)
legend("topright",
       legend = tmp[["type"]],
       fill = gray.colors(2),
       angle = 90)

econ$TOTALDMG <- econ$PROPDMG + econ$CROPDMG
totDMG <- sum(econ$TOTALDMG)

top3CROPDMG <- econ[order(econ$CROPDMG,decreasing = T)[1:3],]
top3PROPDMG <- econ[order(econ$PROPDMG,decreasing = T)[1:3],]
totCROPDMG <- sum(econ$CROPDMG)
totPROPDMG <- sum(econ$PROPDMG)



"TOP 3 CROPDMG (in Million USD)"; top3CROPDMG; paste("Top 3 CROPDMG accounted for",
      round(sum(top3CROPDMG$CROPDMG/totCROPDMG)*100,1),
      "% of Total recorded CROPDMG")
## [1] "TOP 3 CROPDMG (in Million USD)"
## # A tibble: 3 × 4
##   EVENT               PROPDMG CROPDMG TOTALDMG
##   <fct>                 <dbl>   <dbl>    <dbl>
## 1 DROUGHT               1046.  13368.   14414.
## 2 HURRICANE (TYPHOON)  81719.   5350.   87069.
## 3 FLOOD                29130.   4981.   34110.
## [1] "Top 3 CROPDMG accounted for 68.3 % of Total recorded CROPDMG"
"TOP 3 PROPDMG (in Million USD)"; top3PROPDMG; paste("Top 3 PROPDMG accounted for",
      round(sum(top3PROPDMG$PROPDMG/totPROPDMG)*100,1),
      "% of Total recorded PROPDMG")
## [1] "TOP 3 PROPDMG (in Million USD)"
## # A tibble: 3 × 4
##   EVENT               PROPDMG  CROPDMG TOTALDMG
##   <fct>                 <dbl>    <dbl>    <dbl>
## 1 HURRICANE (TYPHOON)  81719. 5350.      87069.
## 2 STORM SURGE/TIDE     47835.    0.855   47836.
## 3 FLOOD                29130. 4981.      34110.
## [1] "Top 3 PROPDMG accounted for 63 % of Total recorded PROPDMG"

\(HURRICANE (TYPHOON)\) Events cause the most Properties Damage at \(8.1718889\times 10^{4}\) Million USD, while \(DROUGHT\) cause the most damage to Crops at \(1.3367566\times 10^{4}\) Million USD.