Synopsis

Overview

This analysis investigates the following questions:

  1. Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?
  2. Across the United States, which types of events have the greatest economic consequences?

Excessive heat appears to be the most harmful event in terms of fatalities per single event (mean) and across all recorded events (total). In terms of injuries, excessive heat again causes the most injuries per event (mean), although tsunamis are not far behind. However, tornadoes come out on top in terms of total injuries.

Hurricanes/typhoons are the most destructive in terms of property damage, per single event, but floods have caused more damage over the recorded events in this dataset. Droughts are by far the most destructive for crops, per single event and across all events.

Limitations

As described in the data processing section, accuracy compromises were made in favour of efficiency and this also extends to fully exploring each events’ remarks and any potential outliers. Also large ranges were observed in the health and economic impact values and simple statistics such as mean and total probably do not fully account for the intricacies in the data and things such as likelihood of more severe events - Further exploratory analysis is recommended.

Load packages

library(tidyverse)
library(knitr)
library(lubridate)
library(fuzzyjoin)
library(stringdist)
library(gridExtra)
## 
## Attaching package: 'gridExtra'
## The following object is masked from 'package:dplyr':
## 
##     combine

Data Processing

This next section details the steps that were taken to source the data and process it in preparation for analysis. Broadly speaking, the steps were:

Download and read data

This first code chunk downloads and reads in the raw data before displaying some summary information.

## Check if data has already been downloaded
# Create object with name of destination file
destfile <- "StormData.csv"

# Run check. If the file DOES NOT exist, then it downloads it.
if (!file.exists(destfile))
    {
      download.file("https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2", 
              destfile = destfile)
    }


## Read data into R object
StormDataRaw <- read.csv("StormData.csv")

## View quick summary of raw data
print(str(StormDataRaw))
## 'data.frame':    902297 obs. of  37 variables:
##  $ STATE__   : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ BGN_DATE  : chr  "4/18/1950 0:00:00" "4/18/1950 0:00:00" "2/20/1951 0:00:00" "6/8/1951 0:00:00" ...
##  $ BGN_TIME  : chr  "0130" "0145" "1600" "0900" ...
##  $ TIME_ZONE : chr  "CST" "CST" "CST" "CST" ...
##  $ COUNTY    : num  97 3 57 89 43 77 9 123 125 57 ...
##  $ COUNTYNAME: chr  "MOBILE" "BALDWIN" "FAYETTE" "MADISON" ...
##  $ STATE     : chr  "AL" "AL" "AL" "AL" ...
##  $ EVTYPE    : chr  "TORNADO" "TORNADO" "TORNADO" "TORNADO" ...
##  $ BGN_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ BGN_AZI   : chr  "" "" "" "" ...
##  $ BGN_LOCATI: chr  "" "" "" "" ...
##  $ END_DATE  : chr  "" "" "" "" ...
##  $ END_TIME  : chr  "" "" "" "" ...
##  $ COUNTY_END: num  0 0 0 0 0 0 0 0 0 0 ...
##  $ COUNTYENDN: logi  NA NA NA NA NA NA ...
##  $ END_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ END_AZI   : chr  "" "" "" "" ...
##  $ END_LOCATI: chr  "" "" "" "" ...
##  $ LENGTH    : num  14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
##  $ WIDTH     : num  100 150 123 100 150 177 33 33 100 100 ...
##  $ F         : int  3 2 2 2 2 2 2 1 3 3 ...
##  $ MAG       : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ FATALITIES: num  0 0 0 0 0 0 0 0 1 0 ...
##  $ INJURIES  : num  15 0 2 2 2 6 1 0 14 0 ...
##  $ PROPDMG   : num  25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
##  $ PROPDMGEXP: chr  "K" "K" "K" "K" ...
##  $ CROPDMG   : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CROPDMGEXP: chr  "" "" "" "" ...
##  $ WFO       : chr  "" "" "" "" ...
##  $ STATEOFFIC: chr  "" "" "" "" ...
##  $ ZONENAMES : chr  "" "" "" "" ...
##  $ LATITUDE  : num  3040 3042 3340 3458 3412 ...
##  $ LONGITUDE : num  8812 8755 8742 8626 8642 ...
##  $ LATITUDE_E: num  3051 0 0 0 0 ...
##  $ LONGITUDE_: num  8806 0 0 0 0 ...
##  $ REMARKS   : chr  "" "" "" "" ...
##  $ REFNUM    : num  1 2 3 4 5 6 7 8 9 10 ...
## NULL
print(head(StormDataRaw))
##   STATE__           BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE  EVTYPE
## 1       1  4/18/1950 0:00:00     0130       CST     97     MOBILE    AL TORNADO
## 2       1  4/18/1950 0:00:00     0145       CST      3    BALDWIN    AL TORNADO
## 3       1  2/20/1951 0:00:00     1600       CST     57    FAYETTE    AL TORNADO
## 4       1   6/8/1951 0:00:00     0900       CST     89    MADISON    AL TORNADO
## 5       1 11/15/1951 0:00:00     1500       CST     43    CULLMAN    AL TORNADO
## 6       1 11/15/1951 0:00:00     2000       CST     77 LAUDERDALE    AL TORNADO
##   BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END COUNTYENDN
## 1         0                                               0         NA
## 2         0                                               0         NA
## 3         0                                               0         NA
## 4         0                                               0         NA
## 5         0                                               0         NA
## 6         0                                               0         NA
##   END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES INJURIES PROPDMG
## 1         0                      14.0   100 3   0          0       15    25.0
## 2         0                       2.0   150 2   0          0        0     2.5
## 3         0                       0.1   123 2   0          0        2    25.0
## 4         0                       0.0   100 2   0          0        2     2.5
## 5         0                       0.0   150 2   0          0        2     2.5
## 6         0                       1.5   177 2   0          0        6     2.5
##   PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES LATITUDE LONGITUDE
## 1          K       0                                         3040      8812
## 2          K       0                                         3042      8755
## 3          K       0                                         3340      8742
## 4          K       0                                         3458      8626
## 5          K       0                                         3412      8642
## 6          K       0                                         3450      8748
##   LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1       3051       8806              1
## 2          0          0              2
## 3          0          0              3
## 4          0          0              4
## 5          0          0              5
## 6          0          0              6

Remove the variables that were not required to answer the key questions

In order to reduce the size of the dataset, variables that were not required to answer the key questions were removed.

## Subset raw data to only include variables needed for analysis to answer questions of interest
StormData <- StormDataRaw %>% 
    select(BGN_DATE,
           EVTYPE,
           FATALITIES,
           INJURIES,
           PROPDMG,
           PROPDMGEXP,
           CROPDMG,
           CROPDMGEXP,
           REFNUM)

Clean the date variable and remove records prior to Jan 1996

The original date variable was formatted as a date variable. Then records prior to Jan 1996 were removed. This is because this dataset only recorded tornadoes up until then which heavily skewed the data towards tornadoes. It was deemed preferable to include years in which all storm events were recorded in order to compare them.

## Convert BGN_DATE into BGN_DATE_CLEAN variable, formatted as a date.
StormData <- 
    StormData %>%
        mutate(BGN_DATE_CLEAN = 
                   mdy(
                       str_extract(
                           StormData$BGN_DATE,"[:graph:]*(?=[:space:])"
                       )
                   )
        )

## Drop old date variable
StormData <- select(StormData, -BGN_DATE)


## Subset to records from Jan 1996 when all weather events are recorded
StormData <- filter(StormData, BGN_DATE_CLEAN >= ymd("1995/01/01"))

Remove records with no injuries, property or crop damage

To further reduce the dataset, records with no recorded harmful consequences were removed.

## Subset to remove records with 0 fatalities, injuries, crop or property damage
StormData <- 
    StormData %>%
    filter((FATALITIES + INJURIES + PROPDMG + CROPDMG) != 0
)

Clean (standarise) the property and crop damage variables

The original dataset included the value and scale as separate variables, so these were combined.

## Change exp variables to lower case so that B and b are the same
StormData$PROPDMGEXP <- tolower(StormData$PROPDMGEXP)
StormData$CROPDMGEXP <- tolower(StormData$CROPDMGEXP)

## Convert DMG and EXP variables into single variable with numeric value
StormData <-
  StormData %>%
    mutate(PROPDMG_CLEAN = 
             case_when(
               PROPDMGEXP == "b" ~ PROPDMG * 10^9,
               PROPDMGEXP == "m" | PROPDMGEXP == "m" ~ PROPDMG * 10^6,
               PROPDMGEXP == "k" ~ PROPDMG * 10^3,
               PROPDMGEXP == "h" ~ PROPDMG * 10^2,
               PROPDMGEXP %in%
                 c("0","1","2","3","4","5","6","7","8","9") 
                   ~ PROPDMG * 10^1,
               PROPDMGEXP == "+" ~ PROPDMG,
               PROPDMGEXP == "-" ~ NA_real_,
               PROPDMGEXP == "?" | PROPDMGEXP == "" ~ PROPDMG * 0,
               TRUE ~ NA_real_
             ),
           CROPDMG_CLEAN  = 
            case_when(
               CROPDMGEXP == "b" ~ CROPDMG * 10^9,
               CROPDMGEXP == "m" | CROPDMGEXP == "m" ~ CROPDMG * 10^6,
               CROPDMGEXP == "k" ~ CROPDMG * 10^3,
               CROPDMGEXP == "h" ~ CROPDMG * 10^2,
               CROPDMGEXP %in%
                 c("0","1","2","3","4","5","6","7","8","9") 
                   ~ CROPDMG * 10^1,
               CROPDMGEXP == "+" ~ CROPDMG,
               CROPDMGEXP == "-" ~ NA_real_,
               CROPDMGEXP == "?" | CROPDMGEXP == "" ~ CROPDMG * 0,
               TRUE ~ NA_real_
            )
           )

## Drop original PROP and CROP dmg variables
StormData <-
  select(StormData, -PROPDMG, -PROPDMGEXP, -CROPDMG, -CROPDMGEXP)

Clean the event type variable “EVTYPE”

Even after the cleaning and dataset reduction conducted thus far, there were numerous errors and inconsistencies within this EVTYPE variable. A number of steps were taken to resolve this to a certain extent but it by no means a complete solution and there will therefore be some event types incorrectly assigned. However, exploratory data analysis suggests that the largest events in terms of fatalities, injuries and damage are accounted for so a more accurate assignment should not change the findings significantly.

The first code chunk loads the official list of storm event types, as transcribed from this url: https://d396qusza40orc.cloudfront.net/repdata%2Fpeer2_doc%2Fpd01016005curr.pdf It then stores it as a dataframe that can be joined with the main data.

## Load official list of storm event types for reference
StormEvents <-
    as.character(
        c(
            "Astronomical Low Tide",
            "Avalanche",
            "Blizzard",
            "Coastal Flood",
            "Cold/Wind Chill",
            "Debris Flow",
            "Dense Fog",
            "Dense Smoke",
            "Drought",
            "Dust Devil",
            "Dust Storm",
            "Excessive Heat",
            "Extreme Cold/Wind Chill",
            "Flash Flood",
            "Flood",
            "Frost/Freeze",
            "Funnel Cloud",
            "Freezing Fog",
            "Hail",
            "Heat",
            "Heavy Rain",
            "Heavy Snow",
            "High Surf",
            "High Wind",
            "Hurricane (Typhoon)",
            "Ice Storm",
            "Lake-Effect Snow",
            "Lakeshore Flood",
            "Lightning",
            "Marine Hail",
            "Marine High Wind",
            "Marine Strong Wind",
            "Marine Thunderstorm Wind",
            "Rip Current",
            "Seiche",
            "Sleet",
            "Storm Surge/Tide",
            "Strong Wind",
            "Thunderstorm Wind",
            "Tornado",
            "Tropical Depression",
            "Tropical Storm",
            "Tsunami",
            "Volcanic Ash",
            "Waterspout",
            "Wildfire",
            "Winter Storm",
            "Winter Weather"
        )
    )

## Create Storm Events dataframe
StormEventsDataFrame <- as.data.frame(StormEvents)
## Change variable name
StormEventsDataFrame <-
    rename(StormEventsDataFrame, EVTYPE = StormEvents)

## Print list
print(StormEventsDataFrame)
##                      EVTYPE
## 1     Astronomical Low Tide
## 2                 Avalanche
## 3                  Blizzard
## 4             Coastal Flood
## 5           Cold/Wind Chill
## 6               Debris Flow
## 7                 Dense Fog
## 8               Dense Smoke
## 9                   Drought
## 10               Dust Devil
## 11               Dust Storm
## 12           Excessive Heat
## 13  Extreme Cold/Wind Chill
## 14              Flash Flood
## 15                    Flood
## 16             Frost/Freeze
## 17             Funnel Cloud
## 18             Freezing Fog
## 19                     Hail
## 20                     Heat
## 21               Heavy Rain
## 22               Heavy Snow
## 23                High Surf
## 24                High Wind
## 25      Hurricane (Typhoon)
## 26                Ice Storm
## 27         Lake-Effect Snow
## 28          Lakeshore Flood
## 29                Lightning
## 30              Marine Hail
## 31         Marine High Wind
## 32       Marine Strong Wind
## 33 Marine Thunderstorm Wind
## 34              Rip Current
## 35                   Seiche
## 36                    Sleet
## 37         Storm Surge/Tide
## 38              Strong Wind
## 39        Thunderstorm Wind
## 40                  Tornado
## 41      Tropical Depression
## 42           Tropical Storm
## 43                  Tsunami
## 44             Volcanic Ash
## 45               Waterspout
## 46                 Wildfire
## 47             Winter Storm
## 48           Winter Weather

This chunk first does some initial cleaning of the text in the EVTYPE variable to remove leading and trailing whitespace, as well as double spaces. It also sets the text to title case to match the official list above.

It then investigates the unique values of EVTYPE, the frequencies against these unique values and the fatalities, injuries and damage associated with each. This helps to identify targets for further cleaning. E.g. which have the highest frequency or damage and therefore should be most pertinent to the analysis. This is under the assumption that it is not feasible as part of this project to clean every single value.

## Initial clean up EVTYPE
## Set EVTYPE to title case and trim whitespace
StormData$EVTYPE <- str_to_title(trimws(StormData$EVTYPE))
## Convert double space to single space
StormData$EVTYPE <- gsub("\\s+", " ", StormData$EVTYPE)



## Initial pass to identify EVTYPE value inconsistencies that might not be picked up
## by stringdist.

## Count unique values of EVTYPE
length(unique(StormData$EVTYPE))
## [1] 326
## Find unique values of EVTYPE
EVTYPEunique <- unique(StormData$EVTYPE)

## Check summary of EVTYPE frequencies
table(StormData$EVTYPE)
## 
##            Agricultural Freeze         Astronomical High Tide 
##                              3                              8 
##          Astronomical Low Tide                      Avalanche 
##                              2                            266 
##                  Beach Erosion                      Black Ice 
##                              1                              1 
##                       Blizzard                   Blowing Dust 
##                            230                              1 
##                   Blowing Snow               Breakup Flooding 
##                              2                              1 
##                     Brush Fire                Coastal Erosion 
##                              1                              1 
##                  Coastal Flood               Coastal Flooding 
##                            156                             36 
##       Coastal Flooding/Erosion                  Coastal Storm 
##                              4                              4 
##                   Coastalstorm                           Cold 
##                              1                             28 
##                  Cold And Snow        Cold And Wet Conditions 
##                              1                              1 
##               Cold Temperature                      Cold Wave 
##                              2                              1 
##                   Cold Weather                Cold/Wind Chill 
##                              4                             90 
##                      Dam Break                Damaging Freeze 
##                              2                              6 
##                      Dense Fog                    Dense Smoke 
##                             64                              1 
##                      Downburst                        Drought 
##                              1                            264 
##         Drought/Excessive Heat                       Drowning 
##                             10                              1 
##                 Dry Microburst                     Dust Devil 
##                             77                             90 
##          Dust Devil Waterspout                     Dust Storm 
##                              1                             98 
##                    Early Frost             Erosion/Cstl Flood 
##                              1                              2 
##                 Excessive Heat             Excessive Rainfall 
##                            698                              1 
##                 Excessive Snow              Excessive Wetness 
##                             25                              1 
##                  Extended Cold                   Extreme Cold 
##                              1                            179 
##        Extreme Cold/Wind Chill                   Extreme Heat 
##                            111                             14 
##             Extreme Wind Chill              Extreme Windchill 
##                              1                             19 
##               Falling Snow/Ice                    Flash Flood 
##                              2                          19851 
##       Flash Flood - Heavy Rain              Flash Flood Winds 
##                              2                              1 
##            Flash Flood/ Street              Flash Flood/Flood 
##                              1                              3 
##                 Flash Flooding                   Flash Floods 
##                            100                              8 
##                          Flood             Flood & Heavy Rain 
##                           9712                              2 
##                    Flood/Flash              Flood/Flash Flood 
##                              1                             88 
##              Flood/Flash/Flood               Flood/Rain/Winds 
##                              1                              5 
##                       Flooding            Flooding/Heavy Rain 
##                             23                              1 
##                         Floods                            Fog 
##                              1                            104 
##                         Freeze               Freezing Drizzle 
##                             16                              7 
##                   Freezing Fog                  Freezing Rain 
##                              7                             18 
##             Freezing Rain/Snow                 Freezing Spray 
##                              3                              1 
##                          Frost                   Frost/Freeze 
##                              2                            117 
##                   Funnel Cloud                          Glaze 
##                              9                             20 
##                      Glaze Ice                  Gradient Wind 
##                              2                              6 
##                    Grass Fires                       Gustnado 
##                              1                              2 
##                     Gusty Wind                Gusty Wind/Hail 
##                             13                              1 
##            Gusty Wind/Hvy Rain                Gusty Wind/Rain 
##                              1                              1 
##                    Gusty Winds                           Hail 
##                             30                          23860 
##                      Hail 0.75                       Hail 075 
##                              1                              1 
##                       Hail 100                       Hail 125 
##                              1                              1 
##                       Hail 150                       Hail 175 
##                              1                              2 
##                       Hail 200                       Hail 275 
##                              1                              3 
##                       Hail 450                        Hail 75 
##                              1                              1 
##                    Hail Damage                      Hailstorm 
##                              1                              3 
##                    Hard Freeze                 Hazardous Surf 
##                              2                              1 
##                           Heat                      Heat Wave 
##                            203                             33 
##              Heat Wave Drought                      Heavy Mix 
##                              1                              2 
##                     Heavy Rain           Heavy Rain And Flood 
##                           1080                              1 
##           Heavy Rain/High Surf      Heavy Rain/Severe Weather 
##                              1                              1 
##                    Heavy Rains                     Heavy Seas 
##                              4                              2 
##                     Heavy Snow             Heavy Snow-Squalls 
##                           1134                             15 
##      Heavy Snow And High Winds    Heavy Snow And Strong Winds 
##                              1                              1 
##              Heavy Snow Shower             Heavy Snow Squalls 
##                              1                             12 
##  Heavy Snow/High Winds & Flood                 Heavy Snow/Ice 
##                              1                              1 
##                     Heavy Surf            Heavy Surf And Wind 
##                             30                              1 
##    Heavy Surf Coastal Flooding           Heavy Surf/High Surf 
##                              1                             50 
##                   Heavy Swells                      High Seas 
##                              1                              6 
##                      High Surf             High Surf Advisory 
##                            132                              1 
##                    High Swells                     High Water 
##                              2                              2 
##                     High Waves                      High Wind 
##                              1                           5416 
##                High Wind (G40)               High Wind Damage 
##                              2                              2 
##                     High Winds         High Winds Heavy Rains 
##                            312                              1 
##                      Hurricane     Hurricane-Generated Swells 
##                            126                              3 
##              Hurricane Edouard                 Hurricane Erin 
##                              1                              7 
##                Hurricane Felix                 Hurricane Opal 
##                              1                              8 
##      Hurricane Opal/High Winds              Hurricane/Typhoon 
##                              1                             72 
##                       Hvy Rain          Hyperthermia/Exposure 
##                              1                              1 
##           Hypothermia/Exposure                            Ice 
##                              6                              2 
##           Ice Jam Flood (Minor               Ice Jam Flooding 
##                              1                              2 
##                    Ice On Road                      Ice Roads 
##                              1                              1 
##                      Ice Storm               Ice/Strong Winds 
##                            653                              1 
##                      Icy Roads               Lake-Effect Snow 
##                             22                            194 
##               Lake Effect Snow                     Lake Flood 
##                              4                              1 
##                Lakeshore Flood                      Landslide 
##                              5                            193 
##                     Landslides                      Landslump 
##                              1                              1 
##                      Landspout               Late Season Snow 
##                              2                              1 
##            Light Freezing Rain                     Light Snow 
##                             22                            141 
##                 Light Snowfall                      Lightning 
##                              1                          12026 
##       Lightning And Heavy Rain Lightning And Thunderstorm Win 
##                              1                              1 
##                 Lightning Fire               Lightning Injury 
##                              1                              1 
##   Lightning Thunderstorm Winds              Lightning Wauseon 
##                              1                              1 
##                     Lightning.                      Ligntning 
##                              1                              1 
##                Marine Accident                    Marine Hail 
##                              1                              2 
##               Marine High Wind                  Marine Mishap 
##                             19                              2 
##             Marine Strong Wind       Marine Thunderstorm Wind 
##                             46                             33 
##               Marine Tstm Wind                     Microburst 
##                            109                              1 
##                   Mixed Precip            Mixed Precipitation 
##                              6                             18 
##                      Mud Slide      Mud Slides Urban Flooding 
##                              2                              1 
##                       Mudslide                      Mudslides 
##                              5                              1 
##         Non-Severe Wind Damage                  Non-Tstm Wind 
##                              1                              1 
##                  Non Tstm Wind                          Other 
##                              1                             34 
##                           Rain                      Rain/Snow 
##                              3                              2 
##           Rapidly Rising Water                    Record Cold 
##                              1                              1 
##                    Record Heat                    Rip Current 
##                              1                            389 
##                   Rip Currents        Rip Currents/Heavy Surf 
##                            241                              2 
##         River And Stream Flood                    River Flood 
##                              2                             95 
##                 River Flooding                     Rock Slide 
##                             14                              1 
##                     Rogue Wave                     Rough Seas 
##                              1                              3 
##                     Rough Surf                    Rural Flood 
##                              2                              1 
##                         Seiche            Severe Thunderstorm 
##                              9                              5 
##      Severe Thunderstorm Winds           Severe Thunderstorms 
##                              3                              2 
##                          Sleet                     Small Hail 
##                              1                             11 
##                           Snow                   Snow And Ice 
##                             31                              3 
##             Snow Freezing Rain                    Snow Squall 
##                             10                              3 
##                   Snow Squalls                       Snow/Ice 
##                             10                              1 
##       Snow/Sleet/Freezing Rain              Storm Force Winds 
##                              2                              1 
##                    Storm Surge               Storm Surge/Tide 
##                            169                             47 
##                    Strong Wind                   Strong Winds 
##                           3370                             48 
##            Thundeerstorm Winds            Thunderestorm Winds 
##                              1                              1 
##                    Thundersnow                   Thunderstorm 
##                              1                             14 
##         Thunderstorm Damage To              Thunderstorm Wind 
##                              1                          43170 
##        Thunderstorm Wind (G40)       Thunderstorm Wind 60 Mph 
##                              1                              2 
##       Thunderstorm Wind 65 Mph        Thunderstorm Wind 65mph 
##                              1                              1 
##       Thunderstorm Wind 98 Mph          Thunderstorm Wind G52 
##                              1                              1 
##          Thunderstorm Wind G55          Thunderstorm Wind G60 
##                              1                              1 
##        Thunderstorm Wind Trees             Thunderstorm Wind. 
##                              1                              1 
##        Thunderstorm Wind/ Tree       Thunderstorm Wind/ Trees 
##                              1                              4 
##       Thunderstorm Wind/Awning    Thunderstorm Wind/Lightning 
##                              1                              1 
##             Thunderstorm Winds      Thunderstorm Winds 63 Mph 
##                           5098                              1 
##         Thunderstorm Winds And         Thunderstorm Winds G60 
##                              1                              1 
##        Thunderstorm Winds Hail   Thunderstorm Winds Lightning 
##                             40                              6 
##      Thunderstorm Winds/ Flood        Thunderstorm Winds/Hail 
##                              2                              3 
##           Thunderstorm Winds53         Thunderstorm Windshail 
##                              1                              1 
##            Thunderstorm Windss              Thunderstorm Wins 
##                              6                              1 
##             Thunderstorms Wind            Thunderstorms Winds 
##                              1                              4 
##                  Thunderstormw              Thundertorm Winds 
##                              1                              3 
##              Thunerstorm Winds                 Tidal Flooding 
##                              1                              4 
##                        Tornado                     Tornado F0 
##                          13000                              5 
##                     Tornado F1                     Tornado F2 
##                              4                              2 
##                     Tornado F3            Torrential Rainfall 
##                              2                              1 
##            Tropical Depression                 Tropical Storm 
##                             35                            412 
##            Tropical Storm Dean           Tropical Storm Jerry 
##                              1                              2 
##                      Tstm Wind                 Tstm Wind (41) 
##                          61903                              1 
##                Tstm Wind (G35)                Tstm Wind (G40) 
##                              1                              9 
##                Tstm Wind (G45)                   Tstm Wind 40 
##                             38                              1 
##                   Tstm Wind 45                   Tstm Wind 55 
##                              1                              2 
##                  Tstm Wind 65)        Tstm Wind And Lightning 
##                              1                              1 
##               Tstm Wind Damage                  Tstm Wind G45 
##                              1                              1 
##                  Tstm Wind G58                 Tstm Wind/Hail 
##                              1                            441 
##                     Tstm Winds                        Tsunami 
##                              5                             14 
##               Tunderstorm Wind                        Typhoon 
##                              1                              9 
##              Unseasonable Cold              Unseasonably Cold 
##                              1                              4 
##              Unseasonably Warm      Unseasonably Warm And Dry 
##                              7                              1 
##                Unseasonal Rain Urban And Small Stream Floodin 
##                              2                              1 
##                    Urban Flood                 Urban Flooding 
##                             62                             21 
##       Urban/Small Stream Flood           Urban/Sml Stream Fld 
##                              6                            702 
##                   Volcanic Ash                   Warm Weather 
##                              2                              1 
##                     Waterspout             Waterspout-Tornado 
##                             30                              1 
##             Waterspout Tornado            Waterspout/ Tornado 
##                              1                              1 
##             Waterspout/Tornado                 Wet Microburst 
##                              2                              3 
##                      Whirlwind                     Wild Fires 
##                              3                              2 
##               Wild/Forest Fire              Wild/Forest Fires 
##                            383                              1 
##                       Wildfire                           Wind 
##                            853                             73 
##                  Wind And Wave                    Wind Damage 
##                              1                              8 
##                     Wind Storm                          Winds 
##                              1                              6 
##                   Winter Storm        Winter Storm High Winds 
##                           1479                              1 
##                 Winter Weather             Winter Weather Mix 
##                            407                              2 
##             Winter Weather/Mix                     Wintry Mix 
##                            139                              4
## Looking for events that may not have high frequency but which did cause big damage

## Create summary dataframe
EvTypeSummary <-
    StormData %>%
        group_by(EVTYPE) %>%
            summarise(Freq = n(),
                      FATALITIES_TOTAL = sum(FATALITIES),
                      INJURIES_TOTAL = sum(INJURIES),
                      PROPDMG_TOTAL = sum(PROPDMG_CLEAN),
                      CROPDMG_TOTAL = sum(CROPDMG_CLEAN)
                      )

Following the above (and partially informed by the stringdist exercise below) a number of replacements were carried out according to observed rules. e.g. TSTM shorthand for Thunderstorm replaced with the full name to standardise the values in this variable.

## Make a few known changes
# TSTM wind
StormData$EVTYPE[grepl("TSTM", StormData$EVTYPE, ignore.case = TRUE)] <- "Thunderstorm Wind"
# Landslide
StormData$EVTYPE[grepl("landslide", StormData$EVTYPE, ignore.case = TRUE)] <- "Debris Flow"
# Glaze
StormData$EVTYPE[grepl("glaze", StormData$EVTYPE, ignore.case = TRUE)] <- "Winter Storm"
# Thunderstorm wind - use "storm wind"
StormData$EVTYPE[grepl("storm wind", StormData$EVTYPE, ignore.case = TRUE)] <- "Thunderstorm Wind"
# Hurricane/typhoon
StormData$EVTYPE[grepl("hurricane", StormData$EVTYPE, ignore.case = TRUE)] <- "Hurricane (Typhoon)"
StormData$EVTYPE[grepl("typhoon", StormData$EVTYPE, ignore.case = TRUE)] <- "Hurricane (Typhoon)"
# Hail
StormData$EVTYPE[grepl("hail", StormData$EVTYPE, ignore.case = TRUE)] <- "Hail"
# Tropical storm
StormData$EVTYPE[grepl("tropical storm", StormData$EVTYPE, ignore.case = TRUE)] <- "Tropical Storm"
# Heat
StormData$EVTYPE[grepl("heat", StormData$EVTYPE, ignore.case = TRUE)] <- "Excessive Heat"
# Fire -> wildfire
StormData$EVTYPE[grepl("fire", StormData$EVTYPE, ignore.case = TRUE)] <- "Wildfire"
# Excessive snow -> heavy snow
StormData$EVTYPE[grepl("Excessive snow", StormData$EVTYPE, ignore.case = TRUE)] <- "Heavy snow"
# Light snow -> winter storm
StormData$EVTYPE[grepl("Light snow", StormData$EVTYPE, ignore.case = TRUE)] <- "Winter storm"
# Surf -> high surf
StormData$EVTYPE[grepl("surf", StormData$EVTYPE, ignore.case = TRUE)] <- "High surf"
## heavy seas -> Marine Strong Wind 
StormData$EVTYPE[grepl("heavy sea", StormData$EVTYPE, ignore.case = TRUE)] <- "Marine Strong Wind"
## high water -> marine high wind.
StormData$EVTYPE[grepl("high water", StormData$EVTYPE, ignore.case = TRUE)] <- "Marine High Wind"


## Find unique values of EVTYPE
EVTYPEunique <- unique(StormData$EVTYPE)

## Check summary of EVTYPE frequencies
table(StormData$EVTYPE)
## 
##            Agricultural Freeze         Astronomical High Tide 
##                              3                              8 
##          Astronomical Low Tide                      Avalanche 
##                              2                            266 
##                  Beach Erosion                      Black Ice 
##                              1                              1 
##                       Blizzard                   Blowing Dust 
##                            230                              1 
##                   Blowing Snow               Breakup Flooding 
##                              2                              1 
##                Coastal Erosion                  Coastal Flood 
##                              1                            156 
##               Coastal Flooding       Coastal Flooding/Erosion 
##                             36                              4 
##                  Coastal Storm                   Coastalstorm 
##                              4                              1 
##                           Cold                  Cold And Snow 
##                             28                              1 
##        Cold And Wet Conditions               Cold Temperature 
##                              1                              2 
##                      Cold Wave                   Cold Weather 
##                              1                              4 
##                Cold/Wind Chill                      Dam Break 
##                             90                              2 
##                Damaging Freeze                    Debris Flow 
##                              6                            194 
##                      Dense Fog                    Dense Smoke 
##                             64                              1 
##                      Downburst                        Drought 
##                              1                            264 
##                       Drowning                 Dry Microburst 
##                              1                             77 
##                     Dust Devil          Dust Devil Waterspout 
##                             90                              1 
##                     Dust Storm                    Early Frost 
##                             98                              1 
##             Erosion/Cstl Flood                 Excessive Heat 
##                              2                            960 
##             Excessive Rainfall              Excessive Wetness 
##                              1                              1 
##                  Extended Cold                   Extreme Cold 
##                              1                            179 
##        Extreme Cold/Wind Chill             Extreme Wind Chill 
##                            111                              1 
##              Extreme Windchill               Falling Snow/Ice 
##                             19                              2 
##                    Flash Flood       Flash Flood - Heavy Rain 
##                          19851                              2 
##              Flash Flood Winds            Flash Flood/ Street 
##                              1                              1 
##              Flash Flood/Flood                 Flash Flooding 
##                              3                            100 
##                   Flash Floods                          Flood 
##                              8                           9712 
##             Flood & Heavy Rain                    Flood/Flash 
##                              2                              1 
##              Flood/Flash Flood              Flood/Flash/Flood 
##                             88                              1 
##               Flood/Rain/Winds                       Flooding 
##                              5                             23 
##            Flooding/Heavy Rain                         Floods 
##                              1                              1 
##                            Fog                         Freeze 
##                            104                             16 
##               Freezing Drizzle                   Freezing Fog 
##                              7                              7 
##                  Freezing Rain             Freezing Rain/Snow 
##                             18                              3 
##                 Freezing Spray                          Frost 
##                              1                              2 
##                   Frost/Freeze                   Funnel Cloud 
##                            117                              9 
##                  Gradient Wind                       Gustnado 
##                              6                              2 
##                     Gusty Wind            Gusty Wind/Hvy Rain 
##                             13                              1 
##                Gusty Wind/Rain                    Gusty Winds 
##                              1                             30 
##                           Hail                    Hard Freeze 
##                          23891                              2 
##                      Heavy Mix                     Heavy Rain 
##                              2                           1080 
##           Heavy Rain And Flood      Heavy Rain/Severe Weather 
##                              1                              1 
##                    Heavy Rains                     Heavy snow 
##                              4                             25 
##                     Heavy Snow             Heavy Snow-Squalls 
##                           1134                             15 
##      Heavy Snow And High Winds    Heavy Snow And Strong Winds 
##                              1                              1 
##              Heavy Snow Shower             Heavy Snow Squalls 
##                              1                             12 
##  Heavy Snow/High Winds & Flood                 Heavy Snow/Ice 
##                              1                              1 
##                   Heavy Swells                      High Seas 
##                              1                              6 
##                      High surf                    High Swells 
##                            221                              2 
##                     High Waves                      High Wind 
##                              1                           5416 
##                High Wind (G40)               High Wind Damage 
##                              2                              2 
##                     High Winds         High Winds Heavy Rains 
##                            312                              1 
##            Hurricane (Typhoon)                       Hvy Rain 
##                            228                              1 
##          Hyperthermia/Exposure           Hypothermia/Exposure 
##                              1                              6 
##                            Ice           Ice Jam Flood (Minor 
##                              2                              1 
##               Ice Jam Flooding                    Ice On Road 
##                              2                              1 
##                      Ice Roads                      Ice Storm 
##                              1                            653 
##               Ice/Strong Winds                      Icy Roads 
##                              1                             22 
##               Lake-Effect Snow               Lake Effect Snow 
##                            194                              4 
##                     Lake Flood                Lakeshore Flood 
##                              1                              5 
##                      Landslump                      Landspout 
##                              1                              2 
##               Late Season Snow            Light Freezing Rain 
##                              1                             22 
##                      Lightning       Lightning And Heavy Rain 
##                          12026                              1 
## Lightning And Thunderstorm Win               Lightning Injury 
##                              1                              1 
##              Lightning Wauseon                     Lightning. 
##                              1                              1 
##                      Ligntning                Marine Accident 
##                              1                              1 
##               Marine High Wind                  Marine Mishap 
##                             21                              2 
##             Marine Strong Wind                     Microburst 
##                             48                              1 
##                   Mixed Precip            Mixed Precipitation 
##                              6                             18 
##                      Mud Slide      Mud Slides Urban Flooding 
##                              2                              1 
##                       Mudslide                      Mudslides 
##                              5                              1 
##         Non-Severe Wind Damage                          Other 
##                              1                             34 
##                           Rain                      Rain/Snow 
##                              3                              2 
##           Rapidly Rising Water                    Record Cold 
##                              1                              1 
##                    Rip Current                   Rip Currents 
##                            389                            241 
##         River And Stream Flood                    River Flood 
##                              2                             95 
##                 River Flooding                     Rock Slide 
##                             14                              1 
##                     Rogue Wave                     Rough Seas 
##                              1                              3 
##                    Rural Flood                         Seiche 
##                              1                              9 
##            Severe Thunderstorm           Severe Thunderstorms 
##                              5                              2 
##                          Sleet                           Snow 
##                              1                             31 
##                   Snow And Ice             Snow Freezing Rain 
##                              3                             10 
##                    Snow Squall                   Snow Squalls 
##                              3                             10 
##                       Snow/Ice       Snow/Sleet/Freezing Rain 
##                              1                              2 
##              Storm Force Winds                    Storm Surge 
##                              1                            169 
##               Storm Surge/Tide                    Strong Wind 
##                             47                           3370 
##                   Strong Winds                    Thundersnow 
##                             48                              1 
##                   Thunderstorm         Thunderstorm Damage To 
##                             14                              1 
##              Thunderstorm Wind              Thunderstorm Wins 
##                         110907                              1 
##             Thunderstorms Wind            Thunderstorms Winds 
##                              1                              4 
##                  Thunderstormw              Thundertorm Winds 
##                              1                              3 
##                 Tidal Flooding                        Tornado 
##                              4                          13000 
##                     Tornado F0                     Tornado F1 
##                              5                              4 
##                     Tornado F2                     Tornado F3 
##                              2                              2 
##            Torrential Rainfall            Tropical Depression 
##                              1                             35 
##                 Tropical Storm                        Tsunami 
##                            415                             14 
##              Unseasonable Cold              Unseasonably Cold 
##                              1                              4 
##              Unseasonably Warm      Unseasonably Warm And Dry 
##                              7                              1 
##                Unseasonal Rain Urban And Small Stream Floodin 
##                              2                              1 
##                    Urban Flood                 Urban Flooding 
##                             62                             21 
##       Urban/Small Stream Flood           Urban/Sml Stream Fld 
##                              6                            702 
##                   Volcanic Ash                   Warm Weather 
##                              2                              1 
##                     Waterspout             Waterspout-Tornado 
##                             30                              1 
##             Waterspout Tornado            Waterspout/ Tornado 
##                              1                              1 
##             Waterspout/Tornado                 Wet Microburst 
##                              2                              3 
##                      Whirlwind                       Wildfire 
##                              3                           1242 
##                           Wind                  Wind And Wave 
##                             73                              1 
##                    Wind Damage                     Wind Storm 
##                              8                              1 
##                          Winds                   Winter storm 
##                              6                            142 
##                   Winter Storm        Winter Storm High Winds 
##                           1501                              1 
##                 Winter Weather             Winter Weather Mix 
##                            407                              2 
##             Winter Weather/Mix                     Wintry Mix 
##                            139                              4
## Create summary dataframe
EvTypeSummary <-
    StormData %>%
        group_by(EVTYPE) %>%
            summarise(Freq = n())

This next chunk uses the stringdist_join function, as part of the stringdist package to determine the string distance between each original EVTYPE value and those in the official list. Those with distances less than a certain value are assigned a value from that official list. The Jaro Winkler distance method was used because the differences are due to typos rather than spoken words being mis-transcribed: https://cran.r-project.org/web/packages/stringdist/stringdist.pdf

This exercise assigned new values from the official list to 210332 records from the original 211775 (99%) although as noted earlier, these 99% will not all have been assigned correctly.

## Create new dataframe with new variables for EVTYPE, informed by stringdist matching.
StormDataDist <- stringdist_join(StormData, StormEventsDataFrame,
                by = "EVTYPE",
                mode = "left",
                ignore_case = TRUE,
                method = "jw",
                max_dist = 0.2444444, # Informed by manually investigating first pass to find cut-off where matching appears                                         inaccurate
                distance_col = "dist") %>%
                    group_by(EVTYPE.x) %>%
                        slice_min(order_by = dist, n = 1)

## Investigating results of stringdist_join
print(paste0("Number of records in new StormDataDist dataframe: ", nrow(StormDataDist)))
## [1] "Number of records in new StormDataDist dataframe: 210332"
# Number of unique evtype values
length(unique(StormDataDist$EVTYPE.x))
## [1] 137
length(unique(StormDataDist$EVTYPE.y))
## [1] 46
# Number of unique events which could not find a perfect match
length(unique(StormDataDist$EVTYPE.x[StormDataDist$dist!=0]))
## [1] 90
# Frequencies of EVTYPE in original and new variable
table(StormDataDist$EVTYPE.x)
## 
##         Astronomical High Tide          Astronomical Low Tide 
##                              8                              2 
##                      Avalanche                       Blizzard 
##                            266                            230 
##                Coastal Erosion                  Coastal Flood 
##                              1                            156 
##               Coastal Flooding       Coastal Flooding/Erosion 
##                             36                              4 
##                  Coastal Storm                   Coastalstorm 
##                              4                              1 
##                Cold/Wind Chill                    Debris Flow 
##                             90                            194 
##                      Dense Fog                    Dense Smoke 
##                             64                              1 
##                        Drought                     Dust Devil 
##                            264                             90 
##          Dust Devil Waterspout                     Dust Storm 
##                              1                             98 
##                 Excessive Heat             Excessive Rainfall 
##                            960                              1 
##              Excessive Wetness                   Extreme Cold 
##                              1                            179 
##        Extreme Cold/Wind Chill             Extreme Wind Chill 
##                            111                              1 
##              Extreme Windchill                    Flash Flood 
##                             19                          19851 
##       Flash Flood - Heavy Rain              Flash Flood Winds 
##                              2                              1 
##            Flash Flood/ Street              Flash Flood/Flood 
##                              1                              3 
##                 Flash Flooding                   Flash Floods 
##                            100                              8 
##                          Flood             Flood & Heavy Rain 
##                           9712                              2 
##                    Flood/Flash              Flood/Flash Flood 
##                              1                             88 
##              Flood/Flash/Flood               Flood/Rain/Winds 
##                              1                              5 
##                       Flooding                         Floods 
##                             23                              1 
##               Freezing Drizzle                   Freezing Fog 
##                              7                              7 
##                  Freezing Rain             Freezing Rain/Snow 
##                             18                              3 
##                 Freezing Spray                          Frost 
##                              1                              2 
##                   Frost/Freeze                   Funnel Cloud 
##                            117                              9 
##                       Gustnado                           Hail 
##                              2                          23891 
##                      Heavy Mix                     Heavy Rain 
##                              2                           1080 
##           Heavy Rain And Flood      Heavy Rain/Severe Weather 
##                              1                              1 
##                    Heavy Rains                     Heavy snow 
##                              4                             25 
##                     Heavy Snow             Heavy Snow-Squalls 
##                           1134                             15 
##      Heavy Snow And High Winds    Heavy Snow And Strong Winds 
##                              1                              1 
##              Heavy Snow Shower             Heavy Snow Squalls 
##                              1                             12 
##  Heavy Snow/High Winds & Flood                 Heavy Snow/Ice 
##                              1                              1 
##                   Heavy Swells                      High Seas 
##                              1                              6 
##                      High surf                      High Wind 
##                            221                           5416 
##                High Wind (G40)               High Wind Damage 
##                              2                              2 
##                     High Winds         High Winds Heavy Rains 
##                            312                              1 
##            Hurricane (Typhoon)                       Hvy Rain 
##                            228                              1 
##                            Ice                      Ice Storm 
##                              2                            653 
##               Ice/Strong Winds               Lake-Effect Snow 
##                              1                            194 
##               Lake Effect Snow                     Lake Flood 
##                              4                              1 
##                Lakeshore Flood                      Lightning 
##                              5                          12026 
##       Lightning And Heavy Rain Lightning And Thunderstorm Win 
##                              1                              1 
##               Lightning Injury              Lightning Wauseon 
##                              1                              1 
##                     Lightning.                      Ligntning 
##                              1                              1 
##                Marine Accident               Marine High Wind 
##                              1                             21 
##                  Marine Mishap             Marine Strong Wind 
##                              2                             48 
##                    Rip Current                   Rip Currents 
##                            389                            241 
##                 River Flooding                    Rural Flood 
##                             14                              1 
##                         Seiche                          Sleet 
##                              9                              1 
##                    Storm Surge               Storm Surge/Tide 
##                            169                             47 
##                    Strong Wind                   Strong Winds 
##                           3370                             48 
##                    Thundersnow                   Thunderstorm 
##                              1                             14 
##         Thunderstorm Damage To              Thunderstorm Wind 
##                              1                         110907 
##              Thunderstorm Wins             Thunderstorms Wind 
##                              1                              1 
##            Thunderstorms Winds                  Thunderstormw 
##                              4                              1 
##              Thundertorm Winds                        Tornado 
##                              3                          13000 
##                     Tornado F0                     Tornado F1 
##                              5                              4 
##                     Tornado F2                     Tornado F3 
##                              2                              2 
##            Tropical Depression                 Tropical Storm 
##                             35                            415 
##                        Tsunami                    Urban Flood 
##                             14                             62 
##                 Urban Flooding                   Volcanic Ash 
##                             21                              2 
##                     Waterspout             Waterspout-Tornado 
##                             30                              1 
##             Waterspout Tornado            Waterspout/ Tornado 
##                              1                              1 
##             Waterspout/Tornado                      Whirlwind 
##                              2                              3 
##                       Wildfire                     Wind Storm 
##                           1242                              1 
##                   Winter storm                   Winter Storm 
##                            142                           1501 
##        Winter Storm High Winds                 Winter Weather 
##                              1                            407 
##             Winter Weather Mix             Winter Weather/Mix 
##                              2                            139 
##                     Wintry Mix 
##                              4
table(StormDataDist$EVTYPE.y)
## 
##   Astronomical Low Tide               Avalanche                Blizzard 
##                      10                     266                     230 
##           Coastal Flood         Cold/Wind Chill             Debris Flow 
##                     198                      90                     194 
##               Dense Fog             Dense Smoke                 Drought 
##                      64                       1                     264 
##              Dust Devil              Dust Storm          Excessive Heat 
##                      91                     102                     962 
## Extreme Cold/Wind Chill             Flash Flood                   Flood 
##                     310                   20029                    9868 
##            Freezing Fog            Frost/Freeze            Funnel Cloud 
##                      36                     119                       9 
##                    Hail              Heavy Rain              Heavy Snow 
##                   23891                    1089                    1192 
##               High Surf               High Wind     Hurricane (Typhoon) 
##                     227                    5736                     228 
##               Ice Storm        Lake-Effect Snow         Lakeshore Flood 
##                     656                     198                       5 
##               Lightning             Marine Hail        Marine High Wind 
##                   12032                       3                      21 
##      Marine Strong Wind             Rip Current                  Seiche 
##                      48                     630                       9 
##                   Sleet        Storm Surge/Tide             Strong Wind 
##                       1                     216                    3419 
##       Thunderstorm Wind                 Tornado     Tropical Depression 
##                  110933                   13015                      35 
##          Tropical Storm                 Tsunami            Volcanic Ash 
##                     416                      14                       2 
##              Waterspout                Wildfire            Winter Storm 
##                      35                    1242                    1648 
##          Winter Weather 
##                     548
# Find duplicates
length(StormDataDist$REFNUM[length(StormDataDist$REFNUM)>1]) == nrow(StormDataDist) # TRUE if no duplicates
## [1] TRUE
print(StormDataDist[duplicated(StormDataDist$REFNUM), ]) # If there are any duplicates
## # A tibble: 0 x 9
## # Groups:   EVTYPE.x [0]
## # ... with 9 variables: EVTYPE.x <chr>, FATALITIES <dbl>, INJURIES <dbl>,
## #   REFNUM <dbl>, BGN_DATE_CLEAN <date>, PROPDMG_CLEAN <dbl>,
## #   CROPDMG_CLEAN <dbl>, EVTYPE.y <chr>, dist <dbl>

The StormDataDist dataframe is then cleaned up further to remove the original EVTYPE variable and sum the fatality, injury and damage values per new EVTYPE.

The new dataframe is then used for analysis and results.

## Rename event type
StormDataDist <- rename(StormDataDist, EventType = EVTYPE.y)

## Ungroup
StormDataDist <- ungroup(StormDataDist)

## Remove unnecessary variables and summarise frequencies
StormDataCleanSummary <-
  StormDataDist %>%
    select(-EVTYPE.x, -REFNUM, -BGN_DATE_CLEAN, -dist) %>%
      group_by(EventType) %>%
        summarise(Freq = n(),
                  FATALITIES_TOTAL = sum(FATALITIES),
                  INJURIES_TOTAL = sum(INJURIES),
                  PROPDMG_TOTAL = sum(PROPDMG_CLEAN)/10^9,
                  CROPDMG_TOTAL = sum(CROPDMG_CLEAN)/10^9,
                  FATALITIES_MEAN = mean(FATALITIES),
                  INJURIES_MEAN = mean(INJURIES),
                  PROPDMG_MEAN = mean(PROPDMG_CLEAN)/10^6,
                  CROPDMG_MEAN = mean(CROPDMG_CLEAN)/10^6
        )

## Remove unnecessary variables but no summary
StormDataClean <-
  StormDataDist %>%
    select(-EVTYPE.x, -REFNUM, -BGN_DATE_CLEAN, -dist)




## Print table (original sort, alphabetical)
print(StormDataClean)
## # A tibble: 210,332 x 5
##    FATALITIES INJURIES PROPDMG_CLEAN CROPDMG_CLEAN EventType            
##         <dbl>    <dbl>         <dbl>         <dbl> <chr>                
##  1          0        0       1000000             0 Astronomical Low Tide
##  2          0        0       5000000             0 Astronomical Low Tide
##  3          0        0         90000             0 Astronomical Low Tide
##  4          0        0       2500000             0 Astronomical Low Tide
##  5          0        0         20000             0 Astronomical Low Tide
##  6          0        0        800000             0 Astronomical Low Tide
##  7          0        0          5000             0 Astronomical Low Tide
##  8          0        0         10000             0 Astronomical Low Tide
##  9          0        0        120000             0 Astronomical Low Tide
## 10          0        0        200000             0 Astronomical Low Tide
## # ... with 210,322 more rows

Results

The following section presents the results for the two key questions.

The mean and sum (total) statistics were chosen to give an indication of impact per event and across the dataset of recorded events. These results are displayed below for fatalities, injuries, property damage and crop damage.

Most harmful weather events in the United States with respect to population health

## Fatalities
  ## Looking at mean
  ## Create initial ggplot object
  PopHealthFatalityMean <- 
    ggplot(
      slice_max(StormDataCleanSummary, order_by = FATALITIES_MEAN, n=10),
      mapping = aes(reorder(EventType, FATALITIES_MEAN), FATALITIES_MEAN),
      ) 
  
  ## Create plot
  FatalityMeanPlot <- 
  PopHealthFatalityMean +
  geom_bar(stat = "identity", fill = "indianred3") +
  labs(y = "Mean Fatalities", x = "Event") +
  geom_text(aes(label=round(FATALITIES_MEAN, 2)),
            position=position_dodge(width=0.9),
            hjust=-0.1
            ) +
  coord_flip() +
  ylim(0, 4)
  
  
  ## Looking at totals
  ## Create initial ggplot object
  PopHealthFatalityTotal <- 
    ggplot(
      slice_max(StormDataCleanSummary, order_by = FATALITIES_TOTAL, n=10),
      mapping = aes(reorder(EventType, FATALITIES_TOTAL), FATALITIES_TOTAL)
      )
  
  ## Create plot
  FatalityTotalPlot <- 
  PopHealthFatalityTotal +
  geom_bar(stat = "identity", fill = "indianred3") +
  labs(y = "Total Fatalities", x = "Event") +
  geom_text(aes(label=round(FATALITIES_TOTAL, 2)),
            position=position_dodge(width=0.9)
            ) +
  coord_flip() +
  ylim(0, 4000)

  
  
  
## Injuries
  ## Looking at mean
  ## Create initial ggplot object
  PopHealthInjuryMean <- 
    ggplot(
      slice_max(StormDataCleanSummary, order_by = INJURIES_MEAN, n=10),
      mapping = aes(reorder(EventType, INJURIES_MEAN), INJURIES_MEAN),
      )
  
  ## Create plot
  InjuryMeanPlot <- 
  PopHealthInjuryMean +
  geom_bar(stat = "identity", fill = "orangered1") +
  labs(y = "Mean Injuries", x = "Event") +
  geom_text(aes(label=round(INJURIES_MEAN, 2)),
            position=position_dodge(width=0.9)
            ) +
  coord_flip()+
  ylim(0, 10)
  
  
  ## Looking at totals
  ## Create initial ggplot object
  PopHealthInjuryTotal <- 
    ggplot(
      slice_max(StormDataCleanSummary, order_by = INJURIES_TOTAL, n=10),
      mapping = aes(reorder(EventType, INJURIES_TOTAL), INJURIES_TOTAL)
      )
  
  ## Create plot
  InjuryTotalPlot <- 
  PopHealthInjuryTotal +
  geom_bar(stat = "identity", fill = "orangered1") +
  labs(y = "Total Injuries", x = "Event") +
  geom_text(aes(label=round(INJURIES_TOTAL, 2)),
            position=position_dodge(width=0.9)
            ) +
  coord_flip()+
  ylim(0, 25000)

## Arrange above plots into 2x2 grid
grid.arrange(FatalityMeanPlot, FatalityTotalPlot,InjuryMeanPlot, InjuryTotalPlot, 
             ncol=2,
             nrow=2,
             top = "Top 10 storm events"
)

Excessive heat appears to be the most harmful event in terms of fatalities per single event (mean) and across all recorded events (total). However individual events such as marine hail, tsunamis and sleet have the potential to cause as many fatalities per event.

In terms of injuries, excessive heat again causes the most injuries per event (mean), although tsunamis are not far behind. However, tornadoes come out on top in terms of total injuries. Further work is required with regards to tornadoes to determine if an outlier is contributing to this extremely high total number. That it is much lower in the mean chart suggests that there have just been more recorded Tornadoes.

Most harmful weather events in the United States with respect to Economic consequences

## Property damage
  ## Looking at mean
  ## Create initial ggplot object
  EconPropMean <- 
    ggplot(
      slice_max(StormDataCleanSummary, order_by = PROPDMG_MEAN, n=10),
      mapping = aes(reorder(EventType, PROPDMG_MEAN), PROPDMG_MEAN),
      ) 
  
  ## Create plot
  EconPropMeanPlot <- 
  EconPropMean +
  geom_bar(stat = "identity", fill = "cornflowerblue") +
  labs(y = "Mean Property Damage ($M)", x = "Event") +
  geom_text(aes(label=round(PROPDMG_MEAN, 2)),
            position=position_dodge(width=0.9),
            hjust=-0.1
            ) +
  coord_flip() +
  ylim(0, 500)
  
  
  ## Looking at totals
  ## Create initial ggplot object
  EconPropTotal <- 
    ggplot(
      slice_max(StormDataCleanSummary, order_by = PROPDMG_TOTAL, n=10),
      mapping = aes(reorder(EventType, PROPDMG_TOTAL), PROPDMG_TOTAL)
      )
  
  ## Create plot
  EconPropTotalPlot <- 
  EconPropTotal +
  geom_bar(stat = "identity", fill = "cornflowerblue") +
  labs(y = "Total Property Damage ($B)", x = "Event") +
  geom_text(aes(label=round(PROPDMG_TOTAL, 2)),
            position=position_dodge(width=0.9)
            ) +
  coord_flip() +
  ylim(0, 200)

  
  
  
## Crop damage
  ## Looking at mean
  ## Create initial ggplot object
  EconCropMean <- 
    ggplot(
      slice_max(StormDataCleanSummary, order_by = CROPDMG_MEAN, n=10),
      mapping = aes(reorder(EventType, CROPDMG_MEAN), CROPDMG_MEAN),
      )
  
  ## Create plot
  EconCropMeanPlot <- 
  EconCropMean +
  geom_bar(stat = "identity", fill = "gold2") +
  labs(y = "Mean Crop Damage ($M)", x = "Event") +
  geom_text(aes(label=round(CROPDMG_MEAN, 2)),
            position=position_dodge(width=0.9)
            ) +
  coord_flip()+
  ylim(0, 60)
  
  
  ## Looking at totals
  ## Create initial ggplot object
  EconCropTotal <- 
    ggplot(
      slice_max(StormDataCleanSummary, order_by = CROPDMG_TOTAL, n=10),
      mapping = aes(reorder(EventType, CROPDMG_TOTAL), CROPDMG_TOTAL)
      )
  
  ## Create plot
  EconCropTotalPlot <- 
  EconCropTotal +
  geom_bar(stat = "identity", fill = "gold2") +
  labs(y = "Total Crop Damage ($B)", x = "Event") +
  geom_text(aes(label=round(CROPDMG_TOTAL, 2)),
            position=position_dodge(width=0.9)
            ) +
  coord_flip()+
  ylim(0, 20)

## Arrange above plots into 2x2 grid
grid.arrange(EconPropMeanPlot, EconPropTotalPlot,EconCropMeanPlot, EconCropTotalPlot, 
             ncol=2,
             nrow=2,
             top = "Top 10 storm events"
)

Hurricanes/typhoons are the most destructive in terms of property damage, per single event, but floods have caused more damage over the recorded events.

Droughts are by far the most destructive for crops, per single event and across all events.