The most harmfull and expensive weather events between 1993 and 2011

Synopsis

For this document the Storm Data dataset by the National Weather Service has been analysed to find the most expensive and the most harmful types of weather events. The analysis is restricted to the years from 1993 and 2011, because the

The most harmfull weather events are tornadoes with on average 1230 injuries per year, while excessive heat is the deadliest weather event with on average 110 fatalities per year. The flash flood caused by Hurricane Madeline (1998) was the single most harmful event with over 6000 injuries.

The weather event with the most damages to properties and crops was Hurricane Kathrina (2005), single-handedly making Hurricanes the most expensive weather event type. Property damage is the prevalent damage type (over 80 % of the combined damage). Droughts caused the most crop damage over the years.

Data Processing

Data has been downloaded from https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2

Load the raw data:

library(tidyverse)

library(lubridate)

raw_df <- read_csv("repdata_data_StormData.csv.bz2",
                   col_types = cols(.default = col_character(),
                                    #BGN_DATE = col_date(format = "%d/%m/%Y %H:%M:%S"),
                                    #BGN_TIME = col_time(format = "%H%M"),
                                    STATE__ = col_double(),
                                    COUNTY = col_double(),
                                    BGN_RANGE = col_double(),
                                    COUNTY_END = col_double(),
                                    END_RANGE = col_double(),
                                    LENGTH = col_double(),
                                    WIDTH = col_double(),
                                    F = col_integer(),
                                    MAG = col_double(),
                                    FATALITIES = col_double(),
                                    INJURIES = col_double(),
                                    PROPDMG = col_double(),
                                    CROPDMG = col_double(),
                                    LATITUDE = col_double(),
                                    LONGITUDE = col_double(),
                                    LATITUDE_E = col_double(),
                                    LONGITUDE_ = col_double(),
                                    REFNUM = col_double()))
# Parse date and time
raw_df <- raw_df %>%
    #mutate(BGN_TIME = parse_time(BGN_TIME, format = "%H%M")) %>%
    mutate(BGN_DATE = parse_datetime(BGN_DATE, format = "%m/%d/%Y %H:%M:%S"))

For both property and crop damage the damage and damage exponent columns are aggregated into a single columen containing the damage in dollars.

# K is interpreted as 1,000, M as 1,000,000 and B as 1,000,000,000
df <- raw_df %>%
    mutate(property_dmg = case_when(
        is.na(PROPDMGEXP) ~ PROPDMG,
        !is.na(as.numeric(PROPDMGEXP)) ~ PROPDMG*(10^as.numeric(PROPDMGEXP)),
        PROPDMGEXP == "K" | PROPDMGEXP == "k" ~ PROPDMG*1000,
        PROPDMGEXP == "M" | PROPDMGEXP == "m" ~ PROPDMG*1000000,
        PROPDMGEXP == "B" | PROPDMGEXP == "b" ~ PROPDMG*1000000000,
        TRUE ~ PROPDMG
    )) %>%
    select(-PROPDMG, -PROPDMGEXP) %>%
    mutate(crop_dmg = case_when(
        is.na(CROPDMGEXP) ~ CROPDMG,
        !is.na(as.numeric(CROPDMGEXP)) ~ CROPDMG*(10^as.numeric(CROPDMGEXP)),
        CROPDMGEXP == "K" | CROPDMGEXP == "k" ~ CROPDMG*1000,
        CROPDMGEXP == "M" | CROPDMGEXP == "m" ~ CROPDMG*1000000,
        CROPDMGEXP == "B" | CROPDMGEXP == "b" ~ CROPDMG*1000000000,
        TRUE ~ CROPDMG
    )) %>%
    select(-CROPDMG, -CROPDMGEXP) 

NAs durch Umwandlung erzeugtNAs durch Umwandlung erzeugtNAs durch Umwandlung erzeugtNAs durch Umwandlung erzeugt

Summarise both damage types into one column.

df <- df %>%
    mutate(all_dmg = property_dmg + crop_dmg)

Before 1993 only 3 event types have been recorded: Thunderstorm, Tornado, and Hail. To be able to compare all event types with each other the data before 1993 is excluded.

df <- df %>%
    filter(BGN_DATE >= as.Date("1993-01-01"))

There are over 900 different event types. Some of them are combined into a single event type.

df <- df %>%
    mutate(EVTYPE = replace(EVTYPE, EVTYPE=="THUNDERSTORM WIND", "TSTM WIND")) %>%
    mutate(EVTYPE = replace(EVTYPE, EVTYPE=="MARINE THUNDERSTORM WIND", "TSTM WIND")) %>%
    mutate(EVTYPE = replace(EVTYPE, EVTYPE=="THUNDERSTORM WINDS", "TSTM WIND")) %>%
    mutate(EVTYPE = replace(EVTYPE, EVTYPE=="RIP CURRENTS", "RIP CURRENT")) %>%
    mutate(EVTYPE = replace(EVTYPE, EVTYPE=="EXTREME HEAT", "EXCESSIVE HEAT")) %>%
    mutate(EVTYPE = replace(EVTYPE, EVTYPE=="HURRICANE/TYPHOON", "HURRICANE")) %>%
    mutate(EVTYPE = replace(EVTYPE, EVTYPE=="FLASH FLOODING", "FLASH FLOOD"))

Results

Research Question 1: Casualities

Across the United States, which types of events (as indicated in the EVTYP variable) are most harmful with respect to population health?

# Plot the 20 events with the most injuries, all other event types are
# aggregated under 'OTHER'
most_dangerous <- df %>%
    group_by(EVTYPE) %>%
    summarise(sum = sum(INJURIES)) %>%
    arrange(desc(sum)) 
most_dangerous <- most_dangerous$EVTYPE[1:20]
p1 <- df %>%
    mutate(new_events = ifelse(EVTYPE %in% most_dangerous, EVTYPE, "OTHER")) %>%
    group_by(new_events) %>%
    summarise(sum = sum(INJURIES)) %>%
    ggplot(aes(x = reorder(new_events, sum), y = sum)) + 
    geom_bar(stat = "identity") + 
    labs(title = "Injuries per Event Type since 1993",
         x = "Event Type",
         y = "Total Injuries") + 
    coord_flip()
# Plot the 20 events with the most fatalities, all other event types are
# aggregated under 'OTHER'
most_dangerous <- df %>%
    group_by(EVTYPE) %>%
    summarise(sum = sum(FATALITIES)) %>%
    arrange(desc(sum)) 
most_dangerous <- most_dangerous$EVTYPE[1:20]
p2 <- df %>%
    mutate(new_events = ifelse(EVTYPE %in% most_dangerous, EVTYPE, "OTHER")) %>%
    group_by(new_events) %>%
    summarise(sum = sum(FATALITIES)) %>%
    ggplot(aes(x = reorder(new_events, sum), y = sum)) + 
    geom_bar(stat = "identity") + 
    labs(title = "Fatalities per Event Type since 1993",
         x = "Event Type",
         y = "Total Fatalities") + 
    coord_flip()
# package cowplot is used arrange both plots inside one figure
cowplot::plot_grid(p1, p2, nrow=2, align='v')

# average number of injuries per year from tornados
df %>%
    mutate(year = year(BGN_DATE)) %>%
    group_by(year, EVTYPE) %>%
    summarise(sum = sum(INJURIES)) %>%
    filter(EVTYPE == 'TORNADO') %>%
    ungroup() %>%
    summarise(average = mean(sum))

# average number of fatalities per year from excessive heat
df %>%
    mutate(year = year(BGN_DATE)) %>%
    group_by(year, EVTYPE) %>%
    summarise(sum = sum(FATALITIES)) %>%
    filter(EVTYPE == 'EXCESSIVE HEAT') %>%
    ungroup() %>%
    summarise(average = mean(sum))

Tornado is the event type with the most injuries (1230 per year on average). Excessive Heat is the most deadly event type with an average of 110 fatalities per year.

Overall Tornado is the most harmfull event type.

Research Question 2: Economic Consequences

Across the United States, which types of events have the greatest economic consequences?

There is an error in the data. The property damage of the Napa River Flooding was not 115 Billion Dollar, but more likely 115 Million Dollar (see e.g. https://en.wikipedia.org/wiki/Napa_River_flood_of_1986 or https://books.google.at/books?id=nh-vCgAAQBAJ&pg=PA193&lpg=PA193&dq=napa+river+flooding+2005+damage&source=bl&ots=1KO-Q1JPMC&sig=yuw_W2Djij7KR1YaOTpWEs1FluQ&hl=de&sa=X&ved=2ahUKEwihqP3u8ZndAhVRJFAKHQ07DhQ4FBDoATAAegQIARAB#v=onepage&q=napa%20river%20flooding%202005%20damage&f=false ).

There are actually 2 entries for the Napa River flooding one with the correct numbers and one with the incorrect ones.

How reliable is the data?

# filter entry about the napa river flooding on the 2006.01.01.
df <- df %>% 
    filter(!(COUNTYNAME == 'NAPA' & BGN_DATE == as.Date('2006-01-01')))

What is the relation between crop and property damage?

sum_crop <- summarise(df, sum = sum(crop_dmg))[[1]]
sum_prop <- summarise(df, sum = sum(property_dmg))[[1]]
sum_crop/(sum_crop + sum_prop)* 100

[1] 14.79407

The crop damage is only 14.8 % of the overall damage. The property damage is dominating.

most_prop_dmg <- df %>%
    group_by(EVTYPE) %>%
    summarise(sum = sum(property_dmg)) %>%
    arrange(desc(sum)) 
most_prop_dmg <- most_prop_dmg$EVTYPE[1:20]
p1 <- df %>%
    mutate(new_events = ifelse(EVTYPE %in% most_prop_dmg, EVTYPE, "OTHER")) %>%
    group_by(new_events) %>%
    summarise(sum = sum(property_dmg)) %>%
    ggplot(aes(x = reorder(new_events, sum), y = sum/(10^9))) + 
    geom_bar(stat = "identity") + 
    labs(title = "Property Damage since 1993.",
         x = "Event Type",
         y = "Damage in Billion USD") + 
    coord_flip()
most_crop_dmg <- df %>%
    group_by(EVTYPE) %>%
    summarise(sum = sum(crop_dmg)) %>%
    arrange(desc(sum)) 
most_crop_dmg <- most_crop_dmg$EVTYPE[1:20]
p2 <- df %>%
    mutate(new_events = ifelse(EVTYPE %in% most_crop_dmg, EVTYPE, "OTHER")) %>%
    group_by(new_events) %>%
    summarise(sum = sum(crop_dmg)) %>%
    ggplot(aes(x = reorder(new_events, sum), y = sum/(10^9))) + 
    geom_bar(stat = "identity") + 
    labs(title = "Crop Damage since 1993.",
         x = "Event Type",
         y = "Damage in Billion USD") + 
    coord_flip()
# package cowplot is used arange both plots inside one figure
cowplot::plot_grid(p1, p2, nrow=2, align='v')

Hurricanes did the most damage to properties, while droughts are the most expensive event type for crop damage.

Yearly casualties and damages

How are casualties and damages distributed over the years?

p1 <- df %>%
    mutate(year = year(BGN_DATE)) %>%
    group_by(year) %>%
    summarise(FAT = sum(FATALITIES), INJ = sum(INJURIES)) %>%
    ggplot() +
    geom_bar(aes(year, INJ, fill = "Casualties"),  stat = "identity") + 
    geom_bar(aes(year, FAT, fill = 'Fatalities'), stat = "identity") + 
    theme(legend.position = "right") + 
    labs(title = "Casualties per year",
         x = "Year",
         y = "Casualties") + 
    scale_fill_manual(name = "Injury Type", 
                    values = c("lightcoral", "black"))
p2 <- df %>%
    mutate(year = year(BGN_DATE)) %>%
    group_by(year) %>%
    summarise(sum = sum(all_dmg)) %>%
    ggplot() +
    geom_bar(aes(year, sum/1000000000, fill = "All Damages"),  stat = "identity") + 
    scale_y_continuous(labels = scales::dollar) + 
    labs(title = "Damage per Year",
         x = "Year",
         y = "Damages in Billion USD") + 
    scale_fill_manual(name = "Damage Type", 
                    values = c("lightcoral", "black"))
cowplot::plot_grid(p1, p2, nrow=2, align='v')

What are the causes for the peaks in 1998, 2005 and 2011, respectively?

# damages during Kathrina
df %>%
    filter(BGN_DATE >= as.Date('2005-08-23') & BGN_DATE <= as.Date('2005-08-30')) %>%
    summarise(sum = sum(all_dmg)/1E9)

The high damages in 2005 are dominated by Hurricane Kathrina (https://en.wikipedia.org/wiki/Hurricane_Katrina). Over USD 75 Billion of damages occured in the week between the 23. and 30. of August.

# Injuries during Hurrican Madeline
df %>%
    filter(BGN_DATE >= as.Date('1998-10-16') & BGN_DATE <= as.Date('1998-10-20')) %>%
    summarise(sum = sum(INJURIES))

The high number of injuries in 1998 can be attributed to Hurricane Madeline ( https://en.wikipedia.org/wiki/Hurricane_Madeline_(1998) )

The large number of injuries in 2011 are related to the 2011 Super Outbreak, one of the largest, costliest and deadliest tornado outbreaks ever recorded (https://en.wikipedia.org/wiki/2011_Super_Outbreak).