Overview

Sever weather events have huge impact on public health and economy. In this analysis we will explore the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database to discover the impact of different events on economy and number of people lost their life and who suffered injuries.

Policies regarding these types of events depends on proper analysis to prioritize resources to face theses events

We used a dplyr work flow with help of “stringr, tidyr and tidytext packages. we used ggplot2 for visualizations. NOAA is a big database with many parameters. Recorded observations are from 1950 to 2011. We concerned with public health variables (number of fatalities and injuries) and the total amount of losses with respect to different kind of events.

Data Processing

1. Getting and loading the data

Our data is relatively large. It is compressed via a bzip2 algorithm. We used bzfile() with read.csv() function to load the data.

data_link <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"

download.file(data_link, destfile = "weather.csv.bz2")

weather_data <- read.csv(bzfile("weather.csv.bz2"))

2. Loading necessary packages

Our work flow will depend mainly on dplyr and stringr packages.

library(dplyr)
library(stringr)
library(ggplot2)
## Warning: package 'ggplot2' was built under R version 4.6.1
library(tidyr)
library(tidytext)
## Warning: package 'tidytext' was built under R version 4.6.1
library(lubridate)
## Warning: package 'lubridate' was built under R version 4.6.1

3. Taking the subset we need for our analysis

We took the subset we need for analysis.

Variables
EVTYPE Weather event type
FATALITIES Number of fatalities
INJURIES Number of injuries
PROPDMG Number multiplied by property exponent value
PROPDMGEXP Property damage exponent
CROPDMG Number multiplied by crops exponent value
CROPDMGEXP Crop damage exponent
BGN_DATE Date of beginning of an event

We used regular expressions with str_detect(),str_to_lower() and str_trim() functions to combine weather events into their main categories. The original data set encoded money losses into characters (k for thousands, m for millions, etc..). We replaced these exponent characters with their actual values. Some exponents are encoded by symbols ( =. -, ?, +).We treated it as a multiplication of one so we don’t loss the actual value recorded for an event.

data_subset <- weather_data %>% select(EVTYPE,BGN_DATE,FATALITIES,INJURIES, 
                                       PROPDMG,PROPDMGEXP,
                                       CROPDMG,CROPDMGEXP) %>%
    mutate(event_type = str_to_lower(EVTYPE),
           year = as.character(year(mdy_hms(BGN_DATE)))) %>%
    mutate(event_type = case_when(
        str_detect(event_type, "thunder|tstm|microburst|tunder|thuder|thuner|thundeerstorm winds|tropical|gustnado|downburst") ~ "thunder/tropical storm",
        str_detect(event_type, "tornado|torndao|landspout") ~ "tornado",
        str_detect(event_type, "heat") ~ "heat",
        str_detect(event_type, "flood|flooding|floods|floodin|urban/sml stream fld|ice jam") ~ "flood",
        str_detect(event_type, "lightning|lighting|ligntning") ~ "lightning",
        str_detect(event_type, "ice storm|glaze|ice/strong winds|freezing rain|ice roads|ice floes|ice") ~ "ice storm",
        str_detect(event_type, "frost|early frost|frost\freeze|freeze") ~ "frost/freeze",
        str_detect(event_type, "blizzard/winter storm|high wind/blizzard") ~ "blizzard",
        str_detect(event_type, "winter storms|winter storm high winds|winter weather") ~ "winter storm",
        str_detect(event_type, "wind|winds") ~ "high wind",
        str_detect(event_type, "small hail") ~ "hail",
        str_detect(event_type, "hurricane") ~ "hurricane",
        str_detect(event_type, "snow") ~ "heavy snow",
        str_detect(event_type, "fog") ~ "fog",
        str_detect(event_type, "rip currents") ~ "rip current",
        str_detect(event_type, "tropical storm") ~ "tropical storm",
        str_detect(event_type, "heavy rain|heavy shower|heavy precipitation") ~ "heavy rain",
        str_detect(event_type, "storm surge/tide|coastal surge") ~ "storm surge",
        str_detect(event_type, "hail") ~ "hail",
        str_detect(event_type, "rain|gusty wind/hvy rain|hvy rain|hvy rainrecord rainfall|rainstorm") ~ "heavy rain",
        str_detect(event_type, "cold") ~ "extreme cold",
        str_detect(event_type, "landslides") ~ "landslide",
        str_detect(event_type, "strong winds") ~ "strong wind",
        str_detect(event_type, "excessive wetness") ~ "drought", 
        str_detect(event_type, "extreme windchill|extreme wind chill|cool and wet") ~ "extreme cold",
        str_detect(event_type, "fire") ~ "wildfire",
        str_detect(event_type, "swells|high seas|surf") ~ "high surf",
        str_detect(event_type, "erosion|coastal storm|high tides") ~ "coastal flood",
        str_detect(event_type, "urban|dam break|high water") ~ "flash flood",
        str_detect(event_type, "apache county|marine accident|severe turbulence|unseasonably warm") ~ "other",
        str_detect(event_type, "wintry|mix|icy roads") ~ "winter weather",
        TRUE ~ event_type),
        property_exponent = str_trim(str_to_lower(PROPDMGEXP)),
        crop_exponent =  str_trim(str_to_lower(CROPDMGEXP))) %>%
    mutate(property_exponent = case_when(
        property_exponent ==  "k" ~  10^3,
        property_exponent == "m" ~  10^6,
        property_exponent == "b" ~  10^9,
        property_exponent == "h" ~  10^2,
        property_exponent == "8" ~  10^8,
        property_exponent =="7" ~  10^7,
        property_exponent =="6" ~  10^6,
        property_exponent =="5" ~  10^5,
        property_exponent =="4" ~  10^4,
        property_exponent =="3" ~  10^3,
        property_exponent == "2" ~  10^2,
        property_exponent == "1" ~  10^1,
        property_exponent == "0" ~  10^0,
        property_exponent == "-" ~  1,
        property_exponent == "?" ~  1,
        property_exponent == "+" ~  1,
        property_exponent == ""  ~  1,
        TRUE ~ 1 ),
        crop_exponent = case_when(
            crop_exponent == "k" ~  10^3,
            crop_exponent == "m" ~  10^6,
            crop_exponent == "b" ~  10^9,
            crop_exponent == "h" ~  10^2,
            crop_exponent == "8" ~  10^8,
            crop_exponent == "7" ~  10^7,
            crop_exponent == "6" ~  10^6,
            crop_exponent == "5" ~  10^5,
            crop_exponent == "4" ~  10^4,
            crop_exponent == "3" ~  10^3,
            crop_exponent == "2" ~  10^2,
            crop_exponent == "1" ~  10^1,
            crop_exponent == "0" ~  10^0,
            crop_exponent == "-" ~  1,
            crop_exponent == "?" ~  1,
            crop_exponent == "+" ~  1,
            crop_exponent == ""  ~  1,
            TRUE ~ 1 ))

4. Health impact

We aggregated the data by weather event type then summarize it by the sum of total fatalities and injuries records per event. The data has many missing values we filtered it out of data for better analysis.

health_impact <- data_subset %>% group_by(event_type) %>%
    summarise(num_fatal = sum(FATALITIES),
              num_injuries = sum(INJURIES),
              num_fatal_injuries = num_fatal + num_injuries) %>%
    filter(num_fatal_injuries > 0 ) %>%
    arrange(desc(num_fatal_injuries))

4.1 Plotting Public health impact by severe weather events

We transformed data first before plotting by pivot_longer() function from tidyr package to split events by impact type (an injury or a fatality).

We used faciting to compare between two plots side by side. reorder_within() and scale_y_reordered() from tidytext package are used to arrange y axis values in a descending order within the plot code for better plot interpretation.

plot_data <- health_impact %>% slice(1:20) %>%
             pivot_longer(cols = c(num_fatal, num_injuries),
                     names_to = "impact_type",
                     values_to = "count")


health_plot <- ggplot(data = plot_data,
                      aes(x = count,
                          y = reorder_within(event_type, count, impact_type),
                          fill = impact_type)) +
               geom_col() +
               labs(title = "Public Health Impact by Severe Weather Event",
                    x = "Total Number", 
                    y = NULL, fill = "Impact type") +
               scale_fill_manual(values = c("num_injuries" = "#1b9e77",
                                          "num_fatal" = "#d95f02")) + 
               theme_minimal() +
               scale_x_continuous(labels = scales::comma) +
               facet_wrap(~ impact_type, scales = "free",
                          labeller = as_labeller(c("num_injuries" = "Injuries",
                                                   "num_fatal" = "Fatalites"))) +
               scale_y_reordered() +
               theme(plot.margin = margin(5, 10, 5, 20, "pt"),
                 legend.position = "none")

5. Economic impact

Data aggregated by event type then property and crops actual damage are calculated

economic_impact <- data_subset %>% group_by(event_type) %>% 
    summarise(property_damage = sum(property_exponent * PROPDMG, na.rm = TRUE) ,
               crop_damage = sum(crop_exponent * CROPDMG, na.rm = TRUE),
               total_damage = property_damage + crop_damage) %>%
    arrange(desc(total_damage))

5.1 Plotting economic impact by severe weather events

Like before, we transformed it first to make 2 plots side by side with facet_wrap() function.

plot2_data <- economic_impact %>% slice(1:15) %>%
    pivot_longer(cols = c(property_damage, crop_damage),
                 names_to = "impact_type",
                 values_to = "losses")


economic_plot <- ggplot(data = plot2_data,
       aes(x = losses/10^9,
           y = reorder_within(event_type, losses, impact_type),
           fill = impact_type)) +
    geom_col() +
    theme_minimal() +
    scale_x_continuous(labels = scales::comma) +
    facet_wrap(~ impact_type, scales = "free",
               labeller = as_labeller(c("crop_damage" = "Crop Damage",
                        "property_damage" = "Property Damage"))) +
    scale_y_reordered() +
    theme(plot.margin = margin(5, 10, 5, 20, "pt"),
          legend.position = "none") +
    labs(title = "Ecnomic Impact by Severe Weather Event",
         x = "Total Losses (In Billions)", 
         y = NULL, fill = "Impact type") +
    scale_fill_manual(values = c("property_damage" = "#1b9e77",
                                 "crop_damage" = "#d95f02"))


total_economic <- ggplot(data = economic_impact[1:15,],
                         aes(y = reorder(event_type, total_damage), 
                             x = total_damage/10^9)) +
                  geom_col() + 
                  labs(title = "Total losses per event",
                  x = "Total Losses (In Billions)",
                  y = NULL)

6.The most catastrophic event in each year

top_data <- data_subset %>% group_by(year,event_type) %>%
    summarise(num_fatal = sum(FATALITIES),
              num_injuries = sum(INJURIES),
              num_fatal_injuries = num_fatal + num_injuries,
              .groups = "drop") %>%
    filter(num_fatal_injuries > 0)

top_event_year <- top_data %>% group_by(year) %>%
    slice_max(order_by = num_fatal_injuries, n = 1, with_ties = FALSE) %>%
    ungroup() %>%
    arrange(desc(year)) %>%
    select(c(1,2))

7.The biggest damage caused by an event in a year

economy_data_damage_year <- data_subset %>% group_by(year,event_type) %>%
    summarise(property_damage =sum(property_exponent * PROPDMG,
                                   na.rm = TRUE),
              crop_damage = sum(crop_exponent * CROPDMG, na.rm = TRUE),
              total_damage = property_damage + crop_damage,
              .groups = "drop")

top_event_damage_year <- economy_data_damage_year %>% group_by(year) %>%
    slice_max(order_by = total_damage, n = 1, with_ties = FALSE) %>%
    ungroup() %>%
    arrange(desc(year)) %>%
    select(c(1,2))

Results

Plots

From 1950 to 2011, Tornado had the greatest impact on public health. It have the biggest number of fatalities and injuries from 1950 to 1993 and 2000 to 2011.

Heat was the 2nd leading cause of fatalities. It had the greatest impact in 1995, 1999, and 2006. In 2004, hurricanes had the greatest impact. In 1998 it were the floods.

Floods caused the most devastating loss from 1950 to 2011 and hurricanes was the 2nd. The biggest property damage caused by floods and the biggest crops damage by drought. Also, Floods caused a substantial damage to crops. From 1950 to 1992, Tornadoes caused the biggst damage. In general, Floods, hurricanes, and Tornadoes had a greater impact on property damage. Drought, floods, hurricanes, and ice storms had the biggest influence on crops.

print(health_plot)

print(total_economic)

print(economic_plot)

Tables

knitr::kable(health_impact[1:10,], caption = "The biggest 10 events")
The biggest 10 events
event_type num_fatal num_injuries num_fatal_injuries
tornado 5636 91407 97043
heat 3138 9224 12362
thunder/tropical storm 825 9956 10781
flood 1553 8683 10236
lightning 817 5231 6048
high wind 696 1979 2675
ice storm 117 2403 2520
winter storm 278 1891 2169
wildfire 90 1608 1698
hurricane 133 1328 1461
knitr::kable(top_event_damage_year, caption = "The biggest damage caused by an event in each year")
The biggest damage caused by an event in each year
year event_type
2011 tornado
2010 flood
2009 hail
2008 flood
2007 flood
2006 flood
2005 hurricane
2004 hurricane
2003 flood
2002 hurricane
2001 thunder/tropical storm
2000 drought
1999 hurricane
1998 hurricane
1997 flood
1996 flood
1995 hurricane
1994 ice storm
1993 flood
1992 tornado
1991 tornado
1990 tornado
1989 tornado
1988 tornado
1987 tornado
1986 tornado
1985 tornado
1984 tornado
1983 tornado
1982 tornado
1981 tornado
1980 tornado
1979 tornado
1978 tornado
1977 tornado
1976 tornado
1975 tornado
1974 tornado
1973 tornado
1972 tornado
1971 tornado
1970 tornado
1969 tornado
1968 tornado
1967 tornado
1966 tornado
1965 tornado
1964 tornado
1963 tornado
1962 tornado
1961 tornado
1960 tornado
1959 tornado
1958 tornado
1957 tornado
1956 tornado
1955 tornado
1954 tornado
1953 tornado
1952 tornado
1951 tornado
1950 tornado
knitr::kable(top_event_year, caption = "The top event in each year ")
The top event in each year
year event_type
2011 tornado
2010 tornado
2009 tornado
2008 tornado
2007 tornado
2006 heat
2005 tornado
2004 hurricane
2003 tornado
2002 tornado
2001 tornado
2000 tornado
1999 heat
1998 flood
1997 tornado
1996 tornado
1995 heat
1994 ice storm
1993 tornado
1992 tornado
1991 tornado
1990 tornado
1989 tornado
1988 tornado
1987 tornado
1986 tornado
1985 tornado
1984 tornado
1983 tornado
1982 tornado
1981 tornado
1980 tornado
1979 tornado
1978 tornado
1977 tornado
1976 tornado
1975 tornado
1974 tornado
1973 tornado
1972 tornado
1971 tornado
1970 tornado
1969 tornado
1968 tornado
1967 tornado
1966 tornado
1965 tornado
1964 tornado
1963 tornado
1962 tornado
1961 tornado
1960 tornado
1959 tornado
1958 tornado
1957 tornado
1956 tornado
1955 tornado
1954 tornado
1953 tornado
1952 tornado
1951 tornado
1950 tornado