Sever weather events have huge impact on public health and economy. In this analysis we will explore the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database to discover the impact of different events on economy and number of people lost their life and who suffered injuries.
Policies regarding these types of events depends on proper analysis to prioritize resources to face theses events
We used a dplyr work flow with help of “stringr,
tidyr and tidytext packages. we used
ggplot2 for visualizations. NOAA is a big database with
many parameters. Recorded observations are from 1950 to 2011. We
concerned with public health variables (number of fatalities and
injuries) and the total amount of losses with respect to different kind
of events.
Our data is relatively large. It is compressed via a bzip2 algorithm.
We used bzfile() with read.csv() function to
load the data.
data_link <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
download.file(data_link, destfile = "weather.csv.bz2")
weather_data <- read.csv(bzfile("weather.csv.bz2"))
Our work flow will depend mainly on dplyr and
stringr packages.
library(dplyr)
library(stringr)
library(ggplot2)
## Warning: package 'ggplot2' was built under R version 4.6.1
library(tidyr)
library(tidytext)
## Warning: package 'tidytext' was built under R version 4.6.1
library(lubridate)
## Warning: package 'lubridate' was built under R version 4.6.1
We took the subset we need for analysis.
EVTYPE |
Weather event type |
FATALITIES |
Number of fatalities |
INJURIES |
Number of injuries |
PROPDMG |
Number multiplied by property exponent value |
PROPDMGEXP |
Property damage exponent |
CROPDMG |
Number multiplied by crops exponent value |
CROPDMGEXP |
Crop damage exponent |
BGN_DATE |
Date of beginning of an event |
We used regular expressions with
str_detect(),str_to_lower() and
str_trim() functions to combine weather events into their
main categories. The original data set encoded money losses into
characters (k for thousands, m for millions,
etc..). We replaced these exponent characters with their actual values.
Some exponents are encoded by symbols ( =. -,
?, +).We treated it as a multiplication of one
so we don’t loss the actual value recorded for an event.
data_subset <- weather_data %>% select(EVTYPE,BGN_DATE,FATALITIES,INJURIES,
PROPDMG,PROPDMGEXP,
CROPDMG,CROPDMGEXP) %>%
mutate(event_type = str_to_lower(EVTYPE),
year = as.character(year(mdy_hms(BGN_DATE)))) %>%
mutate(event_type = case_when(
str_detect(event_type, "thunder|tstm|microburst|tunder|thuder|thuner|thundeerstorm winds|tropical|gustnado|downburst") ~ "thunder/tropical storm",
str_detect(event_type, "tornado|torndao|landspout") ~ "tornado",
str_detect(event_type, "heat") ~ "heat",
str_detect(event_type, "flood|flooding|floods|floodin|urban/sml stream fld|ice jam") ~ "flood",
str_detect(event_type, "lightning|lighting|ligntning") ~ "lightning",
str_detect(event_type, "ice storm|glaze|ice/strong winds|freezing rain|ice roads|ice floes|ice") ~ "ice storm",
str_detect(event_type, "frost|early frost|frost\freeze|freeze") ~ "frost/freeze",
str_detect(event_type, "blizzard/winter storm|high wind/blizzard") ~ "blizzard",
str_detect(event_type, "winter storms|winter storm high winds|winter weather") ~ "winter storm",
str_detect(event_type, "wind|winds") ~ "high wind",
str_detect(event_type, "small hail") ~ "hail",
str_detect(event_type, "hurricane") ~ "hurricane",
str_detect(event_type, "snow") ~ "heavy snow",
str_detect(event_type, "fog") ~ "fog",
str_detect(event_type, "rip currents") ~ "rip current",
str_detect(event_type, "tropical storm") ~ "tropical storm",
str_detect(event_type, "heavy rain|heavy shower|heavy precipitation") ~ "heavy rain",
str_detect(event_type, "storm surge/tide|coastal surge") ~ "storm surge",
str_detect(event_type, "hail") ~ "hail",
str_detect(event_type, "rain|gusty wind/hvy rain|hvy rain|hvy rainrecord rainfall|rainstorm") ~ "heavy rain",
str_detect(event_type, "cold") ~ "extreme cold",
str_detect(event_type, "landslides") ~ "landslide",
str_detect(event_type, "strong winds") ~ "strong wind",
str_detect(event_type, "excessive wetness") ~ "drought",
str_detect(event_type, "extreme windchill|extreme wind chill|cool and wet") ~ "extreme cold",
str_detect(event_type, "fire") ~ "wildfire",
str_detect(event_type, "swells|high seas|surf") ~ "high surf",
str_detect(event_type, "erosion|coastal storm|high tides") ~ "coastal flood",
str_detect(event_type, "urban|dam break|high water") ~ "flash flood",
str_detect(event_type, "apache county|marine accident|severe turbulence|unseasonably warm") ~ "other",
str_detect(event_type, "wintry|mix|icy roads") ~ "winter weather",
TRUE ~ event_type),
property_exponent = str_trim(str_to_lower(PROPDMGEXP)),
crop_exponent = str_trim(str_to_lower(CROPDMGEXP))) %>%
mutate(property_exponent = case_when(
property_exponent == "k" ~ 10^3,
property_exponent == "m" ~ 10^6,
property_exponent == "b" ~ 10^9,
property_exponent == "h" ~ 10^2,
property_exponent == "8" ~ 10^8,
property_exponent =="7" ~ 10^7,
property_exponent =="6" ~ 10^6,
property_exponent =="5" ~ 10^5,
property_exponent =="4" ~ 10^4,
property_exponent =="3" ~ 10^3,
property_exponent == "2" ~ 10^2,
property_exponent == "1" ~ 10^1,
property_exponent == "0" ~ 10^0,
property_exponent == "-" ~ 1,
property_exponent == "?" ~ 1,
property_exponent == "+" ~ 1,
property_exponent == "" ~ 1,
TRUE ~ 1 ),
crop_exponent = case_when(
crop_exponent == "k" ~ 10^3,
crop_exponent == "m" ~ 10^6,
crop_exponent == "b" ~ 10^9,
crop_exponent == "h" ~ 10^2,
crop_exponent == "8" ~ 10^8,
crop_exponent == "7" ~ 10^7,
crop_exponent == "6" ~ 10^6,
crop_exponent == "5" ~ 10^5,
crop_exponent == "4" ~ 10^4,
crop_exponent == "3" ~ 10^3,
crop_exponent == "2" ~ 10^2,
crop_exponent == "1" ~ 10^1,
crop_exponent == "0" ~ 10^0,
crop_exponent == "-" ~ 1,
crop_exponent == "?" ~ 1,
crop_exponent == "+" ~ 1,
crop_exponent == "" ~ 1,
TRUE ~ 1 ))
We aggregated the data by weather event type then summarize it by the sum of total fatalities and injuries records per event. The data has many missing values we filtered it out of data for better analysis.
health_impact <- data_subset %>% group_by(event_type) %>%
summarise(num_fatal = sum(FATALITIES),
num_injuries = sum(INJURIES),
num_fatal_injuries = num_fatal + num_injuries) %>%
filter(num_fatal_injuries > 0 ) %>%
arrange(desc(num_fatal_injuries))
We transformed data first before plotting by
pivot_longer() function from tidyr package to
split events by impact type (an injury or a fatality).
We used faciting to compare between two plots side by
side. reorder_within() and scale_y_reordered()
from tidytext package are used to arrange y axis values in
a descending order within the plot code for better plot
interpretation.
plot_data <- health_impact %>% slice(1:20) %>%
pivot_longer(cols = c(num_fatal, num_injuries),
names_to = "impact_type",
values_to = "count")
health_plot <- ggplot(data = plot_data,
aes(x = count,
y = reorder_within(event_type, count, impact_type),
fill = impact_type)) +
geom_col() +
labs(title = "Public Health Impact by Severe Weather Event",
x = "Total Number",
y = NULL, fill = "Impact type") +
scale_fill_manual(values = c("num_injuries" = "#1b9e77",
"num_fatal" = "#d95f02")) +
theme_minimal() +
scale_x_continuous(labels = scales::comma) +
facet_wrap(~ impact_type, scales = "free",
labeller = as_labeller(c("num_injuries" = "Injuries",
"num_fatal" = "Fatalites"))) +
scale_y_reordered() +
theme(plot.margin = margin(5, 10, 5, 20, "pt"),
legend.position = "none")
Data aggregated by event type then property and crops actual damage are calculated
economic_impact <- data_subset %>% group_by(event_type) %>%
summarise(property_damage = sum(property_exponent * PROPDMG, na.rm = TRUE) ,
crop_damage = sum(crop_exponent * CROPDMG, na.rm = TRUE),
total_damage = property_damage + crop_damage) %>%
arrange(desc(total_damage))
Like before, we transformed it first to make 2 plots side by side
with facet_wrap() function.
plot2_data <- economic_impact %>% slice(1:15) %>%
pivot_longer(cols = c(property_damage, crop_damage),
names_to = "impact_type",
values_to = "losses")
economic_plot <- ggplot(data = plot2_data,
aes(x = losses/10^9,
y = reorder_within(event_type, losses, impact_type),
fill = impact_type)) +
geom_col() +
theme_minimal() +
scale_x_continuous(labels = scales::comma) +
facet_wrap(~ impact_type, scales = "free",
labeller = as_labeller(c("crop_damage" = "Crop Damage",
"property_damage" = "Property Damage"))) +
scale_y_reordered() +
theme(plot.margin = margin(5, 10, 5, 20, "pt"),
legend.position = "none") +
labs(title = "Ecnomic Impact by Severe Weather Event",
x = "Total Losses (In Billions)",
y = NULL, fill = "Impact type") +
scale_fill_manual(values = c("property_damage" = "#1b9e77",
"crop_damage" = "#d95f02"))
total_economic <- ggplot(data = economic_impact[1:15,],
aes(y = reorder(event_type, total_damage),
x = total_damage/10^9)) +
geom_col() +
labs(title = "Total losses per event",
x = "Total Losses (In Billions)",
y = NULL)
top_data <- data_subset %>% group_by(year,event_type) %>%
summarise(num_fatal = sum(FATALITIES),
num_injuries = sum(INJURIES),
num_fatal_injuries = num_fatal + num_injuries,
.groups = "drop") %>%
filter(num_fatal_injuries > 0)
top_event_year <- top_data %>% group_by(year) %>%
slice_max(order_by = num_fatal_injuries, n = 1, with_ties = FALSE) %>%
ungroup() %>%
arrange(desc(year)) %>%
select(c(1,2))
economy_data_damage_year <- data_subset %>% group_by(year,event_type) %>%
summarise(property_damage =sum(property_exponent * PROPDMG,
na.rm = TRUE),
crop_damage = sum(crop_exponent * CROPDMG, na.rm = TRUE),
total_damage = property_damage + crop_damage,
.groups = "drop")
top_event_damage_year <- economy_data_damage_year %>% group_by(year) %>%
slice_max(order_by = total_damage, n = 1, with_ties = FALSE) %>%
ungroup() %>%
arrange(desc(year)) %>%
select(c(1,2))
From 1950 to 2011, Tornado had the greatest impact on public health. It have the biggest number of fatalities and injuries from 1950 to 1993 and 2000 to 2011.
Heat was the 2nd leading cause of fatalities. It had the greatest impact in 1995, 1999, and 2006. In 2004, hurricanes had the greatest impact. In 1998 it were the floods.
Floods caused the most devastating loss from 1950 to 2011 and hurricanes was the 2nd. The biggest property damage caused by floods and the biggest crops damage by drought. Also, Floods caused a substantial damage to crops. From 1950 to 1992, Tornadoes caused the biggst damage. In general, Floods, hurricanes, and Tornadoes had a greater impact on property damage. Drought, floods, hurricanes, and ice storms had the biggest influence on crops.
print(health_plot)
print(total_economic)
print(economic_plot)
knitr::kable(health_impact[1:10,], caption = "The biggest 10 events")
| event_type | num_fatal | num_injuries | num_fatal_injuries |
|---|---|---|---|
| tornado | 5636 | 91407 | 97043 |
| heat | 3138 | 9224 | 12362 |
| thunder/tropical storm | 825 | 9956 | 10781 |
| flood | 1553 | 8683 | 10236 |
| lightning | 817 | 5231 | 6048 |
| high wind | 696 | 1979 | 2675 |
| ice storm | 117 | 2403 | 2520 |
| winter storm | 278 | 1891 | 2169 |
| wildfire | 90 | 1608 | 1698 |
| hurricane | 133 | 1328 | 1461 |
knitr::kable(top_event_damage_year, caption = "The biggest damage caused by an event in each year")
| year | event_type |
|---|---|
| 2011 | tornado |
| 2010 | flood |
| 2009 | hail |
| 2008 | flood |
| 2007 | flood |
| 2006 | flood |
| 2005 | hurricane |
| 2004 | hurricane |
| 2003 | flood |
| 2002 | hurricane |
| 2001 | thunder/tropical storm |
| 2000 | drought |
| 1999 | hurricane |
| 1998 | hurricane |
| 1997 | flood |
| 1996 | flood |
| 1995 | hurricane |
| 1994 | ice storm |
| 1993 | flood |
| 1992 | tornado |
| 1991 | tornado |
| 1990 | tornado |
| 1989 | tornado |
| 1988 | tornado |
| 1987 | tornado |
| 1986 | tornado |
| 1985 | tornado |
| 1984 | tornado |
| 1983 | tornado |
| 1982 | tornado |
| 1981 | tornado |
| 1980 | tornado |
| 1979 | tornado |
| 1978 | tornado |
| 1977 | tornado |
| 1976 | tornado |
| 1975 | tornado |
| 1974 | tornado |
| 1973 | tornado |
| 1972 | tornado |
| 1971 | tornado |
| 1970 | tornado |
| 1969 | tornado |
| 1968 | tornado |
| 1967 | tornado |
| 1966 | tornado |
| 1965 | tornado |
| 1964 | tornado |
| 1963 | tornado |
| 1962 | tornado |
| 1961 | tornado |
| 1960 | tornado |
| 1959 | tornado |
| 1958 | tornado |
| 1957 | tornado |
| 1956 | tornado |
| 1955 | tornado |
| 1954 | tornado |
| 1953 | tornado |
| 1952 | tornado |
| 1951 | tornado |
| 1950 | tornado |
knitr::kable(top_event_year, caption = "The top event in each year ")
| year | event_type |
|---|---|
| 2011 | tornado |
| 2010 | tornado |
| 2009 | tornado |
| 2008 | tornado |
| 2007 | tornado |
| 2006 | heat |
| 2005 | tornado |
| 2004 | hurricane |
| 2003 | tornado |
| 2002 | tornado |
| 2001 | tornado |
| 2000 | tornado |
| 1999 | heat |
| 1998 | flood |
| 1997 | tornado |
| 1996 | tornado |
| 1995 | heat |
| 1994 | ice storm |
| 1993 | tornado |
| 1992 | tornado |
| 1991 | tornado |
| 1990 | tornado |
| 1989 | tornado |
| 1988 | tornado |
| 1987 | tornado |
| 1986 | tornado |
| 1985 | tornado |
| 1984 | tornado |
| 1983 | tornado |
| 1982 | tornado |
| 1981 | tornado |
| 1980 | tornado |
| 1979 | tornado |
| 1978 | tornado |
| 1977 | tornado |
| 1976 | tornado |
| 1975 | tornado |
| 1974 | tornado |
| 1973 | tornado |
| 1972 | tornado |
| 1971 | tornado |
| 1970 | tornado |
| 1969 | tornado |
| 1968 | tornado |
| 1967 | tornado |
| 1966 | tornado |
| 1965 | tornado |
| 1964 | tornado |
| 1963 | tornado |
| 1962 | tornado |
| 1961 | tornado |
| 1960 | tornado |
| 1959 | tornado |
| 1958 | tornado |
| 1957 | tornado |
| 1956 | tornado |
| 1955 | tornado |
| 1954 | tornado |
| 1953 | tornado |
| 1952 | tornado |
| 1951 | tornado |
| 1950 | tornado |