Analysis of the effects of storm events
Jason Schmidt
July 13, 2016
This study answers the question of which storm event types are most harmful to human health, and which have the most significant economic impact. The study uses the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database which covers the period 1950 to 2011. However data prior to January 1996 only contains Tornado, Thunderstorm Wind and Hail event types. In order not to skew the results, the study is limited to data from January 1996 to 2011.
In order to better quantify the effects the study groups the NOAA storm database event types into a cleaned event type classification. This is to correct for duplications, miss-spellings and synonyms in the NOAA event types.The study uses simple bar plots of total fatalities, total injuries and total economic cost to rank event types.
We find that Tornadoes, Extreme heat, and Extreme wind have been most damaging and costly storm events over the 15 year period.
Data processing
Load the R libraries required for analysis
library(data.table)## Warning: package 'data.table' was built under R version 3.2.5library(ggplot2)
library(scales)Download U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database
setwd("~/coursera/data scientist/RepResearchCourseProject")
download.file("https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2","StormData.csv.bz2")
stormdata<-read.csv("StormData.csv.bz2", stringsAsFactors=FALSE)Only include records from Jan 1996 onwards in the data set for analysis, and only set where fatality, injury, property or crop damage are recorded
stormdata$BGN_DATE<-as.Date(stormdata$BGN_DATE,"%m/%d/%Y")
analysis_set<-data.table(stormdata[stormdata$BGN_DATE>="1996-01-01",])
analysis_set<-analysis_set[analysis_set$INJURIES>0|analysis_set$FATALITIES>0|analysis_set$PROPDMG>0|analysis_set$CROPDMG>0,]Analyse and clean up event types
Check the unique set of event types
eventtypes<-unique(analysis_set$EVTYPE)
sort(eventtypes)## [1] " HIGH SURF ADVISORY" " FLASH FLOOD"
## [3] " TSTM WIND" " TSTM WIND (G45)"
## [5] "AGRICULTURAL FREEZE" "ASTRONOMICAL HIGH TIDE"
## [7] "ASTRONOMICAL LOW TIDE" "AVALANCHE"
## [9] "Beach Erosion" "BLACK ICE"
## [11] "BLIZZARD" "BLOWING DUST"
## [13] "blowing snow" "BRUSH FIRE"
## [15] "COASTAL FLOODING/EROSION" "COASTAL EROSION"
## [17] "Coastal Flood" "COASTAL FLOOD"
## [19] "Coastal Flooding" "COASTAL FLOODING"
## [21] "COASTAL FLOODING/EROSION" "Coastal Storm"
## [23] "COASTAL STORM" "COASTALSTORM"
## [25] "Cold" "COLD"
## [27] "COLD AND SNOW" "Cold Temperature"
## [29] "COLD WEATHER" "COLD/WIND CHILL"
## [31] "DAM BREAK" "Damaging Freeze"
## [33] "DAMAGING FREEZE" "DENSE FOG"
## [35] "DENSE SMOKE" "DOWNBURST"
## [37] "DROUGHT" "DROWNING"
## [39] "DRY MICROBURST" "Dust Devil"
## [41] "DUST DEVIL" "DUST STORM"
## [43] "Early Frost" "Erosion/Cstl Flood"
## [45] "EXCESSIVE HEAT" "EXCESSIVE SNOW"
## [47] "Extended Cold" "Extreme Cold"
## [49] "EXTREME COLD" "EXTREME COLD/WIND CHILL"
## [51] "EXTREME WINDCHILL" "FALLING SNOW/ICE"
## [53] "FLASH FLOOD" "FLASH FLOOD/FLOOD"
## [55] "FLOOD" "FLOOD/FLASH/FLOOD"
## [57] "FOG" "Freeze"
## [59] "FREEZE" "Freezing drizzle"
## [61] "Freezing Drizzle" "FREEZING DRIZZLE"
## [63] "FREEZING FOG" "Freezing Rain"
## [65] "FREEZING RAIN" "Freezing Spray"
## [67] "FROST" "Frost/Freeze"
## [69] "FROST/FREEZE" "FUNNEL CLOUD"
## [71] "Glaze" "GLAZE"
## [73] "gradient wind" "Gradient wind"
## [75] "GRADIENT WIND" "GUSTY WIND"
## [77] "GUSTY WIND/HAIL" "GUSTY WIND/HVY RAIN"
## [79] "Gusty wind/rain" "Gusty winds"
## [81] "Gusty Winds" "GUSTY WINDS"
## [83] "HAIL" "HARD FREEZE"
## [85] "HAZARDOUS SURF" "HEAT"
## [87] "Heat Wave" "HEAVY RAIN"
## [89] "Heavy Rain/High Surf" "HEAVY SEAS"
## [91] "HEAVY SNOW" "Heavy snow shower"
## [93] "Heavy Surf" "HEAVY SURF"
## [95] "Heavy surf and wind" "HEAVY SURF/HIGH SURF"
## [97] "HIGH SEAS" "High Surf"
## [99] "HIGH SURF" "HIGH SWELLS"
## [101] "HIGH WATER" "HIGH WIND"
## [103] "HIGH WIND (G40)" "HIGH WINDS"
## [105] "HURRICANE" "Hurricane Edouard"
## [107] "HURRICANE/TYPHOON" "HYPERTHERMIA/EXPOSURE"
## [109] "Hypothermia/Exposure" "HYPOTHERMIA/EXPOSURE"
## [111] "Ice jam flood (minor" "ICE ON ROAD"
## [113] "ICE ROADS" "ICE STORM"
## [115] "ICY ROADS" "LAKE-EFFECT SNOW"
## [117] "Lake Effect Snow" "LAKE EFFECT SNOW"
## [119] "LAKESHORE FLOOD" "LANDSLIDE"
## [121] "LANDSLIDES" "Landslump"
## [123] "LANDSPOUT" "LATE SEASON SNOW"
## [125] "LIGHT FREEZING RAIN" "Light snow"
## [127] "Light Snow" "LIGHT SNOW"
## [129] "Light Snowfall" "LIGHTNING"
## [131] "Marine Accident" "MARINE HAIL"
## [133] "MARINE HIGH WIND" "MARINE STRONG WIND"
## [135] "MARINE THUNDERSTORM WIND" "MARINE TSTM WIND"
## [137] "Microburst" "MIXED PRECIP"
## [139] "Mixed Precipitation" "MIXED PRECIPITATION"
## [141] "MUD SLIDE" "Mudslide"
## [143] "MUDSLIDE" "Mudslides"
## [145] "NON-SEVERE WIND DAMAGE" "NON-TSTM WIND"
## [147] "NON TSTM WIND" "Other"
## [149] "OTHER" "RAIN"
## [151] "RAIN/SNOW" "RECORD HEAT"
## [153] "RIP CURRENT" "RIP CURRENTS"
## [155] "RIVER FLOOD" "River Flooding"
## [157] "RIVER FLOODING" "ROCK SLIDE"
## [159] "ROGUE WAVE" "ROUGH SEAS"
## [161] "ROUGH SURF" "SEICHE"
## [163] "SMALL HAIL" "Snow"
## [165] "SNOW" "SNOW AND ICE"
## [167] "SNOW SQUALL" "Snow Squalls"
## [169] "SNOW SQUALLS" "STORM SURGE"
## [171] "STORM SURGE/TIDE" "Strong Wind"
## [173] "STRONG WIND" "Strong Winds"
## [175] "STRONG WINDS" "THUNDERSTORM"
## [177] "THUNDERSTORM WIND" "THUNDERSTORM WIND (G40)"
## [179] "Tidal Flooding" "TIDAL FLOODING"
## [181] "TORNADO" "Torrential Rainfall"
## [183] "TROPICAL DEPRESSION" "TROPICAL STORM"
## [185] "Tstm Wind" "TSTM WIND"
## [187] "TSTM WIND (G45)" "TSTM WIND (41)"
## [189] "TSTM WIND (G35)" "TSTM WIND (G40)"
## [191] "TSTM WIND (G45)" "TSTM WIND 40"
## [193] "TSTM WIND 45" "TSTM WIND AND LIGHTNING"
## [195] "TSTM WIND G45" "TSTM WIND/HAIL"
## [197] "TSUNAMI" "TYPHOON"
## [199] "Unseasonable Cold" "UNSEASONABLY COLD"
## [201] "UNSEASONABLY WARM" "UNSEASONAL RAIN"
## [203] "URBAN/SML STREAM FLD" "VOLCANIC ASH"
## [205] "WARM WEATHER" "WATERSPOUT"
## [207] "WET MICROBURST" "Whirlwind"
## [209] "WHIRLWIND" "WILD/FOREST FIRE"
## [211] "WILDFIRE" "Wind"
## [213] "WIND" "WIND AND WAVE"
## [215] "Wind Damage" "WINDS"
## [217] "WINTER STORM" "WINTER WEATHER"
## [219] "WINTER WEATHER MIX" "WINTER WEATHER/MIX"
## [221] "Wintry Mix" "WINTRY MIX"Exclude records with “Summary…” and “Monthly..” event types
analysis_set<-analysis_set[!grepl("Summary|SUMMARY|Monthly|MONTHLY",analysis_set$EVTYPE)]Create a cleaned event type field to combine event types which appear to be duplications/ miss-spellings/ synonyms.
analysis_set[,eventType_Cleaned:=""]
analysis_set[EVTYPE %in% c(" HIGH SURF ADVISORY","HAZARDOUS SURF","HEAVY SEAS","Heavy Surf","HEAVY SURF","Heavy surf and wind","HEAVY SURF/HIGH SURF","HIGH SEAS","High Surf","HIGH SURF","HIGH SWELLS","HIGH WAVES","RIP CURRENTS","RIP CURRENT","RIP CURRENTS/HEAVY SURF","ROUGH SURF","ROGUE WAVE","ROUGH SEAS","Heavy Rain/High Surf","HIGH SWELLS","HIGH SURF ADVISORY", "HIGH SURF ADVISORIES","HIGH SWELLS","WIND AND WAVE" ),"eventType_Cleaned"]<-"HIGH SURF/ SEAS"
analysis_set[EVTYPE %in% c(" COASTAL FLOOD","COASTAL FLOOD","Coastal Flooding","COASTAL FLOODING","COASTAL FLOODING/EROSION","STORM SURGE/TIDE","TIDAL FLOODING","STORM SURGE","HIGH WATER","CSTL FLOODING/EROSION","COASTALFLOOD","COASTAL FLOODING/EROSION","Beach Erosion","BEACH EROSION","ASTRONOMICAL HIGH TIDE", "Tidal Flooding","Erosion/Cstl Flood","Coastal Flood","coastal flooding", "COASTAL EROSION","SEICHE" ),"eventType_Cleaned"]<-"COASTAL FLOODING"
analysis_set[EVTYPE %in% c("FLASH FLOOD","FLASH FLOOD/FLOOD","FLASH FLOODING","FLASH FLOODING/FLOOD","FLASH FLOODS","FLOOD/FLASH FLOOD","ICE STORM/FLASH FLOOD"," FLASH FLOOD","FLOOD/FLASH/FLOOD","Flood/Flash Flood" ),"eventType_Cleaned"]<-"FLASH FLOOD"
analysis_set[EVTYPE %in% c("FLOOD","FLOOD/RIVER FLOOD","FLOODING","MINOR FLOODING","RIVER FLOOD","RAPIDLY RISING WATER","RIVER FLOODING","RIVER FLOODING","River Flooding","River Flooding ","URBAN AND SMALL STREAM FLOODIN","URBAN/SML STREAM FLD", "URBAN/STREET FLOODING","URBAN/SML STREAM FLDG","URBAN/SMALL STRM FLDG","Urban Flooding","URBAN FLOOD","Urban Flood", "Urban flood", "STREET FLOODING", "SNOWMELT FLOODING", "Sml Stream Fld","Minor Flooding", "Ice jam flood (minor", "LAKESHORE FLOOD"),"eventType_Cleaned"]<-"OTHER FLOOD"
analysis_set[EVTYPE %in% c("LIGHTNING","LIGHTNING AND THUNDERSTORM WIN","LIGHTNING INJURY","LIGHTNING.","THUNDERSTORM"," LIGHTNING","SEVERE THUNDERSTORM","SEVERE THUNDERSTORMS","TSTM WIND AND LIGHTNING","TSTM","THUNDERSTORMS","Thundersnow shower" ),"eventType_Cleaned"]<-"THUNDERSTORM/ LIGHTNING"
analysis_set[EVTYPE %in% c("GUSTY WIND","Gusty winds","Gusty Winds","GUSTY WINDS","HIGH WIND","HIGH WIND AND SEAS","HIGH WIND/SEAS","HIGH WINDS","HIGH WINDS/COLD","MARINE HIGH WIND","MARINE STRONG WIND","MARINE TSTM WIND","MARINE THUDERSTORM WIND","NON TSTM WIND","STRONG WIND","Strong Winds","STRONG WINDS","THUNDERSTORM WINDS","THUNDERSTORM WIND","THUNDERSTORM WIND (G40)","THUNDERSTORM WIND G52","THUNDERSTORM WINDS","THUNDERSTORM WINDS 13","THUNDERSTORM WINDSS","THUNDERSTORMS WINDS","THUNDERSTORM WINDS","THUNDERSTORMW","THUNDERTORM WINDS","TSTM WIND","TSTM WIND (G53)","TSTM WIND (G40)","TSTM WIND (G45)","WIND","WIND STORM","WINDS","HIGH WIND 48","MARINE THUNDERSTORM WIND","WINTER STORM HIGH WINDS","TSTM WIND (G35)","NON-SEVERE WIND DAMAGE"," TSTM WIND"," TSTM WIND (G45)"," WIND","Wind Damage","WIND GUSTS", "WIND DAMAGE","Wind", "WIND ADVISORY","TSTM WND","TSTM WINDS","TSTM WIND 45","TSTM WIND 40", "TSTM WIND (41)","TSTM WIND (G45)", "Tstm Wind","Thunderstorm Wind", "Strong winds" , "STRONG WIND GUST", "Strong Wind","NON-TSTM WIND", "Strong Wind","TSTM WIND (41)","TSTM WIND 40","TSTM WIND 45","HIGH WIND (G40)","GRADIENT WIND", "Gradient wind","gradient wind","TSTM WIND G45","GUSTY WIND/HAIL","GUSTY WIND/HVY RAIN","Gusty wind/rain","Microburst","WET MICROBURST"),"eventType_Cleaned"]<-"EXTREME WINDS"
analysis_set[EVTYPE %in% c("AGRICULTURAL FREEZE","BLACK ICE","FROST","GLAZE/ICE STORM","ICE ON ROAD","ICE ROADS","ICE","ICE STORM","ICY ROADS","GLAZE","DAMAGING FREEZE","Frost/Freeze","FROST/FREEZE","Damaging Freeze","Early Frost","Freeze","FREEZE","Freezing drizzle","Freezing Drizzle","FREEZING DRIZZLE","Freezing Spray","HARD FREEZE","LIGHT FREEZING RAIN","Freezing Rain" ,"FREEZING RAIN" ,"FREEZING FOG","Glaze" ),"eventType_Cleaned"]<-"ICE/ FROST/ FREEZE"
analysis_set[EVTYPE %in% c("DROWNING","OTHER","Marine Accident","HIGH","MARINE MISHAP","ASTRONOMICAL LOW TIDE","DAM BREAK","Other","VOLCANIC ASH"),"eventType_Cleaned"]<-"OTHER"
analysis_set[EVTYPE %in% c("AVALANCE","AVALANCHE","HEAVY SNOW/BLIZZARD/AVALANCHE"),"eventType_Cleaned"]<-"AVALANCHE"
analysis_set[EVTYPE %in% c("HEAVY SNOW AND HIGH WINDS","Snow","SNOW","SNOW AND ICE","SNOW SQUALL","Snow Squalls","SNOW/ BITTER COLD","SNOW/HIGH WINDS","BLIZZARD","HEAVY SNOW","Heavy snow shower","HEAVY SNOW/ICE","FALLING SNOW/ICE","EXCESSIVE SNOW","COLD AND SNOW","BLOWING SNOW","blowing snow","HIGH WIND/HEAVY SNOW","HIGH WINDS/SNOW","FREEZING RAIN/SNOW","THUNDERSNOW","LIGHT SNOW","RAIN/SNOW","LATE SEASON SNOW","Light Snow","LAKE EFFECT SNOW","Lake Effect Snow","Light Snowfall","LAKE-EFFECT SNOW","Light snow", "SNOW SQUALLS" ),"eventType_Cleaned"]<-"SNOW STORMS"
analysis_set[EVTYPE %in% c("DRY MICROBURST","DRY MIRCOBURST WINDS","DRY MICROBURST WINDS","Dust Devil","DUST DEVIL","DUST STORM","BLOWING DUST"),"eventType_Cleaned"]<-"DUST STORM"
analysis_set[EVTYPE %in% c("BRUSH FIRE","WILD/FOREST FIRE","WILDFIRE","WILD FIRES","DENSE SMOKE"),"eventType_Cleaned"]<-"BRUSH FIRE"
analysis_set[EVTYPE %in% c("Coastal Storm","COASTAL STORM","COASTALSTORM"),"eventType_Cleaned"]<-"COASTAL STORM"
analysis_set[EVTYPE %in% c("Cold","COLD","Cold Temperature","COLD WAVE","COLD WEATHER","COLD/WIND CHILL","COLD/WINDS","Extended Cold","Extreme Cold","EXTREME COLD","EXTREME COLD/WIND CHILL","EXTREME WINDCHILL","UNSEASONABLY COLD","HYPERTHERMIA/EXPOSURE","HYPOTHERMIA","Hypothermia/Exposure","HYPOTHERMIA/EXPOSURE","RECORD COLD","LOW TEMPERATURE", "Unseasonable Cold"),"eventType_Cleaned"]<-"EXTREME COLD"
analysis_set[EVTYPE %in% c("DENSE FOG","FOG","FOG AND COLD TEMPERATURES"),"eventType_Cleaned"]<-"FOG"
analysis_set[EVTYPE %in% c("DROUGHT","DROUGHT/EXCESSIVE HEAT","HEAT WAVE DROUGHT"),"eventType_Cleaned"]<-"DROUGHT"
analysis_set[EVTYPE %in% c("EXCESSIVE RAINFALL","FLOOD & HEAVY RAIN","HEAVY RAIN","HEAVY RAINS","RAIN/WIND","Torrential Rainfall","DOWNBURST","RAIN","UNSEASONAL RAIN","RAIN" ),"eventType_Cleaned"]<-"HEAVY RAIN"
analysis_set[EVTYPE %in% c("EXCESSIVE HEAT","EXTREME HEAT","HEAT","Heat Wave","HEAT WAVE","HEAT WAVES","RECORD HEAT","RECORD/EXCESSIVE HEAT","UNSEASONABLY WARM","WARM WEATHER","UNSEASONABLY WARM AND DRY"),"eventType_Cleaned"]<-"EXTREME HEAT"
analysis_set[EVTYPE %in% c("FUNNEL CLOUD","TORNADO","TORNADO F2","TORNADO F3","TORNADOES, TSTM WIND, HAIL","WATERSPOUT","WATERSPOUT TORNADO", "WATERSPOUT/TORNADO","Whirlwind","WHIRLWIND", "LANDSPOUT"),"eventType_Cleaned"]<-"TORNADO"
analysis_set[EVTYPE %in% c("HAIL","THUNDERSTORM WINDS/HAIL","TSTM WIND/HAIL","SLEET","SMALL HAIL","MIXED PRECIP","MARINE HAIL","Mixed Precipitation" ,"MIXED PRECIPITATION"),"eventType_Cleaned"]<-"HAIL/ SLEET"
analysis_set[EVTYPE %in% c("HURRICANE","HURRICANE-GENERATED SWELLS","Hurricane Edouard","HURRICANE EMILY","HURRICANE ERIN","HURRICANE FELIX","HURRICANE OPAL","HURRICANE OPAL/HIGH WINDS"),"eventType_Cleaned"]<-"HURRICANE"
analysis_set[EVTYPE %in% c("HURRICANE/TYPHOON","TYPHOON"),"eventType_Cleaned"]<-"TYPHOON"
analysis_set[EVTYPE %in% c("LANDSLIDE","LANDSLIDES","Mudslides","Mudslide", "ROCK SLIDE", "Landslump","MUDSLIDE","MUD SLIDE" ),"eventType_Cleaned"]<-"LAND/ ROCK/ MUD SLIDES"
analysis_set[EVTYPE %in% c("TROPICAL STORM","TROPICAL STORM GORDON", "TROPICAL DEPRESSION"),"eventType_Cleaned"]<-"TROPICAL STORM"
analysis_set[EVTYPE %in% c("TSUNAMI"),"eventType_Cleaned"]<-"TSUNAMI"
analysis_set[EVTYPE %in% c("WINTER STORM","WINTER STORMS","WINTER WEATHER","WINTER WEATHER MIX","WINTER WEATHER/MIX","WINTRY MIX","Wintry Mix"),"eventType_Cleaned"]<-"WINTER STORM"Add totals to data table
Calculate total injuries, fatilities, crop and property damages per event type
analysis_set<-analysis_set[,c("totalInjuries","totalFatalities","totalCropDamage","totalPropertyDamage","totalDamage"):=list(sum(INJURIES),sum(FATALITIES),sum(CROPDMG)/1000,sum(PROPDMG)/1000,sum(CROPDMG+PROPDMG)/1000),by=eventType_Cleaned][]Results
Types of events most harmful to population health
The graphics below plot the total fatalities and total injuries caused by the different event types included in the clean list. The totals have been calculated for the period Jan 1996 to Dec 2011 (15 years).
To assess which event types caused the most fatalities we plot a bar chart of number of total number of fatalities caused per event type.
ggplot(analysis_set, aes(x=reorder(eventType_Cleaned,-totalFatalities),y=totalFatalities))+geom_bar(position="dodge",stat="identity")+theme(axis.text.x = element_text(angle = 90, hjust = 1)) + labs(x="Event type", y="Total number of fatalities")+ggtitle("Total number of fatalities by event type")+ scale_y_continuous(labels = comma)
From the bar chart we can see that the most lethal event type is Extreme heat. During the 15 year period over 2,000 people lost their lives as a result of conditions of extreme heat.
The second most lethal event type is Tornadoes with over 1,500 people losing their lives.
The group of big killers which each resulted in around 750 people losing their lives in the 15 year period are Flash floods, Extreme winds, High surf/ Seas, and Thunderstorms/ Lightning.
To assess which event types resulted in the most injuries we plot a bar chart of total injuries suffered per event type.
ggplot(analysis_set, aes(x=reorder(eventType_Cleaned,-totalInjuries),y=totalInjuries))+geom_bar(position="dodge",stat="identity")+theme(axis.text.x = element_text(angle = 90, hjust = 1)) + labs(x="Event type", y="Total number of injuries")+ggtitle("Total number of injuries by event type")+ scale_y_continuous(labels = comma)
By far the event type which caused the most injuries in the 15 year period is Tornadoes with over 20,000 people injured.
There are three event types which caused around 7,500 injuries each. These are Extreme Heat, Other Floods (this excludes flash floods and coastal floods), and Extreme winds.
We may conclude that the tow event types most harmful to human health are the Tornado and Extreme heat.Extreme winds is the third most harmful.
Types of events with the greatest economic consequences
In order to assess which event types have the greatest total economic cost for the period Jan 1996 to Dec 2011 we plot a bar chart of the total economic cost incurred.
ggplot(analysis_set, aes(x=reorder(eventType_Cleaned,-totalDamage),y=totalDamage))+geom_bar(position="dodge",stat="identity")+theme(axis.text.x = element_text(angle = 90, hjust = 1)) + labs(x="Event type", y="Total economic cost in USD (millions)")+ggtitle("Total economic cost per event type USD (millions)")+ scale_y_continuous(labels = comma)
Extreme winds, which includes gusty winds, thunderstorm winds, gradient winds and various other strong wind event types has caused by far the most total damage, with over USD 2.5 billion worth of damage in the 15 year period.
Following that, there are four event types which caused between USD 1 billion and 1.5 billion for the period. These are Flash Floods, Other Floods, Tornado and Hail/ Sleet.
Thuderstorm/ Lightning caused around USD 0.5 billion damage.