An analysis of Severe US Weather Events from 1950 to 2011 using NOAA Storm Database

Synopsis

This is an analysis of US weather events from 1950 to 2011 using the NOAA Storm Database to answer to questions:
(1) Across the United States, which types of events are most harmful with respect to population health?
(2) Across the United States, which types of events have the greatest economic consequences?
The analysis looks at these questions with respect to the magnitude of storm events with respect to the frequencies of occurances and the magnitude of damage, whether in human or economic terms. The analysis also examines the distribution of these types of storm events in terms of the number of US states where this type of event has been recorded.

The analysis finds key differences in event magnitude when examined in terms of human or economic costs, but identifies the most serious weather events as: Floods, Hurricanes, Tornadoes, Heat Waves and Thunderstorms.

Data Processing

The following R libraries are used for this analysis:

library(plyr)
library(ggplot2)
library(scales)

The Storm Tracker Database is provided in a compressed .csv.bz2 file. This file can be read directly from a datastream provided by bzfile() function, passed to read.csv() The dataset size is 902297 observations by 41 variables

if (!exists("storm")) storm<-read.csv(bzfile(description="./repdata-data-StormData.csv.bz2"),sep=",",header=TRUE,stringsAsFactors=FALSE)

Our analysis focuses on questions related to economic and human costs of storms. The dataset classifies cost data according to property damage and crop damage, recorded in the respective variables PROPDMG and CROPDMG. Two additional variables, PROPDMGEXP and CROPDMGEXP, provide unit multiplyers for the damage data. For example, PROPDMG=25 and PROPDMGEXP=K indicates a total property damage value of $25,000.

Considerations on data transformations:

This aspect required considerable data processing as the dataset contains a number of inconsistencies. The following steps were taken:
- Normalizing multiplier variables to upper case (k vs K or m vs M)
- Filling empty multipliers where damage values are present: here, empty values were filled with K, the 1,000s unit. This was (a) The most common unit within the dataset and (b) Reviewing the REMARKS data validated this in a few instances where the damage value was provided
- Establishing multiplier values that are not defined in the data documentation. The documentation defines K,M,B as thousands, millions and billions, respectively; but the set of multipliers include additional values. Values were assigned as defined below in exp_map
- Note: although the multipliers were assigned based on assumption, the number of instances of those values are very infrequent within the dataset; it’s unlikely that the aggregate result will change significantly, regardless of whether these values are included, excluded or set according to the assumptions specified in exp_map
- The EVTYPE variable that defines the weather event type required cleaning. Here, many entries included typos or spelling permutations of similar weather events. Such as " FLOOD" (with a space) versus “FLOOD” or “FLASH FLOOD”, “FLASH/FLOOD”, “FLOOD/FLASH”, etc.
- In addition to data entry permutations, many event types are unnecessarily granular for the purposes of this analysis, or ambiguously specific in some instances and non-specific in others. Such as classifying thunderstorms by wind velocity vs unspecified windspeeds, or size of hail vs general hail storms without size precision. Therefore, the weather events were re-classified into more general categories to analyze. This was a manual process that can be seen in the categories variable below.

Normalizing to upper case and filling in empty values with 1000s multiplier:

storm$EVTYPE <- toupper(storm$EVTYPE) #Some same-type event types are encoded in different case, which R differentiates as separate events.
storm$PROPDMGEXP <- toupper(storm$PROPDMGEXP) #Some EXP codes are mixed upper/lower, eg kK or mM
storm$CROPDMGEXP <- toupper(storm$CROPDMGEXP)

storm$PROPDMGEXP[storm$PROPDMGEXP=="" & storm$PROPDMG>0] <- "K" ## Blank entries are assumed to be omissions in magnitude exp, set as K as the most reasonable fill value
storm$CROPDMGEXP[storm$CROPDMGEXP=="" & storm$CROPDMG>0] <- "K" ## Same for crop damages

Establishing the mapping table for the multiplier values:

exp_map <- list(
     "?"=1,
     "+"=1,
     "-"=1,
     "H"=100,
     "K"=1000,
     "M"=1000000,
     "B"=1000000000,
     "0"=10^0,  
     "1"=10^1,
     "2"=10^2,
     "3"=10^3,
     "4"=10^4,
     "5"=10^5,
     "6"=10^6,
     "7"=10^7,
     "8"=10^8,
     "NA"=0) ##NA for NA values; "NA" is irrelevant; rather, it's referenced at index 17 for match's "nomatch" value.

Recategorizing weather event types to fix data entry issues and create more general analytic groupings:

categories <- data.frame(
EVTYPE = c("   HIGH SURF ADVISORY"," FLASH FLOOD"," TSTM WIND"," TSTM WIND (G45)","?","AGRICULTURAL FREEZE","APACHE COUNTY","ASTRONOMICAL HIGH TIDE","ASTRONOMICAL LOW TIDE",
       "AVALANCE","AVALANCHE","BEACH EROSION","BLACK ICE","BLIZZARD","BLIZZARD/WINTER STORM","BLOWING DUST","BLOWING SNOW","BREAKUP FLOODING","BRUSH FIRE","COASTAL  FLOODING/EROSION",
       "COASTAL EROSION","COASTAL FLOOD","COASTAL FLOODING","COASTAL FLOODING/EROSION","COASTAL STORM","COASTAL SURGE","COASTALSTORM","COLD","COLD AIR TORNADO","COLD AND SNOW",
       "COLD AND WET CONDITIONS","COLD TEMPERATURE","COLD WAVE","COLD WEATHER","COLD/WIND CHILL","COLD/WINDS","COOL AND WET","DAM BREAK","DAMAGING FREEZE","DENSE FOG","DENSE SMOKE",
       "DOWNBURST","DROUGHT","DROUGHT/EXCESSIVE HEAT","DROWNING","DRY MICROBURST","DRY MIRCOBURST WINDS","DUST DEVIL","DUST DEVIL WATERSPOUT","DUST STORM","DUST STORM/HIGH WINDS",
       "EARLY FROST","EROSION/CSTL FLOOD","EXCESSIVE HEAT","EXCESSIVE RAINFALL","EXCESSIVE SNOW","EXCESSIVE WETNESS","EXTENDED COLD","EXTREME COLD","EXTREME COLD/WIND CHILL",
       "EXTREME HEAT","EXTREME WIND CHILL","EXTREME WINDCHILL","FALLING SNOW/ICE","FLASH FLOOD","FLASH FLOOD - HEAVY RAIN","FLASH FLOOD FROM ICE JAMS","FLASH FLOOD LANDSLIDES",
       "FLASH FLOOD WINDS","FLASH FLOOD/","FLASH FLOOD/ STREET","FLASH FLOOD/FLOOD","FLASH FLOOD/LANDSLIDE","FLASH FLOODING","FLASH FLOODING/FLOOD","FLASH FLOODING/THUNDERSTORM WI",
       "FLASH FLOODS","FLOOD","FLOOD & HEAVY RAIN","FLOOD FLASH","FLOOD/FLASH","FLOOD/FLASH FLOOD","FLOOD/FLASH/FLOOD","FLOOD/FLASHFLOOD","FLOOD/RAIN/WINDS","FLOOD/RIVER FLOOD",
       "FLOODING","FLOODING/HEAVY RAIN","FLOODS","FOG","FOG AND COLD TEMPERATURES","FOREST FIRES","FREEZE","FREEZING DRIZZLE","FREEZING FOG","FREEZING RAIN","FREEZING RAIN/SLEET",
       "FREEZING RAIN/SNOW","FREEZING SPRAY","FROST","FROST/FREEZE","FROST\\FREEZE","FROST\\\\FREEZE","FUNNEL CLOUD","GLAZE","GLAZE ICE","GLAZE/ICE STORM","GRADIENT WIND",
       "GRASS FIRES","GROUND BLIZZARD","GUSTNADO","GUSTY WIND","GUSTY WIND/HAIL","GUSTY WIND/HVY RAIN","GUSTY WIND/RAIN","GUSTY WINDS","HAIL","HAIL 0.75","HAIL 075",
       "HAIL 100","HAIL 125","HAIL 150","HAIL 175","HAIL 200","HAIL 275","HAIL 450","HAIL 75","HAIL DAMAGE","HAIL/WIND","HAIL/WINDS","HAILSTORM","HARD FREEZE","HAZARDOUS SURF",
       "HEAT","HEAT WAVE","HEAT WAVE DROUGHT","HEAT WAVES","HEAVY LAKE SNOW","HEAVY MIX","HEAVY PRECIPITATION","HEAVY RAIN","HEAVY RAIN AND FLOOD","HEAVY RAIN/HIGH SURF",
       "HEAVY RAIN/LIGHTNING","HEAVY RAIN/SEVERE WEATHER","HEAVY RAIN/SMALL STREAM URBAN","HEAVY RAIN/SNOW","HEAVY RAINS","HEAVY RAINS/FLOODING","HEAVY SEAS","HEAVY SHOWER",
       "HEAVY SNOW","HEAVY SNOW AND HIGH WINDS","HEAVY SNOW AND STRONG WINDS","HEAVY SNOW SHOWER","HEAVY SNOW SQUALLS","HEAVY SNOW/BLIZZARD","HEAVY SNOW/BLIZZARD/AVALANCHE",
       "HEAVY SNOW/FREEZING RAIN","HEAVY SNOW/HIGH WINDS & FLOOD","HEAVY SNOW/ICE","HEAVY SNOW/SQUALLS","HEAVY SNOW/WIND","HEAVY SNOW/WINTER STORM","HEAVY SNOWPACK","HEAVY SNOW-SQUALLS",
       "HEAVY SURF","HEAVY SURF AND WIND","HEAVY SURF COASTAL FLOODING","HEAVY SURF/HIGH SURF","HEAVY SWELLS","HIGH","HIGH  WINDS","HIGH SEAS","HIGH SURF","HIGH SWELLS","HIGH TIDES",
       "HIGH WATER","HIGH WAVES","HIGH WIND","HIGH WIND (G40)","HIGH WIND 48","HIGH WIND AND SEAS","HIGH WIND DAMAGE","HIGH WIND/BLIZZARD","HIGH WIND/HEAVY SNOW","HIGH WIND/SEAS",
       "HIGH WINDS","HIGH WINDS HEAVY RAINS","HIGH WINDS/","HIGH WINDS/COASTAL FLOOD","HIGH WINDS/COLD","HIGH WINDS/HEAVY RAIN","HIGH WINDS/SNOW","HURRICANE","HURRICANE EDOUARD",
       "HURRICANE EMILY","HURRICANE ERIN","HURRICANE FELIX","HURRICANE GORDON","HURRICANE OPAL","HURRICANE OPAL/HIGH WINDS","HURRICANE/TYPHOON","HURRICANE-GENERATED SWELLS",
       "HVY RAIN","HYPERTHERMIA/EXPOSURE","HYPOTHERMIA","HYPOTHERMIA/EXPOSURE","ICE","ICE AND SNOW","ICE FLOES","ICE JAM","ICE JAM FLOOD (MINOR","ICE JAM FLOODING",
       "ICE ON ROAD","ICE ROADS","ICE STORM","ICE STORM/FLASH FLOOD","ICE/STRONG WINDS","ICY ROADS","LAKE EFFECT SNOW","LAKE FLOOD","LAKE-EFFECT SNOW","LAKESHORE FLOOD",
       "LANDSLIDE","LANDSLIDES","LANDSLUMP","LANDSPOUT","LATE SEASON SNOW","LIGHT FREEZING RAIN","LIGHT SNOW","LIGHT SNOWFALL","LIGHTING","LIGHTNING","LIGHTNING  WAUSEON",
       "LIGHTNING AND HEAVY RAIN","LIGHTNING AND THUNDERSTORM WIN","LIGHTNING FIRE","LIGHTNING INJURY","LIGHTNING THUNDERSTORM WINDS","LIGHTNING.","LIGHTNING/HEAVY RAIN",
       "LIGNTNING","LOW TEMPERATURE","MAJOR FLOOD","MARINE ACCIDENT","MARINE HAIL","MARINE HIGH WIND","MARINE MISHAP","MARINE STRONG WIND","MARINE THUNDERSTORM WIND",
       "MARINE TSTM WIND","MICROBURST","MICROBURST WINDS","MINOR FLOODING","MIXED PRECIP","MIXED PRECIPITATION","MUD SLIDE","MUD SLIDES","MUD SLIDES URBAN FLOODING",
       "MUDSLIDE","MUDSLIDES","NON TSTM WIND","NON-SEVERE WIND DAMAGE","NON-TSTM WIND","OTHER","RAIN","RAIN/SNOW","RAIN/WIND","RAINSTORM","RAPIDLY RISING WATER",
       "RECORD COLD","RECORD HEAT","RECORD RAINFALL","RECORD SNOW","RECORD/EXCESSIVE HEAT","RIP CURRENT","RIP CURRENTS","RIP CURRENTS/HEAVY SURF","RIVER AND STREAM FLOOD",
       "RIVER FLOOD","RIVER FLOODING","ROCK SLIDE","ROGUE WAVE","ROUGH SEAS","ROUGH SURF","RURAL FLOOD","SEICHE","SEVERE THUNDERSTORM","SEVERE THUNDERSTORM WINDS",
       "SEVERE THUNDERSTORMS","SEVERE TURBULENCE","SLEET","SLEET/ICE STORM","SMALL HAIL","SMALL STREAM FLOOD","SNOW","SNOW ACCUMULATION","SNOW AND HEAVY SNOW",
       "SNOW AND ICE","SNOW AND ICE STORM","SNOW FREEZING RAIN","SNOW SQUALL","SNOW SQUALLS","SNOW/ BITTER COLD","SNOW/ ICE","SNOW/BLOWING SNOW","SNOW/COLD",
       "SNOW/FREEZING RAIN","SNOW/HEAVY SNOW","SNOW/HIGH WINDS","SNOW/ICE","SNOW/ICE STORM","SNOW/SLEET","SNOW/SLEET/FREEZING RAIN","SNOWMELT FLOODING","STORM FORCE WINDS",
       "STORM SURGE","STORM SURGE/TIDE","STRONG WIND","STRONG WINDS","THUDERSTORM WINDS","THUNDEERSTORM WINDS","THUNDERESTORM WINDS","THUNDERSNOW","THUNDERSTORM",
       "THUNDERSTORM  WINDS","THUNDERSTORM DAMAGE TO","THUNDERSTORM HAIL","THUNDERSTORM WIND","THUNDERSTORM WIND (G40)","THUNDERSTORM WIND 60 MPH",
       "THUNDERSTORM WIND 65 MPH","THUNDERSTORM WIND 65MPH","THUNDERSTORM WIND 98 MPH","THUNDERSTORM WIND G50","THUNDERSTORM WIND G52","THUNDERSTORM WIND G55",
       "THUNDERSTORM WIND G60","THUNDERSTORM WIND TREES","THUNDERSTORM WIND.","THUNDERSTORM WIND/ TREE","THUNDERSTORM WIND/ TREES","THUNDERSTORM WIND/AWNING",
       "THUNDERSTORM WIND/HAIL","THUNDERSTORM WIND/LIGHTNING","THUNDERSTORM WINDS","THUNDERSTORM WINDS 13","THUNDERSTORM WINDS 63 MPH","THUNDERSTORM WINDS AND",
       "THUNDERSTORM WINDS G60","THUNDERSTORM WINDS HAIL","THUNDERSTORM WINDS LIGHTNING","THUNDERSTORM WINDS.","THUNDERSTORM WINDS/ FLOOD","THUNDERSTORM WINDS/FLOODING",
       "THUNDERSTORM WINDS/FUNNEL CLOU","THUNDERSTORM WINDS/HAIL","THUNDERSTORM WINDS53","THUNDERSTORM WINDSHAIL","THUNDERSTORM WINDSS","THUNDERSTORM WINS","THUNDERSTORMS",
       "THUNDERSTORMS WIND","THUNDERSTORMS WINDS","THUNDERSTORMW","THUNDERSTORMWINDS","THUNDERSTROM WIND","THUNDERTORM WINDS","THUNERSTORM WINDS","TIDAL FLOODING","TORNADO",
       "TORNADO F0","TORNADO F1","TORNADO F2","TORNADO F3","TORNADOES","TORNADOES, TSTM WIND, HAIL","TORNDAO","TORRENTIAL RAINFALL","TROPICAL DEPRESSION","TROPICAL STORM",
       "TROPICAL STORM ALBERTO","TROPICAL STORM DEAN","TROPICAL STORM GORDON","TROPICAL STORM JERRY","TSTM WIND","TSTM WIND  (G45)","TSTM WIND (41)","TSTM WIND (G35)",
       "TSTM WIND (G40)","TSTM WIND (G45)","TSTM WIND 40","TSTM WIND 45","TSTM WIND 55","TSTM WIND 65)","TSTM WIND AND LIGHTNING","TSTM WIND DAMAGE","TSTM WIND G45",
       "TSTM WIND G58","TSTM WIND/HAIL","TSTM WINDS","TSTMW","TSUNAMI","TUNDERSTORM WIND","TYPHOON","UNSEASONABLE COLD","UNSEASONABLY COLD","UNSEASONABLY WARM",
       "UNSEASONABLY WARM AND DRY","UNSEASONAL RAIN","URBAN AND SMALL","URBAN AND SMALL STREAM FLOODIN","URBAN FLOOD","URBAN FLOODING","URBAN FLOODS","URBAN SMALL",
       "URBAN/SMALL STREAM","URBAN/SMALL STREAM FLOOD","URBAN/SML STREAM FLD","VOLCANIC ASH","WARM WEATHER","WATERSPOUT","WATERSPOUT-","WATERSPOUT TORNADO",
       "WATERSPOUT/ TORNADO","WATERSPOUT/TORNADO","WATERSPOUT-TORNADO","WET MICROBURST","WHIRLWIND","WILD FIRES","WILD/FOREST FIRE","WILD/FOREST FIRES","WILDFIRE",
       "WILDFIRES","WIND","WIND AND WAVE","WIND DAMAGE","WIND STORM","WIND/HAIL","WINDS","WINTER STORM","WINTER STORM HIGH WINDS","WINTER STORMS","WINTER WEATHER",
       "WINTER WEATHER MIX","WINTER WEATHER/MIX","WINTRY MIX"),

EV_CATEGORY = c("HIGH SURF/SEAS","FLASH/URBAN FLOOD","HIGH WINDS","HIGH WINDS","UNKNOWN/OTHER","FREEZE/FROST","UNKNOWN/OTHER","HIGH SURF/SEAS","HIGH SURF/SEAS",
                 "AVALANCHE","AVALANCHE","COASTAL FLOODING/EROSION/STORM","ICE/ICY CONDITIONS","BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM",
                 "TORNADO/WATERSPOUT/LANDSPOUD","BLIZZARD/WINTER STORM","FLOOD","FOREST/WILD FIRE","COASTAL FLOODING/EROSION/STORM","COASTAL FLOODING/EROSION/STORM",
                 "COASTAL FLOODING/EROSION/STORM","COASTAL FLOODING/EROSION/STORM","COASTAL FLOODING/EROSION/STORM","COASTAL FLOODING/EROSION/STORM","COASTAL FLOODING/EROSION/STORM",
                 "COASTAL FLOODING/EROSION/STORM","COLD TEMPERATURES","TORNADO/WATERSPOUT/LANDSPOUD","SNOWSTORM","COLD TEMPERATURES","COLD TEMPERATURES","COLD TEMPERATURES","COLD TEMPERATURES","COLD TEMPERATURES","COLD TEMPERATURES","HEAVY RAIN","UNKNOWN/OTHER","FREEZE/FROST","FOG","UNKNOWN/OTHER","THUNDERSTORM, VARIOUS","DROUGHT/EXCESSIVE HEAT","DROUGHT/EXCESSIVE HEAT","UNKNOWN/OTHER","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD","FREEZE/FROST","COASTAL FLOODING/EROSION/STORM","HEAT WAVE","HEAVY RAIN","SNOWSTORM","HEAVY RAIN","COLD TEMPERATURES","COLD TEMPERATURES","COLD TEMPERATURES","HEAT WAVE","COLD TEMPERATURES","COLD TEMPERATURES","SNOWSTORM","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","LAND, MUD, ROCK SLIDES","HIGH WINDS","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","LAND, MUD, ROCK SLIDES","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLOOD","FLOOD","FLOOD","FLOOD","FOG","FOG","FOREST/WILD FIRE","FREEZE/FROST","FREEZING RAIN/SLEET/MIX","FREEZING RAIN/SLEET/MIX","FREEZING RAIN/SLEET/MIX","FREEZING RAIN/SLEET/MIX","FREEZING RAIN/SLEET/MIX","FREEZING RAIN/SLEET/MIX","FREEZE/FROST","FREEZE/FROST","FREEZE/FROST","FREEZE/FROST","TORNADO/WATERSPOUT/LANDSPOUD","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","HIGH WINDS","FOREST/WILD FIRE","BLIZZARD/WINTER STORM","TORNADO/WATERSPOUT/LANDSPOUD","HIGH WINDS","HIGH WINDS","HIGH WINDS","HIGH WINDS","HIGH WINDS","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","HAIL, VARIOUS SIZES","FREEZE/FROST","HIGH SURF/SEAS","HEAT WAVE","HEAT WAVE","DROUGHT/EXCESSIVE HEAT","HEAT WAVE","SNOWSTORM","FREEZING RAIN/SLEET/MIX","HEAVY RAIN","HEAVY RAIN","FLASH/URBAN FLOOD","HIGH SURF/SEAS","HEAVY RAIN","HEAVY RAIN","FLASH/URBAN FLOOD","FREEZING RAIN/SLEET/MIX","HEAVY RAIN","FLASH/URBAN FLOOD","HIGH SURF/SEAS","HEAVY RAIN","SNOWSTORM","BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM","SNOWSTORM","BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM","SNOWSTORM","BLIZZARD/WINTER STORM","SNOWSTORM","SNOWSTORM","BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM","SNOWSTORM","SNOWSTORM","HIGH SURF/SEAS","HIGH SURF/SEAS","COASTAL FLOODING/EROSION/STORM","HIGH SURF/SEAS","HIGH SURF/SEAS","HIGH WINDS","HIGH WINDS","HIGH SURF/SEAS","HIGH SURF/SEAS","HIGH SURF/SEAS","HIGH SURF/SEAS","HIGH SURF/SEAS","HIGH SURF/SEAS","HIGH WINDS","HIGH WINDS","HIGH WINDS","HIGH SURF/SEAS","HIGH WINDS","BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM","HIGH SURF/SEAS","HIGH WINDS","THUNDERSTORM, VARIOUS","HIGH WINDS","COASTAL FLOODING/EROSION/STORM","HIGH WINDS","HIGH WINDS","BLIZZARD/WINTER STORM","HURRICANE","HURRICANE","HURRICANE","HURRICANE","HURRICANE","HURRICANE","HURRICANE","HURRICANE","HURRICANE","HURRICANE","HEAVY RAIN","COLD TEMPERATURES","COLD TEMPERATURES","COLD TEMPERATURES","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","ICE/ICY CONDITIONS","SNOWSTORM","FLOOD","SNOWSTORM","FLOOD","LAND, MUD, ROCK SLIDES","LAND, MUD, ROCK SLIDES","LAND, MUD, ROCK SLIDES","TORNADO/WATERSPOUT/LANDSPOUD","SNOWSTORM","FREEZING RAIN/SLEET/MIX","SNOWSTORM","SNOWSTORM","LIGHTNING","LIGHTNING","LIGHTNING","LIGHTNING","LIGHTNING","LIGHTNING","LIGHTNING","LIGHTNING","LIGHTNING","LIGHTNING","LIGHTNING","COLD TEMPERATURES","FLOOD","UNKNOWN/OTHER","MARINE STORM","MARINE STORM","UNKNOWN/OTHER","MARINE STORM","MARINE STORM","MARINE STORM",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","FLASH/URBAN FLOOD","FREEZING RAIN/SLEET/MIX","FREEZING RAIN/SLEET/MIX","LAND, MUD, ROCK SLIDES",
                "LAND, MUD, ROCK SLIDES","LAND, MUD, ROCK SLIDES","LAND, MUD, ROCK SLIDES","LAND, MUD, ROCK SLIDES","HIGH WINDS","HIGH WINDS","HIGH WINDS","UNKNOWN/OTHER",
                "HEAVY RAIN","SNOWSTORM","THUNDERSTORM, VARIOUS","HEAVY RAIN","FLOOD","COLD TEMPERATURES","HEAT WAVE","HEAVY RAIN","SNOWSTORM","HEAT WAVE",
                "HIGH SURF/SEAS","HIGH SURF/SEAS","HIGH SURF/SEAS","FLOOD","FLOOD","FLOOD","LAND, MUD, ROCK SLIDES","HIGH SURF/SEAS","HIGH SURF/SEAS","HIGH SURF/SEAS",
                "FLOOD","TIDAL WAVE/TYPHOON","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","HIGH WINDS","FREEZING RAIN/SLEET/MIX",
                "ICE/ICY CONDITIONS","HAIL, VARIOUS SIZES","FLASH/URBAN FLOOD","SNOWSTORM","SNOWSTORM","SNOWSTORM","ICE/ICY CONDITIONS","SNOWSTORM",
                "FREEZING RAIN/SLEET/MIX","SNOWSTORM","SNOWSTORM","SNOWSTORM","ICE/ICY CONDITIONS","SNOWSTORM","SNOWSTORM","FREEZING RAIN/SLEET/MIX","SNOWSTORM",
                "BLIZZARD/WINTER STORM","ICE/ICY CONDITIONS","BLIZZARD/WINTER STORM","SNOWSTORM","SNOWSTORM","FLOOD","HIGH WINDS","COASTAL FLOODING/EROSION/STORM",
                "COASTAL FLOODING/EROSION/STORM","HIGH WINDS","HIGH WINDS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","COASTAL FLOODING/EROSION/STORM","TORNADO/WATERSPOUT/LANDSPOUD",
                "TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD",
                "TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD","HEAVY RAIN","TROPICAL STORM","TROPICAL STORM","TROPICAL STORM","TROPICAL STORM",
                "TROPICAL STORM","TROPICAL STORM","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS","THUNDERSTORM, VARIOUS",
                "TIDAL WAVE/TYPHOON","THUNDERSTORM, VARIOUS","TIDAL WAVE/TYPHOON","COLD TEMPERATURES","COLD TEMPERATURES","HEAT WAVE","HEAT WAVE","HEAVY RAIN",
                "FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","FLASH/URBAN FLOOD",
                "FLASH/URBAN FLOOD","FLASH/URBAN FLOOD","UNKNOWN/OTHER","HEAT WAVE","TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD",
                "TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD","TORNADO/WATERSPOUT/LANDSPOUD","THUNDERSTORM, VARIOUS",
                "TORNADO/WATERSPOUT/LANDSPOUD","FOREST/WILD FIRE","FOREST/WILD FIRE","FOREST/WILD FIRE","FOREST/WILD FIRE","FOREST/WILD FIRE","HIGH WINDS","HIGH SURF/SEAS",
                "HIGH WINDS","HIGH WINDS","HIGH WINDS","HIGH WINDS","BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM",
                "BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM","BLIZZARD/WINTER STORM"),

COLOR = c("darkseagreen1","cadetblue","lavender","lavender","black","cyan","black","darkseagreen1","darkseagreen1","ghostwhite","ghostwhite","blanchedalmond","cadetblue1",
          "azure","azure","beige","azure","cornflowerblue","chocolate1","blanchedalmond","blanchedalmond","blanchedalmond","blanchedalmond","blanchedalmond",
          "blanchedalmond","blanchedalmond","blanchedalmond","dodgerblue","beige","gainsboro","dodgerblue","dodgerblue","dodgerblue","dodgerblue","dodgerblue",
          "dodgerblue","darkslateblue","black","cyan","gray90","black","lavenderblush4","orange","orange","black","lavenderblush4","lavenderblush4","beige","beige",
          "beige","beige","cyan","blanchedalmond","firebrick1","darkslateblue","gainsboro","darkslateblue","dodgerblue","dodgerblue","dodgerblue","firebrick1",
          "dodgerblue","dodgerblue","gainsboro","cadetblue","cadetblue","cadetblue","tan3","lavender","cadetblue","cadetblue","cadetblue","tan3","cadetblue","cadetblue",
          "cadetblue","cadetblue","cornflowerblue","cadetblue","cadetblue","cadetblue","cadetblue","cadetblue","cadetblue","cadetblue","cornflowerblue","cornflowerblue",
          "cornflowerblue","cornflowerblue","gray90","gray90","chocolate1","cyan","slategray2","slategray2","slategray2","slategray2","slategray2","slategray2","cyan",
          "cyan","cyan","cyan","beige","cadetblue1","cadetblue1","cadetblue1","lavender","chocolate1","azure","beige","lavender","lavender","lavender","lavender",
          "lavender","skyblue1","skyblue1","skyblue1","skyblue1","skyblue1","skyblue1","skyblue1","skyblue1","skyblue1","skyblue1","skyblue1","skyblue1","skyblue1",
          "skyblue1","skyblue1","cyan","darkseagreen1","firebrick1","firebrick1","orange","firebrick1","gainsboro","slategray2","darkslateblue","darkslateblue",
          "cadetblue","darkseagreen1","darkslateblue","darkslateblue","cadetblue","slategray2","darkslateblue","cadetblue","darkseagreen1","darkslateblue","gainsboro",
          "azure","azure","gainsboro","azure","azure","azure","gainsboro","azure","gainsboro","gainsboro","azure","azure","gainsboro","gainsboro","darkseagreen1",
          "darkseagreen1","blanchedalmond","darkseagreen1","darkseagreen1","lavender","lavender","darkseagreen1","darkseagreen1","darkseagreen1","darkseagreen1",
          "darkseagreen1","darkseagreen1","lavender","lavender","lavender","darkseagreen1","lavender","azure","azure","darkseagreen1","lavender","lavenderblush4",
          "lavender","blanchedalmond","lavender","lavender","azure","navy","navy","navy","navy","navy","navy","navy","navy","navy","navy","darkslateblue","dodgerblue",
          "dodgerblue","dodgerblue","cadetblue1","cadetblue1","cadetblue1","cadetblue1","cadetblue1","cadetblue1","cadetblue1","cadetblue1","cadetblue1","cadetblue1",
          "cadetblue1","cadetblue1","gainsboro","cornflowerblue","gainsboro","cornflowerblue","tan3","tan3","tan3","beige","gainsboro","slategray2","gainsboro",
          "gainsboro","slategray3","slategray3","slategray3","slategray3","slategray3","slategray3","slategray3","slategray3","slategray3","slategray3","slategray3",
          "dodgerblue","cornflowerblue","black","turquoise4","turquoise4","black","turquoise4","turquoise4","turquoise4","lavenderblush4","lavenderblush4","cadetblue",
          "slategray2","slategray2","tan3","tan3","tan3","tan3","tan3","lavender","lavender","lavender","black","darkslateblue","gainsboro","lavenderblush4",
          "darkslateblue","cornflowerblue","dodgerblue","firebrick1","darkslateblue","gainsboro","firebrick1","darkseagreen1","darkseagreen1","darkseagreen1",
          "cornflowerblue","cornflowerblue","cornflowerblue","tan3","darkseagreen1","darkseagreen1","darkseagreen1","cornflowerblue","violet","lavenderblush4",
          "lavenderblush4","lavenderblush4","lavender","slategray2","cadetblue1","skyblue1","cadetblue","gainsboro","gainsboro","gainsboro","cadetblue1","gainsboro",
          "slategray2","gainsboro","gainsboro","gainsboro","cadetblue1","gainsboro","gainsboro","slategray2","gainsboro","azure","cadetblue1","azure","gainsboro",
          "gainsboro","cornflowerblue","lavender","blanchedalmond","blanchedalmond","lavender","lavender","lavenderblush4","lavenderblush4","lavenderblush4",
          "lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4",
          "lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4",
          "lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4",
          "lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4",
          "lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4",
          "lavenderblush4","blanchedalmond","beige","beige","beige","beige","beige","beige","beige","beige","darkslateblue","seagreen","seagreen","seagreen","seagreen",
          "seagreen","seagreen","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4",
          "lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4","lavenderblush4",
          "violet","lavenderblush4","violet","dodgerblue","dodgerblue","firebrick1","firebrick1","darkslateblue","cadetblue","cadetblue","cadetblue","cadetblue",
          "cadetblue","cadetblue","cadetblue","cadetblue","cadetblue","black","firebrick1","beige","beige","beige","beige","beige","beige","lavenderblush4","beige",
          "chocolate1","chocolate1","chocolate1","chocolate1","chocolate1","lavender","darkseagreen1","lavender","lavender","lavender","lavender","azure","azure","azure",
          "azure","azure","azure","azure"))

Analysis

The analysis does the following steps (1) Maps the exp_map table to the matching exponents for property and crop damage, mutiplied by the damage coefficient to get damage cost (2) Create a total DMG_VALUE variable, the sum of property and crop damages (3) Creates a HAS_EFFECT variable, a binary value wether a storm entry has an injury, a fatality or any damage cost (4) A subet data frame, storm_effects, is created – subsetted for only data with effects (5) The category data is bound to the new data frame

storm$PROPDMG_VALUE <- storm$PROPDMG * unlist(exp_map,use.names=F)[match(storm$PROPDMGEXP,names(exp_map),nomatch=17)]
storm$CROPDMG_VALUE <- storm$CROPDMG * unlist(exp_map,use.names=F)[match(storm$CROPDMGEXP,names(exp_map),nomatch=17)]
storm$DMG_VALUE <- storm$PROPDMG_VALUE + storm$CROPDMG_VALUE
storm$HAS_EFFECT <- (storm$PROPDMG_VALUE + storm$CROPDMG_VALUE + storm$INJURIES + storm$FATALITIES)>0

storm_effects <- subset(storm,storm$HAS_EFFECT==T)
storm_effects <- cbind(storm_effects,categories[match(storm_effects$EVTYPE,categories$EVTYPE),c("EV_CATEGORY","COLOR")])

Human and economic cost analysis is done by: (1) Aggregating fatalities and injuries by the EV_CATEGORY, and greating an additional value PEOPLE HARMED variable, the sum of fataliies and injuries (2) Two additional aggregate variables are created, FREQUENCY and AFFECTEDSTATES. These look at the frequency of events, ie, the total number of occurances over the analysis period (3) Some additional sorting and labeling is done for graphing needs (3) The same steps are performed for eonomic value

health_by_event <- ddply(storm_effects, .(EV_CATEGORY,COLOR), summarize, FATALITIES=sum(FATALITIES,na.rm=T), INJURIES=sum(INJURIES,na.rm=T), PEOPLE_HARMED=sum(FATALITIES,na.rm=T)+sum(INJURIES,na.rm=T), FREQUENCY=length(EV_CATEGORY), AFFECTED_STATES=length(unique(STATE)))
health_by_event$severity <- health_by_event$PEOPLE_HARMED/health_by_event$FREQUENCY
health_by_event<-health_by_event[order(health_by_event$PEOPLE_HARMED, decreasing=T),]
health_by_event$LABELS <- paste(1:dim(health_by_event)[1],health_by_event$EV_CATEGORY)

cost_by_event <- ddply(storm_effects, .(EV_CATEGORY,COLOR), summarize, PROPDMG_VALUE=sum(PROPDMG_VALUE,na.rm=T), CROPDMG_VALUE=sum(CROPDMG_VALUE,na.rm=T), DMG_VALUE=sum(PROPDMG_VALUE,na.rm=T)+sum(CROPDMG_VALUE,na.rm=T), FREQUENCY=length(EV_CATEGORY), AFFECTED_STATES=length(unique(STATE)))
cost_by_event$severity <- cost_by_event$DMG_VALUE/cost_by_event$FREQUENCY
cost_by_event<-cost_by_event[order(cost_by_event$DMG_VALUE, decreasing=T),]
cost_by_event$LABELS <- paste(1:dim(cost_by_event)[1],cost_by_event$EV_CATEGORY)

Results

In terms of economic costs, floods are the most harmful, having wreked over $160 billion in damages during the time period, with more than 10,000 occurances. Hurricanes are far less frequent, with only 224 major events recorded; but their cost is extreme, causing about $900 million in damage. Finally, Tornadoes of various types have brought nearly $600 million in damages, with over 40,000 occurances. Thunderstorms, while far less damaging are also much more frequent, with nearly 120,000 recorded.

Human costs compare differently, with Tornados causing the most serious injuries/fatalities to over 97,000 people. Heat waves follow, having affected over 12,000 people; and thunderstorms are the 3rd most harmful, having affected nearly 10,000 people. Flash floods are also of note, having caused over 1000 fatalities, the 3rd most significant in this respect, after Tornados and Heat Waves.

Below, a few notes on interpreting the graphs: These graphs plot frequency of event occurance on the x-axis, which has a log scale. The y-axis is similarly plotted with a log scale, respecitvely plotting number of people harmed (sum of injuries and fatalities) and the total USD costs of events. Additionally, graphs attempt to scale points according to the number of states in which these events are experienced: larger dots indicate the event is experienced in most/all of the US states; whereas smaller dots show that the weather event is more regionally specific. For example, tital waves/typoons are recorded in onyl 8 US states; heavy rains are ubiquitous throughout the United States. In this regard, it’s worth noting that Hurricanes, while causing significant economic damage, can also be seen to affect a relatively smaller region of the united states compared to floods or tornadoes, which are experienced more widely across most states.

1. Across the United States, which types of events are most harmful with respect to population health?

hp1 <- health_by_event[1:20,]
ggplot(data=hp1, aes(x=(hp1$FREQUENCY), y=(hp1$PEOPLE_HARMED), colour=hp1$LABELS))+
     labs(title="Top 20 Severe US Weather Events: Total Human Costs - 1950 to 2011", x="Frequency: Number of Event Occurances [Log Scale]", y="Severity: Number of People Harmed [Log Scale]")+
     scale_colour_manual(name="Event Category\nPoints sized to frequency across US States\nLarge points occur within most states\nSmall points are isolated to few states", labels=hp1$LABELS, values=rep("black",27), breaks=hp1$LABELS)+
     geom_point(shape=21, size=round(hp1$AFFECTED_STATES/5,1), fill = hp1$COLOR)+
     geom_text(aes(label=1:dim(hp1)[1],vjust=-1.0,fontface="bold"),size=4,colour=1)+
     scale_y_continuous(trans=log10_trans(), labels=comma, breaks=c(100,250,500,1000,2000,5000,10000,20000,50000,100000))+
     scale_x_continuous(trans=log10_trans(), labels=comma,breaks=c(10,100,1000,10000,100000))+
     theme_bw()

Caption: Graph shows top 20 most frequent and harmful storm events in human terms in the US from 1950-2011. See above notes on interpreting graphs.

2. Across the United States, which types of events have the greatest economic consequences?

hp2 <- cost_by_event[1:20,]
ggplot(data=hp2, aes(x=(hp2$FREQUENCY), y=round(hp2$DMG_VALUE/1000000000,10), colour=hp2$LABELS))+
     labs(title="Top 20 Severe US Weather Events: Total Economic Costs - 1950 to 2011", x="Frequency: Number of Event Occurances [Log Scale]", y="Severity: Total Damage Cost (Billions USD) [Log Scale]")+
     scale_colour_manual(name="Event Category\nPoints sized to frequency across US States\nLarge points occur within most states\nSmall points are isolated to few states", labels=hp2$LABELS, values=rep("black",27), breaks=hp2$LABELS)+
     geom_point(shape=21, size=round(hp2$AFFECTED_STATES/5,1), fill = hp2$COLOR)+
     geom_text(aes(label=1:dim(hp2)[1],vjust=-1.0,fontface="bold"),size=4,colour=1)+
     scale_y_continuous(trans=log2_trans(), breaks=c(1,2,5,10,25,50,100,150))+
     scale_x_continuous(trans=log10_trans(), labels=comma, breaks=c(100,1000,10000,100000))+
     theme_bw()

Caption: Graph shows top 20 most frequent and economically harmful storm events in the US from 1950-2011. See above notes on interpreting graphs.