Most Severe Weather Events in the US: Health and Economic Issues
Francisco Marco-Serrano
Synopsis
Since 1996, year from which we have full information on the current 48 events NOAA is recording major storms and weather events information, there have been 653530 of such events in the US. On average, the type of event most harmful with respect to population health is Heat Wave, with a combined average of fatalities and injuries of 70 per recorded event. As per the economic consequences, HURRICANE/TYPHOON is the event generating most losses on average, with a 817.20 M$.
Data Processing
From the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database we obtained data on major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage since 1950 until November 2011.
We first read the data from the bz2 compressed file. The data is a CSV file containing 902297 rows and 37 variables.
storm.df = read.csv("repdata-data-StormData.csv.bz2")
dim(storm.df) # how big the data frame is## [1] 902297 37str(storm.df) # which is the structure## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
## $ BGN_TIME : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
## $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
## $ STATE : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ EVTYPE : Factor w/ 985 levels " HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : Factor w/ 35 levels ""," N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_LOCATI: Factor w/ 54429 levels ""," Christiansburg",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_DATE : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_TIME : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_LOCATI: Factor w/ 34506 levels ""," CANTON"," TULIA",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ WFO : Factor w/ 542 levels ""," CI","%SD",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ ZONENAMES : Factor w/ 25112 levels ""," "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : Factor w/ 436781 levels "","\t","\t\t",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...head(storm.df) # show first few rows## STATE__ BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE
## 1 1 4/18/1950 0:00:00 0130 CST 97 MOBILE AL
## 2 1 4/18/1950 0:00:00 0145 CST 3 BALDWIN AL
## 3 1 2/20/1951 0:00:00 1600 CST 57 FAYETTE AL
## 4 1 6/8/1951 0:00:00 0900 CST 89 MADISON AL
## 5 1 11/15/1951 0:00:00 1500 CST 43 CULLMAN AL
## 6 1 11/15/1951 0:00:00 2000 CST 77 LAUDERDALE AL
## EVTYPE BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END
## 1 TORNADO 0 0
## 2 TORNADO 0 0
## 3 TORNADO 0 0
## 4 TORNADO 0 0
## 5 TORNADO 0 0
## 6 TORNADO 0 0
## COUNTYENDN END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES
## 1 NA 0 14.0 100 3 0 0
## 2 NA 0 2.0 150 2 0 0
## 3 NA 0 0.1 123 2 0 0
## 4 NA 0 0.0 100 2 0 0
## 5 NA 0 0.0 150 2 0 0
## 6 NA 0 1.5 177 2 0 0
## INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES
## 1 15 25.0 K 0
## 2 0 2.5 K 0
## 3 2 25.0 K 0
## 4 2 2.5 K 0
## 5 2 2.5 K 0
## 6 6 2.5 K 0
## LATITUDE LONGITUDE LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1 3040 8812 3051 8806 1
## 2 3042 8755 0 0 2
## 3 3340 8742 0 0 3
## 4 3458 8626 0 0 4
## 5 3412 8642 0 0 5
## 6 3450 8748 0 0 6We make sure the date is formated adequately, and create a new variable (YEAR). Keep only the variables for YEAR, STATE, Event Type, Population Health (FATALITIES plus INJURIES), and Economic Consequences (PROPDMG & PROPDMGEXP for property damages plus CROPDMG & CROPDMGEXP for crop damages).
Only data from 1996 to present is used in the analysis since before not all events were recorded. These are a total of 653530 events.
storm.df$BGN_DATE = as.Date(storm.df$BGN_DATE, format = "%m/%d/%Y %H:%M:%S")
storm.df$YEAR = as.integer(format(storm.df$BGN_DATE, "%Y"))
# subset, to keep only the variables we are interested in
keeps = c("YEAR", "STATE", "EVTYPE", "FATALITIES", "INJURIES", "PROPDMG", "PROPDMGEXP", "CROPDMG", "CROPDMGEXP")
storm.df = storm.df[storm.df$YEAR > 1995 , keeps]
dim(storm.df)## [1] 653530 9We clean the event types (NOOA states only 48 types, while the database reflects 985).
# Trim spaces
library(stringr)
storm.df$EVTYPE = str_trim(storm.df$EVTYPE)
# Normalise some events...
pattern = "(THUNDERSTORM|TSTM)"
events.index = grep(pattern, storm.df$EVTYPE)
storm.df$EVTYPE[events.index] = "THUNDERSTORM"
pattern = "(FLOOD)"
events.index = grep(pattern, storm.df$EVTYPE)
storm.df$EVTYPE[events.index] = "FLOOD"We transform PROPDMG and CROPDMG in USD$ according to PROPDMGEXP and CROPDMGEXP (K for thousands, M for millions, B for billions).
# Normalise PROPDMG & CROPDMG according to the EXP variables
# Calculated in millions
pattern = "(K|k)"
exp.index = grep(pattern, storm.df$PROPDMGEXP)
storm.df$PROPDMG[exp.index] = storm.df$PROPDMG[exp.index] * 0.001
exp.index = grep(pattern, storm.df$CROPDMGEXP)
storm.df$CROPDMG[exp.index] = storm.df$CROPDMG[exp.index] * 0.001
pattern = "(M|m)"
exp.index = grep(pattern, storm.df$PROPDMGEXP)
storm.df$PROPDMG[exp.index] = storm.df$PROPDMG[exp.index] * 1
exp.index = grep(pattern, storm.df$CROPDMGEXP)
storm.df$CROPDMG[exp.index] = storm.df$CROPDMG[exp.index] * 1
pattern = "(B|b)"
exp.index = grep(pattern, storm.df$PROPDMGEXP)
storm.df$PROPDMG[exp.index] = storm.df$PROPDMG[exp.index] * 1000
exp.index = grep(pattern, storm.df$CROPDMGEXP)
storm.df$CROPDMG[exp.index] = storm.df$CROPDMG[exp.index] * 1000We group the data adding the four variables of interest per type of event.
# mean of the 1996-2011 period per event
storm.mean = aggregate(cbind(FATALITIES, INJURIES, PROPDMG, CROPDMG) ~ EVTYPE, data = storm.df, FUN="mean")Results
Show Top 10 events with higher average values for each one of the four variables.
head(storm.mean[order(-storm.mean$FATALITIES),], 10)## EVTYPE FATALITIES INJURIES PROPDMG CROPDMG
## 34 COLD AND SNOW 14.000000 0.000000 0.00000000 0.0000000
## 146 Heavy surf and wind 3.000000 0.000000 0.00000000 0.0000000
## 280 ROUGH SEAS 2.666667 1.666667 0.00000000 0.0000000
## 37 COLD WEATHER 2.000000 0.000000 0.00000000 0.0000000
## 397 TSUNAMI 1.650000 6.450000 7.20310000 0.0010000
## 155 HIGH WATER 1.500000 0.000000 0.00000000 0.0000000
## 167 Hypothermia/Exposure 1.333333 0.000000 0.00000000 0.0000000
## 67 EXCESSIVE HEAT 1.085145 3.859300 0.00466407 0.2973442
## 29 COASTALSTORM 1.000000 0.000000 0.00000000 0.0000000
## 35 Cold Temperature 1.000000 0.000000 0.00000000 0.0000000head(storm.mean[order(-storm.mean$INJURIES),], 10)## EVTYPE FATALITIES INJURIES PROPDMG CROPDMG
## 128 Heat Wave 0.00000000 70.00000 0.000000e+00 0.0000000
## 165 HURRICANE/TYPHOON 0.72727273 14.48864 7.875664e+02 29.6349182
## 458 WINTER WEATHER MIX 0.00000000 11.33333 1.000000e-02 0.0000000
## 109 GLAZE 0.04761905 10.09524 2.857143e-03 0.0000000
## 233 NON-SEVERE WIND DAMAGE 0.00000000 7.00000 5.000000e-03 0.0000000
## 304 SNOW SQUALL 0.40000000 7.00000 6.000000e-03 0.0000000
## 397 TSUNAMI 1.65000000 6.45000 7.203100e+00 0.0010000
## 393 Torrential Rainfall 0.00000000 4.00000 0.000000e+00 0.0000000
## 67 EXCESSIVE HEAT 1.08514493 3.85930 4.664070e-03 0.2973442
## 213 MIXED PRECIP 0.20000000 2.60000 0.000000e+00 0.0000000head(storm.mean[order(-storm.mean$PROPDMG),], 10)## EVTYPE FATALITIES INJURIES PROPDMG CROPDMG
## 165 HURRICANE/TYPHOON 0.727272727 14.48863636 787.56636 2.963492e+01
## 312 STORM SURGE 0.007905138 0.14624506 170.72544 1.976285e-05
## 163 HURRICANE 0.358823529 0.27058824 69.48717 1.612594e+01
## 398 TYPHOON 0.000000000 0.45454545 54.56636 7.500000e-02
## 313 STORM SURGE/TIDE 0.074324324 0.03378378 31.35938 5.743243e-03
## 277 River Flooding 0.000000000 0.20000000 21.23100 5.604000e+00
## 137 Heavy Rain/High Surf 0.000000000 0.00000000 13.50000 1.500000e+00
## 395 TROPICAL STORM 0.083577713 0.49560117 11.20598 9.937111e-01
## 65 Erosion/Cstl Flood 0.000000000 0.00000000 8.10000 0.000000e+00
## 43 DAMAGING FREEZE 0.000000000 0.00000000 8.00000 0.000000e+00head(storm.mean[order(-storm.mean$CROPDMG),], 10)## EVTYPE FATALITIES INJURIES PROPDMG CROPDMG
## 61 Early Frost 0.0000000 0.000000000 0.0000000 42.000000
## 165 HURRICANE/TYPHOON 0.7272727 14.488636364 787.5663636 29.634918
## 42 Damaging Freeze 0.0000000 0.000000000 0.0000000 17.065000
## 163 HURRICANE 0.3588235 0.270588235 69.4871706 16.125941
## 74 Extreme Cold 1.0000000 0.000000000 0.0000000 10.000000
## 5 AGRICULTURAL FREEZE 0.0000000 0.000000000 0.0000000 9.606667
## 277 River Flooding 0.0000000 0.200000000 21.2310000 5.604000
## 49 DROUGHT 0.0000000 0.001644061 0.4299634 5.494273
## 89 Freeze 0.0000000 0.000000000 0.0000000 5.250000
## 399 Unseasonable Cold 0.0000000 0.000000000 0.0000000 5.100000We generate the ‘Population Health’ variable adding FATALITIES and INJURIES, and the ‘Economic Consequences’ variable adding PROPDMG and CROPDMG.
storm.mean$POPHEALTH = storm.mean$FATALITIES + storm.mean$INJURIES
storm.mean$ECONCONS = storm.mean$PROPDMG + storm.mean$CROPDMGShow Top 10 events most harmful for population health and with worst economic consequences.
storm.top10H = storm.mean[order(-storm.mean$POPHEALTH),][1:10, ]
storm.top10H## EVTYPE FATALITIES INJURIES PROPDMG CROPDMG
## 128 Heat Wave 0.00000000 70.000000 0.000000e+00 0.0000000
## 165 HURRICANE/TYPHOON 0.72727273 14.488636 7.875664e+02 29.6349182
## 34 COLD AND SNOW 14.00000000 0.000000 0.000000e+00 0.0000000
## 458 WINTER WEATHER MIX 0.00000000 11.333333 1.000000e-02 0.0000000
## 109 GLAZE 0.04761905 10.095238 2.857143e-03 0.0000000
## 397 TSUNAMI 1.65000000 6.450000 7.203100e+00 0.0010000
## 304 SNOW SQUALL 0.40000000 7.000000 6.000000e-03 0.0000000
## 233 NON-SEVERE WIND DAMAGE 0.00000000 7.000000 5.000000e-03 0.0000000
## 67 EXCESSIVE HEAT 1.08514493 3.859300 4.664070e-03 0.2973442
## 280 ROUGH SEAS 2.66666667 1.666667 0.000000e+00 0.0000000
## POPHEALTH ECONCONS
## 128 70.000000 0.000000e+00
## 165 15.215909 8.172013e+02
## 34 14.000000 0.000000e+00
## 458 11.333333 1.000000e-02
## 109 10.142857 2.857143e-03
## 397 8.100000 7.204100e+00
## 304 7.400000 6.000000e-03
## 233 7.000000 5.000000e-03
## 67 4.944444 3.020083e-01
## 280 4.333333 0.000000e+00barplot(storm.top10H$POPHEALTH,
main = "Average total fatalities and injuries (Top 10 Events)",
names.arg = storm.top10H$EVTYPE, las = 2)
storm.top10E = storm.mean[order(-storm.mean$ECONCONS),][1:10, ]
storm.top10E## EVTYPE FATALITIES INJURIES PROPDMG CROPDMG
## 165 HURRICANE/TYPHOON 0.727272727 14.48863636 787.56636 2.963492e+01
## 312 STORM SURGE 0.007905138 0.14624506 170.72544 1.976285e-05
## 163 HURRICANE 0.358823529 0.27058824 69.48717 1.612594e+01
## 398 TYPHOON 0.000000000 0.45454545 54.56636 7.500000e-02
## 61 Early Frost 0.000000000 0.00000000 0.00000 4.200000e+01
## 313 STORM SURGE/TIDE 0.074324324 0.03378378 31.35938 5.743243e-03
## 277 River Flooding 0.000000000 0.20000000 21.23100 5.604000e+00
## 42 Damaging Freeze 0.000000000 0.00000000 0.00000 1.706500e+01
## 137 Heavy Rain/High Surf 0.000000000 0.00000000 13.50000 1.500000e+00
## 395 TROPICAL STORM 0.083577713 0.49560117 11.20598 9.937111e-01
## POPHEALTH ECONCONS
## 165 15.2159091 817.20128
## 312 0.1541502 170.72546
## 163 0.6294118 85.61311
## 398 0.4545455 54.64136
## 61 0.0000000 42.00000
## 313 0.1081081 31.36512
## 277 0.2000000 26.83500
## 42 0.0000000 17.06500
## 137 0.0000000 15.00000
## 395 0.5791789 12.19969barplot(storm.top10E$ECONCONS,
main = "Average economic consequences M.US$ (Top 10 Events)",
names.arg = storm.top10E$EVTYPE, las = 2)