Human and Economic Consequences of Extreme Weather in America

Synopsis

The goal of this analysis is to provide insights, based on the information contained in the NOAA Storm Database, into the real consequences of severe weather for Americans; specifically, we look to assess both human and economic costs.

First, we looked at the overall impact of severe weather and found that Tornodos are the most common form of severe weather (by number of events) while Texas and Florida are the states with the highest numbers of severe weather evetns.

We found that the three most harmful event types, in terms of total economic cost were (1) floods, (2) hurricanes or typhoons, and (3) storm surges. Tornados, the most common form of severe weather, placed fourth overall in economic terms.

In contrast, we found that the three most harmful events, in the context of human costs were (1) Tornados, (2) Excessive Heat and (3) TSTM Wind.

We also examined the distribution of each of the potential consequences (injury, death, economic loss) of severe weather. In this part of the analysis, we found that a large number of storms are not classed as severe weather according to a reasonable definition, and counterintuitively that many severe events do not lead to both human and economic consequences above the severity threshold.

Data Processing

As of the time of this analysis, the NOAA data was available from 1950 to November 2011 with documentation. The data processing begins by accessing the file (from bz2 format) and reading it into a data frame. We progress to the analysis by cleansing the data with the removal of incomplete records.

#load necessary libraries for later use
require(dplyr)

## Loading required package: dplyr
## 
## Attaching package: 'dplyr'
## 
## The following object is masked from 'package:stats':
## 
##     filter
## 
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

require(ggplot2)

## Loading required package: ggplot2

require(lubridate)

## Loading required package: lubridate

# record date and time of download of the compressed file from the internet, then download and uncompress it
sprintf("%s %s", Sys.time(), Sys.timezone())

## [1] "2015-04-23 23:46:39 America/Toronto"

system("curl -o repdata_StormData.csv.bz2 https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2")
system("bzip2 -dk repdata_StormData.csv.bz2")

# read the uncompressed CSV file into a data frame and review its properties
sddf <- read.csv("repdata_StormData.csv")

#select only key attributes from the data frame
sddf <- select(sddf, BGN_DATE, REFNUM, EVTYPE, STATE, FATALITIES, INJURIES, PROPDMG, PROPDMGEXP, CROPDMG, CROPDMGEXP)

#reset key attributes to proper types: dates, times, factors, numerical, etc.
sddf$BGN_DATE <- strptime(x = as.character(sddf$BGN_DATE),format = "%m/%d/%Y %H:%M:%S")

Before proceeding further, we make some assumptions and adjust the data accordingly. The assumptions are:

Data prior to 1982 is not relevant in its current form for the assessment of damages in economic terms. This is likely to be the case because of the significant inflation experienced since the beginning of the recorded data in 1950 through to 1982. The 1982 limit ensures that the dollar values assessed are comparable within a factor of between 2 and 2.5 for all data.
To maintain consistent, comparable analysis we use the same data to calculate all values in this report, even though deaths and injuries are likely to be comparable across decades.
Only storm events tracked with reference to a particular state are tracked. Washington DC, Hawaii and Alaska are considered. Outlying territories of the US are not considered.
Only storms deemed severe enough to be tracked by NOAA will qualify as potential severe weather events for the purposes of our assessment.
All storms that caused no human fatalities or injuries and which resulted in less than $1 000 000 (one million dollars) in economic damages are not considered severe and so can be ignored in our analysis.
To maintain balance, we assign injuries a weight of 35% and fatalities a weight of 100% when estimating total human costs.
The data set is clean and complete, with no missing values denoted by NA.

# We filter the data to match the stated assumptions
sddf <- mutate(sddf, year = as.numeric(year(BGN_DATE)), realinfo=FATALITIES+INJURIES+PROPDMG+CROPDMG)
sddf <- sddf[,2:12]

states <- c("AK", "AL", "AR", "AZ", "CA", "CO", "CT", "DC", "DE", "FL", "GA", "HI", "IA", "ID", "IL", "IN", "KS", "KY", "LA", "MA", "MD", "ME", "MI", "MN", "MO", "MS", "MT", "NC", "ND", "NE", "NH", "NJ", "NM", "NV", "NY", "OH", "OK", "OR", "PA", "RI", "SC", "SD", "TN", "TX", "UT", "VA", "VT", "WA", "WI", "WV", "WY")

sddf <- filter(sddf, year>1981, STATE %in% states,realinfo>0)

# in our reduced data set, we convert all dollar amounts to standard form
sddf <- mutate(sddf, valuelost = 
                PROPDMG * ifelse(PROPDMGEXP=="K",1000,
                                 ifelse(PROPDMGEXP=="M",1000000,
                                        ifelse(PROPDMGEXP=="B",1000000000,
                                               ifelse(PROPDMGEXP=="5",100000,1)))) + 
                CROPDMG * ifelse(CROPDMGEXP=="K",1000,
                                 ifelse(CROPDMGEXP=="M",1000000,
                                        ifelse(CROPDMGEXP=="B",1000000000,
                                               ifelse(CROPDMGEXP=="k",1000,1)))),
               lifeimpact = FATALITIES+INJURIES,
               severe = ifelse(valuelost>= 1000000 | lifeimpact>0,"Severe","Small"))

sevdf <- filter(sddf, severe=="Severe")
printdf <- select(sevdf, EVTYPE, STATE, FATALITIES, INJURIES, valuelost)

tvaluedf <- printdf %>% group_by(EVTYPE) %>% summarize(economicloss=sum(valuelost), count=n()) %>% arrange(desc(economicloss))

thumandf <- printdf %>% group_by(EVTYPE) %>% summarize(humanloss=sum(FATALITIES) + sum(INJURIES)*.35, count=n()) %>% arrange(desc(humanloss))

Results

Severe Weather - Economic Impact

The first figure, shown below, shows some basic statistics concerning the overall impact of severe weather we can assess based on the NOAA Storm Database.

summary(printdf)

##          EVTYPE         STATE         FATALITIES          INJURIES       
##  TORNADO    :5046   TX     : 2021   Min.   :  0.0000   Min.   :   0.000  
##  LIGHTNING  :3433   FL     : 1661   1st Qu.:  0.0000   1st Qu.:   0.000  
##  TSTM WIND  :3411   CA     : 1078   Median :  0.0000   Median :   1.000  
##  FLASH FLOOD:2279   IL     :  960   Mean   :  0.4466   Mean   :   3.294  
##  FLOOD      :1943   GA     :  890   3rd Qu.:  0.0000   3rd Qu.:   2.000  
##  HAIL       :1570   NY     :  872   Max.   :583.0000   Max.   :1568.000  
##  (Other)    :7972   (Other):18172                                        
##    valuelost       
##  Min.   :0.00e+00  
##  1st Qu.:0.00e+00  
##  Median :1.50e+05  
##  Mean   :1.74e+07  
##  3rd Qu.:2.50e+06  
##  Max.   :1.15e+11  
##

The second figure, immediately below, shows some basic statistics concerning the economic and human impact of severe weather we can assess based on the NOAA Storm Database.

print(tvaluedf[1:5,])

## Source: local data frame [5 x 3]
## 
##              EVTYPE economicloss count
## 1             FLOOD 149269625240  1943
## 2 HURRICANE/TYPHOON  71636165800    55
## 3       STORM SURGE  43305955000    51
## 4           TORNADO  39528860678  5046
## 5              HAIL  17598280206  1570

print(thumandf[1:5,])

## Source: local data frame [5 x 3]
## 
##           EVTYPE humanloss count
## 1        TORNADO  14827.60  5046
## 2 EXCESSIVE HEAT   4186.75   679
## 3      TSTM WIND   2938.60  3411
## 4          FLOOD   2839.10  1943
## 5      LIGHTNING   2634.40  3433

Another important piece of information we can assess with this data is the distribution of key consequences of severe weather events: injuries, fatalities and economic costs. The figure below shows the log-distribution of each amount. We use the log to better represent the data since in each case, there is a significant proportion of storms that result in 0 occurences. That is, fatalities are zero and/or injuries are zero and/or the level of economic loss is zero or very small.

Recalling that we defined a severe event as one in which an injury or fatality occured or in which more than $1M was lost, we can interpret the large number of zero values in the distributions to mean that in a significant number of cases, human and economic consequences do not result from the same severe weather.

#Compare occurrences of fatalities, injuries and property damages
par(mfrow=c(1,3))
par(mar=c(4.5,4.5,4.5,4.5))
hist(log(sevdf$INJURIES),main="Distribution of Injuries",breaks=15,xlab="Log(Number of Injuries)")
hist(log(sevdf$FATALITIES),main="Distribution of Fatalities",breaks=15,xlab="Log(Number of Fatalities)")
hist(log(sevdf$valuelost),main="Economic Cost Distribution",breaks=15,xlab="Log(Total Costs)")

To compare the impact of human and economic consequences, other literature has attempted to assign a value to human life. We refrain from doing so here, and so leave the exhibits to inform our audience in their own right, without drawing specific conclusions.

Appendix: Additional Information

The system used to compute the results had the following software and hardware settings, as copied from the command window. They were not invoked as a command here because some private information has been removed (this is indicated):

  System Version: OS X 10.9.5 (13F1077)

  Kernel Version: Darwin 13.4.0
  
  Boot Volume: Macintosh HD
  
  Boot Mode: Normal
  
  Computer Name: <REMOVED>
  
  User Name: <REMOVED>
  
  Secure Virtual Memory: Enabled

About the version of R:

version

##                _                           
## platform       x86_64-apple-darwin13.4.0   
## arch           x86_64                      
## os             darwin13.4.0                
## system         x86_64, darwin13.4.0        
## status                                     
## major          3                           
## minor          1.2                         
## year           2014                        
## month          10                          
## day            31                          
## svn rev        66913                       
## language       R                           
## version.string R version 3.1.2 (2014-10-31)
## nickname       Pumpkin Helmet