The following report tries to analyse the economic and public health impact of different weather events in the United States from 1950 to 2011, based on the storm database collected from the U.S. National Oceanic and Atmospheric Administration. The estimates of fatalities, injuries, property and crop damage from the database are used to find the most devastating types of event on United States economy and population health. The conclusion of this report is that tornadoes have the most devastating effect on population health, while flood, drought have the greatest economic consequences.
We are using some packages for data manipulation and graph plotting in our analysis:
options(scipen = 1) # Turn off scientific notations for numbers
library(R.utils)
library(ggplot2)
library(plyr)
require(gridExtra)
First, we download the data file and unzip it. Then we extract the csv file in the working directory and and read it into the stormData data frame.
read_data <- function() {
file_name = "./stormData.csv.bz2"
source_url = "http://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
if(!file.exists(file_name)) {
download.file(source_url, destfile=file_name)
}
}
read_data()
stormData <- read.csv("stormData.csv", sep = ",")
We look what variables the data frame contains.
str(stormData)
## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
## $ BGN_TIME : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
## $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
## $ STATE : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ EVTYPE : Factor w/ 985 levels " HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : Factor w/ 35 levels ""," N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_LOCATI: Factor w/ 54429 levels "","- 1 N Albion",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_DATE : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_TIME : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_LOCATI: Factor w/ 34506 levels "","- .5 NNW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ WFO : Factor w/ 542 levels ""," CI","$AC",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ ZONENAMES : Factor w/ 25112 levels ""," "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : Factor w/ 436774 levels "","-2 at Deer Park\n",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...
As we can see the data frame contains a number of 902297 observations on 37 variables.
The events in the database start in the year 1950 and end in 2011. In the earlier years of the database there are generally fewer events recorded, most likely due to a lack of good records.
In this section, we are looking at the number of fatalities and injuries that severe weather events are causing.
Since our data frame is big (902297 records) we will use a function to manipulate the data by selecting and sorting records from the data frame for some variables (fatalities and injuries in our case).
sortSelect <- function(fieldName, dataset = stormData) {
workset <- dataset[, which(colnames(dataset) == fieldName)]
df <- aggregate(workset, by = list(dataset$EVTYPE), FUN = "sum")
names(df) <- c("EVTYPE", fieldName)
df <- arrange(df, df[, 2], decreasing = T)
df <- within(df, EVTYPE <- factor(x = EVTYPE, levels = df$EVTYPE))
return(df)
}
fatalities <- sortSelect("FATALITIES", dataset = stormData)
injuries <- sortSelect("INJURIES", dataset = stormData)
As it is described in the code book (Storm Events), the property damages are recorded in the PROPDMG and PROPDMGEXP columns, while the crop damages are stored in the CROPDMG and CROPDMGEXP. For both property and crop damages, the first column holds the value and the second holds the multiplier for that value, in each cases, the multiplier being denominated by (H) for Hundred, (K) for Thousand, (M) for Million and (B) for Billion.
We need to convert them into comparable numerical variables and we do this using a converter function:
convertDamage <- function(dataset = stormData, fieldName, newFieldName) {
totalLen <- dim(dataset)[2]
index <- which(colnames(dataset) == fieldName)
dataset[, index] <- as.character(dataset[, index])
logic <- !is.na(toupper(dataset[, index]))
dataset[logic & toupper(dataset[, index]) == "B", index] <- "1000000000"
dataset[logic & toupper(dataset[, index]) == "M", index] <- "1000000"
dataset[logic & toupper(dataset[, index]) == "K", index] <- "1000"
dataset[logic & toupper(dataset[, index]) == "H", index] <- "100"
dataset[logic & toupper(dataset[, index]) == "", index] <- "1"
dataset[, index] <- as.numeric(dataset[, index])
dataset[is.na(dataset[, index]), index] <- 0
dataset <- cbind(dataset, dataset[, index - 1] * dataset[, index])
names(dataset)[totalLen + 1] <- newFieldName
return(dataset)
}
stormPropDmg <- convertDamage(stormData, "PROPDMGEXP", "propertyDamage")
stormCropDmg <- convertDamage(stormPropDmg, "CROPDMGEXP", "cropDamage")
property <- sortSelect("propertyDamage", dataset = stormPropDmg)
crop <- sortSelect("cropDamage", dataset = stormCropDmg)
For analysis, we generated two sorted lists (fatalities and injuries) for the impact on population health by severe weather events type, counting in each the number of people affected.
head(fatalities)
## EVTYPE FATALITIES
## 1 TORNADO 5633
## 2 EXCESSIVE HEAT 1903
## 3 FLASH FLOOD 978
## 4 HEAT 937
## 5 LIGHTNING 816
## 6 TSTM WIND 504
head(injuries)
## EVTYPE INJURIES
## 1 TORNADO 91346
## 2 TSTM WIND 6957
## 3 FLOOD 6789
## 4 EXCESSIVE HEAT 6525
## 5 LIGHTNING 5230
## 6 HEAT 2100
We plotted these lists in the following graph for total number of fatalities and total number of injuries for these severe weather events.
fatalitiesPlot <- qplot(EVTYPE, data = head(fatalities), weight = FATALITIES, geom = "bar", binwidth = 1) +
scale_y_continuous("Number of Fatalities") +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
xlab("Severe weather event type") +
ggtitle("Number of fatalities by\n severe weather events")
injuriesPlot <- qplot(EVTYPE, data = head(injuries), weight = INJURIES, geom = "bar", binwidth = 1) +
scale_y_continuous("Number of Injuries") +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
xlab("Severe weather event type") +
ggtitle("Number of injuries by\n severe weather events")
grid.arrange(fatalitiesPlot, injuriesPlot, ncol = 2)
Based on the above histograms, we find that tornado is causing the most fatalities and also the most injuries in the United States from 1950 to 2011.
As for the impact on economy, we generated another two sorted lists for the amount of money cost by damage type.
options(scipen=1)
head(property)
## EVTYPE propertyDamage
## 1 FLOOD 144657709807
## 2 HURRICANE/TYPHOON 69305840000
## 3 TORNADO 56937160994
## 4 STORM SURGE 43323536000
## 5 FLASH FLOOD 16140812294
## 6 HAIL 15732267427
head(crop)
## EVTYPE cropDamage
## 1 DROUGHT 13972566000
## 2 FLOOD 5661968450
## 3 RIVER FLOOD 5029459000
## 4 ICE STORM 5022113500
## 5 HAIL 3025954453
## 6 HURRICANE 2741910000
We plotted these lists in the following graph for the total property damage and total crop damage produced by these severe weather events.
options(scipen=999)
propertyPlot <- qplot(EVTYPE, data = head(property), weight = propertyDamage, geom = "bar", binwidth = 1) +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
scale_y_continuous("Property Damage in US dollars")+
xlab("Severe weather event type") +
ggtitle("Total property damage by\n severe weather events")
cropPlot<- qplot(EVTYPE, data = head(crop), weight = cropDamage, geom = "bar", binwidth = 1) +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
scale_y_continuous("Crop Damage in US dollars") +
xlab("Severe weather event type") +
ggtitle("Total crop damage by\n severe weather events")
grid.arrange(propertyPlot, cropPlot, ncol = 2)
Looking at the above histograms, we can see that flood cause most property damage, while drought causes most crop damage in the United States from 1950 to 2011.
From these data, we found that tornado has the most devastating effect on population health, while flood, drought have the greatest economic consequences, based on records in United States, from 1950 to 2011.