Introduction

The goal of the assignment is to explore the NOAA Storm Database and explore the effects of severe weather events on both population and economy.

The database covers the time period between 1950 and November 2011. In the earlier years of the database there are generally fewer events recorded, most likely due to a lack of good records. More recent years should be considered more complete.

The analysis aims to investigate which different types of sever weather events are most harmful on the populations health in respect of general injuries and fatalities. Further the economic consequences will be analyzed by exploring the financial damage done to both general property and agriculture (i.e. crops)

Getting and Loading Data

Following are the sources of information available from the National Oceanic & Atmospheric Administration(NOAA)

The following code contains the packages used to perform our analysis and the code to retrieve and store the data

library(knitr)
library(ggplot2)
library(reshape2)
library(kableExtra)

setwd("/Users/siddharth/Desktop/Coursera/DataScience/C5W4")
url1 <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"

download.file(url = url1, destfile = paste(getwd(), "StormData.csv.bz2", sep = "/"))

stormData <- read.csv(file = "StormData.csv.bz2")

str(stormData)
## 'data.frame':    902297 obs. of  37 variables:
##  $ STATE__   : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ BGN_DATE  : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
##  $ BGN_TIME  : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
##  $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
##  $ COUNTY    : num  97 3 57 89 43 77 9 123 125 57 ...
##  $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
##  $ STATE     : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
##  $ EVTYPE    : Factor w/ 985 levels "   HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
##  $ BGN_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ BGN_AZI   : Factor w/ 35 levels "","  N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ BGN_LOCATI: Factor w/ 54429 levels ""," Christiansburg",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ END_DATE  : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ END_TIME  : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ COUNTY_END: num  0 0 0 0 0 0 0 0 0 0 ...
##  $ COUNTYENDN: logi  NA NA NA NA NA NA ...
##  $ END_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ END_AZI   : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ END_LOCATI: Factor w/ 34506 levels ""," CANTON"," TULIA",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ LENGTH    : num  14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
##  $ WIDTH     : num  100 150 123 100 150 177 33 33 100 100 ...
##  $ F         : int  3 2 2 2 2 2 2 1 3 3 ...
##  $ MAG       : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ FATALITIES: num  0 0 0 0 0 0 0 0 1 0 ...
##  $ INJURIES  : num  15 0 2 2 2 6 1 0 14 0 ...
##  $ PROPDMG   : num  25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
##  $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
##  $ CROPDMG   : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ WFO       : Factor w/ 542 levels ""," CI","%SD",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ ZONENAMES : Factor w/ 25112 levels "","                                                                                                               "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ LATITUDE  : num  3040 3042 3340 3458 3412 ...
##  $ LONGITUDE : num  8812 8755 8742 8626 8642 ...
##  $ LATITUDE_E: num  3051 0 0 0 0 ...
##  $ LONGITUDE_: num  8806 0 0 0 0 ...
##  $ REMARKS   : Factor w/ 436781 levels "","\t","\t\t",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ REFNUM    : num  1 2 3 4 5 6 7 8 9 10 ...

Processing and cleaning Data

For our analysis we are focussing on trying to analyze the events (natural calamities) that hast the most impact on human health/life and econmy.

Hence for our analysis the variables that are of interest are:-

Therefore for each row, we can evaluate the property damage value as: PROPDMG* 10^PROPDMGEXP. Therefore for each row, we can evaluate the property damage value as: CROPDMG* 10^CROPDMGEXP.

The code chunk below evaluates the property damage cost and crop damage costs for each row in the dataset. Furthermore, it also aggregates the total injuries, fatalities, total property damage cost, total crop damage cost caused by differents events. There are store as follows:-

powerExp is the function that takes each expense row and calculates the power 10 value.

Column PROPDMGEXP contains the unique values K, M, , B, m, +, 0, 5, 6, ?, 4, 2, 3, h, 7, H, -, 1, 8. Implementing the function powerExp on the column gives:-

Column CROPDMGEXP contains the unique values , M, K, m, B, ?, 0, k, 2. Implementing the function powerExp on the column gives:-

#Writing a function that will take the power of the 10 from the column PROPDMGEXP and CROPDMGEXP
powerExp <- function(column) {
  if(column %in% c("B","b")) {
    return(9)
  }
  else if(column %in% c("M","m")) {
    return(6)
  }
  else if(column %in% c("K","k")) {
    return(3)
  }
  else if(column %in% c("H","h")) {
    return(2)
  }
  else if(column %in% c("", "-", "+", "?")){
    return(0)
  }
  else {
    return(as.numeric(column))
  }
}

#Convering PROPDMGEXP to character vector
stormData$PROPDMGEXP <- as.character(stormData$PROPDMGEXP)
#Using the function to derive the power of 10 from each row of PROPDMGEXP
PROPDMG_powerValues <- sapply(stormData$PROPDMGEXP, FUN = powerExp)

#Creating a new column names Property.Damage that evaluates the property damage for each row
stormData$Property.Damage <- melt(stormData$PROPDMG * 10^(PROPDMG_powerValues))$value


#Converting CROPDMGEXP to character vector
stormData$CROPDMGEXP <- as.character(stormData$CROPDMGEXP)
#Using the function to derive the power of 10 from each row of CROPDMGEXP
CROPDMG_powerValues <- sapply(stormData$CROPDMGEXP, FUN = powerExp)

#Creating a new column names Crop.Damage that evaluates the crop damage for each row
stormData$Crop.Damage <- melt(stormData$CROPDMG * 10^(CROPDMG_powerValues))$value


#Creating a dataframe for total Fatalities by event-type
stormFatalities <- aggregate(FATALITIES~ EVTYPE, data = stormData, sum)
#Creating a dataframe for total Injuries by event-type
stormInjuries <- aggregate(INJURIES~ EVTYPE, data = stormData, sum)

#Combining the two dataframes
eventPeopleHealth <- merge(stormFatalities, stormInjuries, by = "EVTYPE")

#Summing Injuries and Fatalities and storing the resukt in the column Total.Affected
eventPeopleHealth$Total.Affected <- rowSums(eventPeopleHealth[,c(2,3)])

#Ordering the dataframe in descending order of Total.Affected
eventPeopleHealth <- eventPeopleHealth[order(eventPeopleHealth$Total.Affected, decreasing = T),]


#Creating a dataframe for total Property.Damage by event-type
eventPropDamage <- aggregate(Property.Damage~EVTYPE, data = stormData, sum)
#Creating a dataframe for total Crop.Damage by event-type
eventCropDamage<- aggregate(Crop.Damage~EVTYPE, data = stormData, sum)

#Combining the two dataframes
eventDamageExp <- merge(eventPropDamage, eventCropDamage, by= "EVTYPE")

#Summing Property.Damage and Crop.Damage and storing the resukt in the column Total.Damages
eventDamageExp$Total.Damages <- rowSums(eventDamageExp[,2:3])

#Ordering the dataframe in descending order of Total.Damages
eventDamageExp <- eventDamageExp[order(eventDamageExp$Total.Damages, decreasing = T),]

Results

The objective of this analysis is to summarize and pinpoint the events that have highest impact on:-

1. Consequences of Events on Human Health and Life

The table below summarizes the top 10 events that impact human health and life (Fatalities+Injured)

#Representing the results in a table format to view top 20 events that affect human life

kable(eventPeopleHealth[1:10,], col.names = c("Event.Type", "Fatalities", "Injuries", "Total.Affected"),
      row.names = F, format.args = list(big.mark = ',')) #%>%
Event.Type Fatalities Injuries Total.Affected
TORNADO 5,633 91,346 96,979
EXCESSIVE HEAT 1,903 6,525 8,428
TSTM WIND 504 6,957 7,461
FLOOD 470 6,789 7,259
LIGHTNING 816 5,230 6,046
HEAT 937 2,100 3,037
FLASH FLOOD 978 1,777 2,755
ICE STORM 89 1,975 2,064
THUNDERSTORM WIND 133 1,488 1,621
WINTER STORM 206 1,321 1,527 
    #kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"))


#Representing the results in barplot to view top 20 events that affect human life
ggplot(eventPeopleHealth[1:10,], aes(reorder(EVTYPE, Total.Affected), Total.Affected)) +
  geom_bar(stat = "identity") + coord_flip() + theme_bw() +
  theme(axis.text.x = element_text(face= "bold", angle = 90, hjust = 1, size = 10),
        plot.title = element_text(face = "bold", hjust = 0.5, size = 16)) +
  labs(x = "", y = "Total Affected (Fatalities+Injured)", title = "Top 10 most harmful events to Human Health")
The above barplot shows the top 10 natural calamities that effect the Human Health/Life

The above barplot shows the top 10 natural calamities that effect the Human Health/Life

2. Consequences of Events on Economy

The table below summarizes the top 10 events that impact Economy (Property.Damage.Cost+Crop.Damage.Cost)

#Representing the results in a table format to view top 20 events that affect Economy
kable(eventDamageExp[1:10,], col.names = c("Event.Type", "Property.Damage($)", "Crop.Damage($)","Total.Damage($)"),
      row.names = F, format.args = list(big.mark = ',')) #%>%
Event.Type Property.Damage($) Crop.Damage($) Total.Damage($)
FLOOD 144,657,709,807 5,661,968,450 150,319,678,257
HURRICANE/TYPHOON 69,305,840,000 2,607,872,800 71,913,712,800
TORNADO 56,947,380,676 414,953,270 57,362,333,946
STORM SURGE 43,323,536,000 5,000 43,323,541,000
HAIL 15,735,267,513 3,025,954,473 18,761,221,986
FLASH FLOOD 16,822,673,978 1,421,317,100 18,243,991,078
DROUGHT 1,046,106,000 13,972,566,000 15,018,672,000
HURRICANE 11,868,319,010 2,741,910,000 14,610,229,010
RIVER FLOOD 5,118,945,500 5,029,459,000 10,148,404,500
ICE STORM 3,944,927,860 5,022,113,500 8,967,041,360 
    #kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"))


#Representing the results in barplot to view top 20 events that affect Economy
ggplot(eventDamageExp[1:10,], aes(reorder(EVTYPE, Total.Damages), Total.Damages/1000000)) +
  geom_bar(stat = "identity") + coord_flip() + theme_bw() +
  theme(axis.text.x = element_text(face= "bold", angle = 90, hjust = 1, size = 10),
        plot.title = element_text(face = "bold", hjust = 0.5, size = 16)) +
  labs(x = "", y = "Total Damages ($, in millions)", title = "Top 10 most harmful events to Economy")
The above barplot shows the top 10 natural calamities that effect the economy

The above barplot shows the top 10 natural calamities that effect the economy