Exploratory data analysis on US NOAA storm database

Introduction

Storms and other severe weather events can cause both public health and economic problems for communities and municipalities. Many severe events can result in fatalities, injuries, and property damage, and preventing such outcomes to the extent possible is a key concern.

This project involves exploring the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. This database tracks characteristics of major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage.

Data

The data for this assignment come in the form of a comma-separated-value file compressed via the bzip2 algorithm to reduce its size. You can download the file from the course web site:

Storm Data download here There is also some documentation of the database available. Here you will find how some of the variables are constructed/defined.

National Weather Service Storm Data Documentation National Climatic Data Center Storm Events FAQ

The events in the database start in the year 1950 and end in November 2011. In the earlier years of the database there are generally fewer events recorded, most likely due to a lack of good records. More recent years should be considered more complete.

Data Preprocessing

knitr::opts_chunk$set(echo = TRUE)
library(ggplot2)
library(dplyr)
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
library(gridExtra)
## Warning: package 'gridExtra' was built under R version 4.0.3
## 
## Attaching package: 'gridExtra'
## The following object is masked from 'package:dplyr':
## 
##     combine
setwd("C:/Users/virar/Downloads/repdata_data_StormData.csv")
data<-read.csv('repdata_data_StormData.csv',header=TRUE)
str(data)
## 'data.frame':    902297 obs. of  37 variables:
##  $ STATE__   : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ BGN_DATE  : chr  "4/18/1950 0:00:00" "4/18/1950 0:00:00" "2/20/1951 0:00:00" "6/8/1951 0:00:00" ...
##  $ BGN_TIME  : chr  "0130" "0145" "1600" "0900" ...
##  $ TIME_ZONE : chr  "CST" "CST" "CST" "CST" ...
##  $ COUNTY    : num  97 3 57 89 43 77 9 123 125 57 ...
##  $ COUNTYNAME: chr  "MOBILE" "BALDWIN" "FAYETTE" "MADISON" ...
##  $ STATE     : chr  "AL" "AL" "AL" "AL" ...
##  $ EVTYPE    : chr  "TORNADO" "TORNADO" "TORNADO" "TORNADO" ...
##  $ BGN_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ BGN_AZI   : chr  "" "" "" "" ...
##  $ BGN_LOCATI: chr  "" "" "" "" ...
##  $ END_DATE  : chr  "" "" "" "" ...
##  $ END_TIME  : chr  "" "" "" "" ...
##  $ COUNTY_END: num  0 0 0 0 0 0 0 0 0 0 ...
##  $ COUNTYENDN: logi  NA NA NA NA NA NA ...
##  $ END_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ END_AZI   : chr  "" "" "" "" ...
##  $ END_LOCATI: chr  "" "" "" "" ...
##  $ LENGTH    : num  14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
##  $ WIDTH     : num  100 150 123 100 150 177 33 33 100 100 ...
##  $ F         : int  3 2 2 2 2 2 2 1 3 3 ...
##  $ MAG       : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ FATALITIES: num  0 0 0 0 0 0 0 0 1 0 ...
##  $ INJURIES  : num  15 0 2 2 2 6 1 0 14 0 ...
##  $ PROPDMG   : num  25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
##  $ PROPDMGEXP: chr  "K" "K" "K" "K" ...
##  $ CROPDMG   : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CROPDMGEXP: chr  "" "" "" "" ...
##  $ WFO       : chr  "" "" "" "" ...
##  $ STATEOFFIC: chr  "" "" "" "" ...
##  $ ZONENAMES : chr  "" "" "" "" ...
##  $ LATITUDE  : num  3040 3042 3340 3458 3412 ...
##  $ LONGITUDE : num  8812 8755 8742 8626 8642 ...
##  $ LATITUDE_E: num  3051 0 0 0 0 ...
##  $ LONGITUDE_: num  8806 0 0 0 0 ...
##  $ REMARKS   : chr  "" "" "" "" ...
##  $ REFNUM    : num  1 2 3 4 5 6 7 8 9 10 ...
na_info<-apply(is.na(data),2,which)
str(na_info) #gives info about every missing value
## List of 37
##  $ STATE__   : int(0) 
##  $ BGN_DATE  : int(0) 
##  $ BGN_TIME  : int(0) 
##  $ TIME_ZONE : int(0) 
##  $ COUNTY    : int(0) 
##  $ COUNTYNAME: int(0) 
##  $ STATE     : int(0) 
##  $ EVTYPE    : int(0) 
##  $ BGN_RANGE : int(0) 
##  $ BGN_AZI   : int(0) 
##  $ BGN_LOCATI: int(0) 
##  $ END_DATE  : int(0) 
##  $ END_TIME  : int(0) 
##  $ COUNTY_END: int(0) 
##  $ COUNTYENDN: int [1:902297] 1 2 3 4 5 6 7 8 9 10 ...
##  $ END_RANGE : int(0) 
##  $ END_AZI   : int(0) 
##  $ END_LOCATI: int(0) 
##  $ LENGTH    : int(0) 
##  $ WIDTH     : int(0) 
##  $ F         : int [1:843563] 24 37 54 55 56 57 58 59 60 61 ...
##  $ MAG       : int(0) 
##  $ FATALITIES: int(0) 
##  $ INJURIES  : int(0) 
##  $ PROPDMG   : int(0) 
##  $ PROPDMGEXP: int(0) 
##  $ CROPDMG   : int(0) 
##  $ CROPDMGEXP: int(0) 
##  $ WFO       : int(0) 
##  $ STATEOFFIC: int(0) 
##  $ ZONENAMES : int(0) 
##  $ LATITUDE  : int [1:47] 647217 647219 647220 647221 690294 690295 690297 690298 690303 690304 ...
##  $ LONGITUDE : int(0) 
##  $ LATITUDE_E: int [1:40] 647217 647219 647220 647221 690297 690298 690307 746023 746027 746028 ...
##  $ LONGITUDE_: int(0) 
##  $ REMARKS   : int(0) 
##  $ REFNUM    : int(0)

We’ll be primarily addressing the following questions:

  • Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?
  • Across the United States, which types of events have the greatest economic consequences?
data_1<- data %>% select(EVTYPE, FATALITIES) %>% group_by(EVTYPE) %>%
  summarise(TOTAL_FATALITIES=sum(FATALITIES)) %>% arrange(desc(TOTAL_FATALITIES))
## `summarise()` ungrouping output (override with `.groups` argument)
head(data_1)
## # A tibble: 6 x 2
##   EVTYPE         TOTAL_FATALITIES
##   <chr>                     <dbl>
## 1 TORNADO                    5633
## 2 EXCESSIVE HEAT             1903
## 3 FLASH FLOOD                 978
## 4 HEAT                        937
## 5 LIGHTNING                   816
## 6 TSTM WIND                   504
data_2<- data %>% select(EVTYPE, INJURIES) %>% group_by(EVTYPE) %>%
  summarise(TOTAL_INJURIES=sum(INJURIES)) %>% arrange(desc(TOTAL_INJURIES))
## `summarise()` ungrouping output (override with `.groups` argument)
head(data_2)
## # A tibble: 6 x 2
##   EVTYPE         TOTAL_INJURIES
##   <chr>                   <dbl>
## 1 TORNADO                 91346
## 2 TSTM WIND                6957
## 3 FLOOD                    6789
## 4 EXCESSIVE HEAT           6525
## 5 LIGHTNING                5230
## 6 HEAT                     2100

Results

fatalaties.plot<- data_1[1:10,] %>% ggplot(aes(EVTYPE,TOTAL_FATALITIES)) + 
  theme(axis.text.x=element_text(angle=30))+geom_bar(stat='identity') +ggtitle('Top 10 Events with Highest Total Fatalities')

injuries.plot<- data_2[1:10,] %>% ggplot(aes(EVTYPE,TOTAL_INJURIES)) + 
  theme(axis.text.x=element_text(angle=30))+geom_bar(stat='identity') +ggtitle('Top 10 Events with Highest Total Injuries')

grid.arrange(fatalaties.plot,injuries.plot,ncol=2)

economic_damage <- data[,c('EVTYPE','PROPDMG','PROPDMGEXP','CROPDMG','CROPDMGEXP')]
Symbol <- sort(unique(as.character(economic_damage$PROPDMGEXP)))
Multiplier <- c(0,0,0,1,10,10,10,10,10,10,10,10,10,10^9,10^2,10^2,10^3,10^6,10^6)
# refer the link to know the weightage of each notation

Multiplier_df <- data.frame(Symbol, Multiplier)
economic_damage$prop_damage <- Multiplier_df$Multiplier[match(economic_damage$PROPDMGEXP,Multiplier_df$Symbol)]
economic_damage$crop_damage <- Multiplier_df$Multiplier[match(economic_damage$CROPDMGEXP, Multiplier_df$Symbol)]

economic_damage <- economic_damage %>% 
  mutate(PROPDMG = PROPDMG*prop_damage) %>% 
  mutate(CROPDMG = CROPDMG*crop_damage) %>% 
  mutate(TOTAL.DMG = PROPDMG+CROPDMG)

data_3 <- economic_damage %>% group_by(EVTYPE) %>% summarize(TOTAL_DAMAGE = sum(TOTAL.DMG))%>% arrange(-TOTAL_DAMAGE) 
## `summarise()` ungrouping output (override with `.groups` argument)

Results

dmg.plot<-data_3[1:10,] %>% ggplot(aes(EVTYPE,TOTAL_DAMAGE)) + 
  theme(axis.text.x=element_text(angle=30))+geom_bar(stat='identity') +ggtitle('Top 10 Events with Highest Economic Impact')
dmg.plot