Set up environment

U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database - Health and Economic Impacts

This project involves exploring the NOAA storm database.

This is the final project for the Reproducible Research course, that is part of the Coursera’s Data Science Specialization.

Weather events and natural dissasters can have an impact not only in people´s health but in economics and other areas. This database tracks characteristics of major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage.

The data for this assignment come in the form of a comma-separated-value file compressed via the bzip2 algorithm to reduce its size. You can download the file from the course web site:

Storm Data

There is also some documentation of the database available. Here you will find how some of the variables are constructed/defined.

National Weather Service Storm Data Documentation

National Climatic Data Center Storm Events FAQ

Data Processing

Assignment

The basic goal of this assignment is to explore the NOAA Storm Database and answer the following basic questions about severe weather events.

  • Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?
  • Across the United States, which types of events have the greatest economic consequences?

Loading data

Download the data on the local directory.

data <- read.csv("repdata_data_StormData.csv.bz2")

Explore the data

dim(data)
## [1] 902297     37
str(data)
## 'data.frame':    902297 obs. of  37 variables:
##  $ STATE__   : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ BGN_DATE  : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
##  $ BGN_TIME  : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
##  $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
##  $ COUNTY    : num  97 3 57 89 43 77 9 123 125 57 ...
##  $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
##  $ STATE     : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
##  $ EVTYPE    : Factor w/ 985 levels "   HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
##  $ BGN_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ BGN_AZI   : Factor w/ 35 levels "","  N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ BGN_LOCATI: Factor w/ 54429 levels "","- 1 N Albion",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ END_DATE  : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ END_TIME  : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ COUNTY_END: num  0 0 0 0 0 0 0 0 0 0 ...
##  $ COUNTYENDN: logi  NA NA NA NA NA NA ...
##  $ END_RANGE : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ END_AZI   : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ END_LOCATI: Factor w/ 34506 levels "","- .5 NNW",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ LENGTH    : num  14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
##  $ WIDTH     : num  100 150 123 100 150 177 33 33 100 100 ...
##  $ F         : int  3 2 2 2 2 2 2 1 3 3 ...
##  $ MAG       : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ FATALITIES: num  0 0 0 0 0 0 0 0 1 0 ...
##  $ INJURIES  : num  15 0 2 2 2 6 1 0 14 0 ...
##  $ PROPDMG   : num  25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
##  $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
##  $ CROPDMG   : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ WFO       : Factor w/ 542 levels ""," CI","$AC",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ ZONENAMES : Factor w/ 25112 levels "","                                                                                                               "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ LATITUDE  : num  3040 3042 3340 3458 3412 ...
##  $ LONGITUDE : num  8812 8755 8742 8626 8642 ...
##  $ LATITUDE_E: num  3051 0 0 0 0 ...
##  $ LONGITUDE_: num  8806 0 0 0 0 ...
##  $ REMARKS   : Factor w/ 436781 levels "","-2 at Deer Park\n",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ REFNUM    : num  1 2 3 4 5 6 7 8 9 10 ...

Getting important variables for the analysis

Health related variables: - FATALITIES: approx. number of deaths - INJURIES: approx. number of injuries

Economic related variables: - PROPDMG: approx. property damags - PROPDMGEXP: the units for property damage value - CROPDMG: approx. crop damages - CROPDMGEXP: the units for crop damage value

SubData <- data[, c( "EVTYPE", "FATALITIES", "INJURIES", "PROPDMG", "PROPDMGEXP", "CROPDMG", "CROPDMGEXP")]
head(SubData)
##    EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## 1 TORNADO          0       15    25.0          K       0           
## 2 TORNADO          0        0     2.5          K       0           
## 3 TORNADO          0        2    25.0          K       0           
## 4 TORNADO          0        2     2.5          K       0           
## 5 TORNADO          0        2     2.5          K       0           
## 6 TORNADO          0        6     2.5          K       0

Missing values

There is no NAs values in the data

sum(!complete.cases(SubData))
## [1] 0

Transforming data

Count the number of events

length(unique(SubData$EVTYPE))
## [1] 985

There are too many events, we will group them in less event and in new variable> EV.

SubData$EVENT <- "other"
# group by keyword in EVTYPE
SubData$EVENT[grep("HAIL", SubData$EVTYPE, ignore.case = TRUE)] <- "hail"
SubData$EVENT[grep("HEAT", SubData$EVTYPE, ignore.case = TRUE)] <- "heat"
SubData$EVENT[grep("FLOOD", SubData$EVTYPE, ignore.case = TRUE)] <- "flood"
SubData$EVENT[grep("WIND", SubData$EVTYPE, ignore.case = TRUE)] <- "wind"
SubData$EVENT[grep("STORM", SubData$EVTYPE, ignore.case = TRUE)] <- "storm"
SubData$EVENT[grep("SNOW", SubData$EVTYPE, ignore.case = TRUE)] <- "snow"
SubData$EVENT[grep("TORNADO", SubData$EVTYPE, ignore.case = TRUE)] <- "tornado"
SubData$EVENT[grep("WINTER", SubData$EVTYPE, ignore.case = TRUE)] <- "winter"
SubData$EVENT[grep("RAIN", SubData$EVTYPE, ignore.case = TRUE)] <- "rain"
# listing the transformed event types 
table(SubData$EVENT)
## 
##   flood    hail    heat   other    rain    snow   storm tornado    wind  winter 
##   82686  289270    2648   48970   12241   17660  113156   60700  255362   19604
head(SubData)
##    EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP   EVENT
## 1 TORNADO          0       15    25.0          K       0            tornado
## 2 TORNADO          0        0     2.5          K       0            tornado
## 3 TORNADO          0        2    25.0          K       0            tornado
## 4 TORNADO          0        2     2.5          K       0            tornado
## 5 TORNADO          0        2     2.5          K       0            tornado
## 6 TORNADO          0        6     2.5          K       0            tornado
colnames(SubData)
## [1] "EVTYPE"     "FATALITIES" "INJURIES"   "PROPDMG"    "PROPDMGEXP"
## [6] "CROPDMG"    "CROPDMGEXP" "EVENT"
SubData <- SubData %>% 
  mutate(Dam_prop = ifelse(grepl("k", PROPDMGEXP, ignore.case=TRUE), PROPDMG*1000,
         ifelse(grepl("m", PROPDMGEXP, ignore.case=TRUE),PROPDMG*1000000,
         ifelse(grepl("", PROPDMGEXP, ignore.case=TRUE),0,NA))))

SubData <- SubData %>% 
  mutate(Dam_crops = ifelse(grepl("k", CROPDMGEXP, ignore.case=TRUE), CROPDMG*1000,
         ifelse(grepl("m", CROPDMGEXP, ignore.case=TRUE),CROPDMG*1000000,
         ifelse(grepl("", CROPDMGEXP, ignore.case=TRUE),0,NA))))
  
head(SubData)
##    EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP   EVENT
## 1 TORNADO          0       15    25.0          K       0            tornado
## 2 TORNADO          0        0     2.5          K       0            tornado
## 3 TORNADO          0        2    25.0          K       0            tornado
## 4 TORNADO          0        2     2.5          K       0            tornado
## 5 TORNADO          0        2     2.5          K       0            tornado
## 6 TORNADO          0        6     2.5          K       0            tornado
##   Dam_prop Dam_crops
## 1    25000         0
## 2     2500         0
## 3    25000         0
## 4     2500         0
## 5     2500         0
## 6     2500         0

Results

health effects

inj<-SubData %>% 
  group_by(EVENT) %>% 
  summarise(val = sum(INJURIES))
inj$type <- "fatalities"
fat<-SubData %>% 
  group_by(EVENT) %>% 
  summarise(val = sum(FATALITIES)) 
fat$type <- "injuries"
x <- rbind(inj,fat)

ggplot(data=x, aes(x=EVENT, y=val, fill=type)) +
  geom_bar(stat="identity", position=position_dodge())+
  theme_minimal()

The most harmful event for health are the tornados, by a significan margin.

economics effects

prop<-SubData %>% 
  group_by(EVENT) %>% 
  summarise(val = sum(Dam_prop))
prop$type <- "property"

crops<-SubData %>% 
  group_by(EVENT) %>% 
  summarise(val = sum(Dam_crops)) 
crops$type <- "crop"
y <- rbind(prop,crops)


ggplot(data=y, aes(x=EVENT, y=val, fill=type)) +
  geom_bar(stat="identity", position=position_dodge())+
  theme_minimal()

The plot shows that property is vulnerable to tornado’s, floods, hail and other events; while crops are less affected by them.