Assignment US Storm Impacts on Health and Economics
Ramon Schildknecht
20 April 2017
Synopsis
This document adresses the second assignment of the cousera course Reproducible Research.Storms and other severe weather events can cause both public health and economic problems for communities and municipalities. Many severe events can result in fatalities, injuries, and property damage, and preventing such outcomes to the extent possible is a key concern.
This project involves exploring the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. This database tracks characteristics of major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage.
The result section answers the following two questions:
1. Across the United States, which types of events are most harmful with respect to population health?
2. Across the United States, which types of events have the greatest economic consequences?
Hint: The answers of the questions are directly included in the questions paragraphs
Load packages and set global options
Global options echo = TRUE means that everybody can read the codes and results = ‘hold’ that the results will be displayed below the code.
library(knitr)
library(dplyr)## Warning: package 'dplyr' was built under R version 3.3.3##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(ggplot2)
source("multiplot function.R")
opts_chunk$set(echo = TRUE, results = 'hold')Loading and (pre)processing the data
[This is the source of the data.] (https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2)
For answering the question we need the following variables:
- CROPDMG contains numbers of crop damage and hence economic damage (USD)
- CROPDMGEXP contains magnitude of crop damage (e.g. thousands, millions USD, …)
- EVTYPE: contains event type like hail, rain, snow, …
- FATALITIES: contains measurementes of harm to health
- INJURIES: contains measurementes of harm to health
- PROPDMG: contains numbers of property damage and therefore economic damage (USD)
- PROPDMGEXP: contains magnitude of property damage (for example thousands, millions USD, …)
The details steps are shown below.
Load the data
filename <- "Data for Peer Assessment.csv.bz2"
if (!file.exists(filename)){
fileURL <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
download.file(fileURL, filename, mode = "wb")
}
raw.d <- read.csv(filename)Explore the data
Get dimensions, variable metadata and first observations. Get statistical summary. Get unique values for date which represents the number of days.
str(raw.d)
summary(raw.d)
length(unique(raw.d$date))## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
## $ BGN_TIME : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
## $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
## $ STATE : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ EVTYPE : Factor w/ 985 levels " HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : Factor w/ 35 levels ""," N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_LOCATI: Factor w/ 54429 levels "","- 1 N Albion",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_DATE : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_TIME : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_LOCATI: Factor w/ 34506 levels "","- .5 NNW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ WFO : Factor w/ 542 levels ""," CI","$AC",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ ZONENAMES : Factor w/ 25112 levels ""," "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : Factor w/ 436781 levels "","-2 at Deer Park\n",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...
## STATE__ BGN_DATE BGN_TIME
## Min. : 1.0 5/25/2011 0:00:00: 1202 12:00:00 AM: 10163
## 1st Qu.:19.0 4/27/2011 0:00:00: 1193 06:00:00 PM: 7350
## Median :30.0 6/9/2011 0:00:00 : 1030 04:00:00 PM: 7261
## Mean :31.2 5/30/2004 0:00:00: 1016 05:00:00 PM: 6891
## 3rd Qu.:45.0 4/4/2011 0:00:00 : 1009 12:00:00 PM: 6703
## Max. :95.0 4/2/2006 0:00:00 : 981 03:00:00 PM: 6700
## (Other) :895866 (Other) :857229
## TIME_ZONE COUNTY COUNTYNAME STATE
## CST :547493 Min. : 0.0 JEFFERSON : 7840 TX : 83728
## EST :245558 1st Qu.: 31.0 WASHINGTON: 7603 KS : 53440
## MST : 68390 Median : 75.0 JACKSON : 6660 OK : 46802
## PST : 28302 Mean :100.6 FRANKLIN : 6256 MO : 35648
## AST : 6360 3rd Qu.:131.0 LINCOLN : 5937 IA : 31069
## HST : 2563 Max. :873.0 MADISON : 5632 NE : 30271
## (Other): 3631 (Other) :862369 (Other):621339
## EVTYPE BGN_RANGE BGN_AZI
## HAIL :288661 Min. : 0.000 :547332
## TSTM WIND :219940 1st Qu.: 0.000 N : 86752
## THUNDERSTORM WIND: 82563 Median : 0.000 W : 38446
## TORNADO : 60652 Mean : 1.484 S : 37558
## FLASH FLOOD : 54277 3rd Qu.: 1.000 E : 33178
## FLOOD : 25326 Max. :3749.000 NW : 24041
## (Other) :170878 (Other):134990
## BGN_LOCATI END_DATE END_TIME
## :287743 :243411 :238978
## COUNTYWIDE : 19680 4/27/2011 0:00:00: 1214 06:00:00 PM: 9802
## Countywide : 993 5/25/2011 0:00:00: 1196 05:00:00 PM: 8314
## SPRINGFIELD : 843 6/9/2011 0:00:00 : 1021 04:00:00 PM: 8104
## SOUTH PORTION: 810 4/4/2011 0:00:00 : 1007 12:00:00 PM: 7483
## NORTH PORTION: 784 5/30/2004 0:00:00: 998 11:59:00 PM: 7184
## (Other) :591444 (Other) :653450 (Other) :622432
## COUNTY_END COUNTYENDN END_RANGE END_AZI
## Min. :0 Mode:logical Min. : 0.0000 :724837
## 1st Qu.:0 NA's:902297 1st Qu.: 0.0000 N : 28082
## Median :0 Median : 0.0000 S : 22510
## Mean :0 Mean : 0.9862 W : 20119
## 3rd Qu.:0 3rd Qu.: 0.0000 E : 20047
## Max. :0 Max. :925.0000 NE : 14606
## (Other): 72096
## END_LOCATI LENGTH WIDTH
## :499225 Min. : 0.0000 Min. : 0.000
## COUNTYWIDE : 19731 1st Qu.: 0.0000 1st Qu.: 0.000
## SOUTH PORTION : 833 Median : 0.0000 Median : 0.000
## NORTH PORTION : 780 Mean : 0.2301 Mean : 7.503
## CENTRAL PORTION: 617 3rd Qu.: 0.0000 3rd Qu.: 0.000
## SPRINGFIELD : 575 Max. :2315.0000 Max. :4400.000
## (Other) :380536
## F MAG FATALITIES INJURIES
## Min. :0.0 Min. : 0.0 Min. : 0.0000 Min. : 0.0000
## 1st Qu.:0.0 1st Qu.: 0.0 1st Qu.: 0.0000 1st Qu.: 0.0000
## Median :1.0 Median : 50.0 Median : 0.0000 Median : 0.0000
## Mean :0.9 Mean : 46.9 Mean : 0.0168 Mean : 0.1557
## 3rd Qu.:1.0 3rd Qu.: 75.0 3rd Qu.: 0.0000 3rd Qu.: 0.0000
## Max. :5.0 Max. :22000.0 Max. :583.0000 Max. :1700.0000
## NA's :843563
## PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## Min. : 0.00 :465934 Min. : 0.000 :618413
## 1st Qu.: 0.00 K :424665 1st Qu.: 0.000 K :281832
## Median : 0.00 M : 11330 Median : 0.000 M : 1994
## Mean : 12.06 0 : 216 Mean : 1.527 k : 21
## 3rd Qu.: 0.50 B : 40 3rd Qu.: 0.000 0 : 19
## Max. :5000.00 5 : 28 Max. :990.000 B : 9
## (Other): 84 (Other): 9
## WFO STATEOFFIC
## :142069 :248769
## OUN : 17393 TEXAS, North : 12193
## JAN : 13889 ARKANSAS, Central and North Central: 11738
## LWX : 13174 IOWA, Central : 11345
## PHI : 12551 KANSAS, Southwest : 11212
## TSA : 12483 GEORGIA, North and Central : 11120
## (Other):690738 (Other) :595920
## ZONENAMES
## :594029
## :205988
## GREATER RENO / CARSON CITY / M - GREATER RENO / CARSON CITY / M : 639
## GREATER LAKE TAHOE AREA - GREATER LAKE TAHOE AREA : 592
## JEFFERSON - JEFFERSON : 303
## MADISON - MADISON : 302
## (Other) :100444
## LATITUDE LONGITUDE LATITUDE_E LONGITUDE_
## Min. : 0 Min. :-14451 Min. : 0 Min. :-14455
## 1st Qu.:2802 1st Qu.: 7247 1st Qu.: 0 1st Qu.: 0
## Median :3540 Median : 8707 Median : 0 Median : 0
## Mean :2875 Mean : 6940 Mean :1452 Mean : 3509
## 3rd Qu.:4019 3rd Qu.: 9605 3rd Qu.:3549 3rd Qu.: 8735
## Max. :9706 Max. : 17124 Max. :9706 Max. :106220
## NA's :47 NA's :40
## REMARKS REFNUM
## :287433 Min. : 1
## : 24013 1st Qu.:225575
## Trees down.\n : 1110 Median :451149
## Several trees were blown down.\n : 568 Mean :451149
## Trees were downed.\n : 446 3rd Qu.:676723
## Large trees and power lines were blown down.\n: 432 Max. :902297
## (Other) :588295
## [1] 0Subset relevant variables & lowercase variables
I subsetted the data including just the seven mentioned variables above. After I transformed the uppercase variable names to lowercase for easier analysis.
d <- raw.d %>% select(CROPDMG, CROPDMGEXP, EVTYPE, FATALITIES, INJURIES, PROPDMG, PROPDMGEXP)
colnames(d) <- tolower(names(d))Question 1 including result: Across the United States, which types of events are most harmful with respect to population health?
We analyse the following three variables: - evtype - fatalities - injuries
# facilities
d.q1 <- d %>% select(evtype, fatalities, injuries) %>% group_by(evtype) %>% summarise(sum.fatalities = sum(fatalities), sum.injuries = sum(injuries)) %>% arrange(desc(sum.fatalities), desc(sum.injuries))
d.q1.plot <- head(d.q1, 10)
g <- ggplot(d.q1.plot, aes(x = reorder(evtype, - sum.fatalities), y = sum.fatalities, fill = evtype))
p1 <- g + geom_bar(stat = "identity") + labs(x = "Event type", y = "Total Fatalities", title = "Most health harmful natural events I/II") + theme(axis.text.x = element_blank())
# injuries
d.q1.2 <- d %>% select(evtype, fatalities, injuries) %>% group_by(evtype) %>% summarise(sum.fatalities = sum(fatalities), sum.injuries = sum(injuries)) %>% arrange( desc(sum.injuries), desc(sum.fatalities))
d.q1.2.plot <- head(d.q1)
g2 <- ggplot(d.q1.2.plot, aes(x = reorder(evtype, - sum.injuries), y = sum.injuries, fill = evtype))
p2 <- g2 + geom_bar(stat = "identity") + labs(x = "Event type", y = "Total Injuries", title = "Most health harmful natural events II/II") + theme(axis.text.x = element_blank())
multiplot(p1, p2, layout = matrix(data = 1:2, nrow = 2, ncol = 1))
As we can see as a result tornado is the by far the most dangerous event type for the human health. Regarding fatalities the events excessive heat as well flash flood, flood and lightning are pretty dangerous, too. Regarding injuries Wind, excessive hat and lightning are very dangerous, too.
Question 2 including result: Across the United States, which types of events have the greatest economic consequences?
We need the following variables to answer this question:
- CROPDMG
- CROPDMGEXP
- EVTYPE
- PROPDMG
- PROPDMGEXP
First we need to change the values in variables cropdmgexp and propdmgexp to their real values. The transformation is shown below.
d.q2 <-
d %>% select(evtype, cropdmg, cropdmgexp, propdmg, propdmgexp) %>%
group_by(evtype)
table(d.q2$cropdmgexp)
table(d.q2$propdmgexp)
# change data type into character
d.q2$cropdmgexp <- as.character(d.q2$cropdmgexp)
d.q2$propdmgexp <- as.character(d.q2$propdmgexp)##
## ? 0 2 B k K m M
## 618413 7 19 1 9 21 281832 1 1994
##
## - ? + 0 1 2 3 4 5
## 465934 1 8 5 216 25 13 4 4 28
## 6 7 8 B h H K m M
## 4 5 1 40 1 6 424665 7 11330Run replacements:
- B/b repladed with 9
- M/m replaced with 6
- K/k replaced with 3
- H/h replaced with 2
- -/+/? replaced with 1
- “” replaced with 0
Now we can calculate to total damages for crop and properties.
d.q2 <- as.data.frame(d.q2)
eco.dam <- d.q2$propdmg * 10^d.q2$cropdmgexp + d.q2$propdmg * 10^d.q2$propdmgexp
d.q2$eco.dam <- eco.damSubset and aggregate necessary variables. Afterwords select top 11 values, all above 10 billion USD:
d.q2.2 <- d.q2 %>% select(evtype, eco.dam) %>% group_by(evtype) %>% summarise(sum.eco.dam = sum(eco.dam)) %>% arrange(desc(sum.eco.dam))
d.q2.plot <- as.data.frame(d.q2.2)
d.q2.plot <- d.q2.plot[1:11,]
g3 <- ggplot(d.q2.plot, aes(x = reorder(evtype, - sum.eco.dam), y = sum.eco.dam, fill = evtype))
g3 + geom_bar(stat = "identity") + labs(x = "Event type", y = "Damage in USD", title = "Most harmful natural events with respect to economic damage") + theme(axis.text.x = element_blank())
Hint to plot: the number after e+ is standing for amount of zeros. Example: 2e+11 means 200’000’000’000 or 200 billions
As a result we can see that Ice Storm is by far the greatest cause for economic damage. Flood, Hurricane/Typhoons, Tornados and Hail are pretty bad, too. The other causes are shown in the graph above.