Harmful and Costly US Weather Events from 1950 to 2011
Veronica Vaca
18 de julio de 2017
Loading packages needed
require(R.utils)## Loading required package: R.utils## Warning: package 'R.utils' was built under R version 3.4.1## Loading required package: R.oo## Loading required package: R.methodsS3## R.methodsS3 v1.7.1 (2016-02-15) successfully loaded. See ?R.methodsS3 for help.## R.oo v1.21.0 (2016-10-30) successfully loaded. See ?R.oo for help.##
## Attaching package: 'R.oo'## The following objects are masked from 'package:methods':
##
## getClasses, getMethods## The following objects are masked from 'package:base':
##
## attach, detach, gc, load, save## R.utils v2.5.0 (2016-11-07) successfully loaded. See ?R.utils for help.##
## Attaching package: 'R.utils'## The following object is masked from 'package:utils':
##
## timestamp## The following objects are masked from 'package:base':
##
## cat, commandArgs, getOption, inherits, isOpen, parse, warningsrequire(Hmisc)## Loading required package: Hmisc## Loading required package: lattice## Loading required package: survival## Loading required package: Formula## Loading required package: ggplot2##
## Attaching package: 'Hmisc'## The following object is masked from 'package:R.utils':
##
## capitalize## The following objects are masked from 'package:base':
##
## format.pval, round.POSIXt, trunc.POSIXt, unitsSynopsis
Severe weather events, like storms, can cause extensive harm to public health and economic damage.Determine which severe weather events have caused the most harm to public heath (i.e., fatalities and injuries) and the most costly economic damages (i.e., property and crop) can help us to prevent them and take actions before it happens.
THe Data that is used is the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database, which described the major storms and weather events in the United States, as well as estimates of any fatalities, injuries, and property and crop damages in a period of time from 1950 until the end of November 2011. The steps followed were: 1. Processed and cleaned the data 2. Do de Analysis 3. Present Results
Data Processing
Getting the Data
Please Download the data from: https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2
Once the data is downloaded to the destination, it will be extracted.
setwd("~/Coursera/Reproducible research")
if (!file.exists("repdata-data-StormData.csv")) {
library(R.utils)
bunzip2("repdata_data_StormData.csv.bz2", "repdata-data-StormData.csv", remove = FALSE)
}
# load data into R
data_storm <- read.csv("repdata-data-StormData.csv")
summary(data_storm)## STATE__ BGN_DATE BGN_TIME
## Min. : 1.0 5/25/2011 0:00:00: 1202 12:00:00 AM: 10163
## 1st Qu.:19.0 4/27/2011 0:00:00: 1193 06:00:00 PM: 7350
## Median :30.0 6/9/2011 0:00:00 : 1030 04:00:00 PM: 7261
## Mean :31.2 5/30/2004 0:00:00: 1016 05:00:00 PM: 6891
## 3rd Qu.:45.0 4/4/2011 0:00:00 : 1009 12:00:00 PM: 6703
## Max. :95.0 4/2/2006 0:00:00 : 981 03:00:00 PM: 6700
## (Other) :895866 (Other) :857229
## TIME_ZONE COUNTY COUNTYNAME STATE
## CST :547493 Min. : 0.0 JEFFERSON : 7840 TX : 83728
## EST :245558 1st Qu.: 31.0 WASHINGTON: 7603 KS : 53440
## MST : 68390 Median : 75.0 JACKSON : 6660 OK : 46802
## PST : 28302 Mean :100.6 FRANKLIN : 6256 MO : 35648
## AST : 6360 3rd Qu.:131.0 LINCOLN : 5937 IA : 31069
## HST : 2563 Max. :873.0 MADISON : 5632 NE : 30271
## (Other): 3631 (Other) :862369 (Other):621339
## EVTYPE BGN_RANGE BGN_AZI
## HAIL :288661 Min. : 0.000 :547332
## TSTM WIND :219940 1st Qu.: 0.000 N : 86752
## THUNDERSTORM WIND: 82563 Median : 0.000 W : 38446
## TORNADO : 60652 Mean : 1.484 S : 37558
## FLASH FLOOD : 54277 3rd Qu.: 1.000 E : 33178
## FLOOD : 25326 Max. :3749.000 NW : 24041
## (Other) :170878 (Other):134990
## BGN_LOCATI END_DATE END_TIME
## :287743 :243411 :238978
## COUNTYWIDE : 19680 4/27/2011 0:00:00: 1214 06:00:00 PM: 9802
## Countywide : 993 5/25/2011 0:00:00: 1196 05:00:00 PM: 8314
## SPRINGFIELD : 843 6/9/2011 0:00:00 : 1021 04:00:00 PM: 8104
## SOUTH PORTION: 810 4/4/2011 0:00:00 : 1007 12:00:00 PM: 7483
## NORTH PORTION: 784 5/30/2004 0:00:00: 998 11:59:00 PM: 7184
## (Other) :591444 (Other) :653450 (Other) :622432
## COUNTY_END COUNTYENDN END_RANGE END_AZI
## Min. :0 Mode:logical Min. : 0.0000 :724837
## 1st Qu.:0 NA's:902297 1st Qu.: 0.0000 N : 28082
## Median :0 Median : 0.0000 S : 22510
## Mean :0 Mean : 0.9862 W : 20119
## 3rd Qu.:0 3rd Qu.: 0.0000 E : 20047
## Max. :0 Max. :925.0000 NE : 14606
## (Other): 72096
## END_LOCATI LENGTH WIDTH
## :499225 Min. : 0.0000 Min. : 0.000
## COUNTYWIDE : 19731 1st Qu.: 0.0000 1st Qu.: 0.000
## SOUTH PORTION : 833 Median : 0.0000 Median : 0.000
## NORTH PORTION : 780 Mean : 0.2301 Mean : 7.503
## CENTRAL PORTION: 617 3rd Qu.: 0.0000 3rd Qu.: 0.000
## SPRINGFIELD : 575 Max. :2315.0000 Max. :4400.000
## (Other) :380536
## F MAG FATALITIES INJURIES
## Min. :0.0 Min. : 0.0 Min. : 0.0000 Min. : 0.0000
## 1st Qu.:0.0 1st Qu.: 0.0 1st Qu.: 0.0000 1st Qu.: 0.0000
## Median :1.0 Median : 50.0 Median : 0.0000 Median : 0.0000
## Mean :0.9 Mean : 46.9 Mean : 0.0168 Mean : 0.1557
## 3rd Qu.:1.0 3rd Qu.: 75.0 3rd Qu.: 0.0000 3rd Qu.: 0.0000
## Max. :5.0 Max. :22000.0 Max. :583.0000 Max. :1700.0000
## NA's :843563
## PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## Min. : 0.00 :465934 Min. : 0.000 :618413
## 1st Qu.: 0.00 K :424665 1st Qu.: 0.000 K :281832
## Median : 0.00 M : 11330 Median : 0.000 M : 1994
## Mean : 12.06 0 : 216 Mean : 1.527 k : 21
## 3rd Qu.: 0.50 B : 40 3rd Qu.: 0.000 0 : 19
## Max. :5000.00 5 : 28 Max. :990.000 B : 9
## (Other): 84 (Other): 9
## WFO STATEOFFIC
## :142069 :248769
## OUN : 17393 TEXAS, North : 12193
## JAN : 13889 ARKANSAS, Central and North Central: 11738
## LWX : 13174 IOWA, Central : 11345
## PHI : 12551 KANSAS, Southwest : 11212
## TSA : 12483 GEORGIA, North and Central : 11120
## (Other):690738 (Other) :595920
## ZONENAMES
## :594029
## :205988
## GREATER RENO / CARSON CITY / M - GREATER RENO / CARSON CITY / M : 639
## GREATER LAKE TAHOE AREA - GREATER LAKE TAHOE AREA : 592
## JEFFERSON - JEFFERSON : 303
## MADISON - MADISON : 302
## (Other) :100444
## LATITUDE LONGITUDE LATITUDE_E LONGITUDE_
## Min. : 0 Min. :-14451 Min. : 0 Min. :-14455
## 1st Qu.:2802 1st Qu.: 7247 1st Qu.: 0 1st Qu.: 0
## Median :3540 Median : 8707 Median : 0 Median : 0
## Mean :2875 Mean : 6940 Mean :1452 Mean : 3509
## 3rd Qu.:4019 3rd Qu.: 9605 3rd Qu.:3549 3rd Qu.: 8735
## Max. :9706 Max. : 17124 Max. :9706 Max. :106220
## NA's :47 NA's :40
## REMARKS REFNUM
## :287433 Min. : 1
## : 24013 1st Qu.:225575
## Trees down.\n : 1110 Median :451149
## Several trees were blown down.\n : 569 Mean :451149
## Trees were downed.\n : 446 3rd Qu.:676723
## Large trees and power lines were blown down.\n: 432 Max. :902297
## (Other) :588294names(data_storm)## [1] "STATE__" "BGN_DATE" "BGN_TIME" "TIME_ZONE" "COUNTY"
## [6] "COUNTYNAME" "STATE" "EVTYPE" "BGN_RANGE" "BGN_AZI"
## [11] "BGN_LOCATI" "END_DATE" "END_TIME" "COUNTY_END" "COUNTYENDN"
## [16] "END_RANGE" "END_AZI" "END_LOCATI" "LENGTH" "WIDTH"
## [21] "F" "MAG" "FATALITIES" "INJURIES" "PROPDMG"
## [26] "PROPDMGEXP" "CROPDMG" "CROPDMGEXP" "WFO" "STATEOFFIC"
## [31] "ZONENAMES" "LATITUDE" "LONGITUDE" "LATITUDE_E" "LONGITUDE_"
## [36] "REMARKS" "REFNUM"Cleaning the Data
In this section we are going to prepare de data base for the analysis, managing the missing values, giving sense to the data and taking only the informations that is useful for our target.
# subset the data to health and economic impact analysis against weather
# event
mycol <- c("EVTYPE", "FATALITIES", "INJURIES", "PROPDMG", "PROPDMGEXP", "CROPDMG",
"CROPDMGEXP")
storm <- data_storm[mycol]
describe(storm)## storm
##
## 7 Variables 902297 Observations
## ---------------------------------------------------------------------------
## EVTYPE
## n missing distinct
## 902297 0 985
##
## lowest : HIGH SURF ADVISORY COASTAL FLOOD FLASH FLOOD LIGHTNING TSTM WIND
## highest: WINTERY MIX Wintry mix Wintry Mix WINTRY MIX WND
## ---------------------------------------------------------------------------
## FATALITIES
## n missing distinct Info Mean Gmd .05 .10
## 902297 0 52 0.023 0.01678 0.03344 0 0
## .25 .50 .75 .90 .95
## 0 0 0 0 0
##
## lowest : 0 1 2 3 4, highest: 99 114 116 158 583
## ---------------------------------------------------------------------------
## INJURIES
## n missing distinct Info Mean Gmd .05 .10
## 902297 0 200 0.057 0.1557 0.3101 0 0
## .25 .50 .75 .90 .95
## 0 0 0 0 0
##
## lowest : 0 1 2 3 4, highest: 800 1150 1228 1568 1700
## ---------------------------------------------------------------------------
## PROPDMG
## n missing distinct Info Mean Gmd .05 .10
## 902297 0 1390 0.603 12.06 22.74 0.0 0.0
## .25 .50 .75 .90 .95
## 0.0 0.0 0.5 15.0 50.0
##
## lowest : 0.00 0.01 0.02 0.03 0.04
## highest: 3200.00 3500.00 4410.00 4800.00 5000.00
## ---------------------------------------------------------------------------
## PROPDMGEXP
## n missing distinct
## 902297 0 19
##
## (465934, 0.516), - (1, 0.000), ? (8, 0.000), + (5, 0.000), 0 (216, 0.000),
## 1 (25, 0.000), 2 (13, 0.000), 3 (4, 0.000), 4 (4, 0.000), 5 (28, 0.000), 6
## (4, 0.000), 7 (5, 0.000), 8 (1, 0.000), B (40, 0.000), h (1, 0.000), H (6,
## 0.000), K (424665, 0.471), m (7, 0.000), M (11330, 0.013)
## ---------------------------------------------------------------------------
## CROPDMG
## n missing distinct Info Mean Gmd .05 .10
## 902297 0 432 0.072 1.527 3.036 0 0
## .25 .50 .75 .90 .95
## 0 0 0 0 0
##
## lowest : 0.00 0.01 0.02 0.03 0.05, highest: 950.00 975.00 978.00 985.00 990.00
## ---------------------------------------------------------------------------
## CROPDMGEXP
## n missing distinct
## 902297 0 9
##
## Value ? 0 2 B k K m M
## Frequency 618413 7 19 1 9 21 281832 1 1994
## Proportion 0.685 0.000 0.000 0.000 0.000 0.000 0.312 0.000 0.002
## ---------------------------------------------------------------------------Property and Crop damage
According to the database’s documentation, the variables propdmgexp and cropdmgexp should be alphabetical characters that signifying the magnitude of the number (i.e., K" for thousands, “M” for millions, and “B” for billions). However, other characters were also used to indicate magnitude (see below).
# property damage magnitude
unique(storm$PROPDMGEXP)## [1] K M B m + 0 5 6 ? 4 2 3 h 7 H - 1 8
## Levels: - ? + 0 1 2 3 4 5 6 7 8 B h H K m M# crop damage magnitude
unique(storm$CROPDMGEXP)## [1] M K m B ? 0 k 2
## Levels: ? 0 2 B k K m Mstorm$propdmgexp <- toupper(storm$PROPDMGEXP)
storm$propdmg10 <- ifelse(grepl("[0-9]", storm$propdmgexp), 10^as.numeric(storm$propdmgexp), 1)## Warning in ifelse(grepl("[0-9]", storm$propdmgexp), 10^as.numeric(storm
## $propdmgexp), : NAs introducidos por coerciónstorm$propdmg10[storm$propdmgexp=="H"] <- 10^2
storm$propdmg10[storm$propdmgexp=="K"] <- 10^3
storm$propdmg10[storm$propdmgexp=="M"] <- 10^6
storm$propdmg10[storm$propdmgexp=="B"] <- 10^9
# recode crop damage magnitude
storm$cropdmgexp <- toupper(storm$CROPDMGEXP)
storm$cropdmg10 <- ifelse(grepl("[0-9]", storm$cropdmgexp), 10^as.numeric(storm$cropdmgexp), 1)## Warning in ifelse(grepl("[0-9]", storm$cropdmgexp), 10^as.numeric(storm
## $cropdmgexp), : NAs introducidos por coerciónstorm$cropdmg10[storm$cropdmgexp=="K"] <- 10^3
storm$cropdmg10[storm$cropdmgexp=="M"] <- 10^6
storm$cropdmg10[storm$cropdmgexp=="B"] <- 10^9
storm$Property <- storm$PROPDMG*storm$propdmg10
storm$Crop <- storm$CROPDMG*storm$cropdmg10Event type Data
You can also embed plots, for example:
describe(storm$EVTYPE)## storm$EVTYPE
## n missing distinct
## 902297 0 985
##
## lowest : HIGH SURF ADVISORY COASTAL FLOOD FLASH FLOOD LIGHTNING TSTM WIND
## highest: WINTERY MIX Wintry mix Wintry Mix WINTRY MIX WNDnum.events <- length(unique(storm$EVTYPE))
head(sort(unique(storm$EVTYPE)), 10)## [1] HIGH SURF ADVISORY COASTAL FLOOD FLASH FLOOD
## [4] LIGHTNING TSTM WIND TSTM WIND (G45)
## [7] WATERSPOUT WIND ?
## [10] ABNORMAL WARMTH
## 985 Levels: HIGH SURF ADVISORY COASTAL FLOOD ... WNDBefore any pre-processing has been conducted, there are a total of 985 unique event type descriptions. It is clear that the same event type may be described in multiple ways with shorthand (e.g., TSTM vs.THUNDERSTORM), different word stems (e.g., WIND vs.WINDS) or typos (see below).
storm$EVTYPE <- toupper(storm$EVTYPE)
# Combine similar event types.
storm$EVTYPE <- gsub('.*HURRICANE.*', 'HURRICANE', storm$EVTYPE)
storm$EVTYPE <- gsub('.*HEAT.*', 'HEAT', storm$EVTYPE)
storm$EVTYPE <- gsub('.*WARM.*', 'HEAT', storm$EVTYPE)
storm$EVTYPE <- gsub('.*HIGH.*TEMP.*', 'EXTREME HEAT', storm$EVTYPE)
storm$EVTYPE <- gsub('.*RECORD HIGH TEMPERATURES.*', 'EXTREME HEAT', storm$EVTYPE)
storm$EVTYPE <- gsub('.*STORM.*', 'STORM', storm$EVTYPE)
storm$EVTYPE <- gsub('.*THUNDERSTORM.*', 'THUNDERSTORM', storm$EVTYPE)
storm$EVTYPE <- gsub('.*WATERSPOUT.*', 'THUNDERSTORM', storm$EVTYPE)
storm$EVTYPE <- gsub('.* WATERSPOU.*', 'THUNDERSTORM', storm$EVTYPE)
storm$EVTYPE <- gsub('.*FLOOD.*', 'FLOOD', storm$EVTYPE)
storm$EVTYPE <- gsub('.*WIND.*', 'WIND', storm$EVTYPE)
storm$EVTYPE <- gsub('.*TORNADO.*', 'TORNADO', storm$EVTYPE)
storm$EVTYPE <- gsub('.*CLOUD.*', 'CLOUD', storm$EVTYPE)
storm$EVTYPE <- gsub('.*MICROBURST.*', 'MICROBURST', storm$EVTYPE)
storm$EVTYPE <- gsub('.*BLIZZARD.*', 'BLIZZARD', storm$EVTYPE)
storm$EVTYPE <- gsub('.*COLD.*', 'COLD', storm$EVTYPE)
storm$EVTYPE <- gsub('.*SNOW.*', 'COLD', storm$EVTYPE)
storm$EVTYPE <- gsub('.*FREEZ.*', 'COLD', storm$EVTYPE)
storm$EVTYPE <- gsub('.*LOW TEMPERATURE RECORD.*', 'COLD', storm$EVTYPE)
storm$EVTYPE <- gsub('.*ICE.*', 'COLD', storm$EVTYPE)
storm$EVTYPE <- gsub('.*FROST.*', 'COLD', storm$EVTYPE)
storm$EVTYPE <- gsub('.*LO.*TEMP.*', 'COLD', storm$EVTYPE)
storm$EVTYPE <- gsub('.*HAIL.*', 'HAIL', storm$EVTYPE)
storm$EVTYPE <- gsub('.*DRY.*', 'DRY', storm$EVTYPE)
storm$EVTYPE <- gsub('.*DUST.*', 'DUST', storm$EVTYPE)
storm$EVTYPE <- gsub('.*RAIN.*', 'RAIN', storm$EVTYPE)
storm$EVTYPE <- gsub('.*LIGHTNING.*', 'LIGHTNING', storm$EVTYPE)
storm$EVTYPE <- gsub('.*SUMMARY.*', 'SUMMARY', storm$EVTYPE)
storm$EVTYPE <- gsub('.*WET.*', 'WET', storm$EVTYPE)
storm$EVTYPE <- gsub('.*FIRE.*', 'FIRE', storm$EVTYPE)
storm$EVTYPE <- gsub('.*FOG.*', 'FOG', storm$EVTYPE)
storm$EVTYPE <- gsub('.*VOLCANIC.*', 'VOLCANIC', storm$EVTYPE)
storm$EVTYPE <- gsub('.*SURF.*', 'SURF', storm$EVTYPE)
length(unique(storm$EVTYPE))## [1] 147We left only 150 subtypes
RESULTS
Top 10 harmful weather event type by infjuries and fatalities
Subset the data, harmful weather respect to the fatalities and injuries
storm1 <- aggregate( x = list(Health_Impact = storm$FATALITIES + storm$INJURIES),
by=list(EVENT_TYPE=storm$EVTYPE),
FUN=sum, na.rm=TRUE)
storm1 <- storm1[order(storm1$Health_Impact, decreasing=T),]
head(storm1,10)## EVENT_TYPE Health_Impact
## 117 TORNADO 96997
## 40 HEAT 12421
## 140 WIND 10282
## 32 FLOOD 10127
## 112 STORM 7325
## 68 LIGHTNING 6048
## 16 COLD 2011
## 30 FIRE 1698
## 58 HURRICANE 1463
## 39 HAIL 1386HealthImpactChart <- ggplot(head(storm1,10), aes(x=reorder(EVENT_TYPE, -Health_Impact), y=Health_Impact, fill = EVENT_TYPE)) +
geom_bar(stat="identity") +
xlab("Event Type") + ylab("Total Fatalities & Injures Qty.") +
ggtitle("Top 10 Weather Events in US - Health Impacts")
print(HealthImpactChart) 
Top 10 weather event types with the greatest economic consequences
storm2 <- aggregate( x = list(Damage_Cost = storm$Property + storm$Crop), by=list(EVENT_TYPE=storm$EVTYPE),
FUN=sum, na.rm=TRUE)
storm2 <- storm2[order(storm2$Damage_Cost, decreasing=T),]
head(storm2,10)## EVENT_TYPE Damage_Cost
## 32 FLOOD 180591769935
## 58 HURRICANE 90271472810
## 112 STORM 79670614754
## 117 TORNADO 57367113447
## 39 HAIL 18782880986
## 23 DROUGHT 15018672000
## 140 WIND 13747954768
## 30 FIRE 8899910130
## 16 COLD 4726752550
## 89 RAIN 4036524490DamageCostChart <- ggplot(head(storm2,10), aes(x=reorder(EVENT_TYPE, -Damage_Cost), y=Damage_Cost, fill = EVENT_TYPE)) +
geom_bar(stat="identity") +
xlab("Event Type") + ylab("Total Damage Cost ($)") +
ggtitle("Top 10 Weather Events in US - Economic Impacts")
print(DamageCostChart)
Conclusions
The outcome of NOAA database study between 1950 and 2011 shows the worst weather events type are presented as below:
Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?
Tornado was the most harmful weather event type in terms of population health.
Across the United States, which types of events have the greatest economic consequences?
Flood was the worst weather event type in terms of economic consequences.