Exploring the US National Oceanic and Atmospheric Administrator’s Storm Database
The goal is to discover the public health and economic impact that storms and severe weather poses to the US
Xtrangeloop
October 24, 2019
Synopsis
Storms and other severe weather events can cause extreme damage to both public health and the economy of communities and municipalities. In this report we explore the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database to understand which types of weather events damaged the public health and economy the most between 1950 and 2011.
This project revealed that tornados caused the most direct threat to public health, while floods had the highest economic impact.
Data Processing
First, we prepared some basic libraries for clenaning and sorting data, and for plotting. Then, we read in NOAA’s Storm Database and explored the data with dim, head, names, and summary.
#Load the libraries
library(ggplot2)
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(tidyr)
##Read the Data
Storm <- read.csv("repdata_data_StormData.csv")
dim(Storm)## [1] 902297 37head(Storm)## STATE__ BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE
## 1 1 4/18/1950 0:00:00 0130 CST 97 MOBILE AL
## 2 1 4/18/1950 0:00:00 0145 CST 3 BALDWIN AL
## 3 1 2/20/1951 0:00:00 1600 CST 57 FAYETTE AL
## 4 1 6/8/1951 0:00:00 0900 CST 89 MADISON AL
## 5 1 11/15/1951 0:00:00 1500 CST 43 CULLMAN AL
## 6 1 11/15/1951 0:00:00 2000 CST 77 LAUDERDALE AL
## EVTYPE BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END
## 1 TORNADO 0 0
## 2 TORNADO 0 0
## 3 TORNADO 0 0
## 4 TORNADO 0 0
## 5 TORNADO 0 0
## 6 TORNADO 0 0
## COUNTYENDN END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES
## 1 NA 0 14.0 100 3 0 0
## 2 NA 0 2.0 150 2 0 0
## 3 NA 0 0.1 123 2 0 0
## 4 NA 0 0.0 100 2 0 0
## 5 NA 0 0.0 150 2 0 0
## 6 NA 0 1.5 177 2 0 0
## INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES
## 1 15 25.0 K 0
## 2 0 2.5 K 0
## 3 2 25.0 K 0
## 4 2 2.5 K 0
## 5 2 2.5 K 0
## 6 6 2.5 K 0
## LATITUDE LONGITUDE LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1 3040 8812 3051 8806 1
## 2 3042 8755 0 0 2
## 3 3340 8742 0 0 3
## 4 3458 8626 0 0 4
## 5 3412 8642 0 0 5
## 6 3450 8748 0 0 6names(Storm)## [1] "STATE__" "BGN_DATE" "BGN_TIME" "TIME_ZONE" "COUNTY"
## [6] "COUNTYNAME" "STATE" "EVTYPE" "BGN_RANGE" "BGN_AZI"
## [11] "BGN_LOCATI" "END_DATE" "END_TIME" "COUNTY_END" "COUNTYENDN"
## [16] "END_RANGE" "END_AZI" "END_LOCATI" "LENGTH" "WIDTH"
## [21] "F" "MAG" "FATALITIES" "INJURIES" "PROPDMG"
## [26] "PROPDMGEXP" "CROPDMG" "CROPDMGEXP" "WFO" "STATEOFFIC"
## [31] "ZONENAMES" "LATITUDE" "LONGITUDE" "LATITUDE_E" "LONGITUDE_"
## [36] "REMARKS" "REFNUM"summary(Storm)## STATE__ BGN_DATE BGN_TIME
## Min. : 1.0 5/25/2011 0:00:00: 1202 12:00:00 AM: 10163
## 1st Qu.:19.0 4/27/2011 0:00:00: 1193 06:00:00 PM: 7350
## Median :30.0 6/9/2011 0:00:00 : 1030 04:00:00 PM: 7261
## Mean :31.2 5/30/2004 0:00:00: 1016 05:00:00 PM: 6891
## 3rd Qu.:45.0 4/4/2011 0:00:00 : 1009 12:00:00 PM: 6703
## Max. :95.0 4/2/2006 0:00:00 : 981 03:00:00 PM: 6700
## (Other) :895866 (Other) :857229
## TIME_ZONE COUNTY COUNTYNAME STATE
## CST :547493 Min. : 0.0 JEFFERSON : 7840 TX : 83728
## EST :245558 1st Qu.: 31.0 WASHINGTON: 7603 KS : 53440
## MST : 68390 Median : 75.0 JACKSON : 6660 OK : 46802
## PST : 28302 Mean :100.6 FRANKLIN : 6256 MO : 35648
## AST : 6360 3rd Qu.:131.0 LINCOLN : 5937 IA : 31069
## HST : 2563 Max. :873.0 MADISON : 5632 NE : 30271
## (Other): 3631 (Other) :862369 (Other):621339
## EVTYPE BGN_RANGE BGN_AZI
## HAIL :288661 Min. : 0.000 :547332
## TSTM WIND :219940 1st Qu.: 0.000 N : 86752
## THUNDERSTORM WIND: 82563 Median : 0.000 W : 38446
## TORNADO : 60652 Mean : 1.484 S : 37558
## FLASH FLOOD : 54277 3rd Qu.: 1.000 E : 33178
## FLOOD : 25326 Max. :3749.000 NW : 24041
## (Other) :170878 (Other):134990
## BGN_LOCATI END_DATE END_TIME
## :287743 :243411 :238978
## COUNTYWIDE : 19680 4/27/2011 0:00:00: 1214 06:00:00 PM: 9802
## Countywide : 993 5/25/2011 0:00:00: 1196 05:00:00 PM: 8314
## SPRINGFIELD : 843 6/9/2011 0:00:00 : 1021 04:00:00 PM: 8104
## SOUTH PORTION: 810 4/4/2011 0:00:00 : 1007 12:00:00 PM: 7483
## NORTH PORTION: 784 5/30/2004 0:00:00: 998 11:59:00 PM: 7184
## (Other) :591444 (Other) :653450 (Other) :622432
## COUNTY_END COUNTYENDN END_RANGE END_AZI
## Min. :0 Mode:logical Min. : 0.0000 :724837
## 1st Qu.:0 NA's:902297 1st Qu.: 0.0000 N : 28082
## Median :0 Median : 0.0000 S : 22510
## Mean :0 Mean : 0.9862 W : 20119
## 3rd Qu.:0 3rd Qu.: 0.0000 E : 20047
## Max. :0 Max. :925.0000 NE : 14606
## (Other): 72096
## END_LOCATI LENGTH WIDTH
## :499225 Min. : 0.0000 Min. : 0.000
## COUNTYWIDE : 19731 1st Qu.: 0.0000 1st Qu.: 0.000
## SOUTH PORTION : 833 Median : 0.0000 Median : 0.000
## NORTH PORTION : 780 Mean : 0.2301 Mean : 7.503
## CENTRAL PORTION: 617 3rd Qu.: 0.0000 3rd Qu.: 0.000
## SPRINGFIELD : 575 Max. :2315.0000 Max. :4400.000
## (Other) :380536
## F MAG FATALITIES INJURIES
## Min. :0.0 Min. : 0.0 Min. : 0.0000 Min. : 0.0000
## 1st Qu.:0.0 1st Qu.: 0.0 1st Qu.: 0.0000 1st Qu.: 0.0000
## Median :1.0 Median : 50.0 Median : 0.0000 Median : 0.0000
## Mean :0.9 Mean : 46.9 Mean : 0.0168 Mean : 0.1557
## 3rd Qu.:1.0 3rd Qu.: 75.0 3rd Qu.: 0.0000 3rd Qu.: 0.0000
## Max. :5.0 Max. :22000.0 Max. :583.0000 Max. :1700.0000
## NA's :843563
## PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## Min. : 0.00 :465934 Min. : 0.000 :618413
## 1st Qu.: 0.00 K :424665 1st Qu.: 0.000 K :281832
## Median : 0.00 M : 11330 Median : 0.000 M : 1994
## Mean : 12.06 0 : 216 Mean : 1.527 k : 21
## 3rd Qu.: 0.50 B : 40 3rd Qu.: 0.000 0 : 19
## Max. :5000.00 5 : 28 Max. :990.000 B : 9
## (Other): 84 (Other): 9
## WFO STATEOFFIC
## :142069 :248769
## OUN : 17393 TEXAS, North : 12193
## JAN : 13889 ARKANSAS, Central and North Central: 11738
## LWX : 13174 IOWA, Central : 11345
## PHI : 12551 KANSAS, Southwest : 11212
## TSA : 12483 GEORGIA, North and Central : 11120
## (Other):690738 (Other) :595920
## ZONENAMES
## :594029
## :205988
## GREATER RENO / CARSON CITY / M - GREATER RENO / CARSON CITY / M : 639
## GREATER LAKE TAHOE AREA - GREATER LAKE TAHOE AREA : 592
## JEFFERSON - JEFFERSON : 303
## MADISON - MADISON : 302
## (Other) :100444
## LATITUDE LONGITUDE LATITUDE_E LONGITUDE_
## Min. : 0 Min. :-14451 Min. : 0 Min. :-14455
## 1st Qu.:2802 1st Qu.: 7247 1st Qu.: 0 1st Qu.: 0
## Median :3540 Median : 8707 Median : 0 Median : 0
## Mean :2875 Mean : 6940 Mean :1452 Mean : 3509
## 3rd Qu.:4019 3rd Qu.: 9605 3rd Qu.:3549 3rd Qu.: 8735
## Max. :9706 Max. : 17124 Max. :9706 Max. :106220
## NA's :47 NA's :40
## REMARKS REFNUM
## :287433 Min. : 1
## : 24013 1st Qu.:225575
## Trees down.\n : 1110 Median :451149
## Several trees were blown down.\n : 568 Mean :451149
## Trees were downed.\n : 446 3rd Qu.:676723
## Large trees and power lines were blown down.\n: 432 Max. :902297
## (Other) :588295We decided to tackle Data Processing in two phases, each corresponding to our two questions, the type of weather most damaging to health and the economy.
The Danger Posed to Public Health
We extracted the subset of data requried to analyze the severe weather impact posed to public health, these included the columns:
#Extract the Events, Injuries, and Fatalities
Danger <- select(Storm, one_of(c("EVTYPE", "FATALITIES", "INJURIES")))Next, we verified that the Fatalities and Injuries columns were numeric, and then added the two columns together under the title Health.Damage.
#Veryify that the Fatalities and Injuries are numbers
head(as.numeric(Danger$FATALITIES))## [1] 0 0 0 0 0 0head(as.numeric(Danger$INJURIES))## [1] 15 0 2 2 2 6#Combine the Fatalities and Injuries
Danger$HEALTH.DAMAGE <- Danger$FATALITIES + Danger$INJURIESNext we prepped and cleaned the data. First extracting just the two main columns of the Weather Type and the Health Damage (the combined fatalities and injuries). Then, we removed all weather events that caused no public injury or fatality. Finally, we summed the public health concern by weather event and arranged the table by highest public health concern.
#Extract the Events and newly combined Health Damage
Danger <- select(Danger, one_of(c("EVTYPE", "HEALTH.DAMAGE")))
#Remove No Health Damage
Danger <- filter(Danger, HEALTH.DAMAGE>0)
#Aggregate the Health Damage by Event
Danger <- aggregate(HEALTH.DAMAGE~EVTYPE, Danger, sum)
#Sort by the most Health Damage
Danger <- arrange(Danger, desc(HEALTH.DAMAGE))Next, we explored the data in order to grasp what kind of numbers and events we were dealing with. Afterwards, we decided to cut that data down to the weather events that meet or exceed the mean of damage to all public health. These numbers were a bit unruly, so for plotting purposes, we reduced them to thousands.
#Explore the Data dimensions
head(Danger$HEALTH.DAMAGE)## [1] 96979 8428 7461 7259 6046 3037quantile(Danger$HEALTH.DAMAGE)## 0% 25% 50% 75% 100%
## 1.00 1.75 5.00 44.25 96979.00n_distinct(Danger$HEALTH.DAMAGE)## [1] 80mean(Danger$HEALTH.DAMAGE)## [1] 707.6045#Filter to all Health Damage greater than or equal to the mean
Danger <- filter(Danger, HEALTH.DAMAGE>=mean(Danger$HEALTH.DAMAGE))
#Reduce Economic Damage by a thousand
Danger$HEALTH.DAMAGE <- Danger$HEALTH.DAMAGE/1000The Danger Posed to the Economy
We extracted the subset of data requried to analyze the severe weather impact posed to the economy, these included the columns:
#Extract the Events and the Crop and Property Damage along with their exponents
Economy <- select(Storm, one_of(c("EVTYPE", "PROPDMG", "PROPDMGEXP", "CROPDMG", "CROPDMGEXP")))Next, we verified that the two Exponent columns were characters, and then converted the contents appropriatel, i.e. H to 100, K to 1000, M to 100000, B to 1000000000, and we converted blanks to 1, so that when we multipled later our numbers wouldn’t go wonky.
#Verify the Exponents are characters
Economy$PROPDMGEXP <- as.character(Economy$PROPDMGEXP)
Economy$CROPDMGEXP <- as.character(Economy$CROPDMGEXP)
#Replace the exponents with numerical data
Economy$PROPDMGEXP[Economy$PROPDMGEXP == ""] <- "1"
Economy$PROPDMGEXP[Economy$PROPDMGEXP == "H"] <- "100"
Economy$PROPDMGEXP[Economy$PROPDMGEXP == "K"] <- "1000"
Economy$PROPDMGEXP[Economy$PROPDMGEXP == "M"] <- "1000000"
Economy$PROPDMGEXP[Economy$PROPDMGEXP == "B"] <- "1000000000"
Economy$CROPDMGEXP[Economy$CROPDMGEXP == ""] <- "1"
Economy$CROPDMGEXP[Economy$CROPDMGEXP == "H"] <- "100"
Economy$CROPDMGEXP[Economy$CROPDMGEXP == "K"] <- "1000"
Economy$CROPDMGEXP[Economy$CROPDMGEXP == "M"] <- "1000000"
Economy$CROPDMGEXP[Economy$CROPDMGEXP == "B"] <- "1000000000" Next, we verified that the Crop and Property and Exponents columns were numeric, so that we could start our math. We multiplied the Crop Damage by its Exponent, and the same for the Property Damage and its Exponent. Then, we added the columns together under the title Econ.Damage.
#Verify the Damage and Exponents are numeric
Economy$PROPDMGEXP <- as.numeric(Economy$PROPDMGEXP)## Warning: NAs introduced by coercionEconomy$CROPDMGEXP <- as.numeric(Economy$CROPDMGEXP)## Warning: NAs introduced by coercionEconomy$PROPDMG <- as.numeric(Economy$PROPDMG)
Economy$CROPDMG <- as.numeric(Economy$CROPDMG)
#Multiply the Property Damage with Exponent
Economy$PROP.DAMAGE <- Economy$PROPDMG*Economy$PROPDMGEXP
Economy$CROP.DAMAGE <- Economy$CROPDMG*Economy$CROPDMGEXP
#Combine the Crop and Property Damage
Economy$ECON.DAMAGE <- Economy$CROP.DAMAGE + Economy$PROP.DAMAGENext we continued to prepare and cleane the data. First extracting just the two main columns of the Weather Type and the Economy Damage (the combined crop and property damage). Then, we removed all weather events that caused no economic damage. Finally, we summed the economic damage by weather event and arranged the table by highest economic damage concern.
#Extract the Events and the newly combined Economy Damage
Economy <- select(Economy, one_of(c("EVTYPE", "ECON.DAMAGE")))
#Remove No Economic Damage
Economy <- filter(Economy, ECON.DAMAGE>0)
#Aggregate the Economic Damage by Event
Economy <- aggregate(ECON.DAMAGE~EVTYPE, Economy, sum)
#Sort by the most Economic Damage
Economy <- arrange(Economy, desc(ECON.DAMAGE))Next, we explored the data in order to grasp what kind of numbers and events we were dealing with. Afterwards, we decided to cut that data down to the weather events that meet or exceed the mean of all economic damage to. These numbers were a bit unruly, so for plotting purposes, we reduced them to billions.
#Explore the Data dimensions
head(Economy$ECON.DAMAGE)## [1] 150319678257 71913712800 57340614104 43323541000 18752708880
## [6] 17562129394quantile(Economy$ECON.DAMAGE)## 0% 25% 50% 75% 100%
## 50 17250 242500 6360250 150319678257n_distinct(Economy$ECON.DAMAGE)## [1] 238mean(Economy$ECON.DAMAGE)## [1] 1118244564#Filter to all Economic Damage greater than or equal to the mean
Economy <- filter(Economy, ECON.DAMAGE>=mean(Economy$ECON.DAMAGE))
#Reduce Economic Damage by a billion
Economy$ECON.DAMAGE <- Economy$ECON.DAMAGE/1000000000Results
The Most Harmful Events with Respect to Population Health
The Data
#Explore the Data dimensions
head(Danger)## EVTYPE HEALTH.DAMAGE
## 1 TORNADO 96.979
## 2 EXCESSIVE HEAT 8.428
## 3 TSTM WIND 7.461
## 4 FLOOD 7.259
## 5 LIGHTNING 6.046
## 6 HEAT 3.037The Graph
The Most Harmful Events with Respect to Economic Damage
The Data
#Explore the Data dimensions
head(Economy)## EVTYPE ECON.DAMAGE
## 1 FLOOD 150.31968
## 2 HURRICANE/TYPHOON 71.91371
## 3 TORNADO 57.34061
## 4 STORM SURGE 43.32354
## 5 HAIL 18.75271
## 6 FLASH FLOOD 17.56213The Graph
