Exploratory Analysis of U.S. NOAA Storm Database
Victoria O.
14/07/2020
1: Synopsis
Storms and other severe weather events can cause both public health and economic problems for communities and municipalities. Many severe events can result in fatalities, injuries, and property damage, and preventing such outcomes to the extent possible is a key concern.
This project involves exploring the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. This database tracks characteristics of major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage.
2: Data
The data for this project come in the form of a comma-separated-value file compressed It can downloaded from Storm Data
There is also a documentation of the database available where some of the variables are constructed/defined.
National Weather Service Storm Data Documentation National Climatic Data Center Storm Events FAQ
The events in the database start in the year 1950 and end in November 2011. In the earlier years of the database there are generally fewer events recorded, most likely due to a lack of good records. More recent years should be considered more complete.
3: Data Processing
- Download
Download the raw data file and extract the data into a dataframe.Then convert to a data.table
library("data.table")
library("ggplot2")
download.file("https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2",destfile="./StormData.csv.bz2")
storm_dt_file <-read.csv(bzfile("StormData.csv.bz2"), sep=",", header=T)
stormDT <- as.data.table(storm_dt_file)
head(storm_dt_file)## STATE__ BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE EVTYPE
## 1 1 4/18/1950 0:00:00 0130 CST 97 MOBILE AL TORNADO
## 2 1 4/18/1950 0:00:00 0145 CST 3 BALDWIN AL TORNADO
## 3 1 2/20/1951 0:00:00 1600 CST 57 FAYETTE AL TORNADO
## 4 1 6/8/1951 0:00:00 0900 CST 89 MADISON AL TORNADO
## 5 1 11/15/1951 0:00:00 1500 CST 43 CULLMAN AL TORNADO
## 6 1 11/15/1951 0:00:00 2000 CST 77 LAUDERDALE AL TORNADO
## BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END COUNTYENDN
## 1 0 0 NA
## 2 0 0 NA
## 3 0 0 NA
## 4 0 0 NA
## 5 0 0 NA
## 6 0 0 NA
## END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES INJURIES PROPDMG
## 1 0 14.0 100 3 0 0 15 25.0
## 2 0 2.0 150 2 0 0 0 2.5
## 3 0 0.1 123 2 0 0 2 25.0
## 4 0 0.0 100 2 0 0 2 2.5
## 5 0 0.0 150 2 0 0 2 2.5
## 6 0 1.5 177 2 0 0 6 2.5
## PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES LATITUDE LONGITUDE
## 1 K 0 3040 8812
## 2 K 0 3042 8755
## 3 K 0 3340 8742
## 4 K 0 3458 8626
## 5 K 0 3412 8642
## 6 K 0 3450 8748
## LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1 3051 8806 1
## 2 0 0 2
## 3 0 0 3
## 4 0 0 4
## 5 0 0 5
## 6 0 0 6- Examine number of Rows and Columns
dim(storm_dt_file)## [1] 902297 37- Data Subsetting
Select only the relevant columns.
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:data.table':
##
## between, first, last## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionstorm_dt <- storm_dt_file[,c(8,23:28)]
rm(storm_dt_file)
head(storm_dt)## EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## 1 TORNADO 0 15 25.0 K 0
## 2 TORNADO 0 0 2.5 K 0
## 3 TORNADO 0 2 25.0 K 0
## 4 TORNADO 0 2 2.5 K 0
## 5 TORNADO 0 2 2.5 K 0
## 6 TORNADO 0 6 2.5 K 0- Print Summary of Fatalities
summary(storm_dt$FATALITIES)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.0000 0.0000 0.0168 0.0000 583.0000- Print Summary of Injuries
summary(storm_dt$INJURIES)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.0000 0.0000 0.1557 0.0000 1700.0000- Calcuate Total Fatalities
total_injuries<-aggregate(INJURIES~EVTYPE,storm_dt,sum)
total_injuries<-arrange(total_injuries,desc(INJURIES))
total_injuries<-total_injuries[1:20,]
total_injuries## EVTYPE INJURIES
## 1 TORNADO 91346
## 2 TSTM WIND 6957
## 3 FLOOD 6789
## 4 EXCESSIVE HEAT 6525
## 5 LIGHTNING 5230
## 6 HEAT 2100
## 7 ICE STORM 1975
## 8 FLASH FLOOD 1777
## 9 THUNDERSTORM WIND 1488
## 10 HAIL 1361
## 11 WINTER STORM 1321
## 12 HURRICANE/TYPHOON 1275
## 13 HIGH WIND 1137
## 14 HEAVY SNOW 1021
## 15 WILDFIRE 911
## 16 THUNDERSTORM WINDS 908
## 17 BLIZZARD 805
## 18 FOG 734
## 19 WILD/FOREST FIRE 545
## 20 DUST STORM 440- Calcuate Total Injuries
total_fatalities<-aggregate(FATALITIES~EVTYPE,storm_dt,sum)
total_fatalities<-arrange(total_fatalities,desc(FATALITIES))
total_fatalities<-total_fatalities[1:20,]
total_fatalities## EVTYPE FATALITIES
## 1 TORNADO 5633
## 2 EXCESSIVE HEAT 1903
## 3 FLASH FLOOD 978
## 4 HEAT 937
## 5 LIGHTNING 816
## 6 TSTM WIND 504
## 7 FLOOD 470
## 8 RIP CURRENT 368
## 9 HIGH WIND 248
## 10 AVALANCHE 224
## 11 WINTER STORM 206
## 12 RIP CURRENTS 204
## 13 HEAT WAVE 172
## 14 EXTREME COLD 160
## 15 THUNDERSTORM WIND 133
## 16 HEAVY SNOW 127
## 17 EXTREME COLD/WIND CHILL 125
## 18 STRONG WIND 103
## 19 BLIZZARD 101
## 20 HIGH SURF 1014: Results
- Merge fatalities and injuries in one data set to print results
total <- merge(total_fatalities,total_injuries ,by.x = "EVTYPE", by.y = "EVTYPE")
total <- arrange(total,desc(FATALITIES+INJURIES))
names_events <- total$EVTYPE
barplot(t(total[,-1]),names.arg = names_events,ylim = c(0,95000),beside = T , cex.names = 0.8, las=2,col = c("light green","light blue"),main = "Top Disaster Casualities")
legend("topright",c("Fatalities","Injuries"),fill = c("light green","light blue"),bty = "n")
- Events that have the Greatest Economic Consequences
Convert property and crop damage into numbers where H=10^2, K=10^3, M=10^6, and B=10^9. Create two new variables: PROPDG, CROPDG
storm_dt$PROPDG = 0
storm_dt[storm_dt$PROPDMGEXP == "H", ]$PROPDG = storm_dt[storm_dt$PROPDMGEXP == "H", ]$PROPDMG * 10^2
storm_dt[storm_dt$PROPDMGEXP == "K", ]$PROPDG = storm_dt[storm_dt$PROPDMGEXP == "K", ]$PROPDMG * 10^3
storm_dt[storm_dt$PROPDMGEXP == "M", ]$PROPDG = storm_dt[storm_dt$PROPDMGEXP == "M", ]$PROPDMG * 10^6
storm_dt[storm_dt$PROPDMGEXP == "B", ]$PROPDG = storm_dt[storm_dt$PROPDMGEXP == "B", ]$PROPDMG * 10^9
storm_dt$CROPDG = 0
storm_dt[storm_dt$CROPDMGEXP == "H", ]$CROPDG = storm_dt[storm_dt$CROPDMGEXP == "H", ]$CROPDMG * 10^2
storm_dt[storm_dt$CROPDMGEXP == "K", ]$CROPDG = storm_dt[storm_dt$CROPDMGEXP == "K", ]$CROPDMG * 10^3
storm_dt[storm_dt$CROPDMGEXP == "M", ]$CROPDG = storm_dt[storm_dt$CROPDMGEXP == "M", ]$CROPDMG * 10^6
storm_dt[storm_dt$CROPDMGEXP == "B", ]$CROPDG = storm_dt[storm_dt$CROPDMGEXP == "B", ]$CROPDMG * 10^9- Combine property and crop damage into one variable. Arrange and select the top 20.
economic_dg <- aggregate(PROPDG + CROPDG ~ EVTYPE, storm_dt, sum)
names(economic_dg) = c("EVENT_TYPE", "TOTAL_DAMAGE")
economic_dg <- arrange(economic_dg, desc(TOTAL_DAMAGE))
economic_dg <- economic_dg[1:20, ]
economic_dg$TOTAL_DAMAGE <- economic_dg$TOTAL_DAMAGE/10^9
economic_dg$EVENT_TYPE <- factor(economic_dg$EVENT_TYPE, levels = economic_dg$EVENT_TYPE)
head(economic_dg)## EVENT_TYPE TOTAL_DAMAGE
## 1 FLOOD 150.31968
## 2 HURRICANE/TYPHOON 71.91371
## 3 TORNADO 57.34061
## 4 STORM SURGE 43.32354
## 5 HAIL 18.75290
## 6 FLASH FLOOD 17.56213- Plot Results
# Create chart
with(economic_dg, barplot(TOTAL_DAMAGE, names.arg = EVENT_TYPE, beside = T, cex.names = 0.8, las=2, col = "light blue", main = "Total Property and Crop Damage by Top 20 Event Types", ylab = "Total Damage in USD (10^9)"))
4: Conclusion
Tornado caused the maximum number of fatalities and injuries. It was followed by Excessive Heat for fatalities and Thunderstorm wind for injuries.
Floods caused the maximum property damage where as Drought caused the maximum crop damage. Second major events that caused the maximum damage was Hurricanes/Typhoos for property damage and Floods for crop damage.