Analysis of Storms and other severe weather events on public health and economic problems using NOAA Storm Database
You!
2022-12-30
Course Project
Reproducible Research Course Project 2
Synopsis
Storms and other severe weather events can cause both public health and economic problems for communities and municipalities. Many severe events can result in fatalities, injuries, and property damage, and preventing such outcomes to the extent possible is a key concern. The basic goal of this assignment is to explore the NOAA Storm Database and answer some basic questions about which types of events are most harmful with respect to population health and which have the greatest economic consequences.
Data Processing
Download File
Download file from the Internet:
url <- 'https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2'
if (!file.exists('repdata_data_StormData.csv.bz2')) {
download.file(url = url, destfile = 'repdata_data_StormData.csv.bz2')
}Read the file
load the data file via read.csv
stormData <- read.csv(bzfile('repdata_data_StormData.csv.bz2'), sep = ',', header = TRUE)Display summary
str(stormData)## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : chr "4/18/1950 0:00:00" "4/18/1950 0:00:00" "2/20/1951 0:00:00" "6/8/1951 0:00:00" ...
## $ BGN_TIME : chr "0130" "0145" "1600" "0900" ...
## $ TIME_ZONE : chr "CST" "CST" "CST" "CST" ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: chr "MOBILE" "BALDWIN" "FAYETTE" "MADISON" ...
## $ STATE : chr "AL" "AL" "AL" "AL" ...
## $ EVTYPE : chr "TORNADO" "TORNADO" "TORNADO" "TORNADO" ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : chr "" "" "" "" ...
## $ BGN_LOCATI: chr "" "" "" "" ...
## $ END_DATE : chr "" "" "" "" ...
## $ END_TIME : chr "" "" "" "" ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : chr "" "" "" "" ...
## $ END_LOCATI: chr "" "" "" "" ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: chr "K" "K" "K" "K" ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: chr "" "" "" "" ...
## $ WFO : chr "" "" "" "" ...
## $ STATEOFFIC: chr "" "" "" "" ...
## $ ZONENAMES : chr "" "" "" "" ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : chr "" "" "" "" ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...head(stormData)## STATE__ BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE EVTYPE
## 1 1 4/18/1950 0:00:00 0130 CST 97 MOBILE AL TORNADO
## 2 1 4/18/1950 0:00:00 0145 CST 3 BALDWIN AL TORNADO
## 3 1 2/20/1951 0:00:00 1600 CST 57 FAYETTE AL TORNADO
## 4 1 6/8/1951 0:00:00 0900 CST 89 MADISON AL TORNADO
## 5 1 11/15/1951 0:00:00 1500 CST 43 CULLMAN AL TORNADO
## 6 1 11/15/1951 0:00:00 2000 CST 77 LAUDERDALE AL TORNADO
## BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END COUNTYENDN
## 1 0 0 NA
## 2 0 0 NA
## 3 0 0 NA
## 4 0 0 NA
## 5 0 0 NA
## 6 0 0 NA
## END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES INJURIES PROPDMG
## 1 0 14.0 100 3 0 0 15 25.0
## 2 0 2.0 150 2 0 0 0 2.5
## 3 0 0.1 123 2 0 0 2 25.0
## 4 0 0.0 100 2 0 0 2 2.5
## 5 0 0.0 150 2 0 0 2 2.5
## 6 0 1.5 177 2 0 0 6 2.5
## PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES LATITUDE LONGITUDE
## 1 K 0 3040 8812
## 2 K 0 3042 8755
## 3 K 0 3340 8742
## 4 K 0 3458 8626
## 5 K 0 3412 8642
## 6 K 0 3450 8748
## LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1 3051 8806 1
## 2 0 0 2
## 3 0 0 3
## 4 0 0 4
## 5 0 0 5
## 6 0 0 6Create Subset of Data
For this analysis, the dataset will be trimmed to only include the
necessary variables (listed below). In addition, only observations with
value > 0 will be included.
- EVTYPE - Event type (Flood, Heat, Hurricane, Tornado, …)
- FATALITIES - Number of fatalities resulting from event
- INJURIES - Number of injuries resulting from event
- PROPDMG - Property damage in USD
- PROPDMGEXP - Unit multiplier for property damage (K, M, or B)
- CROPDMG - Crop damage in USD
- CROPDMGEXP - Unit multiplier for property damage (K, M, or B)
- BGN_DATE - Begin date of the event
- END_DATE - End date of the event
- STATE - State where the event occurred
stormDataSubset <- subset(stormData, EVTYPE != "?" & (FATALITIES > 0 | INJURIES > 0 | PROPDMG > 0 | CROPDMG > 0), select = c("EVTYPE", "FATALITIES", "INJURIES", "PROPDMG", "PROPDMGEXP", "CROPDMG", "CROPDMGEXP", "BGN_DATE", "END_DATE", "STATE"))
dim(stormDataSubset)## [1] 254632 10sum(is.na(stormDataSubset))## [1] 0The working (tidy) dataset contains 254632 observations, 10 variables and no missing values.
Clean Event Type Data
There are a total of 487 unique Event Type values in the current dataset.
Exploring the Event Type data revealed many values that appeared to
be similar; however, they were entered with different spellings,
pluralization, mixed case and even misspellings. For example,
Strong Wind, STRONG WIND,
Strong Winds, and STRONG WINDS.
Convert all Event Type values to uppercase and combine similar Event Type values into unique categories.
stormDataSubset$EVTYPE <- toupper(stormDataSubset$EVTYPE)# AVALANCHE
stormDataSubset$EVTYPE <- gsub('.*AVALANCE.*', 'AVALANCHE', stormDataSubset$EVTYPE)
# BLIZZARD
stormDataSubset$EVTYPE <- gsub('.*BLIZZARD.*', 'BLIZZARD', stormDataSubset$EVTYPE)
# CLOUD
stormDataSubset$EVTYPE <- gsub('.*CLOUD.*', 'CLOUD', stormDataSubset$EVTYPE)
# COLD
stormDataSubset$EVTYPE <- gsub('.*COLD.*', 'COLD', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*FREEZ.*', 'COLD', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*FROST.*', 'COLD', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*ICE.*', 'COLD', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*LOW TEMPERATURE RECORD.*', 'COLD', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*LO.*TEMP.*', 'COLD', stormDataSubset$EVTYPE)
# DRY
stormDataSubset$EVTYPE <- gsub('.*DRY.*', 'DRY', stormDataSubset$EVTYPE)
# DUST
stormDataSubset$EVTYPE <- gsub('.*DUST.*', 'DUST', stormDataSubset$EVTYPE)
# FIRE
stormDataSubset$EVTYPE <- gsub('.*FIRE.*', 'FIRE', stormDataSubset$EVTYPE)
# FLOOD
stormDataSubset$EVTYPE <- gsub('.*FLOOD.*', 'FLOOD', stormDataSubset$EVTYPE)
# FOG
stormDataSubset$EVTYPE <- gsub('.*FOG.*', 'FOG', stormDataSubset$EVTYPE)
# HAIL
stormDataSubset$EVTYPE <- gsub('.*HAIL.*', 'HAIL', stormDataSubset$EVTYPE)
# HEAT
stormDataSubset$EVTYPE <- gsub('.*HEAT.*', 'HEAT', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*WARM.*', 'HEAT', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*HIGH.*TEMP.*', 'HEAT', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*RECORD HIGH TEMPERATURES.*', 'HEAT', stormDataSubset$EVTYPE)
# HYPOTHERMIA/EXPOSURE
stormDataSubset$EVTYPE <- gsub('.*HYPOTHERMIA.*', 'HYPOTHERMIA/EXPOSURE', stormDataSubset$EVTYPE)
# LANDSLIDE
stormDataSubset$EVTYPE <- gsub('.*LANDSLIDE.*', 'LANDSLIDE', stormDataSubset$EVTYPE)
# LIGHTNING
stormDataSubset$EVTYPE <- gsub('^LIGHTNING.*', 'LIGHTNING', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('^LIGNTNING.*', 'LIGHTNING', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('^LIGHTING.*', 'LIGHTNING', stormDataSubset$EVTYPE)
# MICROBURST
stormDataSubset$EVTYPE <- gsub('.*MICROBURST.*', 'MICROBURST', stormDataSubset$EVTYPE)
# MUDSLIDE
stormDataSubset$EVTYPE <- gsub('.*MUDSLIDE.*', 'MUDSLIDE', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*MUD SLIDE.*', 'MUDSLIDE', stormDataSubset$EVTYPE)
# RAIN
stormDataSubset$EVTYPE <- gsub('.*RAIN.*', 'RAIN', stormDataSubset$EVTYPE)
# RIP CURRENT
stormDataSubset$EVTYPE <- gsub('.*RIP CURRENT.*', 'RIP CURRENT', stormDataSubset$EVTYPE)
# STORM
stormDataSubset$EVTYPE <- gsub('.*STORM.*', 'STORM', stormDataSubset$EVTYPE)
# SUMMARY
stormDataSubset$EVTYPE <- gsub('.*SUMMARY.*', 'SUMMARY', stormDataSubset$EVTYPE)
# TORNADO
stormDataSubset$EVTYPE <- gsub('.*TORNADO.*', 'TORNADO', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*TORNDAO.*', 'TORNADO', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*LANDSPOUT.*', 'TORNADO', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*WATERSPOUT.*', 'TORNADO', stormDataSubset$EVTYPE)
# SURF
stormDataSubset$EVTYPE <- gsub('.*SURF.*', 'SURF', stormDataSubset$EVTYPE)
# VOLCANIC
stormDataSubset$EVTYPE <- gsub('.*VOLCANIC.*', 'VOLCANIC', stormDataSubset$EVTYPE)
# WET
stormDataSubset$EVTYPE <- gsub('.*WET.*', 'WET', stormDataSubset$EVTYPE)
# WIND
stormDataSubset$EVTYPE <- gsub('.*WIND.*', 'WIND', stormDataSubset$EVTYPE)
# WINTER
stormDataSubset$EVTYPE <- gsub('.*WINTER.*', 'WINTER', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*WINTRY.*', 'WINTER', stormDataSubset$EVTYPE)
stormDataSubset$EVTYPE <- gsub('.*SNOW.*', 'WINTER', stormDataSubset$EVTYPE)After tidying the dataset, the number of unique Event Type values are reduced to 81
Clean Date Data
In the raw dataset, the BNG_START and
END_DATE variables are stored as characters. These must be
converted to actual date types. For now, time variables will be
ignored.
Create four new variables based on date variables in the tidy dataset:
| Variable | Description |
|---|---|
| DATE_START | Event start date stored as a date |
| DATE_END | End date of the event stored as a date |
| YEAR | Year the event started |
| DURATION | Duration (in hours) of the event |
stormDataSubset$DATE_START <- as.Date(stormDataSubset$BGN_DATE, format = "%m/%d/%Y")
stormDataSubset$DATE_END <- as.Date(stormDataSubset$END_DATE, format = "%m/%d/%Y")
stormDataSubset$YEAR <- as.integer(format(stormDataSubset$DATE_START, "%Y"))
stormDataSubset$DURATION <- as.numeric(stormDataSubset$DATE_END - stormDataSubset$DATE_START)/3600Clean Economic Data
According to the “National Weather Service Storm
Data Documentation” (page 12), information about Property Damage is
logged using two variables: PROPDMG and
PROPDMGEXP. PROPDMG is the mantissa (the
significand) rounded to three significant digits and
PROPDMGEXP is the exponent (the multiplier). The same
approach is used for Crop Damage where the CROPDMG variable
is encoded by the CROPDMGEXP variable.
The documentation also specifies that the PROPDMGEXP and
CROPDMGEXP are supposed to contain an alphabetical
character used to signify magnitude such as “K” for thousands, “M” for
millions, and “B” for billions. A quick review of the data, however,
shows that there are several other characters being logged.
table(toupper(stormDataSubset$PROPDMGEXP))##
## - + 0 2 3 4 5 6 7 B
## 11585 1 5 210 1 1 4 18 3 3 40
## H K M
## 7 231427 11327table(toupper(stormDataSubset$CROPDMGEXP))##
## ? 0 B K M
## 152663 6 17 7 99953 1986In order to calculate costs, the PROPDMGEXP and
CROPDMGEXP variables will be mapped to a multiplier factor
which will then be used to calculate the actual costs for both property
and crop damage. Two new variables will be created to store damage
costs:
- PROP_COST
- CROP_COST
# function to get multiplier factor
getMultiplier <- function(exp) {
exp <- toupper(exp);
if (exp == "") return (10^0);
if (exp == "-") return (10^0);
if (exp == "?") return (10^0);
if (exp == "+") return (10^0);
if (exp == "0") return (10^0);
if (exp == "1") return (10^1);
if (exp == "2") return (10^2);
if (exp == "3") return (10^3);
if (exp == "4") return (10^4);
if (exp == "5") return (10^5);
if (exp == "6") return (10^6);
if (exp == "7") return (10^7);
if (exp == "8") return (10^8);
if (exp == "9") return (10^9);
if (exp == "H") return (10^2);
if (exp == "K") return (10^3);
if (exp == "M") return (10^6);
if (exp == "B") return (10^9);
return (NA);
}
# calculate property damage and crop damage costs (in billions)
stormDataSubset$PROP_COST <- with(stormDataSubset, as.numeric(PROPDMG) * sapply(PROPDMGEXP, getMultiplier))/10^9
stormDataSubset$CROP_COST <- with(stormDataSubset, as.numeric(CROPDMG) * sapply(CROPDMGEXP, getMultiplier))/10^9Summarize Data
Create a summarized dataset of health impact data (fatalities + injuries). Sort the results in descending order by health impact.
healthImpactData <- aggregate(x = list(HEALTH_IMPACT = stormDataSubset$FATALITIES + stormDataSubset$INJURIES), by = list(EVENT_TYPE = stormDataSubset$EVTYPE), FUN = sum, na.rm = TRUE)
healthImpactData <- healthImpactData[order(healthImpactData$HEALTH_IMPACT, decreasing = TRUE),]Create a summarized dataset of damage impact costs (property damage + crop damage). Sort the results in descending order by damage cost.
damageCostImpactData <- aggregate(x = list(DAMAGE_IMPACT = stormDataSubset$PROP_COST + stormDataSubset$CROP_COST), by = list(EVENT_TYPE = stormDataSubset$EVTYPE), FUN = sum, na.rm = TRUE)
damageCostImpactData <- damageCostImpactData[order(damageCostImpactData$DAMAGE_IMPACT, decreasing = TRUE),]Results
Event Types Most Harmful to Population Health
Fatalities and injuries have the most harmful impact on population health. The results below display the 10 most harmful weather events in terms of population health in the U.S.
print(xtable(head(healthImpactData, 10), caption = "Top 10 Most Harmful to Population Health Weather Events"), caption.placement = 'top', type = "html", include.rownames = FALSE, html.table.attributes='class="table-bordered", width="100%"')| EVENT_TYPE | HEALTH_IMPACT |
|---|---|
| TORNADO | 97075.00 |
| HEAT | 12392.00 |
| FLOOD | 10127.00 |
| WIND | 9893.00 |
| LIGHTNING | 6049.00 |
| STORM | 4780.00 |
| COLD | 3100.00 |
| WINTER | 1924.00 |
| FIRE | 1698.00 |
| HAIL | 1512.00 |
healthImpactChart <- ggplot(head(healthImpactData, 10), aes(x = reorder(EVENT_TYPE, HEALTH_IMPACT), y = HEALTH_IMPACT, fill = EVENT_TYPE)) + coord_flip() + geom_bar(stat = "identity") + xlab("Event Type") + ylab("Total Fatalities and Injures") + theme(plot.title = element_text(size = 14, hjust = 0.5)) + ggtitle("Top 10 Weather Events Most Harmful to\nPopulation Health")
print(healthImpactChart)
Event Types with Greatest Economic Consequences
Property and crop damage have the most harmful impact on the economy. The results below display the 10 most harmful weather events in terms economic consequences in the U.S.
print(xtable(head(damageCostImpactData, 10), caption = "Top 10 Weather Events with Greatest Economic Consequences"), caption.placement = 'top', type = "html", include.rownames = FALSE, html.table.attributes='class="table-bordered", width="100%"')| EVENT_TYPE | DAMAGE_IMPACT |
|---|---|
| FLOOD | 180.58 |
| HURRICANE/TYPHOON | 71.91 |
| STORM | 70.45 |
| TORNADO | 57.43 |
| HAIL | 20.74 |
| DROUGHT | 15.02 |
| HURRICANE | 14.61 |
| COLD | 12.70 |
| WIND | 12.01 |
| FIRE | 8.90 |
damageCostImpactChart <- ggplot(head(damageCostImpactData, 10), aes(x = reorder(EVENT_TYPE, DAMAGE_IMPACT), y = DAMAGE_IMPACT, fill = EVENT_TYPE)) + coord_flip() + geom_bar(stat = "identity") + xlab("Event Type") + ylab("Total Property / Crop Damage Cost\n(in Billions)") + theme(plot.title = element_text(size = 14, hjust = 0.5)) + ggtitle("Top 10 Weather Events with\nGreatest Economic Consequences")
print(damageCostImpactChart)
Conclusion
Based on the evidence demonstrated in this analysis and supported by the included data and graphs, the following conclusions can be drawn:
Which types of events (as indicated in the EVTYPE variable) are most harmful to population health?
Tornadoes are responsible for the greatest number of fatalities and injuries.
Which types of events have the greatest economic consequences?
Floods are responsible for causing the most property damage and crop damage costs.