The analysis below is applied on the NOAA Storm database which contains information about natural events occurred in USA between years 1950-2011. After processing and cleaning the data, the results are shown in 3 separate figures which indicate the frequency of events, health problems, and the damage caused to the properties or crops.
First we load the raw data
raw_data <- read.csv(file = "~/R Working Directory/repdata_data_StormData.csv.bz2")
Let’s take a look at first 5 rows
head(raw_data)
## STATE__ BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE EVTYPE
## 1 1 4/18/1950 0:00:00 0130 CST 97 MOBILE AL TORNADO
## 2 1 4/18/1950 0:00:00 0145 CST 3 BALDWIN AL TORNADO
## 3 1 2/20/1951 0:00:00 1600 CST 57 FAYETTE AL TORNADO
## 4 1 6/8/1951 0:00:00 0900 CST 89 MADISON AL TORNADO
## 5 1 11/15/1951 0:00:00 1500 CST 43 CULLMAN AL TORNADO
## 6 1 11/15/1951 0:00:00 2000 CST 77 LAUDERDALE AL TORNADO
## BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END COUNTYENDN
## 1 0 0 NA
## 2 0 0 NA
## 3 0 0 NA
## 4 0 0 NA
## 5 0 0 NA
## 6 0 0 NA
## END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES INJURIES PROPDMG
## 1 0 14.0 100 3 0 0 15 25.0
## 2 0 2.0 150 2 0 0 0 2.5
## 3 0 0.1 123 2 0 0 2 25.0
## 4 0 0.0 100 2 0 0 2 2.5
## 5 0 0.0 150 2 0 0 2 2.5
## 6 0 1.5 177 2 0 0 6 2.5
## PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES LATITUDE LONGITUDE
## 1 K 0 3040 8812
## 2 K 0 3042 8755
## 3 K 0 3340 8742
## 4 K 0 3458 8626
## 5 K 0 3412 8642
## 6 K 0 3450 8748
## LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1 3051 8806 1
## 2 0 0 2
## 3 0 0 3
## 4 0 0 4
## 5 0 0 5
## 6 0 0 6
If we take a look to variety of event types { table(raw_data$EVTYPE)}, we’d see that there are some mistakes in entering the values(i.e. Spelling mistakes) and also some of them are in uppercase or lowercase. Some have same meaning but have been written differently. So, let’s fix that:
raw_data$EVTYPE <- tolower(raw_data$EVTYPE)
Flood: an overflow of water onto normally dry land.
Thunderstorm: A storm with thunder and lightning and typically so heavy rain or hail.
Lightning: a visible electrical discharge from a cloud.
Wind: Moving air; can be soft, moderate, or strong.
Extreme Heat
Drought: a deficiency in precipitation over an extended period.
Astronomical Tides: The tidal levels and water motion which would result from Earth’s rotation and gravitational effects of, the Earth, Sun and Moon, without any atmospheric influences.
Avalanche: a mass of snow, ice, and rocks falling rapidly down a mountainside.
Blizzard: a winter storm + wind + reduced visibility.
Fire
Extreme Cold
Fog
Dust
Frost: Different than blizzards and winter storms. Usually occur after snowfalls, winter storms, or rainfalls.
Heavy Rain
Snowfall
Funnel Cloud: a cone-shaped cloud which extends from the base of a cloud towards the ground without actually reaching the surface.
Hail: a form of precipitation consisting of solid ice.
Hurricane: Different than tornado.
Ice Storm: a storm of freezing rain that leaves a coating of ice.
Landslide: a collapse of a mass of earth or rock from a mountain or cliff.
Tornado: a violent destructive whirling wind accompanied by a funnel-shaped cloud.
High Surfs: ocean waves that are larger and more powerful than usual.
Winter Storm: Blizzards are winter storms but more dangerous.
Waterspout: a whirling column of air and water mist
Others
library(stringr)
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "thunderstorm")] <- "Thunderstorm"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "tornado")] <- "Tornado"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "flood")] <- "Flood"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "wind")] <- "Wind"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "lightning")] <- "Lightning"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "snow")] <- "Snowfall"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "rain")] <- "Heavy Rain"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "winter")] <- "Winter Storm"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "funnel")] <- "Funnel Cloud"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "waterspout")] <- "Waterspout"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "fld")] <- "Flood"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "fire")] <- "Fire"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "blizzard")] <- "Blizzard"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "drought")] <- "Drought"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "ice storm")] <- "Ice Storm"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "heat")] <- "Extreme Heat"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "frost")] <- "Frost"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "fog")] <- "Fog"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "high surf")] <- "High Surf"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "tropical storm")] <- "Hurricane"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "hurricane")] <- "Hurricane"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "cold")] <- "Extreme Cold"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "landslide")] <- "Landslide"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "rip current")] <- "High Surf"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "dust")] <- "Dust"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "avalanche")] <- "Avalanche"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "storm surge")] <- "High Surf"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "tstm")] <- "Thunderstorm"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "hail")] <- "Hail"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "dry")] <- "Drought"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "astronomical")] <- "Astronomical Tides"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "warm")] <- "Extreme Heat"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "wintry")] <- "Winter Storm"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "surf")] <- "High Surf"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "freez")] <- "Frost"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "ice")] <- "Ice Storm"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "tropical depression")] <- "Hurricane"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "sleet")] <- "Hail"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "typhoon")] <- "Hurricane"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "tsunami")] <- "High Surf"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "glaze")] <- "Frost"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "mudslide")] <- "Landslide"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "mud slide")] <- "Landslide"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "icy")] <- "Frost"
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = "hot")] <- "Extreme Heat"
others <- names(sort(table(raw_data$EVTYPE)))
for (i in others[2:180]) {
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = i)] <- "Others"
}
raw_data$EVTYPE[str_detect(string = raw_data$EVTYPE ,pattern = '\\?')] <- "Others"
There are k(Thousand), M(Million), B(Billion), and H(Hundred). Let’s make them all uppercase:
raw_data$PROPDMGEXP <- toupper(raw_data$PROPDMGEXP)
raw_data$CROPDMGEXP <- toupper(raw_data$CROPDMGEXP)
We consider other signs as 1-unit. Now, let’s turn these numbers to their real values:
for (i in 1:902297) {
if (raw_data$PROPDMG[i] == 0) {
}
else if (raw_data$PROPDMGEXP[i] == "K") {
raw_data$PROPDMG[i] <- raw_data$PROPDMG[i]*1000
}
else if (raw_data$PROPDMGEXP[i] == "H") {
raw_data$PROPDMG[i] <- raw_data$PROPDMG[i]*100
}
else if (raw_data$PROPDMGEXP[i] == "M") {
raw_data$PROPDMG[i] <- raw_data$PROPDMG[i]*1000000
}
else if (raw_data$PROPDMGEXP[i] == "B") {
raw_data$PROPDMG[i] <- raw_data$PROPDMG[i]*1000000000
}
}
and the same operation for crop damage:
for (k in 1:902297) {
if (raw_data$CROPDMG[k] == 0) {
}
else if (raw_data$CROPDMGEXP[k] == "K") {
raw_data$CROPDMG[k] <- raw_data$CROPDMG[i]*1000
}
else if (raw_data$CROPDMGEXP[k] == "H") {
raw_data$CROPDMG[k] <- raw_data$CROPDMG[i]*100
}
else if (raw_data$CROPDMGEXP[k] == "M") {
raw_data$CROPDMG[k] <- raw_data$CROPDMG[i]*1000000
}
else if (raw_data$CROPDMGEXP[k] == "B") {
raw_data$CROPDMG[k] <- raw_data$CROPDMG[k]*1000000000
}
}
library(ggplot2)
g <- ggplot(raw_data, aes(EVTYPE)) + theme(axis.text.x = element_text(angle = 90)) + xlab("Event") + ylab("Count") + ggtitle("The frequency of events in US" , subtitle = "since 1950 to 2011") + scale_y_continuous(labels = scales::comma) + theme(plot.title = element_text(hjust = 0.5)) +theme(plot.subtitle = element_text(hjust = 0.5))
g + geom_bar()
library(dplyr)
##
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
##
## filter, lag
## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, union
Health <- summarise(group_by(.data = raw_data , EVTYPE) , Fatalities = sum(FATALITIES) , Injuries = sum(INJURIES))
f <- ggplot(Health, aes(x = EVTYPE)) + theme(axis.text.x = element_text(angle = 90)) + xlab("Event") + ylab("Count") + ggtitle("Injuries and Fatalities causing by each event" , subtitle = "since 1950 to 2011") + scale_y_continuous(labels = scales::comma) + theme(plot.title = element_text(hjust = 0.5)) +theme(plot.subtitle = element_text(hjust = 0.5))
f + geom_bar(aes(y = Injuries, fill = "Injuries"), stat = "identity", position = "dodge") + geom_bar(aes(y = Fatalities, fill = "Fatalities"), stat = "identity", position = "dodge")
Considering the graph above, we now know that Tornado is the deadliest event.
Damage <- summarise(group_by(.data = raw_data , EVTYPE) , PropDMG = sum(PROPDMG) , CropDMG = sum(CROPDMG))
h <- ggplot(Damage, aes(x = EVTYPE)) + theme(axis.text.x = element_text(angle = 90)) + xlab("Event") + ylab("Damage in $") + ggtitle("Economic Consequences causing by each event" , subtitle = "since 1950 to 2011") + scale_y_continuous(labels = scales::comma) + theme(plot.title = element_text(hjust = 0.5)) +theme(plot.subtitle = element_text(hjust = 0.5))
h + geom_bar(aes(y = PropDMG, fill = "PropDMG"), stat = "identity", position = "dodge") + geom_bar(aes(y = CropDMG, fill = "CropDMG"), stat = "identity", position = "dodge")
Considering the graph above, we now know that Flood is the most destructive event.