Reproducible Research - Course Project No.2
IMGA
November 24 2020
Analysis: A look into the U.S. National Oceanic and Atmospheric Administration's (NOAA) storm database
Synopsis
This document presents an analysis based on data obtained from the U.S. National Oceanic and Atmospheric Administration's (NOAA) storm database. This data presents characteristics of major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage.
The report is separated in three parts. First, we present the process of getting and cleaning the data, reviewing its characteristics and making the corresponding transformations, if applicably. Next, we proceed to analize the effects of these weather events on two aspects: Population Health and the Economy.
The results show that, regarding the population health, the events that have the highest impact are the tornados. When we speak about the economy, the events related with water, such as floods cause the highest damage, with the maximum cost for damage caused to properties and agricultural resources.
Part 1: Data Processing
First we download the data from the U.S. National Oceanic and Atmospheric Administration's (NOAA) storm database. After reviewing the structure, we make the corresponding transformations to adapt it into a suitable form for our analysis.
#Path to read files
path1 <- "C:/Users/Economics05/Documents/Coursera/RR_CP2"
fileUrl1 <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
filename <- "StormData.csv.bz2"
download.file(fileUrl1, filename, method = "curl")
#Read the file
stormdata <- read.csv(file.path(path1,filename),header = TRUE)After reading the file, we proceed to check the data structure and review it for any changes.
dim(stormdata)## [1] 902297 37head(stormdata)## STATE__ BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE EVTYPE
## 1 1 4/18/1950 0:00:00 0130 CST 97 MOBILE AL TORNADO
## 2 1 4/18/1950 0:00:00 0145 CST 3 BALDWIN AL TORNADO
## 3 1 2/20/1951 0:00:00 1600 CST 57 FAYETTE AL TORNADO
## 4 1 6/8/1951 0:00:00 0900 CST 89 MADISON AL TORNADO
## 5 1 11/15/1951 0:00:00 1500 CST 43 CULLMAN AL TORNADO
## 6 1 11/15/1951 0:00:00 2000 CST 77 LAUDERDALE AL TORNADO
## BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END COUNTYENDN
## 1 0 0 NA
## 2 0 0 NA
## 3 0 0 NA
## 4 0 0 NA
## 5 0 0 NA
## 6 0 0 NA
## END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES INJURIES PROPDMG
## 1 0 14.0 100 3 0 0 15 25.0
## 2 0 2.0 150 2 0 0 0 2.5
## 3 0 0.1 123 2 0 0 2 25.0
## 4 0 0.0 100 2 0 0 2 2.5
## 5 0 0.0 150 2 0 0 2 2.5
## 6 0 1.5 177 2 0 0 6 2.5
## PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES LATITUDE LONGITUDE
## 1 K 0 3040 8812
## 2 K 0 3042 8755
## 3 K 0 3340 8742
## 4 K 0 3458 8626
## 5 K 0 3412 8642
## 6 K 0 3450 8748
## LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1 3051 8806 1
## 2 0 0 2
## 3 0 0 3
## 4 0 0 4
## 5 0 0 5
## 6 0 0 6str(stormdata)## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
## $ BGN_TIME : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
## $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
## $ STATE : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ EVTYPE : Factor w/ 985 levels " HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : Factor w/ 35 levels ""," N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_LOCATI: Factor w/ 54429 levels "","- 1 N Albion",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_DATE : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_TIME : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_LOCATI: Factor w/ 34506 levels "","- .5 NNW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ WFO : Factor w/ 542 levels ""," CI","$AC",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ ZONENAMES : Factor w/ 25112 levels ""," "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : Factor w/ 436781 levels "","-2 at Deer Park\n",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...As we can see, we need to transform date variables into a suitable format.
#Transform the dates
stormdata$BGN_DATE <- as.Date(stormdata$BGN_DATE,"%m/%d/%Y")
stormdata$END_DATE <- as.Date(stormdata$END_DATE,"%m/%d/%Y")Part 2: Events and its effects on population health - Results
In order to show the effects of severe weather events on the population, we analyze which one of them are the most harmful with respect to population heath, across the United States.
Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?
library(ggplot2)
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionWe analyze which are the events that have cause the higher amount of injuries among the population. First, we filter the top 1% of events that have caused the higher amount of injuries, by individual event. Then we filter the data to get the events that belong to this group.
topq <- quantile(stormdata$INJURIES, probs = 0.99)
topq## 99%
## 2storm1 <- tbl_df(stormdata)
top <- storm1 %>% select(EVTYPE,INJURIES,BGN_DATE,STATE) %>% filter(INJURIES>topq) %>% arrange(desc(INJURIES))
top20_injuries <- top[1:20, ]
head(top20_injuries)## # A tibble: 6 x 4
## EVTYPE INJURIES BGN_DATE STATE
## <fct> <dbl> <date> <fct>
## 1 TORNADO 1700 1979-04-10 TX
## 2 ICE STORM 1568 1994-02-08 OH
## 3 TORNADO 1228 1953-06-09 MA
## 4 TORNADO 1150 1974-04-03 OH
## 5 TORNADO 1150 2011-05-22 MO
## 6 FLOOD 800 1998-10-17 TXmaxinjuries <- prettyNum(max(top$INJURIES), big.mark = ",")
maxinjuries## [1] "1,700"maxinjuriestate <- as.vector(top20_injuries$STATE[1])
maxinjuriestate## [1] "TX"As we can see the 1% of these events, when filtered, gives you the event with the highest number of injuries among the population and it shows that a tornado had the highest impact, reaching 1,700 injuries and it ocurred in the state of TX.
When we procced to group the events by type, we get a look at which ones have been the most dangerous to the population.
storm1 <- tbl_df(stormdata)
topinjuries <- storm1 %>% select(EVTYPE,INJURIES,BGN_DATE,STATE) %>% group_by(EVTYPE, .drop= FALSE) %>% summarise(numinjuries = sum(INJURIES)) %>% arrange(desc(numinjuries))
top5_injuries <- topinjuries[1:5, ]
maxinj <- prettyNum(max(topinjuries$numinjuries), big.mark = ",")
maxinj## [1] "91,346"maxinjevent <- as.vector(top5_injuries$EVTYPE[1])
maxinjevent## [1] "TORNADO"top5_injuries## # A tibble: 5 x 2
## EVTYPE numinjuries
## <fct> <dbl>
## 1 TORNADO 91346
## 2 TSTM WIND 6957
## 3 FLOOD 6789
## 4 EXCESSIVE HEAT 6525
## 5 LIGHTNING 5230graph2 <- ggplot(data = top5_injuries, aes(EVTYPE, numinjuries)) + geom_bar(stat = "identity", fill = "Dark Green") + labs(x="Type of event", y="Number of injuries") + ggtitle("Weather events with highest number of injuries in the US: Top 5") + theme(axis.text.x = element_text(angle = 90))+ theme_minimal()
graph2
As the graph shows, the highest number of injuries belongs, again, to the TORNADO with 91,346 .
When we talk about population health, we need also to review the number of fatalities caused by these events. For this section, we proceed to review which events have the highest number of attribuited deaths.
storm1 <- tbl_df(stormdata)
topfat <- storm1 %>% select(EVTYPE,FATALITIES,BGN_DATE,STATE) %>% group_by(EVTYPE) %>% summarise(numfatalities = sum(FATALITIES)) %>% arrange(desc(numfatalities)) ## `summarise()` ungrouping output (override with `.groups` argument)top20_fatalities <- topfat[1:20, ]
maxfat <- prettyNum(max(topfat$numfatalities), big.mark = ",")
maxfat## [1] "5,633"maxfatalitiesevent <- as.vector(top20_fatalities$EVTYPE[1])
maxfatalitiesevent## [1] "TORNADO"top5fat <- topfat[1:5, ]
top5fat## # A tibble: 5 x 2
## EVTYPE numfatalities
## <fct> <dbl>
## 1 TORNADO 5633
## 2 EXCESSIVE HEAT 1903
## 3 FLASH FLOOD 978
## 4 HEAT 937
## 5 LIGHTNING 816graph3 <- ggplot(data = top20_fatalities, aes(EVTYPE, numfatalities)) + geom_bar(stat = "identity", fill = "Dark Green") + labs(x="Type of event", y="Number of fatalities") + ggtitle("Weather events with highest number of fatalities in the US: Top 20")+ theme(axis.text.x = element_text(angle = 90))
graph3
When we review the top 5 number of fatalities caused by weather events, we can see again the influence of the wind-related events among the population health. The highest number of casualities, 5,633, are caused by the TORNADO. Also, we see events related to hotter climat, with the Excessive Heat and Heat, comming in second and fourth place. Flash Flood, in third and Lightning in fifth.
Part 3: Events and its effects on the economy - Results
Across the United States, which types of events have the greatest economic consequences?
First, we transform the data of the property and crop damage into comparable magnitudes, using as reference the alfabethical characters "K" for thousands, "M" for millions, and "B" for billions, that has been established by the NOAA data documentation. The cells with no character (NA, ?,-,+) get a factor value of one (10^0), the numeric ones get a factor of 10 (10^1). After this, we can proceed to compare the effects.
storm1$PROPDMGEXP <- toupper(storm1$PROPDMGEXP)
storm1$CROPDMGEXP <- toupper(storm1$CROPDMGEXP)
storm1$PROPDMGEXP <- as.character(storm1$PROPDMGEXP)
storm1$CROPDMGEXP <- as.character(storm1$CROPDMGEXP)
mag1 <- table(storm1$PROPDMGEXP)
mag2 <- table(storm1$CROPDMGEXP)
mag1##
## - ? + 0 1 2 3 4 5 6
## 465934 1 8 5 216 25 13 4 4 28 4
## 7 8 B H K M
## 5 1 40 7 424665 11337mag2##
## ? 0 2 B K M
## 618413 7 19 1 9 281853 1995#Every value must be converted to USD Dollars with a factor, when we have numbers, we assume of 10^1 for the factor, since the meaning is not specified in the data documentation manual.
storm1$factorprop[(storm1$PROPDMGEXP == "")] <- 10^0## Warning: Unknown or uninitialised column: `factorprop`.storm1$factorprop[(storm1$PROPDMGEXP == "-")] <- 10^0
storm1$factorprop[(storm1$PROPDMGEXP == "?")] <- 10^0
storm1$factorprop[(storm1$PROPDMGEXP == "+")] <- 10^0
storm1$factorprop[(storm1$PROPDMGEXP == "0")] <- 10^0
storm1$factorprop[(storm1$PROPDMGEXP == "1")] <- 10^1
storm1$factorprop[(storm1$PROPDMGEXP == "2")] <- 10^1
storm1$factorprop[(storm1$PROPDMGEXP == "3")] <- 10^1
storm1$factorprop[(storm1$PROPDMGEXP == "4")] <- 10^1
storm1$factorprop[(storm1$PROPDMGEXP == "5")] <- 10^1
storm1$factorprop[(storm1$PROPDMGEXP == "6")] <- 10^1
storm1$factorprop[(storm1$PROPDMGEXP == "7")] <- 10^1
storm1$factorprop[(storm1$PROPDMGEXP == "8")] <- 10^1
storm1$factorprop[(storm1$PROPDMGEXP == "B")] <- 10^9
storm1$factorprop[(storm1$PROPDMGEXP == "H")] <- 10^2
storm1$factorprop[(storm1$PROPDMGEXP == "K")] <- 10^3
storm1$factorprop[(storm1$PROPDMGEXP == "M")] <- 10^6
storm1$factorcrop[(storm1$CROPDMGEXP == "")] <- 10^0## Warning: Unknown or uninitialised column: `factorcrop`.storm1$factorcrop[(storm1$CROPDMGEXP == "?")] <- 10^0
storm1$factorcrop[(storm1$CROPDMGEXP == "0")] <- 10^0
storm1$factorcrop[(storm1$CROPDMGEXP == "2")] <- 10^0
storm1$factorcrop[(storm1$CROPDMGEXP == "B")] <- 10^9
storm1$factorcrop[(storm1$CROPDMGEXP == "K")] <- 10^3
storm1$factorcrop[(storm1$CROPDMGEXP == "M")] <- 10^6
#Convert Property Damage and Crop Damage in comparable magnitudes
ecdamage <- storm1 %>% select(EVTYPE,INJURIES,FATALITIES,BGN_DATE,STATE, PROPDMG, factorprop, CROPDMG, factorcrop) %>% mutate(Proptotal = PROPDMG * factorprop) %>% mutate(Croptotal = CROPDMG * factorcrop) %>% mutate(Totaldmg = Proptotal + Croptotal)
top2 <- quantile(ecdamage$Totaldmg, probs = 0.99)
top2## 99%
## 2e+06top_economy <- ecdamage %>% filter(Totaldmg>top2) %>% arrange(desc(Totaldmg))
top20_economy <- top_economy[1:20, ]
graph4 <- ggplot(data = top20_economy, aes(x= EVTYPE,Totaldmg)) + geom_bar(stat = "identity", fill = "Dark Green") + labs(x="Type of event", y="Total damage expressed in US Dollars") + ggtitle("Weather events with the greatest economic consequenses in the US") + theme(axis.text.x = element_text(angle = 90))
graph4
topcostevent <- as.vector(top20_economy$EVTYPE[1])
topcostevent## [1] "FLOOD"topcostusd <- prettyNum(top20_economy$Totaldmg[1], big.mark = ",")
topcostusd## [1] "115,032,500,000"As we can see, the highest cost for the economy is directly related to FLOOD with a total amount of 115,032,500,000 US dollars.
Conclusion
After reviewing the previous data, we can say that weather-related events have a lasting impact on population health and the economy. That is why, is always important to be alert and prepared with all the available information, to prevent major consequences.