Assignment Two: Reproducible Research

SYNOPSIS

This project explore the relationship between severe weather events that are being harmful for the health of persons living in the United States, and the health and economic impact of this kind of events. For that, the project resolves two main questions:

“1. Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?”

“2. Across the United States, which types of events have the greatest economic consequences?”

This project followed the next steps to make possible the analysis that is presented:

PREVIOUS STEPS

  1. Setting the work directory
  2. Installing packages and loading libraries

DATA PROCESSING

  1. Getting the database
  2. Cleaning and fixing the database

RESULTS PRESENTATION

CONCLUSION

—————————————————-

DEVELOPMENT

PREVIOUS STEPS

1. Setting the work directory
## here() starts at /Users/ESLAVA/Desktop/Programming in R/Learning2
## [1] "/Users/ESLAVA/Desktop/Programming in R/Learning2"

2. Installing packages and loading libraries

library(data.table)
library(R.utils) 
## Loading required package: R.oo
## Loading required package: R.methodsS3
## R.methodsS3 v1.7.1 (2016-02-15) successfully loaded. See ?R.methodsS3 for help.
## R.oo v1.22.0 (2018-04-21) successfully loaded. See ?R.oo for help.
## 
## Attaching package: 'R.oo'
## The following objects are masked from 'package:methods':
## 
##     getClasses, getMethods
## The following objects are masked from 'package:base':
## 
##     attach, detach, gc, load, save
## R.utils v2.6.0 (2017-11-04) successfully loaded. See ?R.utils for help.
## 
## Attaching package: 'R.utils'
## The following object is masked from 'package:utils':
## 
##     timestamp
## The following objects are masked from 'package:base':
## 
##     cat, commandArgs, getOption, inherits, isOpen, parse, warnings
library(downloader)
library(lubridate)
## Warning: package 'lubridate' was built under R version 3.4.4
## 
## Attaching package: 'lubridate'
## The following objects are masked from 'package:data.table':
## 
##     hour, isoweek, mday, minute, month, quarter, second, wday,
##     week, yday, year
## The following object is masked from 'package:here':
## 
##     here
## The following object is masked from 'package:base':
## 
##     date
library(plyr)
## 
## Attaching package: 'plyr'
## The following object is masked from 'package:lubridate':
## 
##     here
## The following object is masked from 'package:here':
## 
##     here
library(stats)
library(graphics)
library(lattice)
library(ggplot2)

DATA PROCESSING

1. Getting the database

        if(!file.exists("repdata-data-StormData.csv.bz2")) {
        download.file("https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2", destfile = "repdata-data-StormData.csv.bz2", method = "curl")
}

        url <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
        bz2.data <- "repdata_data_StormData.csv.bz2"
        download(url, bz2.data, mode = "wb")  

        bunzip2(bz2.data, "repdata-data-StormData.csv", overwrite=TRUE, remove = FALSE)

        storm <- read.table("repdata-data-StormData.csv", sep = ",", header = TRUE)

2. Cleaning and fixing the database

Given that the process of downloading the original file is delayed, I will work on a second database, to facilitate the process of running the command several times.

CHANGING THE DATES FORMAT
        storm1 = storm
        

        
        storm1$Begin.Time <- as.POSIXct(strptime(storm1$BGN_DATE, format ="%m/%d/%Y %H:%M:%S"))
        
        ##storm1$End.Time <- as.POSIXct(strptime(storm1$END_DATE, format="%m/%d/%Y %H:%M:%S"))
        
        storm1$Year.Begin <- year(as.Date(storm1$Begin.Time))

Adjusting more frequent names, according with the cathegories in the “NATIONAL WEATHER SERVICE INSTRUCTION 10-1605 document”

In this step, transformations of bad or old terms to describe the events, are changed for the terms included in the official documentations of the owners of the database.

        ## Changing to AVALANCHE
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("AVALANCE" = "Avalanche", "AVALANCHE" = "AVALANCHE")))
        
        ## Changing to Thunderstorm Wind"
        
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("TSTM WIND" = "THUNDERSTORM WIND")))
        
        ## Changing to Rip Current
        
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("RIP CURRENTS" = "RIP CURRENT")))
        
        ## Changing to Extreme Cold/Wind Chill
        
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("EXTREME COLD" = "EXTREME COLD/WIND CHILL", "EXTREME WINDCHILL" = "EXTREME COLD/WIND CHILL")))
        
        ## Changing to Excessive Heat
        
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("EXTREME HEAT" = "EXCESSIVE HEAT", "RECORD/EXCESSIVE HEAT" = "EXCESSIVE HEAT")))
        
        ## Changing to Hurricane/Typhoon cathegory
        
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("HURRICANE" = "HURRICANE/TYPHOON")))

        ## Changing to High Surf
                
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("HEAVY SURF/HIGH SURF" = "HIGH SURF")))
        
        ## Changing to the new name "Debris Flow"
        
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("LANDSLIDE" = "DEBRIS FLOW")))
        
        ## Changing to Cold/Wind Chill cathegory
        
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("COLD" = "COLD/WIND CHILL")))
        
        ## Changing to High Wind
        
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("HIGH WINDS" = "HIGH WIND")))
        
        ## Changing to Heat
        
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("UNSEASONABLY WARM AND DRY" = "HEAT")))
        
        ## Changing to Heavy Rain, according to the document
        
        storm1 <- transform(storm1,
                            EVTYPE = revalue(storm1$EVTYPE, c("URBAN/SML STREAM FLD" = "HEAVY RAIN")))
        
        ## Changing to Winter Weather
        
        storm1 <- transform(storm1,
                           EVTYPE = revalue(storm1$EVTYPE, c("WINTER WEATHER/MIX" = "WINTER WEATHER")))

Note: The adjustment of names was done until position 38 of events, from highest to lowest, within the ranking of events frequency, without names adjusted

RESULTS

First Question: Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?

For resolving this question, I show first, how Fatalities and Injuries happened per year, during the period, and after, I show how Fatalities and Injuries happened across United States, per Event.

1. Estimating total fatalities and injuries during the period, per year

        library(data.table)
        fatalities.year <- aggregate(storm1$FATALITIES, by = list(storm1$Year.Begin), FUN = sum)
        colnames(fatalities.year) <- c("Year", "Fatalities")
        head(fatalities.year)
##   Year Fatalities
## 1 1950         70
## 2 1951         34
## 3 1952        230
## 4 1953        519
## 5 1954         36
## 6 1955        129
        injuries.year <- aggregate(storm1$INJURIES, by = list(storm1$Year.Begin), FUN = sum)
        colnames(injuries.year) <- c("Year", "Injuries")
        head(injuries.year)
##   Year Injuries
## 1 1950      659
## 2 1951      524
## 3 1952     1915
## 4 1953     5131
## 5 1954      715
## 6 1955      926
        morbimortality.year <- data.frame(cbind(fatalities.year, injuries.year))
        
        ## total cases of mortality and morbidity per year
        
        morbimortality.year$Total <- as.data.table(morbimortality.year$Fatalities + morbimortality.year$Injuries)
        head(morbimortality.year)
##   Year Fatalities Year.1 Injuries   V1
## 1 1950         70   1950      659  729
## 2 1951         34   1951      524  558
## 3 1952        230   1952     1915 2145
## 4 1953        519   1953     5131 5650
## 5 1954         36   1954      715  751
## 6 1955        129   1955      926 1055

2. Estimating total fatalities and injuries, per event type

        fatalities.event <- aggregate(storm1$FATALITIES, by = list(storm1$EVTYPE), FUN = sum)
        colnames(fatalities.event) <- c("Event", "Fatalities")
        fatalities.event.sorted <- fatalities.event[order(-fatalities.event$Fatalities, na.last=TRUE), ][1:30, ]
        head(fatalities.event.sorted)
##                 Event Fatalities
## 825           TORNADO       5633
## 129    EXCESSIVE HEAT       2016
## 149       FLASH FLOOD        978
## 271              HEAT        966
## 457         LIGHTNING        816
## 751 THUNDERSTORM WIND        637
        injuries.event <- aggregate(storm1$INJURIES, by = list(storm1$EVTYPE), FUN = sum)
        colnames(injuries.event) <- c("Event", "Injuries")
        injuries.event.ordered <- injuries.event[order(-injuries.event$Injuries, na.last=TRUE), ][1:30, ]
        head(injuries.event.ordered)
##                 Event Injuries
## 825           TORNADO    91346
## 751 THUNDERSTORM WIND     8445
## 166             FLOOD     6789
## 129    EXCESSIVE HEAT     6680
## 457         LIGHTNING     5230
## 271              HEAT     2100

Now the plot with fatalities and injuries per year

Figure 1. “Weather events most harmful and Fatalities/Injuries Trend, per year, USA, 1950 -2011”

        ##par = c(mfrow = c(1,1), margin = c(50,2,2,1))
        par(mfrow = c(2, 2), mar = c(11.5, 5, 4, 2), las = 3, cex = 0.5, cex.main = 1.4, cex.lab = 1.2)
        
        with(fatalities.event.sorted, barplot(Fatalities, names.arg = fatalities.event.sorted$Event, col = "red", ylab = "Fatalities", main = "Total Fatalities"))
        
        with(injuries.event.ordered, barplot(Injuries, names.arg = injuries.event.ordered$Event, col = "green", ylab = "Injuries", main = "Total Injuries"))
        
        ## Fatalities related with Weather events, USA, 1950-2011
        reg <- lm(Fatalities ~ Year, data = morbimortality.year)
        with(morbimortality.year, plot(Year, Fatalities, ylab = "Fatalities", xlim = c(1945, 2015), ylim = c(0,1200), pch = 19, col ="red", main = "Total Fatalities per year"))
        par(xpd = FALSE)
        abline(reg, col="black")
        
        ## Injuries related with Weather events, USA, 1950-2011
        reg1 <- lm(Injuries ~ Year, data = morbimortality.year)
        with(morbimortality.year, plot(Year, Injuries, ylab = "Injuries", xlim = c(1945, 2015), ylim = c(0,9000),  pch = 19, col = "green", main = "Total Injuries per year"))
             par(xpd = FALSE)
             abline(reg1, col="black")

Note: Source: Author using Storm Data, National Oceanic and Atmospheric Administration (NOAA)

Estimation of total amount of fatalities and injuries during the whole period

             totalfatalities <- sum(morbimortality.year$Fatalities)
             totalinjuries <- sum(morbimortality.year$Injuries)
             ratioif <-  totalinjuries/totalfatalities

Answer: The amount of fatalities and injuries related with these weather events has increased from 1950 to 2011, being higher (this increase) in the group of fatalities. However, the total amount of injuries (1.4052810^{5}) is 9.2788379 times higher than the total amount of fatalities (1.514510^{4}) during the period (Figure 1).

“Tornado” is the event with more fatalities and injuries related during the period (Figure 1).

Excessive heat, flash flood, heat, lightning, thunderstorm wind, rip current, flood, high wind and avalanche, complete the list of the top 10 events causing more fatalities, during the period (Figure 1).

In the case of Injuries, the other weather events that are in the top 10 of more frequent causing them, are: Thunderstorm wind, flood, excessive heat, lightning, heat, ice storm, flash flood, high wind and hail (Figure 1). In the figure 1 the top 30 events related with injuries and fatalities is showed.

Second Question: Across the United States, which types of events have the greatest economic consequences?

For resolving this question, I show first, related total costs of weather events per year, during the period, and after, I show related total costs across United States, per Event. It is relevant to be clear that costs across years are in constant values: they have not been adjusted by inflation or devaluation. This could be done in a future assignment.

Finally, plot of the relationship between total costs with total morbimortality

        morbimortality.year$Total <- morbimortality.year$Fatalities + morbimortality.year$Injuries
        
        total.sum <- sum(morbimortality.year$Total)
        
        head(morbimortality.year)
##   Year Fatalities Year.1 Injuries Total
## 1 1950         70   1950      659   729
## 2 1951         34   1951      524   558
## 3 1952        230   1952     1915  2145
## 4 1953        519   1953     5131  5650
## 5 1954         36   1954      715   751
## 6 1955        129   1955      926  1055
        costs.sum <- sum(total.costs.event$Costs)
        
        sum.costs <- sum(costs.event.ordered$Costs[1:5])
        
        cost5ratio <- (sum.costs/costs.sum)*100

Comment: During the period, there were 1.5567310^{5} fatalities and injuries (together), caused by weather events. The total amount of money lost, in constant values, during the period, were 2.184293510^{21}. In other words, more than 2 thousand trillion of dollars. Among High Wind, Flood Hail, Thunderstorm Wind, Winter Storm and Tornado, the top 5 of weather events, were responsible of losses of U$ 2.006785610^{21}, that represent the 91.8734402 percent of the total costs

Now, I merge both files (morbimortality and costs per year) in a data.frame, to facilitate plotting them together in the next figure

             morbimortality.year <-  as.data.table(morbimortality.year)
             
             total.costs.year <- as.data.table(costs.year)
             
             df <- merge(morbimortality.year, costs.year, by = "Year")
             
             head(df)
##    Year Fatalities Year.1 Injuries Total        Costs
## 1: 1993        298   1993     2149  2447 3.185798e+13
## 2: 1994        344   1994     4161  4505 2.228175e+13
## 3: 1995       1491   1995     4480  5971 3.407079e+16
## 4: 1996        542   1996     2717  3259 2.001576e+16
## 5: 1997        601   1997     3800  4401 8.990892e+15
## 6: 1998        687   1998    11177 11864 1.870919e+17

Figure 2. Economic consequences of weather events, per year and per event, USA, 1950-2011

             par(mfrow = c(2, 2), mar = c(11.5, 5, 4, 2), las = 3, cex = 0.5, cex.main = 1.4, cex.lab = 1.2)
             
             reg2 <- lm(log10(Costs) ~ Year, data = df)
             with(df, plot(log10(Costs) ~ Year, pch = 19, col = "red", ylab = "Total Log(Costs)", main = "Log10(Costs) per Year"))
             abline(reg2, col = "black")
             
             ## log10(Total costs) per event 
             
             with(costs.event.ordered, barplot(log10(Costs), names.arg = costs.event.ordered$Event, col = "red", ylab = "log10(Costs)", main = "Total Costs per event type"))
             
             ## Log(Costs) of Fatalities
             
             with(df, scatter.smooth(Fatalities, log10(Costs), pch = 20, col = "red", main = "Log10(Costs) of Fatalities"))
             
             ## Log(Costs) of Injuries
             
             with(df, scatter.smooth(Injuries, log10(Costs), pch = 20, col = "green", main = "Log10(Costs) of Injuries"))

#### Note: Source: Author using Storm Data, National Oceanic and Atmospheric Administration (NOAA)

Answer: Economic consequences of weather events have increased from 1993. Total costs per event type, show higher consequences in events like High Wind, Flood, Hail, Thunderstorm/Wind and Winter Storm, in the 5 Top events with higher economic consequences (Figure 2).

Costs related with Fatalities and Injuries show higher variability under 600 and 400 fatalities and injuries respectively, as it can be seen in Figure 2.

CONCLUSION

Severe wheather events are frequent in the United States of America, with health and economic consequences. The number of fatalities and injuries caused by these events have increased in the the last years, thing that can be explained by the simultaneous growth of the population, especially in areas with high population density, where the health and economic consequences of severe climatic events may be greater.

Tornados, excessive heat and flash flood cause more fatalities, and Tornado, Thunderstorm wind and Flood caused more Injuries, being the the most harmful weather events with respect to population health.

High Wind, Flood, Hail, Thunderstorm/Wind and Winter Storm were the events that caused more economic losses.

Severe weather events caused economic losses for more than 2 thousand trillion dollars during the period, and the top 5 of them, mentioned above, were responsible for more than 90% of these economic losses.

Acknowledgments: This work was based on :

  1. U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database

  2. “NATIONAL WEATHER SERVICE INSTRUCTION 10-1605 document”.