Reproducible research

Peer-graded Assignment: Course Project 2

Analysis of weather conditions and their impact on public health and economic consequences

Synopsis

Storms and other severe weather events can cause both public health and economic problems for communities and municipalities. Many severe events can result in fatalities, injuries, and property damage, and preventing such outcomes to the extent possible is a key concern.

This project involves exploring the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. The database can be found here. This database tracks characteristics of major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage.

We have performed exploratory data analysis, we analysed the data to see what types of storm events has the biggest impact on public health and greatest economic consequences. We have also checked how many events there are for every year starting from 1950 and up to 2011. In the earlier years of the database there are generally fewer events recorded, most likely due to a lack of good records. More recent years should be considered more complete.

Data Processing

Data download and processing

Let’s download the file:

download.file("https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2", destfile="./data")

And then read the csv file into storm variable, read.csv function does unzipping of the archive under the hood:

storm <- read.csv("stormdata.csv.bz2")

Let’s see how big the data is and check it’s head:

dim(storm)
## [1] 902297     37
head(storm)
##   STATE__           BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE  EVTYPE
## 1       1  4/18/1950 0:00:00     0130       CST     97     MOBILE    AL TORNADO
## 2       1  4/18/1950 0:00:00     0145       CST      3    BALDWIN    AL TORNADO
## 3       1  2/20/1951 0:00:00     1600       CST     57    FAYETTE    AL TORNADO
## 4       1   6/8/1951 0:00:00     0900       CST     89    MADISON    AL TORNADO
## 5       1 11/15/1951 0:00:00     1500       CST     43    CULLMAN    AL TORNADO
## 6       1 11/15/1951 0:00:00     2000       CST     77 LAUDERDALE    AL TORNADO
##   BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END COUNTYENDN
## 1         0                                               0         NA
## 2         0                                               0         NA
## 3         0                                               0         NA
## 4         0                                               0         NA
## 5         0                                               0         NA
## 6         0                                               0         NA
##   END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES INJURIES PROPDMG
## 1         0                      14.0   100 3   0          0       15    25.0
## 2         0                       2.0   150 2   0          0        0     2.5
## 3         0                       0.1   123 2   0          0        2    25.0
## 4         0                       0.0   100 2   0          0        2     2.5
## 5         0                       0.0   150 2   0          0        2     2.5
## 6         0                       1.5   177 2   0          0        6     2.5
##   PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES LATITUDE LONGITUDE
## 1          K       0                                         3040      8812
## 2          K       0                                         3042      8755
## 3          K       0                                         3340      8742
## 4          K       0                                         3458      8626
## 5          K       0                                         3412      8642
## 6          K       0                                         3450      8748
##   LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1       3051       8806              1
## 2          0          0              2
## 3          0          0              3
## 4          0          0              4
## 5          0          0              5
## 6          0          0              6

Data processing

From the dataset documentation and available columns I assume that the most interesting columns are: EVTYPE, FATALITIES, INJURIES, PROPDMG, CROPDMG

Let’s see how many event types (EVTYPE) we have:

length(unique(storm$EVTYPE))
## [1] 985

Let’s see the values:

table(storm$EVTYPE)

The command above generates a long output of 985 values, thus the output is not printed here. The amount of different values is too large, let’s see whether we can do something to reduce the amount of weather event types. We see that for some types of the events there’s plenty of variations of their (same) naming, example: “RECORD WARM,”RECORD WARM TEMPS.”, “RECORD WARMTH”, etc. Also there’s a mix of upper case / lower case variations of spelling of the same events. Let’s convert all to upper case:

storm$EVTYPE <- toupper(storm$EVTYPE)

Let’s see how many event types we have now:

length(unique(storm$EVTYPE))
## [1] 898

So making all values upper case reduced the amount of unique values down to 898, which is good, since Event Type serves us as a factor variable.

Now I will try to fix some spelling errors, reduce spelling variations of the same types of the events. I will write a function that will do that for me:

replaceSpellingVariations <- function(variations, replacement) {
     for(i in variations) {
           storm$EVTYPE[grep(i, storm$EVTYPE)] <- replacement
     }
     storm$EVTYPE
}

And now I am going to use the function defined above:

storm$EVTYPE <- replaceSpellingVariations(c("\\\\", "/", "/ ", " / ", " /", "-", "- ", " -", " - "), " ")
storm$EVTYPE <- replaceSpellingVariations(c("^THUNDERSTORM WIND*" ), "THUNDERSTORM WIND")
storm$EVTYPE <- replaceSpellingVariations(c("UNSEASONABLE COLD"), "UNSEASONABLY COLD")
storm$EVTYPE <- replaceSpellingVariations(c("THUNDERSTORM  WINDS", "THUNDERSTORMS WINDS", "THUNDERSTORMW", "THUNDERSTORMW WINDS", "THUNDERSTORMWINDS", " THUNDERSTROM WIND", "THUNDERSTROM WINDS", "THUNDERTORM WINDS", "THUNDERTSORM WIND", "THUNDESTORM WINDS", "THUNERSTORM WINDS"), "THUNDERSTORM WIND")
storm$EVTYPE <- replaceSpellingVariations(c("TSTM WIND*", "TSTM WND" ), "TSTM WIND")
storm$EVTYPE <- replaceSpellingVariations(c("HIGH WINDS*"), "HIGH WIND")
storm$EVTYPE <- replaceSpellingVariations(c("HAIL*"), "HAIL")

storm$EVTYPE <- replaceSpellingVariations(c("CLOUD.", "CLOUDS"), "CLOUD")

storm$EVTYPE <- replaceSpellingVariations(c("^SUMMARY OF*"), "SUMMARY OF *")
storm$EVTYPE <- replaceSpellingVariations(c("^SUMMARY*"), "SUMMARY ***")

storm$EVTYPE <- replaceSpellingVariations(c("FLOODING", "FLOODINGS", "FLOOODING", "FLOODS", "FLOODIN"), "FLOOD")
storm$EVTYPE <- replaceSpellingVariations(c("STORMS"), "STORM")
storm$EVTYPE <- replaceSpellingVariations(c("WINDS"), "WIND")
storm$EVTYPE <- replaceSpellingVariations(c("TEMPERATURES"), "TEMPERATURE")
storm$EVTYPE <- replaceSpellingVariations(c("ROADS"), "ROAD")
storm$EVTYPE <- replaceSpellingVariations(c("HIGH SURF*"), "HIGH SURF")

length(unique(storm$EVTYPE))
## [1] 452

These generic replacements reduced amount of unique events to 452. Nice work!

Now let’s see how many events are recorded for different years:

storm$bgnDate <- as.Date(substr(storm$BGN_DATE,1,10), format = "%m/%d/%Y")
hist(storm$bgnDate, breaks = 62, freq = TRUE, xlab="year", main="Amount of weather events recorded in different years")

Checking what event causes biggest economical damage

Let’s see which event brings the biggest economical damage. I assume we can calculate economical damage as a sum of property damage PROPDMG and crop damage CROPDMG. However, first let’s look at these 2 columns separately.

Let’s look at PROPDMG column:

res <- tapply(storm$PROPDMG, storm$EVTYPE, FUN=sum)
df <- data.frame(event=names(res),sumPropDmg=res)
index <- which(df$sumPropDmg == max(df$sumPropDmg))  
df[index, ]
##           event sumPropDmg
## TORNADO TORNADO    3212258

So TORNADO seems to be the most dangerous event in respect to property damage.

Let’s look at CROPDMG column:

res2 <- tapply(storm$CROPDMG, storm$EVTYPE, FUN=sum)
df2 <- data.frame(event=names(res2),sumCropDmg=res2)
index2 <- which(df2$sumCropDmg == max(df2$sumCropDmg))   
df2[index2, ]
##      event sumCropDmg
## HAIL  HAIL   581468.4

So HAIL seems to be the most dangerous event in respect to crop damage.

Now, let’s look at the summary of CROPDMG and PROPDMG columns:

res3 <- tapply(storm$CROPDMG + storm$PROPDMG, storm$EVTYPE, FUN=sum)
df3 <- data.frame(event=names(res3),sumEconomicDmg=res3)
index3 <- which(df3$sumEconomicDmg == max(df3$sumEconomicDmg)) 
df3[index3, ]
##           event sumEconomicDmg
## TORNADO TORNADO        3312277

So if we assume that the economical consequences can be calculated as a sum of property damage and crop damage, then TORNADO seems to be the most dangerous event in respect to economical consequences.

Checking what event causes biggest impact on public health

Now let’s look at impact on human health. Similarly economical consequences, I thing impact on public health should take 2 numbers into consideration: fatalities and injuries

res4 <- tapply(storm$FATALITIES + storm$INJURIES, storm$EVTYPE, FUN=sum)
df4 <- data.frame(event=names(res4),sumHealthImpact=res4)
index4 <- which(df4$sumHealthImpact == max(df4$sumHealthImpact)) 
df4[index4, ]
##           event sumHealthImpact
## TORNADO TORNADO           96979

So, as we see TORNADO is the most dangerous weather event in respect to public health.

It is also interesting to see the sum of injuries by year:

injYears <- tapply(storm$INJURIES, format(as.Date(storm$bgnDate), format = "%Y"), FUN=sum)
dfInjYears <- data.frame(year=names(injYears),injuries=injYears)
plot(x = dfInjYears$year, y = dfInjYears$injuries, type="l", xlab="year", ylab="injuries", main="Amount of injuries recorded yearly")

Let’s also see sum of fatalities by year:

fatYears <- tapply(storm$FATALITIES, format(as.Date(storm$bgnDate), format = "%Y"), FUN=sum)
dfInjFat <- data.frame(year=names(fatYears),fatalities=fatYears)
plot(x = dfInjFat$year, y = dfInjFat$fatalities, type="l", xlab="year", ylab="fatalities", main="Amount of fatalities recorded yearly")

Results

As we see from the analysis, the most dangerous event in respect to economical impact and impact on public health is TORNADO as it cases the biggest impact on these both fields among all the other events.