Analysis for harmful and greatest economic consequence events from NOAA storm database
Karthik
Thursday, April 16, 2015
Synopsis
Storm and other natural calamities always causes a great level of distruction and economic loss for any country. Though materialistic loss one can recover over a period of time but human life is more precious than anything. Since these calamities are not avoidable even if we predict it in advance but we can always save many life’s with appropiate pre-cautions.
This preliminary analysis of U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database data is about answering the below questions:
1.Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health ?
2.Across the United States, which types of events have the greatest economic consequences ?
Data Processing
StormData <- read.csv(bzfile("repdata-data-StormData.csv.bz2"),stringsAsFactors=FALSE)Analysis
1. Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health ?
In order to answer this question, let’s look at the structure of our data
str(StormData)## 'data.frame': 902297 obs. of 37 variables:
## $ STATE__ : num 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_DATE : Factor w/ 16335 levels "1/1/1966 0:00:00",..: 6523 6523 4242 11116 2224 2224 2260 383 3980 3980 ...
## $ BGN_TIME : Factor w/ 3608 levels "00:00:00 AM",..: 272 287 2705 1683 2584 3186 242 1683 3186 3186 ...
## $ TIME_ZONE : Factor w/ 22 levels "ADT","AKS","AST",..: 7 7 7 7 7 7 7 7 7 7 ...
## $ COUNTY : num 97 3 57 89 43 77 9 123 125 57 ...
## $ COUNTYNAME: Factor w/ 29601 levels "","5NM E OF MACKINAC BRIDGE TO PRESQUE ISLE LT MI",..: 13513 1873 4598 10592 4372 10094 1973 23873 24418 4598 ...
## $ STATE : Factor w/ 72 levels "AK","AL","AM",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ EVTYPE : Factor w/ 985 levels " HIGH SURF ADVISORY",..: 834 834 834 834 834 834 834 834 834 834 ...
## $ BGN_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ BGN_AZI : Factor w/ 35 levels ""," N"," NW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ BGN_LOCATI: Factor w/ 54429 levels "","- 1 N Albion",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_DATE : Factor w/ 6663 levels "","1/1/1993 0:00:00",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_TIME : Factor w/ 3647 levels ""," 0900CST",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ COUNTY_END: num 0 0 0 0 0 0 0 0 0 0 ...
## $ COUNTYENDN: logi NA NA NA NA NA NA ...
## $ END_RANGE : num 0 0 0 0 0 0 0 0 0 0 ...
## $ END_AZI : Factor w/ 24 levels "","E","ENE","ESE",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ END_LOCATI: Factor w/ 34506 levels "","- .5 NNW",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LENGTH : num 14 2 0.1 0 0 1.5 1.5 0 3.3 2.3 ...
## $ WIDTH : num 100 150 123 100 150 177 33 33 100 100 ...
## $ F : int 3 2 2 2 2 2 2 1 3 3 ...
## $ MAG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ FATALITIES: num 0 0 0 0 0 0 0 0 1 0 ...
## $ INJURIES : num 15 0 2 2 2 6 1 0 14 0 ...
## $ PROPDMG : num 25 2.5 25 2.5 2.5 2.5 2.5 2.5 25 25 ...
## $ PROPDMGEXP: Factor w/ 19 levels "","-","?","+",..: 17 17 17 17 17 17 17 17 17 17 ...
## $ CROPDMG : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CROPDMGEXP: Factor w/ 9 levels "","?","0","2",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ WFO : Factor w/ 542 levels ""," CI","$AC",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ STATEOFFIC: Factor w/ 250 levels "","ALABAMA, Central",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ ZONENAMES : Factor w/ 25112 levels ""," "| __truncated__,..: 1 1 1 1 1 1 1 1 1 1 ...
## $ LATITUDE : num 3040 3042 3340 3458 3412 ...
## $ LONGITUDE : num 8812 8755 8742 8626 8642 ...
## $ LATITUDE_E: num 3051 0 0 0 0 ...
## $ LONGITUDE_: num 8806 0 0 0 0 ...
## $ REMARKS : Factor w/ 436781 levels "","-2 at Deer Park\n",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ REFNUM : num 1 2 3 4 5 6 7 8 9 10 ...Looking carefully, we observer there are two variables which can help us answer the question, namely FATALITIES and INJURIES. Now lets do some pre-processing to get our data ready for the analysis.
- Clean Data
StormData1 = StormData
StormData1$FATALITIES = as.numeric(StormData1$FATALITIES)
StormData1$INJURIES = as.numeric(StormData1$INJURIES)
StormData1 = StormData1[(!is.na(StormData1$FATALITIES) & StormData1$FATALITIES > 0) | (!is.na(StormData1$INJURIES) & StormData1$INJURIES > 0), c("EVTYPE","FATALITIES","INJURIES")]- Pre-Process
library(dplyr)
library(ggplot2)
library(tidyr)StormData1 = group_by(StormData1,EVTYPE) %>%
summarise(FATALITIES = sum(FATALITIES), INJURIES = sum(INJURIES)) %>%
arrange(desc(FATALITIES + INJURIES))
StormData1 = StormData1[1:10,]
StormData1 = gather(StormData1, TYPE, VALUE, FATALITIES:INJURIES)- Plotting the top 10 harmful events using ggplot
ggplot(StormData1, aes(x = reorder(EVTYPE, -VALUE), y = VALUE, fill = TYPE)) +
geom_bar(stat = "identity") +
labs(x = "Event Type", y = "Count") +
labs(title = "Top 10 harmful events") +
labs(fill = "Type") +
theme(axis.text.x = element_text(angle = 90, hjust = 1, vjust = 0))
2. Across the United States, which types of events have the greatest economic consequences ?
Refer above at the output of str(StormData), and the document from NOAA, you will see that PROPDMG and CROPDMG are data about property and crop damage respectively. And these two variables seems to be the perfect choice for answering this question.
Following the same approach as above,
- Clean Data
The starting four commands does the same cleaning activity we did above. But the extra bunch of code you see below it, is to make sure PROPDMGEXP and CROPDMGEXP variables which are the multiplying factor are converted to numerical values before we can actually use it to compute PROPDMG and CROPDMG respectively.
StormData$PROPDMG = as.numeric(StormData$PROPDMG)
StormData$CROPDMG = as.numeric(StormData$CROPDMG)
StormData_PROP = StormData[!is.na(StormData$PROPDMG) & StormData$PROPDMG > 0, c("EVTYPE", "PROPDMG", "PROPDMGEXP")]
StormData_CROP = StormData[!is.na(StormData$CROPDMG) & StormData$CROPDMG > 0, c("EVTYPE", "CROPDMG", "CROPDMGEXP")]- Property Damage
StormData_PROP[StormData_PROP$PROPDMGEXP == "" |
StormData_PROP$PROPDMGEXP == "-" |
StormData_PROP$PROPDMGEXP == "+" |
StormData_PROP$PROPDMGEXP == "0" |
StormData_PROP$PROPDMGEXP == "1" |
StormData_PROP$PROPDMGEXP == "2" |
StormData_PROP$PROPDMGEXP == "3" |
StormData_PROP$PROPDMGEXP == "4" |
StormData_PROP$PROPDMGEXP == "5" |
StormData_PROP$PROPDMGEXP == "6" |
StormData_PROP$PROPDMGEXP == "7" ,
c("PROPDMGEXP")] = "1"
StormData_PROP[StormData_PROP$PROPDMGEXP == "H" | StormData_PROP$PROPDMGEXP == "h", c("PROPDMGEXP")] = "100"
StormData_PROP[StormData_PROP$PROPDMGEXP == "K" , c("PROPDMGEXP")] = "1000"
StormData_PROP[StormData_PROP$PROPDMGEXP == "M" | StormData_PROP$PROPDMGEXP == "m", c("PROPDMGEXP")] = "1000000"
StormData_PROP[StormData_PROP$PROPDMGEXP == "B", c("PROPDMGEXP")] = "1000000000"
StormData_PROP[StormData_PROP$PROPDMGEXP != "1" &
StormData_PROP$PROPDMGEXP != "100" &
StormData_PROP$PROPDMGEXP != "1000" &
StormData_PROP$PROPDMGEXP != "1000000" &
StormData_PROP$PROPDMGEXP != "1000000000",
c("PROPDMGEXP")] <- "0"- Crop Damage
StormData_CROP[StormData_CROP$CROPDMGEXP == "" | StormData_CROP$CROPDMGEXP == "0", c("CROPDMGEXP")] <- "1"
StormData_CROP[StormData_CROP$CROPDMGEXP == "K" | StormData_CROP$CROPDMGEXP == "k", c("CROPDMGEXP")] <- "1000"
StormData_CROP[StormData_CROP$CROPDMGEXP == "M" | StormData_CROP$CROPDMGEXP == "m", c("CROPDMGEXP")] <- "1000000"
StormData_CROP[StormData_CROP$CROPDMGEXP == "B", c("CROPDMGEXP")] <- "1000000000"
StormData_CROP[StormData_CROP$CROPDMGEXP != "1" &
StormData_CROP$CROPDMGEXP != "1000" &
StormData_CROP$CROPDMGEXP != "1000000" &
StormData_CROP$CROPDMGEXP != "1000000000",
c("CROPDMGEXP")] <- "0"- Pre-Processing ( after calculating the damage value , we group, summarise, and arrage the data for plotting)
StormData_PROP$PROPDMG = StormData_PROP$PROPDMG * as.numeric(StormData_PROP$PROPDMGEXP)
StormData_PROP = StormData_PROP[, 1:2]
StormData_PROP = group_by(StormData_PROP,EVTYPE) %>%
summarise(DMG = sum(PROPDMG)) %>%
mutate(Type = "PROPDMG")
StormData_CROP$CROPDMG = StormData_CROP$CROPDMG * as.numeric(StormData_CROP$CROPDMGEXP)
StormData_CROP = StormData_CROP[, 1:2]
StormData_CROP = group_by(StormData_CROP,EVTYPE) %>%
summarise(DMG = sum(CROPDMG)) %>%
mutate(Type = "CROPDMG")
StormData2 <- rbind(StormData_PROP, StormData_CROP)
StormData2 <- spread(StormData2, Type, DMG);
StormData2[is.na(StormData2$CROPDMG), c("CROPDMG")] <- 0;
StormData2[is.na(StormData2$PROPDMG), c("PROPDMG")] <- 0;
StormData2 <- arrange(StormData2, desc(PROPDMG + CROPDMG))- Plot the top 10 event causing the greatest economical consequences
StormData2 <- StormData2[1:10,]
StormData2 <- gather(StormData2, TYPE, VALUE, CROPDMG:PROPDMG)
ggplot(StormData2, aes(x = reorder(EVTYPE, -VALUE), y = VALUE/1000000000, fill = TYPE)) +
geom_bar(stat = "identity") +
labs(x = "Event Type", y = "Value $ (Billions)") +
labs(title = "Top 10 events which have the greatest economic consequences") +
labs(fill = "Type") +
theme(axis.text.x = element_text(angle = 90, hjust = 1, vjust = 0))
Results
So, its clearly emerges out that TORNADO and FLOOD are the two most damaging natural calamities of all. First plot, shows that TORNADO has caused maximum number of deaths and injuries whereas second plot tells us that FLOOD being the reason for greatest crop & property damage. One interesting observation though is that both TORNADO and FLOOD occupies first and third spot interchangeably in both the plots, which implies, these two calamity are most harmful and causes the greatest economical loss.