Reproducible Research: Storm Data
Erin Cisneros
2024-06-05
##Synopsis
The intent of this paper is to explore the NOAA Storm Database, which contains information about severe weather events from 1950 - 2011. This information should be used to provide basic context around which types of events have had the largest impacts on human health, as derived from number of injuries and number of fatalities, as well economic impacts, as derived from property and crop damage estimates. The data shows that Floods have had the largest impact on health, while high winds/cold have had the largest economic impact. However, if we consider the anomaly in 1998 caused by flooding in TX, tornados dominates the health impact category.
##Data Processing
First, we extract the file containing the specific data we used and read it in from https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2. The file is a comma-separated-value file compressed via the bzip2 algorithm, so it must be unzipped.
download.file("https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2", destfile = "StormData.csv.bz2")
unzip("StormData.csv.bz2", exdir = "StormData")## Warning in unzip("StormData.csv.bz2", exdir = "StormData"): error 1 in
## extracting from zip filedata <- read.csv("StormData.csv.bz2", na.strings = "")Then we begin exploring the data to understand what type of data cleaning is necessary. We look at the dimensions and first several rows of data, and discover that the file has 902,297 rows of data.
dim(data)## [1] 902297 37head(data)## STATE__ BGN_DATE BGN_TIME TIME_ZONE COUNTY COUNTYNAME STATE EVTYPE
## 1 1 4/18/1950 0:00:00 0130 CST 97 MOBILE AL TORNADO
## 2 1 4/18/1950 0:00:00 0145 CST 3 BALDWIN AL TORNADO
## 3 1 2/20/1951 0:00:00 1600 CST 57 FAYETTE AL TORNADO
## 4 1 6/8/1951 0:00:00 0900 CST 89 MADISON AL TORNADO
## 5 1 11/15/1951 0:00:00 1500 CST 43 CULLMAN AL TORNADO
## 6 1 11/15/1951 0:00:00 2000 CST 77 LAUDERDALE AL TORNADO
## BGN_RANGE BGN_AZI BGN_LOCATI END_DATE END_TIME COUNTY_END COUNTYENDN
## 1 0 <NA> <NA> <NA> <NA> 0 NA
## 2 0 <NA> <NA> <NA> <NA> 0 NA
## 3 0 <NA> <NA> <NA> <NA> 0 NA
## 4 0 <NA> <NA> <NA> <NA> 0 NA
## 5 0 <NA> <NA> <NA> <NA> 0 NA
## 6 0 <NA> <NA> <NA> <NA> 0 NA
## END_RANGE END_AZI END_LOCATI LENGTH WIDTH F MAG FATALITIES INJURIES PROPDMG
## 1 0 <NA> <NA> 14.0 100 3 0 0 15 25.0
## 2 0 <NA> <NA> 2.0 150 2 0 0 0 2.5
## 3 0 <NA> <NA> 0.1 123 2 0 0 2 25.0
## 4 0 <NA> <NA> 0.0 100 2 0 0 2 2.5
## 5 0 <NA> <NA> 0.0 150 2 0 0 2 2.5
## 6 0 <NA> <NA> 1.5 177 2 0 0 6 2.5
## PROPDMGEXP CROPDMG CROPDMGEXP WFO STATEOFFIC ZONENAMES LATITUDE LONGITUDE
## 1 K 0 <NA> <NA> <NA> <NA> 3040 8812
## 2 K 0 <NA> <NA> <NA> <NA> 3042 8755
## 3 K 0 <NA> <NA> <NA> <NA> 3340 8742
## 4 K 0 <NA> <NA> <NA> <NA> 3458 8626
## 5 K 0 <NA> <NA> <NA> <NA> 3412 8642
## 6 K 0 <NA> <NA> <NA> <NA> 3450 8748
## LATITUDE_E LONGITUDE_ REMARKS REFNUM
## 1 3051 8806 <NA> 1
## 2 0 0 <NA> 2
## 3 0 0 <NA> 3
## 4 0 0 <NA> 4
## 5 0 0 <NA> 5
## 6 0 0 <NA> 6It seems that the columns that would provide us with health impacts are INJURIES and FATALITIES, while the columnns that provide economic impacts are PROPDMG and CROPDMG. The latter two have a separate column that denotes thousands or millions, so we need to format and combine these values appropriately.
data$PROPDMG <- as.numeric(data$PROPDMG)
data$PropertyDamage <- ifelse(data$PROPDMGEXP == "K", data$PROPDMG * 1000, data$PROPDMG * 1000000)
data$CROPDMG <- as.numeric(data$CROPDMG)
data$CropDamage <- ifelse(data$CROPDMGEXP == "K", data$CROPDMG * 1000, data$CROPDMG * 1000000)We want to look at the data by year, so we will add a formatted date column, and normalize the case formatting of the event type column, and then look at summaries of each of the columns we are interested in.
data$Year<-format(as.Date(data$BGN_DATE, format="%m/%d/%Y"),"%Y")
data$EVTYPE <- casefold(data$EVTYPE)
summary(data$INJURIES)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.0000 0.0000 0.1557 0.0000 1700.0000summary(data$FATALITIES)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.0000 0.0000 0.0168 0.0000 583.0000summary(data$PropertyDamage)## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 0 0 1000 365328 10000 929000000 465934summary(data$CropDamage)## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 0 0 0 127529 0 596000000 618413Since we want to look at the health impacts and economic impacts as a combination of two columns each, we’ll also create combined columns in the data.
data$HealthImpacts<-data$INJURIES + data$FATALITIES
data$EconImpacts <- data$PropertyDamage + data$CropDamageNow we’ll explore these two categories of data by Year.
library(dplyr) ## Warning: package 'dplyr' was built under R version 4.3.3##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionsummary_df <- data %>%
group_by(Year) %>%
summarise(Total_HealthImpacts = sum(HealthImpacts, na.rm = TRUE))
print(summary_df, n=62)## # A tibble: 62 × 2
## Year Total_HealthImpacts
## <chr> <dbl>
## 1 1950 729
## 2 1951 558
## 3 1952 2145
## 4 1953 5650
## 5 1954 751
## 6 1955 1055
## 7 1956 1438
## 8 1957 2169
## 9 1958 602
## 10 1959 792
## 11 1960 783
## 12 1961 1139
## 13 1962 581
## 14 1963 569
## 15 1964 1221
## 16 1965 5498
## 17 1966 2128
## 18 1967 2258
## 19 1968 2653
## 20 1969 1377
## 21 1970 1428
## 22 1971 2882
## 23 1972 1003
## 24 1973 2495
## 25 1974 7190
## 26 1975 1517
## 27 1976 1239
## 28 1977 814
## 29 1978 972
## 30 1979 3098
## 31 1980 1185
## 32 1981 822
## 33 1982 1340
## 34 1983 853
## 35 1984 3018
## 36 1985 1625
## 37 1986 966
## 38 1987 1505
## 39 1988 1085
## 40 1989 1754
## 41 1990 1920
## 42 1991 1428
## 43 1992 1808
## 44 1993 2447
## 45 1994 4505
## 46 1995 5971
## 47 1996 3259
## 48 1997 4401
## 49 1998 11864
## 50 1999 6056
## 51 2000 3280
## 52 2001 3190
## 53 2002 3653
## 54 2003 3374
## 55 2004 2796
## 56 2005 2303
## 57 2006 3967
## 58 2007 2612
## 59 2008 3191
## 60 2009 1687
## 61 2010 2280
## 62 2011 8794 summary_df <- data %>%
group_by(Year) %>%
summarise(Total_EconImpacts = sum(EconImpacts, na.rm = TRUE))
print(summary_df, n=62)## # A tibble: 62 × 2
## Year Total_EconImpacts
## <chr> <dbl>
## 1 1950 0
## 2 1951 0
## 3 1952 0
## 4 1953 0
## 5 1954 0
## 6 1955 0
## 7 1956 0
## 8 1957 0
## 9 1958 0
## 10 1959 0
## 11 1960 0
## 12 1961 0
## 13 1962 0
## 14 1963 0
## 15 1964 0
## 16 1965 0
## 17 1966 0
## 18 1967 0
## 19 1968 0
## 20 1969 0
## 21 1970 0
## 22 1971 0
## 23 1972 0
## 24 1973 0
## 25 1974 0
## 26 1975 0
## 27 1976 0
## 28 1977 0
## 29 1978 0
## 30 1979 0
## 31 1980 0
## 32 1981 0
## 33 1982 0
## 34 1983 0
## 35 1984 0
## 36 1985 0
## 37 1986 0
## 38 1987 0
## 39 1988 0
## 40 1989 0
## 41 1990 0
## 42 1991 0
## 43 1992 0
## 44 1993 1439941950
## 45 1994 1810668250
## 46 1995 2130664350
## 47 1996 2789148520
## 48 1997 1351694000
## 49 1998 3891008460
## 50 1999 3548517700
## 51 2000 1490102950
## 52 2001 761239110
## 53 2002 1305648560
## 54 2003 2643605960
## 55 2004 4287817750
## 56 2005 1977875700
## 57 2006 3345575350
## 58 2007 7480086160
## 59 2008 12783176080
## 60 2009 5749424130
## 61 2010 7735073640
## 62 2011 13264023960##Anomolies
From the summaries above, we can see that although the counts are quite volatile for the total, there is a large jump in the number of health impacts in 1998. Let’s see what the top event types were in this year, and in what state they occurred.
filtered_data <- data %>%
filter(Year == 1998)
summary_df <- filtered_data %>%
group_by(EVTYPE) %>%
summarise(Total_HealthImpacts = sum(HealthImpacts, na.rm = TRUE))
N <- 10
top_1998EVTYPE <- head(summary_df[order(-summary_df$Total_HealthImpacts), ], N)
print (top_1998EVTYPE)## # A tibble: 10 × 2
## EVTYPE Total_HealthImpacts
## <chr> <dbl>
## 1 flood 6190
## 2 tornado 2004
## 3 tstm wind 854
## 4 excessive heat 801
## 5 flash flood 374
## 6 lightning 327
## 7 glaze 213
## 8 winter storm 178
## 9 fog 158
## 10 high wind 97summary_df2 <- filtered_data %>%
group_by(STATE) %>%
summarise(Total_HealthImpacts = sum(HealthImpacts, na.rm = TRUE))
N <- 10
top_1998STATE <- head(summary_df2[order(-summary_df2$Total_HealthImpacts), ], N)
print (top_1998STATE)## # A tibble: 10 × 2
## STATE Total_HealthImpacts
## <chr> <dbl>
## 1 TX 6563
## 2 FL 601
## 3 MO 351
## 4 AL 348
## 5 GA 339
## 6 CA 294
## 7 PA 284
## 8 IL 239
## 9 IA 219
## 10 OH 206This shows a very large number of floods in TX in 1998. This is outside of the normal distribution, shown below.
library(ggplot2)## Warning: package 'ggplot2' was built under R version 4.3.3filtered_data2 <- data[data$EVTYPE == 'flood', ]
ggplot(filtered_data2, aes(x = Year, y = HealthImpacts)) +
geom_bar(stat = 'identity') +
labs(x = 'Year', y = 'Health Impacts', title = 'Health Impacts for Flood Events')
##Results
Now we’ll take a look at the top health impacts and economic impacts overall.
selected_columns <- c("EVTYPE", "HealthImpacts", "EconImpacts")
results_data <- data[, selected_columns]
library(dplyr)
summarized_data <- results_data %>%
group_by(EVTYPE) %>%
summarise(
HealthImpacts_Sum = sum(HealthImpacts),
EconImpacts_Sum = sum(EconImpacts)
)
N <- 10
top_HealthImpacts <- head(summarized_data[order(-summarized_data$HealthImpacts_Sum), ], N)
print(top_HealthImpacts)## # A tibble: 10 × 3
## EVTYPE HealthImpacts_Sum EconImpacts_Sum
## <chr> <dbl> <dbl>
## 1 tornado 96979 NA
## 2 excessive heat 8428 NA
## 3 tstm wind 7461 NA
## 4 flood 7259 NA
## 5 lightning 6046 NA
## 6 heat 3037 NA
## 7 flash flood 2755 NA
## 8 ice storm 2064 NA
## 9 thunderstorm wind 1621 NA
## 10 winter storm 1527 NAN <- 10
top_EconImpacts <- head(summarized_data[order(-summarized_data$EconImpacts_Sum), ], N)
print(top_EconImpacts)## # A tibble: 10 × 3
## EVTYPE HealthImpacts_Sum EconImpacts_Sum
## <chr> <dbl> <dbl>
## 1 high winds/cold 4 117500000
## 2 winter storm high winds 16 65000000
## 3 heavy rain/high surf 0 15000000
## 4 hurricane opal/high winds 2 10100000
## 5 lakeshore flood 0 7540000
## 6 high winds heavy rains 0 7510000
## 7 forest fires 0 5500000
## 8 tornadoes, tstm wind, hail 25 4100000
## 9 flash flooding/flood 5 1925000
## 10 heavy snow/high winds & flood 0 1520000If we remove 1998 and run the data, we see that tornados have the greatest health impacts, while high winds/cold have the greatest economic impacts.
selected_columns2 <- c("EVTYPE", "HealthImpacts", "EconImpacts", "Year")
results_data2 <- data[, selected_columns2]
library(dplyr)
summarized_data2 <- results_data2 %>%
group_by(Year, EVTYPE) %>%
summarise(
HealthImpacts_Sum = sum(HealthImpacts),
EconImpacts_Sum = sum(EconImpacts)
)## `summarise()` has grouped output by 'Year'. You can override using the
## `.groups` argument.N <- 10
top_HealthImpacts2 <- summarized_data2 %>%
filter(Year != 1998) %>%
arrange(desc(HealthImpacts_Sum)) %>%
head(N)
ggplot(top_HealthImpacts2, aes(x = reorder(EVTYPE, -HealthImpacts_Sum), y = HealthImpacts_Sum)) +
geom_bar(stat = "identity") +
labs(title = "Top Storm Types by Health Impacts", x = "Storm Type", y = "Health Impacts")