week4project2

NOAA Storm Database Analysis: Health and Economic Impacts of Severe Weather Events

For this assignment, I first loaded the data and examined the column names, noting that the event type column (EVTYPE) contained many duplicate entries. The data required cleaning, so I consulted the data codebook to define the event types and performed transformations to map EVTYPE to standardized event types. I analyzed human damage, economic impact, and property damage separately. I combined the PROPDMG, PROPDMGEXP, CROPDMG, and CROPDMGEXP columns, converting the “K”, “M”, and “B” factors into numeric values for analysis. The results show that Tornado has the greatest impact on human damage, while Flood has the most significant economic consequences.

Data Processing

## Read the data
df <- read.csv("repdata_data_StormData.csv.bz2", header = TRUE)

I would like to answer the following two questions:

1. Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?
2. Across the United States, which types of events have the greatest economic consequences?

The EVTYPE column in the dataframe contains many duplicate names, it need to be cleaned. By reading the data codebook, one can create an event vector based on the codebook and map the original EVTYPE column to a unique event type.

## the event_names vector

event_names <- c(
  "Astronomical Low Tide",
  "Avalanche",
  "Blizzard",
  "Coastal Flood",
  "Cold/Wind Chill",
  "Debris Flow",
  "Dense Fog",
  "Dense Smoke",
  "Drought",
  "Dust Devil",
  "Dust Storm",
  "Excessive Heat",
  "Extreme Cold/Wind Chill",
  "Flash Flood",
  "Flood",
  "Frost/Freeze",
  "Funnel Cloud",
  "Freezing Fog",
  "Hail",
  "Heat",
  "Heavy Rain",
  "Heavy Snow",
  "High Surf",
  "High Wind",
  "Hurricane (Typhoon)",
  "Ice Storm",
  "Lake-Effect Snow",
  "Lakeshore Flood",
  "Lightning",
  "Marine Hail",
  "Marine High Wind",
  "Marine Strong Wind",
  "Marine Thunderstorm Wind",
  "Rip Current",
  "Seiche",
  "Sleet",
  "Storm Surge/Tide",
  "Strong Wind",
  "Thunderstorm Wind",
  "Tornado",
  "Tropical Depression",
  "Tropical Storm",
  "Tsunami",
  "Volcanic Ash",
  "Waterspout",
  "Wildfire",
  "Winter Storm",
  "Winter Weather"
)

to clean and pre-process the original EVTYPE, I first create a function which can convert the event_names to partern. and use the function to extract the partern, and clean EVTYPE then create a new event type call clean_EVTYPE

#define the parten extraction function
create_pattern <- function(name) {
  # Split name into words
  words <- str_split(name, "\\s+", simplify = TRUE)
  # If multiple words, create pattern requiring all words in any order
  if (length(words) > 1) {
    # Use lookaheads for each word
    pattern <- paste0("(?=.*", words, ")", collapse = "")
  } else {
    # Single word: exact match
    pattern <- paste0("^", str_escape(name), "$")
  }
  return(pattern)
}

#extract the parten from event_names.
event_names_pattern <- unlist(sapply(event_names,create_pattern))

#code maps patterns in event_names_pattern to row indices of df$EVTYPE.
#Each element in the output list corresponds to an event name pattern 
#and contains the indices of df$EVTYPE rows that match that pattern.
patterns_index <- sapply(event_names_pattern,function (x) grep(x,df$EVTYPE,ignore.case=TRUE,perl=TRUE))

#create NA vector clean_evtype, to store the evtype name and process it
clean_evtype <- rep(NA,nrow(df))

for (i in seq_along(patterns_index)) {
  clean_evtype[patterns_index[[i]]] <- names(patterns_index[i])
  
}

after the clean_EVTYPE is created, let’s observe the percentage of data which be discard or overlook

#check the percentage for clearned data
mean(is.na(clean_evtype))*100

## [1] 26.6399

this result is not bad, we successfully remain 75% of the data to for analysis, considering the original data is huge (902297), this 75% of data should be reasonable enough for our analysis.

filter out the necessary columns for later analysis. STATE, EVTYPE, FATALITIES, INJURIES will be used to study the population health impact. STATE, EVTYPE, PROPDMG, PROPDMGEXP, CROPDMG, CROPDMGEXP will be used to study the economic consequences.

#assignment the clean_EVTYPE to the original dataframe.
#filter out the NA type
df <- df %>% mutate(clean_EVTYPE = clean_evtype)

df_clean <- df %>% select(STATE,clean_EVTYPE,FATALITIES,INJURIES,PROPDMG,PROPDMGEXP,CROPDMG,CROPDMGEXP) %>% filter(!is.na(clean_EVTYPE))

the final result of the data transformation is here:

head(df_clean)

##   STATE clean_EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## 1    AL      Tornado          0       15    25.0          K       0           
## 2    AL      Tornado          0        0     2.5          K       0           
## 3    AL      Tornado          0        2    25.0          K       0           
## 4    AL      Tornado          0        2     2.5          K       0           
## 5    AL      Tornado          0        2     2.5          K       0           
## 6    AL      Tornado          0        6     2.5          K       0

Results

1. Across the United States, which types of events (as indicated in the EVTYPE variable) are most harmful with respect to population health?

#select the relative data, group it and sum across countries
df_harmful <- df_clean %>% group_by(clean_EVTYPE) %>% summarise(Total_Fatalities=sum(FATALITIES),Total_Injuries=sum(INJURIES))

#plot the bar plot for sum_fatalities and sum_injuries
library(ggplot2)
library(tidyr)

data_long <- pivot_longer(df_harmful, cols = c(Total_Fatalities, Total_Injuries), 
                         names_to = "Metric", values_to = "Count")

gp1 <- ggplot(data_long, aes(y = clean_EVTYPE, x = Count, fill = Metric)) +
  geom_bar(stat = "identity") +
  facet_wrap(~ Metric, scales = "free_x") +
  theme_minimal() +
  labs(title = "Fatalities and Injuries by Event Type", y = "Event Type", x = "Count")

print(gp1)

From the above Figure, we can see that Tornado has the most impact on Fatalities and Injuries across US countries. The maximum total fatalities is 5,633. The maximum total injuries is 91,346.

2. Across the United States, which types of events have the greatest economic consequences? First, we need to do the data transformation to convert the “K”, “M”, “B”, for thousand, million and billion, for both PROPDMG and CROPDMG.

#data transformation
df_clean <- df_clean %>% 
  filter(PROPDMGEXP %in% c("K", "M", "B")) %>% 
  mutate(clean_PROPDMG = PROPDMG * case_when(
    PROPDMGEXP == "K" ~ 1000,
    PROPDMGEXP == "M" ~ 1000000,
    PROPDMGEXP == "B" ~ 1000000000
  ))

df_clean <- df_clean %>% 
  filter(CROPDMGEXP %in% c("K", "M", "B")) %>% 
  mutate(clean_CROPDMG = CROPDMG * case_when(
    CROPDMGEXP == "K" ~ 1000,
    CROPDMGEXP == "M" ~ 1000000,
    CROPDMGEXP == "B" ~ 1000000000
  ))

#select the relative data, group it and sum across countries
df_prop <- df_clean %>% filter(PROPDMGEXP %in% c("K", "M", "B"))%>% group_by(clean_EVTYPE) %>% summarise(Total_Damage_PROP=sum(clean_PROPDMG))
df_crop <- df_clean %>% filter(CROPDMGEXP %in% c("K", "M", "B"))%>% group_by(clean_EVTYPE) %>% summarise(Total_Damage_CROP=sum(clean_CROPDMG))

data_combined <- merge(df_prop, df_crop, by = "clean_EVTYPE", all = TRUE)
data_long <- pivot_longer(data_combined, cols = c(Total_Damage_PROP, Total_Damage_CROP), 
                         names_to = "Metric", values_to = "Count")

# Plot
gp2 <- ggplot(data_long, aes(y = clean_EVTYPE, x = Count, fill = Metric)) +
  geom_bar(stat = "identity") +
  facet_wrap(~ Metric, scales = "free_x") +
  theme_minimal() +
  labs(title = "Property and Crop Damage by Event Type", y = "Event Type", x = "Count")

print(gp2)

from the above graph, we can see that Ice Storm and Flood has the most Crop damage impact and Flood also has the most property damage.