5 Capstone Analysis

Author

Luke Green

Introduction

Volunteer firefighters in rural communities must be able to establish and operate effective water-supply systems in areas where municipal hydrants are limited or unavailable. However, opportunities to practice skills such as drafting, configuring fill sites, and maintaining an efficient tanker shuttle may be limited.

This analysis examines simulated pre-test and post-test scores from 75 firefighters who completed rural water-supply training. It also considers participants’ years of experience and their confidence after training. Understanding these results would benefit fire-department training officers, department leaders, and instructors by helping them evaluate whether the training improved firefighter knowledge and identify areas that may require additional instruction or practice.

Data Overview

firefighter_data <- read.csv(
  "CUED7540_LearningAnalytics/data/Rural_Water_Training_Data.csv"
)

firefighter_data$Firefighter_ID <- as.character(
  firefighter_data$Firefighter_ID
)

head(firefighter_data)

  Firefighter_ID Years_Experience Pre_Test Post_Test Confidence
1            300               40       65        83          4
2            301               28       60        80          4
3            302               16       60        73          4
4            303                3       66        78          5
5            304                8       64        85          4
6            305               12       67        88          4

# Display the structure of the dataset
str(firefighter_data)

'data.frame':   75 obs. of  5 variables:
 $ Firefighter_ID  : chr  "300" "301" "302" "303" ...
 $ Years_Experience: int  40 28 16 3 8 12 5 6 10 16 ...
 $ Pre_Test        : int  65 60 60 66 64 67 56 65 72 60 ...
 $ Post_Test       : int  83 80 73 78 85 88 66 87 88 72 ...
 $ Confidence      : int  4 4 4 5 4 4 3 4 4 4 ...

# Display summary statistics
summary(firefighter_data)

 Firefighter_ID     Years_Experience    Pre_Test      Post_Test   
 Length:75          Min.   : 0.00    Min.   :52.0   Min.   :65.0  
 Class :character   1st Qu.: 5.00    1st Qu.:62.0   1st Qu.:78.0  
 Mode  :character   Median :10.00    Median :66.0   Median :82.0  
                    Mean   :10.52    Mean   :68.0   Mean   :83.0  
                    3rd Qu.:16.00    3rd Qu.:73.5   3rd Qu.:88.5  
                    Max.   :40.00    Max.   :90.0   Max.   :99.0  
   Confidence  
 Min.   :3.00  
 1st Qu.:4.00  
 Median :4.00  
 Mean   :4.04  
 3rd Qu.:4.00  
 Max.   :5.00

# Check for missing values
colSums(is.na(firefighter_data))

  Firefighter_ID Years_Experience         Pre_Test        Post_Test 
               0                0                0                0 
      Confidence 
               0

# Check for duplicate firefighter IDs
sum(duplicated(firefighter_data$Firefighter_ID))

[1] 0

Analysis

# Calculate each firefighter's improvement
firefighter_data$Improvement <- 
  firefighter_data$Post_Test - firefighter_data$Pre_Test

# Display summary statistics for improvement
summary(firefighter_data$Improvement)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
      6      12      15      15      17      26

# Calculate important averages
mean(firefighter_data$Pre_Test)

[1] 68

mean(firefighter_data$Post_Test)

[1] 83

mean(firefighter_data$Improvement)

[1] 15

A descriptive analysis was used to examine whether firefighter knowledge increased after the rural water-supply training. An improvement score was calculated for each firefighter by subtracting the pre-test score from the post-test score. The average pre-test score was 68, while the average post-test score was 83. Therefore, firefighters improved by an average of 15 points after completing the training.

library(ggplot2)

# Create a small dataset containing the average scores
average_scores <- data.frame(
  Test = c("Pre-Test", "Post-Test"),
  Average_Score = c(
    mean(firefighter_data$Pre_Test),
    mean(firefighter_data$Post_Test)
  )
)

# Create the bar chart
ggplot(
  average_scores,
  aes(x = Test, y = Average_Score, fill = Test)
) +
  geom_col(width = 0.6) +
  geom_text(
    aes(label = round(Average_Score, 1)),
    vjust = -0.5,
    size = 5
  ) +
  scale_fill_manual(values = c(
    "Pre-Test" = "#4472C4",
    "Post-Test" = "#70AD47"
  )) +
  scale_y_continuous(
    limits = c(0, 100),
    breaks = seq(0, 100, 20)
  ) +
  labs(
    title = "Average Firefighter Scores Before and After Training",
    x = NULL,
    y = "Average Score"
  ) +
  theme_minimal() +
  theme(
    legend.position = "none",
    plot.title = element_text(face = "bold", hjust = 0.5)
  )

# Calculate the relationship between experience and improvement
correlation <- cor(
  firefighter_data$Years_Experience,
  firefighter_data$Improvement
)

correlation

[1] 0.008049809

# Create a scatterplot
ggplot(
  firefighter_data,
  aes(x = Years_Experience, y = Improvement)
) +
  geom_point(
    color = "#4472C4",
    size = 2.5,
    alpha = 0.7
  ) +
  geom_smooth(
    method = "lm",
    se = TRUE,
    color = "#C00000"
  ) +
  labs(
    title = "Training Improvement by Years of Experience",
    x = "Years of Fire-Service Experience",
    y = "Improvement in Test Score"
  ) +
  theme_minimal() +
  theme(
    plot.title = element_text(face = "bold", hjust = 0.5)
  )

`geom_smooth()` using formula = 'y ~ x'

The second visualization examines the relationship between firefighters’ years of experience and their improvement in test scores. The trend line is nearly flat, indicating little to no relationship between experience and score improvement. Firefighters with both low and high levels of experience demonstrated gains following the training. This suggests that the rural water-supply training may benefit firefighters across different experience levels rather than primarily benefiting only newer or more experienced members.

Findings Summary

The analysis of the simulated data showed that firefighters’ average scores increased from 68 before training to 83 after training. This represents an average improvement of 15 points, suggesting that the rural water-supply training was associated with increased participant knowledge. Firefighters demonstrated improvement across different levels of fire-service experience, and years of experience had little relationship with the amount of improvement. Overall, the findings suggest that this training could be beneficial for both newer and more experienced firefighters who need instruction or reinforcement in rural water-supply operations.