Assignment 2
Christabel
2026-09-02
# Load the required package to read CSV files
library(readr)## Warning: package 'readr' was built under R version 4.5.2# 1. Load the dataset into R
# Make sure the CSV file is in the same folder as your Quarto document!
df <- read.csv("garments_worker_productivity.csv")
# 2. Check the structure to see measurement types (numeric vs. character)
str(df)## 'data.frame': 1197 obs. of 15 variables:
## $ date : chr "1/1/2015" "1/1/2015" "1/1/2015" "1/1/2015" ...
## $ quarter : chr "Quarter1" "Quarter1" "Quarter1" "Quarter1" ...
## $ department : chr "sweing" "finishing " "sweing" "sweing" ...
## $ day : chr "Thursday" "Thursday" "Thursday" "Thursday" ...
## $ team : int 8 1 11 12 6 7 2 3 2 1 ...
## $ targeted_productivity: num 0.8 0.75 0.8 0.8 0.8 0.8 0.75 0.75 0.75 0.75 ...
## $ smv : num 26.16 3.94 11.41 11.41 25.9 ...
## $ wip : int 1108 NA 968 968 1170 984 NA 795 733 681 ...
## $ over_time : int 7080 960 3660 3660 1920 6720 960 6900 6000 6900 ...
## $ incentive : int 98 0 50 50 50 38 0 45 34 45 ...
## $ idle_time : num 0 0 0 0 0 0 0 0 0 0 ...
## $ idle_men : int 0 0 0 0 0 0 0 0 0 0 ...
## $ no_of_style_change : int 0 0 0 0 0 0 0 0 0 0 ...
## $ no_of_workers : num 59 8 30.5 30.5 56 56 8 57.5 55 57.5 ...
## $ actual_productivity : num 0.941 0.886 0.801 0.801 0.8 ...# 3. Generate summary statistics for every column
summary(df)## date quarter department day
## Length:1197 Length:1197 Length:1197 Length:1197
## Class :character Class :character Class :character Class :character
## Mode :character Mode :character Mode :character Mode :character
##
##
##
##
## team targeted_productivity smv wip
## Min. : 1.000 Min. :0.0700 Min. : 2.90 Min. : 7.0
## 1st Qu.: 3.000 1st Qu.:0.7000 1st Qu.: 3.94 1st Qu.: 774.5
## Median : 6.000 Median :0.7500 Median :15.26 Median : 1039.0
## Mean : 6.427 Mean :0.7296 Mean :15.06 Mean : 1190.5
## 3rd Qu.: 9.000 3rd Qu.:0.8000 3rd Qu.:24.26 3rd Qu.: 1252.5
## Max. :12.000 Max. :0.8000 Max. :54.56 Max. :23122.0
## NA's :506
## over_time incentive idle_time idle_men
## Min. : 0 Min. : 0.00 Min. : 0.0000 Min. : 0.0000
## 1st Qu.: 1440 1st Qu.: 0.00 1st Qu.: 0.0000 1st Qu.: 0.0000
## Median : 3960 Median : 0.00 Median : 0.0000 Median : 0.0000
## Mean : 4567 Mean : 38.21 Mean : 0.7302 Mean : 0.3693
## 3rd Qu.: 6960 3rd Qu.: 50.00 3rd Qu.: 0.0000 3rd Qu.: 0.0000
## Max. :25920 Max. :3600.00 Max. :300.0000 Max. :45.0000
##
## no_of_style_change no_of_workers actual_productivity
## Min. :0.0000 Min. : 2.00 Min. :0.2337
## 1st Qu.:0.0000 1st Qu.: 9.00 1st Qu.:0.6503
## Median :0.0000 Median :34.00 Median :0.7733
## Mean :0.1504 Mean :34.61 Mean :0.7351
## 3rd Qu.:0.0000 3rd Qu.:57.00 3rd Qu.:0.8503
## Max. :2.0000 Max. :89.00 Max. :1.1204
## # Load the knitr package for clean table formatting
library(knitr)
# Build the table data
summary_table <- data.frame(
Variable = c("date", "quarter", "department", "day", "team", "targeted_productivity",
"smv", "wip", "over_time", "incentive", "idle_time", "idle_men",
"no_of_style_change", "no_of_workers", "actual_productivity"),
Type = c("Categorical", "Categorical", "Categorical", "Categorical", "Categorical",
"Numeric", "Numeric", "Numeric", "Numeric", "Numeric", "Numeric", "Numeric",
"Numeric", "Numeric", "Numeric"),
Stats = c("59 unique dates", "Mode: Quarter1", "Mode: sewing", "Mode: Wednesday",
"Min: 1, Max: 12", "Mean: 0.730", "Mean: 15.06, Max: 54.56", "506 Missing (NAs)",
"Mean: 4567.4", "Mean: 38.21", "Mean: 0.73", "Mean: 0.37, Max: 45",
"Mean: 0.15, Max: 2", "Mean: 34.61", "Mean: 0.735"),
Description = c("Date of production", "A portion of the month", "Associated department",
"Day of the week", "The specific team number", "Target productivity set",
"Standard Minute Value", "Work in progress", "Overtime in minutes",
"Financial incentive", "Time production interrupted", "Number of idle workers",
"Changes in garment style", "Total workers on team", "Actual measured productivity"),
Rationale = c("Exclude: Too granular for a general predictive model.",
"Include: Predictor for seasonal/time trends.",
"Include: Departments have different baselines.",
"Include: Identifies daily productivity dips.",
"Include: Teams may perform differently.",
"Include: Strongly tied to actual outcomes.",
"Include: Represents garment complexity.",
"Exclude: 506 NAs. Imputing could skew models.",
"Include: Captures extra effort/fatigue.",
"Include: Directly impacts motivation.",
"Include: Negative impact on productivity.",
"Include: Represents wasted labor capacity.",
"Include: Style changes slow production.",
"Include: Scales with production capacity.",
"Include: Target Variable to predict.")
)
# Output the clean table
kable(summary_table,
col.names = c("Variable", "Measurement Type", "Summary Stats", "Description", "Decision & Rationale"),
align = "l",
caption = "Task 1: Variable Summary and Rationale")| Variable | Measurement Type | Summary Stats | Description | Decision & Rationale |
|---|---|---|---|---|
| date | Categorical | 59 unique dates | Date of production | Exclude: Too granular for a general predictive model. |
| quarter | Categorical | Mode: Quarter1 | A portion of the month | Include: Predictor for seasonal/time trends. |
| department | Categorical | Mode: sewing | Associated department | Include: Departments have different baselines. |
| day | Categorical | Mode: Wednesday | Day of the week | Include: Identifies daily productivity dips. |
| team | Categorical | Min: 1, Max: 12 | The specific team number | Include: Teams may perform differently. |
| targeted_productivity | Numeric | Mean: 0.730 | Target productivity set | Include: Strongly tied to actual outcomes. |
| smv | Numeric | Mean: 15.06, Max: 54.56 | Standard Minute Value | Include: Represents garment complexity. |
| wip | Numeric | 506 Missing (NAs) | Work in progress | Exclude: 506 NAs. Imputing could skew models. |
| over_time | Numeric | Mean: 4567.4 | Overtime in minutes | Include: Captures extra effort/fatigue. |
| incentive | Numeric | Mean: 38.21 | Financial incentive | Include: Directly impacts motivation. |
| idle_time | Numeric | Mean: 0.73 | Time production interrupted | Include: Negative impact on productivity. |
| idle_men | Numeric | Mean: 0.37, Max: 45 | Number of idle workers | Include: Represents wasted labor capacity. |
| no_of_style_change | Numeric | Mean: 0.15, Max: 2 | Changes in garment style | Include: Style changes slow production. |
| no_of_workers | Numeric | Mean: 34.61 | Total workers on team | Include: Scales with production capacity. |
| actual_productivity | Numeric | Mean: 0.735 | Actual measured productivity | Include: Target Variable to predict. |
Bivariate Associations
# Load the necessary libraries
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(GGally)## Warning: package 'GGally' was built under R version 4.5.3## Loading required package: ggplot2## Warning: package 'ggplot2' was built under R version 4.5.3# Clean the dataset based on our Task 1 decisions
df_clean <- df %>%
select(-date, -wip) %>%
mutate(across(c(quarter, department, day, team), as.factor))
# Matrix 1: Core Categories (Department & Quarter)
ggpairs(df_clean %>% select(actual_productivity, department, quarter),
title = "Matrix 1: Core Categories vs Productivity") +
theme(axis.text.x = element_text(angle = 45, hjust = 1))## `stat_bin()` using `bins = 30`. Pick better value `binwidth`.## `stat_bin()` using `bins = 30`. Pick better value `binwidth`.
# Matrix 2: Targets and Time (Targeted Prod. & SMV)
ggpairs(df_clean %>% select(actual_productivity, targeted_productivity, smv),
title = "Matrix 2: Targets and Time vs Productivity") +
theme(axis.text.x = element_text(angle = 45, hjust = 1))
# Matrix 3: Extra Effort (Overtime & Incentives)
ggpairs(df_clean %>% select(actual_productivity, over_time, incentive),
title = "Matrix 3: Overtime and Incentives vs Productivity") +
theme(axis.text.x = element_text(angle = 45, hjust = 1))
Matrix Explanations Matrix 1: Core Categories vs. Productivity This
matrix visualizes how categorical variables impact our target variable
using boxplots and density curves.
Department: The boxplots reveal the median differences and spread of productivity between departments (e.g., sewing vs. finishing). If one department has a significantly higher median or tighter interquartile range, it indicates more consistent performance.
Quarter: Comparing the quarters allows us to observe if productivity fluctuates based on the time of the month, helping identify potential seasonal fatigue or end-of-month pushes.
Matrix 2: Targets and Time vs. Productivity This matrix examines the continuous production metrics using scatterplots and correlation coefficients.
Targeted Productivity: We expect a strong positive correlation here, as teams generally aim to meet their assigned goals. The scatterplot helps identify if teams consistently underperform or overperform their specific targets.
SMV (Standard Minute Value): This represents the allocated time for a task. Observing its relationship with actual productivity shows whether more complex, time-consuming garments (higher SMV) negatively impact the final productivity rate.
Matrix 3: Overtime and Incentives vs. Productivity This matrix highlights the impact of extra effort and financial motivation.
Incentives: The data shows a slight positive correlation (0.077) between financial incentives and actual productivity. However, the density plot is highly right-skewed, meaning most production runs receive little to no incentive, with only a few extreme outliers.
Overtime: Overtime has a slight negative correlation (-0.054) with productivity. This suggests that simply working longer hours does not necessarily equate to higher efficiency, possibly due to worker fatigue.
Predictive Modeling & Resampling
To predict actual productivity, we are comparing a Baseline Linear Regression model against two regularized models (Ridge and LASSO). Regularization helps prevent overfitting by penalizing overly complex models, which is particularly useful when dealing with highly correlated predictors.
To ensure our models are robust and generalize well to new data, we evaluate their Root Mean Squared Error (RMSE) across three different resampling techniques: 1. 10-Fold Cross-Validation: Splits the data into 10 equal parts, training on 9 and testing on 1, rotating until all parts are tested. 2. Leave-One-Out Cross-Validation (LOOCV): The most exhaustive method, where the model is trained on all data points except one, iterating through every single observation in the dataset. 3. Bootstrapping: Creates 100 random samples (with replacement) from the dataset to train the model, providing a strong measure of stability.
# Load required library for regularized regression
library(glmnet)## Warning: package 'glmnet' was built under R version 4.5.3## Loading required package: Matrix## Loaded glmnet 5.0# Prepare matrix for glmnet (automatically handles dummy variables for categories)
x <- model.matrix(actual_productivity ~ ., data = df_clean)[, -1]
y <- df_clean$actual_productivity
n <- nrow(x)
set.seed(42)
# ==========================================
# 1. BASELINE LINEAR MODEL
# ==========================================
# K-Fold (10-fold)
k <- 10
folds <- sample(1:k, n, replace = TRUE)
baseline_kfold <- mean(sapply(1:k, function(i) {
fit <- lm(actual_productivity ~ ., data = df_clean[folds != i, ])
pred <- predict(fit, df_clean[folds == i, ])
sqrt(mean((df_clean$actual_productivity[folds == i] - pred)^2))
}))
# LOOCV (Using the hat-values shortcut to save computing time)
fit_full <- lm(actual_productivity ~ ., data = df_clean)
baseline_loocv <- sqrt(mean((residuals(fit_full) / (1 - hatvalues(fit_full)))^2))
# Bootstrap (100 iterations)
baseline_boot <- mean(sapply(1:100, function(i) {
idx <- sample(1:n, n, replace = TRUE)
fit <- lm(actual_productivity ~ ., data = df_clean[idx, ])
pred <- predict(fit, df_clean)
sqrt(mean((df_clean$actual_productivity - pred)^2))
}))
# ==========================================
# 2. RIDGE REGRESSION (alpha = 0)
# ==========================================
# K-Fold
ridge_cv_kfold <- cv.glmnet(x, y, alpha = 0, nfolds = 10)
ridge_kfold <- sqrt(min(ridge_cv_kfold$cvm))
# LOOCV
ridge_cv_loocv <- cv.glmnet(x, y, alpha = 0, nfolds = n, grouped = FALSE)
ridge_loocv <- sqrt(min(ridge_cv_loocv$cvm))
# Bootstrap (100 iterations)
ridge_boot <- mean(sapply(1:100, function(i) {
idx <- sample(1:n, n, replace = TRUE)
fit <- cv.glmnet(x[idx, ], y[idx], alpha = 0, nfolds = 5)
pred <- predict(fit, s = "lambda.min", newx = x)
sqrt(mean((y - pred)^2))
}))
# ==========================================
# 3. LASSO REGRESSION (alpha = 1)
# ==========================================
# K-Fold
lasso_cv_kfold <- cv.glmnet(x, y, alpha = 1, nfolds = 10)
lasso_kfold <- sqrt(min(lasso_cv_kfold$cvm))
# LOOCV
lasso_cv_loocv <- cv.glmnet(x, y, alpha = 1, nfolds = n, grouped = FALSE)
lasso_loocv <- sqrt(min(lasso_cv_loocv$cvm))
# Bootstrap (100 iterations)
lasso_boot <- mean(sapply(1:100, function(i) {
idx <- sample(1:n, n, replace = TRUE)
fit <- cv.glmnet(x[idx, ], y[idx], alpha = 1, nfolds = 5)
pred <- predict(fit, s = "lambda.min", newx = x)
sqrt(mean((y - pred)^2))
}))
# ==========================================
# FINAL RESULTS TABLE
# ==========================================
results_table <- data.frame(
Model = c("Baseline (Linear)", "Ridge Regression", "LASSO Regression"),
K_Fold_RMSE = round(c(baseline_kfold, ridge_kfold, lasso_kfold), 4),
LOOCV_RMSE = round(c(baseline_loocv, ridge_loocv, lasso_loocv), 4),
Bootstrap_RMSE = round(c(baseline_boot, ridge_boot, lasso_boot), 4)
)
print(results_table)## Model K_Fold_RMSE LOOCV_RMSE Bootstrap_RMSE
## 1 Baseline (Linear) 0.1460 0.1467 0.1477
## 2 Ridge Regression 0.1481 0.1474 0.1472
## 3 LASSO Regression 0.1461 0.1466 0.1448The RMSE scores are all incredibly close (hovering between 0.145 and 0.149). This means that all three models have a similar level of accuracy when predicting productivity.
LASSO Regression performed slightly better during the Bootstrap resampling (with the lowest RMSE of 0.1458), but the Baseline Linear model actually held its own very well across the board.
Executive Summary
This report analyzes the garments_worker_productivity
dataset to identify key drivers of manufacturing efficiency and to build
predictive models for actual productivity.
Task 1: Data Preparation and Cleaning
To establish a robust foundation for modeling, the dataset underwent
targeted cleaning: * Feature Selection: The
date and wip (work in progress) columns were
removed to focus on relevant operational metrics and mitigate excessive
missing values. * Data Integrity: All remaining rows
with missing values (NAs) were omitted to ensure complete cases for
cross-validation. * Categorical Formatting: Core
categorical variables (department, quarter,
day, and team) were explicitly converted into
factors to ensure they were correctly interpreted as dummy variables
during regression modeling.
Task 2: Exploratory Data Analysis (Bivariate Associations)
To evaluate the relationships between productivity and other operational features without visual overcrowding, the bivariate analysis was segmented into three focused matrices: * Core Categories: Analyzed the distribution of productivity across departments and quarters, highlighting median performance differences. * Targets and Time: Evaluated the linear relationships between allocated time (SMV), target productivity goals, and the actual productivity achieved. * Operational Effort: Assessed the impact of overtime and financial incentives, revealing a slight positive correlation between incentives and productivity (0.077), alongside a highly skewed distribution of bonus payouts.
Task 3: Predictive Modeling & Resampling
Three regression models were developed to predict actual productivity: a Baseline Linear Model, Ridge Regression (\(\alpha = 0\)), and LASSO Regression (\(\alpha = 1\)). To rigorously evaluate model stability and prevent overfitting, each model’s Root Mean Squared Error (RMSE) was calculated across three distinct resampling methodologies: * 10-Fold Cross-Validation: Partitioning the dataset into 10 rotating training and testing sets. * Leave-One-Out Cross-Validation (LOOCV): Iteratively testing on a single observation while training on the remainder of the dataset. * Bootstrapping: Utilizing 100 iterations of random sampling with replacement to validate performance stability.
Conclusion: All three models performed with high similarity, yielding RMSE scores clustering tightly between 0.145 and 0.149 across all resampling methods, with the LASSO model demonstrating a negligible edge during bootstrap validation.