LANE THEORY LAB

Predicting Dota 2 victory from draft composition: what worked, what did not, and what the next model needs.

1 Executive Summary

Final project story: This project began as a player-informed hypothesis about Dota 2 lane composition, but the final Sprint 3 result became a stronger analytics lesson: simple public draft-composition features alone are not enough to reliably predict match outcomes. The value of the project is the reproducible pipeline, model comparison, honest baseline evaluation, and improved next-step design.

The original gameplay theory was that certain lane and draft compositions, especially melee/ranged mismatches, create measurable win/loss risk. In Sprint 3, I expanded the dataset, engineered domain-informed draft-risk features, and compared two model types against a majority-class baseline.

The results were intentionally interpreted conservatively. The models were close to coin-flip performance and did not clearly outperform the Radiant-side majority baseline. That does not make the project useless. It shows that the first feature set was too broad and that the next version should focus more directly on lane-versus-lane matchups, behavior/communication context, role fidelity, patch context, and rank bracket.

2 Requirement Map

library(knitr)

requirement_map <- tibble::tribble(
  ~Rubric_Area, ~Weight, ~Where_This_Report_Addresses_It,
  "EDA & data preparation", "20%", "Dataset checks, target balance, engineered composition features, draft-risk summaries",
  "Model development", "30%", "Interpretable logistic regression plus tree-based comparison model",
  "Evaluation & comparison", "20%", "Train/test split, majority-class baseline, accuracy, ROC AUC, confusion matrix",
  "Insights & limitations", "15%", "Plain-English interpretation, weak/null-result framing, limitations, next iteration",
  "Presentation quality", "15%", "Executive summary, compact tables, visual EDA, clear conclusion and Q&A-ready framing"
)

kable(requirement_map, caption = "Kaggle Project Rubric Alignment")

Kaggle Project Rubric Alignment
Rubric_Area	Weight	Where_This_Report_Addresses_It
EDA & data preparation	20%	Dataset checks, target balance, engineered composition features, draft-risk summaries
Model development	30%	Interpretable logistic regression plus tree-based comparison model
Evaluation & comparison	20%	Train/test split, majority-class baseline, accuracy, ROC AUC, confusion matrix
Insights & limitations	15%	Plain-English interpretation, weak/null-result framing, limitations, next iteration
Presentation quality	15%	Executive summary, compact tables, visual EDA, clear conclusion and Q&A-ready framing

3 Research Question

The polished project question is:

Can public Dota 2 draft-composition data reveal meaningful win/loss signals, or are simple composition features too blunt to explain match outcomes?

The more specific next-iteration question is:

Can lane-versus-lane melee/ranged matchups predict draft risk better than overall team composition?

4 Load Packages

required_packages <- c(
  "tidyverse",
  "knitr",
  "broom",
  "scales",
  "rpart"
)

missing_packages <- required_packages[
  !required_packages %in% rownames(installed.packages())
]

if (length(missing_packages) > 0) {
  stop(
    paste0(
      "Missing required package(s): ",
      paste(missing_packages, collapse = ", "),
      "\nInstall them with: install.packages(c('",
      paste(missing_packages, collapse = "', '"),
      "'))"
    )
  )
}

library(tidyverse)
library(knitr)
library(broom)
library(scales)
library(rpart)

has_ranger <- FALSE  # Fast-knit mode: force lightweight decision tree for reliable classroom submission

# Dota-inspired visual system for charts.
radiant_green <- "#7CFC00"
radiant_green_soft <- "#B7FF5A"
dire_red <- "#C23B22"
dire_red_bright <- "#FF4B3E"
void_bg <- "#05070B"
panel_bg <- "#0B111C"
arcane_blue <- "#38BDF8"
muted_text <- "#D1D5DB"
neutral_gold <- "#FBBF24"

lane_theme <- function(base_size = 13) {
  theme_minimal(base_size = base_size) %+replace%
    theme(
      plot.background = element_rect(fill = void_bg, color = NA),
      panel.background = element_rect(fill = panel_bg, color = NA),
      panel.grid.major = element_line(color = "#253044", linewidth = 0.28),
      panel.grid.minor = element_line(color = "#182033", linewidth = 0.18),
      plot.title = element_text(color = radiant_green_soft, face = "bold", size = base_size + 4,
                                margin = margin(b = 7)),
      plot.subtitle = element_text(color = muted_text, size = base_size, margin = margin(b = 12)),
      axis.title = element_text(color = "#E5E7EB", face = "bold"),
      axis.text = element_text(color = "#D1D5DB"),
      strip.background = element_rect(fill = "#111827", color = "#374151"),
      strip.text = element_text(color = radiant_green_soft, face = "bold"),
      legend.background = element_rect(fill = "#0B111C", color = "#263244"),
      legend.key = element_rect(fill = "#0B111C", color = NA),
      legend.title = element_text(color = "#F8FAFC", face = "bold"),
      legend.text = element_text(color = "#E5E7EB"),
      plot.margin = margin(16, 24, 18, 18)
    )
}

set.seed(580)

5 Load Data

possible_paths <- c(
  "data/lane_theory_modeling_dataset_sprint3_expanded.csv",
  "lane_theory_modeling_dataset_sprint3_expanded.csv",
  "data/lane_theory_modeling_dataset.csv",
  "lane_theory_modeling_dataset.csv"
)

data_path <- possible_paths[file.exists(possible_paths)][1]

if (is.na(data_path)) {
  stop(
    paste0(
      "Could not find the modeling dataset. Expected one of:\n",
      paste(possible_paths, collapse = "\n")
    )
  )
}

model_raw <- readr::read_csv(data_path, show_col_types = FALSE)

cat("Using dataset:", data_path, "\n")

## Using dataset: data/lane_theory_modeling_dataset_sprint3_expanded.csv

cat("Rows:", nrow(model_raw), "\n")

## Rows: 5000

cat("Columns:", ncol(model_raw), "\n")

## Columns: 36

6 Data Preparation and Quality Checks

expected_columns <- c(
  "match_id",
  "radiant_win",
  "radiant_melee_count",
  "dire_melee_count",
  "radiant_ranged_count",
  "dire_ranged_count",
  "radiant_str_count",
  "dire_str_count",
  "radiant_agi_count",
  "dire_agi_count",
  "radiant_int_count",
  "dire_int_count",
  "radiant_all_count",
  "dire_all_count",
  "melee_count_difference",
  "str_count_difference",
  "agi_count_difference",
  "int_count_difference",
  "all_count_difference"
)

quality_checks <- tibble(
  check = c(
    "Dataset has rows",
    "Dataset has expected columns",
    "match_id is present",
    "radiant_win target is present",
    "No missing values in expected columns",
    "At least 1,000 matches available"
  ),
  passed = c(
    nrow(model_raw) > 0,
    all(expected_columns %in% names(model_raw)),
    "match_id" %in% names(model_raw),
    "radiant_win" %in% names(model_raw),
    all(colSums(is.na(model_raw[, expected_columns[expected_columns %in% names(model_raw)]])) == 0),
    nrow(model_raw) >= 1000
  )
) %>%
  mutate(result = if_else(passed, "PASS", "CHECK"))

kable(quality_checks, caption = "Initial Dataset Quality Checks")

Initial Dataset Quality Checks
check	passed	result
Dataset has rows	TRUE	PASS
Dataset has expected columns	TRUE	PASS
match_id is present	TRUE	PASS
radiant_win target is present	TRUE	PASS
No missing values in expected columns	TRUE	PASS
At least 1,000 matches available	TRUE	PASS

if (!all(expected_columns %in% names(model_raw))) {
  stop("One or more expected modeling columns is missing. Review the dataset schema before knitting.")
}

model_prepped <- model_raw %>%
  mutate(
    radiant_win_text = str_to_lower(as.character(radiant_win)),
    radiant_win = case_when(
      radiant_win_text %in% c("true", "1", "radiant_win", "radiant", "win", "yes") ~ "Radiant_Win",
      radiant_win_text %in% c("false", "0", "dire_win", "dire", "loss", "no") ~ "Dire_Win",
      TRUE ~ NA_character_
    ),
    radiant_win = factor(radiant_win, levels = c("Dire_Win", "Radiant_Win"))
  ) %>%
  select(-radiant_win_text) %>%
  drop_na(radiant_win)

target_balance <- model_prepped %>%
  count(radiant_win, name = "matches") %>%
  mutate(percent = matches / sum(matches))

kable(target_balance, digits = 3, caption = "Target Balance")

Target Balance
radiant_win	matches	percent
Dire_Win	2357	0.471
Radiant_Win	2643	0.529

Why this matters: A model should not be judged only against 50%. If one side already wins slightly more often in the sample, the model must be compared with that majority-class baseline.

7 Domain-Informed Feature Engineering

The original project idea came from gameplay experience, but this report uses the term domain-informed rather than expert-deterministic. The goal is to use Dota knowledge to ask better questions while allowing the data to limit the conclusion.

attribute_imbalance <- function(str_count, agi_count, int_count, all_count) {
  attr_mean <- (str_count + agi_count + int_count + all_count) / 4
  abs(str_count - attr_mean) +
    abs(agi_count - attr_mean) +
    abs(int_count - attr_mean) +
    abs(all_count - attr_mean)
}

model_domain <- model_prepped %>%
  mutate(
    melee_advantage_category = case_when(
      melee_count_difference <= -2 ~ "Radiant much less melee",
      melee_count_difference == -1 ~ "Radiant slightly less melee",
      melee_count_difference == 0 ~ "Even melee count",
      melee_count_difference == 1 ~ "Radiant slightly more melee",
      melee_count_difference >= 2 ~ "Radiant much more melee",
      TRUE ~ NA_character_
    ),
    melee_advantage_category = factor(
      melee_advantage_category,
      levels = c(
        "Radiant much less melee",
        "Radiant slightly less melee",
        "Even melee count",
        "Radiant slightly more melee",
        "Radiant much more melee"
      )
    ),
    radiant_heavy_melee = as.integer(radiant_melee_count >= 4),
    dire_heavy_melee = as.integer(dire_melee_count >= 4),
    radiant_all_ranged = as.integer(radiant_melee_count == 0),
    dire_all_ranged = as.integer(dire_melee_count == 0),
    radiant_all_melee = as.integer(radiant_ranged_count == 0),
    dire_all_melee = as.integer(dire_ranged_count == 0),
    radiant_low_frontline_proxy = as.integer(radiant_str_count <= 1 & radiant_all_count <= 1),
    dire_low_frontline_proxy = as.integer(dire_str_count <= 1 & dire_all_count <= 1),
    frontline_proxy_difference = radiant_low_frontline_proxy - dire_low_frontline_proxy,
    radiant_attribute_imbalance = attribute_imbalance(
      radiant_str_count,
      radiant_agi_count,
      radiant_int_count,
      radiant_all_count
    ),
    dire_attribute_imbalance = attribute_imbalance(
      dire_str_count,
      dire_agi_count,
      dire_int_count,
      dire_all_count
    ),
    attribute_imbalance_difference = radiant_attribute_imbalance - dire_attribute_imbalance,
    radiant_high_agi_proxy = as.integer(radiant_agi_count >= 3),
    dire_high_agi_proxy = as.integer(dire_agi_count >= 3),
    high_agi_proxy_difference = radiant_high_agi_proxy - dire_high_agi_proxy,
    radiant_high_int_proxy = as.integer(radiant_int_count >= 4),
    dire_high_int_proxy = as.integer(dire_int_count >= 4),
    high_int_proxy_difference = radiant_high_int_proxy - dire_high_int_proxy
  ) %>%
  drop_na()

feature_summary <- tibble::tribble(
  ~Feature_Group, ~Examples, ~Interpretation,
  "Attack type balance", "melee_count_difference, radiant_heavy_melee", "Tests whether broad melee/ranged draft shape has signal",
  "Frontline proxy", "radiant_low_frontline_proxy, frontline_proxy_difference", "Rough proxy for durability/initiation risk; not a perfect role label",
  "Attribute structure", "attribute_imbalance_difference", "Tests whether skewed STR/AGI/INT/Universal profiles matter",
  "Scaling/spell pressure proxies", "high_agi_proxy_difference, high_int_proxy_difference", "Exploratory proxies for greedy or spell-heavy draft shapes"
)

kable(feature_summary, caption = "Engineered Domain-Informed Features")

Engineered Domain-Informed Features
Feature_Group	Examples	Interpretation
Attack type balance	melee_count_difference, radiant_heavy_melee	Tests whether broad melee/ranged draft shape has signal
Frontline proxy	radiant_low_frontline_proxy, frontline_proxy_difference	Rough proxy for durability/initiation risk; not a perfect role label
Attribute structure	attribute_imbalance_difference	Tests whether skewed STR/AGI/INT/Universal profiles matter
Scaling/spell pressure proxies	high_agi_proxy_difference, high_int_proxy_difference	Exploratory proxies for greedy or spell-heavy draft shapes

8 Exploratory Data Analysis

8.1 Target Distribution

target_balance %>%
  ggplot(aes(x = radiant_win, y = matches, fill = radiant_win)) +
  geom_col(width = 0.62, color = "#E5E7EB", linewidth = 0.35) +
  geom_text(
    aes(label = paste0(matches, "\n", percent(percent, accuracy = 0.1))),
    vjust = -0.35,
    color = "#F8FAFC",
    fontface = "bold",
    size = 4.2
  ) +
  scale_fill_manual(values = c("Dire_Win" = dire_red_bright, "Radiant_Win" = radiant_green), guide = "none") +
  scale_y_continuous(expand = expansion(mult = c(0.02, 0.20))) +
  coord_cartesian(clip = "off") +
  labs(
    title = "Match Outcome Distribution",
    subtitle = "The sample has a slight Radiant-side majority, so baseline comparison matters.",
    x = "Outcome",
    y = "Matches"
  ) +
  lane_theme(base_size = 13)

8.2 Win Rate by Melee Advantage Category

melee_category_summary <- model_domain %>%
  group_by(melee_advantage_category) %>%
  summarise(
    matches = n(),
    radiant_win_rate = mean(radiant_win == "Radiant_Win"),
    .groups = "drop"
  ) %>%
  mutate(win_rate_label = percent(radiant_win_rate, accuracy = 0.1))

kable(melee_category_summary, digits = 3, caption = "Radiant Win Rate by Melee Advantage Category")

Radiant Win Rate by Melee Advantage Category
melee_advantage_category	matches	radiant_win_rate	win_rate_label
Radiant much less melee	589	0.487	48.7%
Radiant slightly less melee	1177	0.517	51.7%
Even melee count	1466	0.550	55.0%
Radiant slightly more melee	1144	0.525	52.5%
Radiant much more melee	624	0.546	54.6%

melee_category_summary %>%
  mutate(
    lane_pressure = case_when(
      str_detect(as.character(melee_advantage_category), "less") ~ "Dire pressure",
      str_detect(as.character(melee_advantage_category), "more") ~ "Radiant pressure",
      TRUE ~ "Even lane shape"
    )
  ) %>%
  ggplot(aes(x = melee_advantage_category, y = radiant_win_rate, fill = lane_pressure)) +
  geom_col(width = 0.68, color = "#E5E7EB", linewidth = 0.28) +
  geom_text(
    aes(label = paste0(win_rate_label, "\nn=", matches)),
    vjust = -0.35,
    size = 3.5,
    color = "#F8FAFC",
    fontface = "bold"
  ) +
  scale_fill_manual(
    values = c("Dire pressure" = dire_red_bright, "Even lane shape" = neutral_gold, "Radiant pressure" = radiant_green),
    name = "Draft pressure"
  ) +
  scale_y_continuous(labels = percent_format(), expand = expansion(mult = c(0.02, 0.18))) +
  coord_cartesian(ylim = c(0, max(melee_category_summary$radiant_win_rate, na.rm = TRUE) + 0.10), clip = "off") +
  labs(
    title = "Radiant Win Rate by Melee Advantage Category",
    subtitle = "This broad team-level view is useful for exploration but may be too blunt for the real lane theory.",
    x = "Melee Advantage Category",
    y = "Radiant Win Rate"
  ) +
  lane_theme(base_size = 13) +
  theme(axis.text.x = element_text(angle = 22, hjust = 1))

8.3 Draft-Risk Flag Summary

risk_flag_summary <- model_domain %>%
  summarise(
    overall_radiant_wr = mean(radiant_win == "Radiant_Win"),
    radiant_heavy_melee_wr = mean(radiant_win == "Radiant_Win" & radiant_heavy_melee == 1) / mean(radiant_heavy_melee == 1),
    radiant_all_ranged_wr = mean(radiant_win == "Radiant_Win" & radiant_all_ranged == 1) / mean(radiant_all_ranged == 1),
    radiant_low_frontline_proxy_wr = mean(radiant_win == "Radiant_Win" & radiant_low_frontline_proxy == 1) / mean(radiant_low_frontline_proxy == 1),
    radiant_high_agi_proxy_wr = mean(radiant_win == "Radiant_Win" & radiant_high_agi_proxy == 1) / mean(radiant_high_agi_proxy == 1),
    radiant_high_int_proxy_wr = mean(radiant_win == "Radiant_Win" & radiant_high_int_proxy == 1) / mean(radiant_high_int_proxy == 1)
  ) %>%
  pivot_longer(everything(), names_to = "draft_condition", values_to = "radiant_win_rate") %>%
  mutate(
    radiant_win_rate = replace_na(radiant_win_rate, 0),
    draft_condition = str_replace_all(draft_condition, "_", " ")
  )

kable(risk_flag_summary, digits = 3, caption = "Radiant Win Rate Under Selected Draft-Risk Conditions")

Radiant Win Rate Under Selected Draft-Risk Conditions
draft_condition	radiant_win_rate
overall radiant wr	0.529
radiant heavy melee wr	0.540
radiant all ranged wr	0.435
radiant low frontline proxy wr	0.511
radiant high agi proxy wr	0.549
radiant high int proxy wr	0.563

8.4 Attribute Imbalance

attribute_imbalance_summary <- model_domain %>%
  mutate(
    radiant_more_imbalanced = case_when(
      attribute_imbalance_difference > 0 ~ "Radiant more imbalanced",
      attribute_imbalance_difference < 0 ~ "Dire more imbalanced",
      TRUE ~ "Equal imbalance"
    )
  ) %>%
  group_by(radiant_more_imbalanced) %>%
  summarise(
    matches = n(),
    radiant_win_rate = mean(radiant_win == "Radiant_Win"),
    .groups = "drop"
  ) %>%
  mutate(win_rate_label = percent(radiant_win_rate, accuracy = 0.1))

kable(attribute_imbalance_summary, digits = 3, caption = "Radiant Win Rate by Attribute Imbalance Direction")

Radiant Win Rate by Attribute Imbalance Direction
radiant_more_imbalanced	matches	radiant_win_rate	win_rate_label
Dire more imbalanced	1802	0.530	53.0%
Equal imbalance	1460	0.508	50.8%
Radiant more imbalanced	1738	0.545	54.5%

attribute_imbalance_summary %>%
  mutate(
    imbalance_side = case_when(
      radiant_more_imbalanced == "Radiant more imbalanced" ~ "Radiant",
      radiant_more_imbalanced == "Dire more imbalanced" ~ "Dire",
      TRUE ~ "Equal"
    )
  ) %>%
  ggplot(aes(x = radiant_more_imbalanced, y = radiant_win_rate, fill = imbalance_side)) +
  geom_col(width = 0.66, color = "#E5E7EB", linewidth = 0.28) +
  geom_text(
    aes(label = paste0(win_rate_label, "\nn=", matches)),
    vjust = -0.35,
    size = 3.6,
    color = "#F8FAFC",
    fontface = "bold"
  ) +
  scale_fill_manual(values = c("Dire" = dire_red_bright, "Equal" = neutral_gold, "Radiant" = radiant_green), guide = "none") +
  scale_y_continuous(labels = percent_format(), expand = expansion(mult = c(0.02, 0.18))) +
  coord_cartesian(ylim = c(0, max(attribute_imbalance_summary$radiant_win_rate, na.rm = TRUE) + 0.10), clip = "off") +
  labs(
    title = "Radiant Win Rate by Attribute Imbalance Direction",
    subtitle = "Attribute composition is an exploratory proxy, not a direct measure of execution or role quality.",
    x = "Attribute Imbalance Category",
    y = "Radiant Win Rate"
  ) +
  lane_theme(base_size = 13)

9 Modeling Setup

model_predictors <- c(
  "melee_count_difference",
  "str_count_difference",
  "agi_count_difference",
  "int_count_difference",
  "all_count_difference",
  "attribute_imbalance_difference",
  "frontline_proxy_difference",
  "high_agi_proxy_difference",
  "high_int_proxy_difference",
  "radiant_heavy_melee",
  "dire_heavy_melee",
  "radiant_all_ranged",
  "dire_all_ranged",
  "radiant_low_frontline_proxy",
  "dire_low_frontline_proxy"
)

model_df <- model_domain %>%
  select(radiant_win, all_of(model_predictors)) %>%
  drop_na()

modeling_columns <- tibble(
  column = names(model_df),
  type = map_chr(model_df, ~ class(.x)[1])
)

kable(modeling_columns, caption = "Final Modeling Columns")

Final Modeling Columns
column	type
radiant_win	factor
melee_count_difference	numeric
str_count_difference	numeric
agi_count_difference	numeric
int_count_difference	numeric
all_count_difference	numeric
attribute_imbalance_difference	numeric
frontline_proxy_difference	integer
high_agi_proxy_difference	integer
high_int_proxy_difference	integer
radiant_heavy_melee	integer
dire_heavy_melee	integer
radiant_all_ranged	integer
dire_all_ranged	integer
radiant_low_frontline_proxy	integer
dire_low_frontline_proxy	integer

set.seed(580)

model_df_with_id <- model_df %>%
  mutate(row_id = row_number())

train_ids <- model_df_with_id %>%
  group_by(radiant_win) %>%
  slice_sample(prop = 0.80) %>%
  ungroup() %>%
  pull(row_id)

train_data <- model_df_with_id %>%
  filter(row_id %in% train_ids) %>%
  select(-row_id)

test_data <- model_df_with_id %>%
  filter(!row_id %in% train_ids) %>%
  select(-row_id)

split_summary <- tibble(
  dataset = c("Training", "Testing"),
  rows = c(nrow(train_data), nrow(test_data)),
  radiant_win_rate = c(
    mean(train_data$radiant_win == "Radiant_Win"),
    mean(test_data$radiant_win == "Radiant_Win")
  )
)

kable(split_summary, digits = 3, caption = "Train/Test Split Summary")

Train/Test Split Summary
dataset	rows	radiant_win_rate
Training	3999	0.529
Testing	1001	0.528

calc_auc <- function(actual_factor, predicted_probability) {
  actual <- as.integer(actual_factor == "Radiant_Win")
  n_pos <- sum(actual == 1)
  n_neg <- sum(actual == 0)
  if (n_pos == 0 || n_neg == 0) {
    return(NA_real_)
  }
  ranks <- rank(predicted_probability, ties.method = "average")
  auc <- (sum(ranks[actual == 1]) - n_pos * (n_pos + 1) / 2) / (n_pos * n_neg)
  auc
}

make_metrics <- function(model_name, actual, predicted_class, predicted_probability = NA_real_) {
  tibble(
    model = model_name,
    accuracy = mean(actual == predicted_class),
    roc_auc = if (all(is.na(predicted_probability))) NA_real_ else calc_auc(actual, predicted_probability)
  )
}

10 Baseline Model

majority_class <- train_data %>%
  count(radiant_win) %>%
  mutate(rate = n / sum(n)) %>%
  arrange(desc(rate)) %>%
  slice(1)

baseline_class <- factor(
  rep(as.character(majority_class$radiant_win), nrow(test_data)),
  levels = levels(test_data$radiant_win)
)

baseline_metrics <- make_metrics(
  "Majority-class baseline",
  test_data$radiant_win,
  baseline_class
)

kable(
  majority_class,
  digits = 3,
  caption = "Majority-Class Baseline from Training Data"
)

Majority-Class Baseline from Training Data
radiant_win	n	rate
Radiant_Win	2114	0.529

11 Model 1: Logistic Regression

Logistic regression is the interpretable baseline model. It is useful because it shows direction and magnitude of association, even when predictive lift is weak.

log_train <- train_data %>%
  mutate(radiant_win_binary = as.integer(radiant_win == "Radiant_Win"))

log_formula <- as.formula(
  paste("radiant_win_binary ~", paste(model_predictors, collapse = " + "))
)

log_fit <- glm(log_formula, data = log_train, family = binomial())

log_prob <- predict(log_fit, newdata = test_data, type = "response")
log_class <- factor(
  if_else(log_prob >= 0.50, "Radiant_Win", "Dire_Win"),
  levels = levels(test_data$radiant_win)
)

log_metrics <- make_metrics(
  "Logistic regression",
  test_data$radiant_win,
  log_class,
  log_prob
)

kable(log_metrics, digits = 3, caption = "Logistic Regression Performance")

Logistic Regression Performance
model	accuracy	roc_auc
Logistic regression	0.519	0.492

log_coefficients <- broom::tidy(log_fit) %>%
  filter(term != "(Intercept)") %>%
  mutate(
    odds_ratio = exp(estimate),
    direction = case_when(
      estimate > 0 ~ "Higher estimated odds of Radiant win",
      estimate < 0 ~ "Lower estimated odds of Radiant win",
      TRUE ~ "No estimated direction"
    )
  ) %>%
  arrange(desc(abs(estimate)))

kable(
  log_coefficients %>%
    select(term, estimate, odds_ratio, p.value, direction) %>%
    head(12),
  digits = 3,
  caption = "Top Logistic Regression Coefficients by Absolute Size"
)

Top Logistic Regression Coefficients by Absolute Size
term	estimate	odds_ratio	p.value	direction
dire_heavy_melee	0.250	1.285	0.060	Higher estimated odds of Radiant win
high_int_proxy_difference	0.248	1.282	0.322	Higher estimated odds of Radiant win
dire_all_ranged	-0.242	0.785	0.389	Lower estimated odds of Radiant win
radiant_all_ranged	-0.215	0.806	0.387	Lower estimated odds of Radiant win
high_agi_proxy_difference	0.144	1.155	0.202	Higher estimated odds of Radiant win
frontline_proxy_difference	-0.135	0.874	0.128	Lower estimated odds of Radiant win
int_count_difference	0.076	1.079	0.100	Higher estimated odds of Radiant win
agi_count_difference	0.068	1.070	0.171	Higher estimated odds of Radiant win
str_count_difference	0.053	1.055	0.223	Higher estimated odds of Radiant win
radiant_low_frontline_proxy	-0.042	0.959	0.647	Lower estimated odds of Radiant win
melee_count_difference	0.028	1.028	0.469	Higher estimated odds of Radiant win
radiant_heavy_melee	-0.014	0.986	0.917	Lower estimated odds of Radiant win

12 Model 2: Tree-Based Comparison Model

A tree-based model is included because Dota draft logic is interaction-heavy. A hero trait may matter differently depending on the other nine heroes in the match. If nonlinear feature interactions were strong in this feature set, the tree-based model would have a chance to improve over logistic regression.

# Fast-knit version for presentation night.
# A decision tree still satisfies the "second model type" requirement while avoiding
# the long random-forest/permutation-importance step that can time out in Posit Cloud.

tree_train <- train_data %>% select(radiant_win, all_of(model_predictors))

tree_fit <- rpart::rpart(
  radiant_win ~ .,
  data = tree_train,
  method = "class",
  control = rpart.control(
    cp = 0.01,
    minsplit = 50,
    maxdepth = 5,
    xval = 0
  )
)

tree_pred <- predict(tree_fit, newdata = test_data, type = "prob")
tree_prob <- tree_pred[, "Radiant_Win"]
tree_model_name <- "Decision tree"

tree_importance <- tibble(
  feature = names(tree_fit$variable.importance),
  importance = as.numeric(tree_fit$variable.importance)
) %>%
  arrange(desc(importance))

tree_class <- factor(
  if_else(tree_prob >= 0.50, "Radiant_Win", "Dire_Win"),
  levels = levels(test_data$radiant_win)
)

tree_metrics <- make_metrics(
  tree_model_name,
  test_data$radiant_win,
  tree_class,
  tree_prob
)

kable(tree_metrics, digits = 3, caption = "Decision Tree Model Performance")

Decision Tree Model Performance
model	accuracy	roc_auc
Decision tree	0.528	0.5

if (nrow(tree_importance) > 0) {
  top_tree_importance <- tree_importance %>%
    slice_max(importance, n = 10) %>%
    mutate(feature = forcats::fct_reorder(feature, importance))

  kable(
    top_tree_importance,
    digits = 3,
    caption = "Top Tree-Based Model Feature Importance Values"
  )

  top_tree_importance %>%
    ggplot(aes(x = feature, y = importance)) +
    geom_col(fill = radiant_green, color = "#E5E7EB", linewidth = 0.22, width = 0.68) +
    coord_flip(clip = "off") +
    labs(
      title = "Decision Tree Feature Importance",
      subtitle = "A fast-knit tree model keeps the second-model comparison stable for presentation night.",
      x = "Feature",
      y = "Importance"
    ) +
    lane_theme(base_size = 13)
}

13 Model Evaluation and Comparison

model_comparison <- bind_rows(
  baseline_metrics,
  log_metrics,
  tree_metrics
) %>%
  mutate(
    accuracy = round(accuracy, 3),
    roc_auc = round(roc_auc, 3),
    interpretation = case_when(
      model == "Majority-class baseline" ~ "Benchmark: always predicts the majority side",
      model == "Logistic regression" ~ "Interpretable draft-risk baseline",
      TRUE ~ "Checks whether nonlinear interactions improve prediction"
    )
  )

kable(model_comparison, caption = "Model Comparison Against Baseline")

Model Comparison Against Baseline
model	accuracy	roc_auc	interpretation
Majority-class baseline	0.528	NA	Benchmark: always predicts the majority side
Logistic regression	0.519	0.492	Interpretable draft-risk baseline
Decision tree	0.528	0.500	Checks whether nonlinear interactions improve prediction

confusion_log <- table(
  Actual = test_data$radiant_win,
  Predicted = log_class
)

confusion_tree <- table(
  Actual = test_data$radiant_win,
  Predicted = tree_class
)

cat("Logistic Regression Confusion Matrix\n")

## Logistic Regression Confusion Matrix

print(confusion_log)

##              Predicted
## Actual        Dire_Win Radiant_Win
##   Dire_Win         111         361
##   Radiant_Win      120         409

cat("\nTree-Based Model Confusion Matrix\n")

## 
## Tree-Based Model Confusion Matrix

print(confusion_tree)

##              Predicted
## Actual        Dire_Win Radiant_Win
##   Dire_Win           0         472
##   Radiant_Win        0         529

probability_df <- bind_rows(
  tibble(model = "Logistic regression", probability = log_prob, actual = test_data$radiant_win),
  tibble(model = tree_model_name, probability = tree_prob, actual = test_data$radiant_win)
)

probability_df %>%
  ggplot(aes(x = probability, fill = actual)) +
  geom_histogram(bins = 28, alpha = 0.78, color = "#E5E7EB", linewidth = 0.18, position = "identity") +
  geom_vline(xintercept = 0.50, color = neutral_gold, linewidth = 0.9, linetype = "dashed") +
  facet_wrap(~ model, ncol = 1, scales = "free_y") +
  scale_fill_manual(values = c("Dire_Win" = dire_red_bright, "Radiant_Win" = radiant_green), name = "Actual outcome") +
  scale_x_continuous(labels = percent_format(accuracy = 1), limits = c(0, 1), breaks = seq(0, 1, by = 0.10)) +
  labs(
    title = "Predicted Radiant Win Probability Distribution",
    subtitle = "The dashed line marks coin-flip territory; clustering near 50% suggests weak class separation.",
    x = "Predicted probability of Radiant win",
    y = "Test-set matches"
  ) +
  lane_theme(base_size = 13)

14 Plain-English Interpretation

Main result: The project did not strongly validate the original gameplay theory as a predictive claim. Instead, the models showed that broad team-level draft-composition features are too blunt to reliably explain match outcomes by themselves.

The most important comparison is not whether a model exceeds 50% accuracy. The right comparison is whether it improves on the majority-class baseline. If a model cannot beat the baseline, that means the feature set is not yet capturing enough useful signal.

That result is still valuable. It prevents overclaiming and points to a better version of the project. My original theory is not really about total team composition. It is more specifically about lane-versus-lane interaction: which heroes are facing each other in the first several minutes, whether the support is actually supporting, whether the lane can trade, whether initiation exists, and whether the draft requires coordination that public matches may not provide.

15 Insights

The current model is a useful exploratory baseline, not a final prediction engine.
It creates a reproducible starting point for Dota draft analytics.
Overall composition is probably too broad.
A total melee/ranged count can miss the actual gameplay mechanism if the real pressure happens in one lane.
Baseline comparison changed the story.
A model can appear acceptable near 50%, but still fail to outperform the default probability structure of the dataset.
The next model needs richer context.
Lane assignments, role fidelity, rank bracket, patch version, communication score, behavior score, and hero-specific archetypes would make the analysis more aligned with actual Dota gameplay.

16 Limitations

Boundary of the claim: This report should not be read as proof that any single lane composition causes wins or losses. It is a first-pass public-data modeling pipeline that shows what simple draft-composition data can and cannot explain.

Key limitations:

The data uses match-level draft-composition features, not actual lane-versus-lane assignments.
Melee/ranged status is a simplified proxy and does not fully describe lane strength, trading, stun reliability, initiation, or survivability.
Primary attribute is also a proxy and does not directly equal role, scaling, or frontline function.
The model does not observe communication, behavior score, griefing, role abuse, bad pulls, inopportune dives, support quality, itemization, or player execution.
Public match data may mix game modes, patches, ranks, regions, and player intentions.
Correlation should not be interpreted as causation.

17 Recommendations and Next Iteration

The next version should shift from a broad draft-composition model to a lane-versus-lane draft-risk model.

Recommended improvements:

Build lane-specific matchup features rather than only team-level composition features.
Add behavior score and communication score if available.
Add rank bracket, patch version, lobby type, and game mode controls.
Add hero archetypes: initiation, stun reliability, sustain, lane harassment, scaling pressure, and frontline.
Build an interactive draft-risk dashboard after improving the dataset.

18 Presentation Talk Track

18.1 30-second opening

I started this project with a Dota theory from my own gameplay experience: certain lane and draft compositions, especially melee/ranged mismatches, feel fragile and often lose. I wanted to test whether that intuition showed up in public match data. The project evolved into a domain-informed predictive analytics pipeline using the Sprint 3 expanded dataset, engineered draft-risk features, and two model types. The final result did not strongly verify my original theory, but it taught me how to ask the question better and what additional data would be needed for a stronger esports BI tool.

18.2 30-second conclusion

The models were close to coin-flip and did not clearly outperform the majority-class baseline. That is not a failed project; it is an honest analytical result. It suggests that simple public draft-count features are not enough by themselves. The next iteration should add lane assignment, role fidelity, rank bracket, patch context, game mode, behavior score, communication score, and better hero archetypes. This project now serves as a reproducible baseline for a future Dota draft-risk dashboard.

19 Q&A Prep

qa_table <- tibble::tribble(
  ~Question, ~Prepared_Response,
  "Why keep the project if the models were weak?", "Weak results are useful when evaluated honestly. The model showed that broad draft-composition features are not enough, which directly improves the next project design.",
  "Why logistic regression?", "It is an interpretable baseline for a binary win/loss outcome and helps explain feature direction rather than only producing a prediction.",
  "Why a tree-based model?", "Dota is interaction-heavy. A tree-based model checks whether nonlinear feature combinations improve prediction.",
  "What would improve the project most?", "Lane-versus-lane features, behavior score, communication score, role fidelity, rank bracket, patch context, and hero archetypes.",
  "What is the business analytics angle?", "The project frames esports as a decision-support problem: can public data become a draft-risk dashboard for players, analysts, or content creators?"
)

kable(qa_table, caption = "Presentation Q&A Preparation")

Presentation Q&A Preparation
Question	Prepared_Response
Why keep the project if the models were weak?	Weak results are useful when evaluated honestly. The model showed that broad draft-composition features are not enough, which directly improves the next project design.
Why logistic regression?	It is an interpretable baseline for a binary win/loss outcome and helps explain feature direction rather than only producing a prediction.
Why a tree-based model?	Dota is interaction-heavy. A tree-based model checks whether nonlinear feature combinations improve prediction.
What would improve the project most?	Lane-versus-lane features, behavior score, communication score, role fidelity, rank bracket, patch context, and hero archetypes.
What is the business analytics angle?	The project frames esports as a decision-support problem: can public data become a draft-risk dashboard for players, analysts, or content creators?

20 Reflection

This project gave me practical experience with API-style data pulls, larger public datasets, feature engineering, and predictive modeling. The final result was not the clean validation of my gameplay theory that I originally hoped for. However, that made the project more useful as a learning exercise because it showed me how important problem framing is.

The biggest improvement in my thinking was moving from overall team composition toward lane-versus-lane analysis. My actual theory is not just that one team has more melee heroes than the other. It is that certain lane matchups create pressure patterns that are difficult to survive in public matches. The current model does not fully capture that, which explains why the results were weak and why the next version needs more precise data.

A passion project can still demonstrate serious analytics skills. Not every analytics project has to be about stocks, finance, or traditional business examples. Esports data can still show data collection, reproducibility, model comparison, baseline evaluation, honest limitations, and decision-support storytelling.

21 AI Use Disclosure

I used ChatGPT to support project planning, debugging, requirement checking, narrative refinement, and presentation preparation. All analysis, interpretation, modeling decisions, and conclusions are my own.

22 Reproducibility Notes

To reproduce this report:

Place this .Rmd file in the Lane Theory project root folder.
Place the final dataset at data/lane_theory_modeling_dataset_sprint3_expanded.csv or in the project root as lane_theory_modeling_dataset_sprint3_expanded.csv.
Install missing packages shown in the package chunk.
Knit to HTML.
Publish the knitted HTML to RPubs and submit the RPubs link or requested files in Canvas.

Lane Theory Lab: Dota 2 Draft Composition Analytics

A domain-informed Kaggle-style project on public-match draft risk, model comparison, and honest limitations

Shep

2026-07-08