August 24, 2026

Welcome to the Model Inventory Challenge

How to play: Model 1 (Baseline) is already in every ensemble. For each case study, pick 2 of Models 2–6 to add to it. On “vote,” hold up that many fingers on each hand at once (e.g., two fingers + four fingers = Model 2 and Model 4). We’ll count the room’s majority against the winning combo. Three rounds, one per business area — let’s see how many the room gets.

Meet the Models

1 Baseline
Logistic regression on standard structured data. Transparent, well understood. Always included.

2 Random Forest
Same data as baseline, different algorithm. Captures nonlinear patterns. Stable and robust.

3 XGBoost + New Features
Same data as baseline, plus newly engineered features. Highest accuracy on historical data.

4 Alternate Data Model
Built on a different but relevant data source. Weaker standalone accuracy than baseline.

5 Random Guess Model
Simple Rule-of-Thumb Model

6 Deep Neural Network
Highly complex, black-box. Needs a lot of data and compute. Hard to interpret.

Case Study 1: Fraud Detection

Real-time card-present transaction screening. The bank has years of stable, high-volume labeled fraud data. Decisions must be made in milliseconds at checkout, and the logic must be explainable to examiners and declined customers.

2 Random Forest
Same data as baseline, different algorithm. Captures nonlinear patterns. Stable and robust.

3 XGBoost + New Features
Same data as baseline, plus newly engineered features. Highest accuracy on historical data.

4 Alternate Data Model
Built on a different but relevant data source. Weaker standalone accuracy than baseline.

5 Random Guess Model
Simple Rule-of-Thumb Model.

6 Deep Neural Network
Highly complex, black-box. Needs a lot of data and compute. Hard to interpret.

Vote now: which 2 models join the baseline?

Case Study 1: Reveal

Winning ensemble: Baseline + Model 3 + Model 4
  • Model 3 — real accuracy lift on data the bank already trusts, fast enough to score in real time.
  • Model 4 — different data source catches fraud patterns the transaction data alone would miss.
  • Model 6 — best paper accuracy, but too slow and too opaque for a millisecond, examiner-facing decision.
  • Model 2 — not wrong, just redundant next to Model 3.
  • Model 5 — no signal to add.

Case Study 2: Marketing

Retention campaign for a long-established product line. The bank holds a solid but moderate volume of interaction history, and response is driven by complex interactions among several factors (tenure, engagement, channel) rather than any single indicator. A new consent policy means the external data source used in other campaigns cannot be applied to this customer segment.

2 Random Forest
Same data as baseline, different algorithm. Captures nonlinear patterns. Stable and robust.

3 XGBoost + New Features
Same data as baseline, plus newly engineered features. Highest accuracy on historical data.

4 Alternate Data Model
Built on a different but relevant data source. Weaker standalone accuracy than baseline.

5 Random Guess Model
Simple Rule-of-Thumb Model

6 Deep Neural Network
Highly complex, black-box. Needs a lot of data and compute. Hard to interpret.

Vote now: which 2 models join the baseline?

Case Study 2: Reveal

Winning ensemble: Baseline + Model 2 + Model 3
  • Model 2 — its splits naturally pick up interaction effects among tenure, engagement, and channel that a single engineered feature can’t represent.
  • Model 3 — still contributes proven, well-understood individual risk indicators that complement what Model 2 finds on its own.
  • Model 4 — the external data source is now off-limits for this segment under the new consent policy.
  • Model 6 — a moderate volume of history isn’t enough to support a deep model without overfitting.
  • Model 5 — no signal to add.

Case Study 3: Cybersecurity

Detecting network intrusions. The bank captures billions of network events per day, and adversaries are constantly changing tactics to evade known signatures.

2 Random Forest
Same data as baseline, different algorithm. Captures nonlinear patterns. Stable and robust.

3 XGBoost + New Features
Same data as baseline, plus newly engineered features. Highest accuracy on historical data.

4 Alternate Data Model
Built on a different but relevant data source. Weaker standalone accuracy than baseline.

5 Random Guess Model
Simple Rule-of-Thumb Model.

6 Deep Neural Network
Highly complex, black-box. Needs a lot of data and compute. Hard to interpret.

Vote now: which 2 models join the baseline?

Case Study 3: Reveal

Winning ensemble: Baseline + Model 6 + Model 4
  • Model 6 — billions of events remove the overfitting risk, so its capacity to catch complex, evolving patterns becomes an asset.
  • Model 4 — an independent detection source that can flag what the network logs have never seen before.
  • Model 3 — engineered features reflect known signatures and can miss genuinely new attacks.
  • Model 2 — too limited in capacity for fast-evolving, complex attack patterns.
  • Model 5 — no signal to add.
Same Model 6 that lost Round 2 wins Round 3 — the constraint changed, not the model’s quality.

Final Scoreboard

Tally how many of the 3 rounds the room’s majority vote matched the winning ensemble.

3/3: strong intuition for context-dependent model risk.
1–2/3: good instincts, but watch the pull toward the “best-performing” model regardless of context.
0/3: a great discussion starter — even experienced practitioners default to the highest-accuracy model on paper.

No single model, including the baseline, is universally the best choice. Inventory and validation review must weigh intended use, data availability, interpretability needs, and operating constraints for each specific application.