Predicting Credit Card Fraud

Author

Rohan Gulati, Aidan Ho, Michael Chen

Report

Outcome to Predict

The primary goal of this report is to predict transaction fraud status (is_fraud)—a Boolean variable where \(Y = 1\) denotes a confirmed fraudulent credit card transaction and \(Y = 0\) represents a legitimate transaction.

Why This Outcome Is Important

Credit card fraud accounts for tens of billions of dollars in annual losses across merchants, payment gateways, and issuing banks. Fraud detection models must carefully balance security against user convenience:

  • False Negatives (Missed Fraud): Result in direct charge-backs, legal compliance penalties, and stolen funds.

  • False Positives (False Alarms): Lead to declined legitimate purchases, customer annoyance, and lost merchant revenue.

Target Audience

  1. Risk & Fraud Operations Teams: To flag anomalous activity to prevent fraud.
  2. Card Issuers & Payment Networks: To identify emerging fraud patterns, including AI-generated scam attempts.
  3. Credit Card Users: To prevent them from being a victim of fraud and understanding signs to look for.

Download Libraries and Data File

I import the credit_card_fraud_2026 data from Kaggle. The dataset contains 20,000 transaction records capturing modern credit card transaction dynamics and security threat indicators (such as AI-generated scams, VPN masking, and geolocation mismatches). These variables help in predicting in whether a transaction is a fraud or not.

Variable Descriptions, Variable Types, and Levels of Measurement

Variable Name Variable Type Level of Measurement Description
is_fraud Qualitative Nominal Target binary dependent variable (1 if fraudulent, 0 if legitimate).
amount_usd Quantitative (Continuous) Ratio Total transaction monetary value in US Dollars.
velocity_score Quantitative (Continuous) Ratio Calculated metric measuring card activity speed and frequency.
merchant_risk_score Quantitative (Continuous) Ratio Automated risk rating score assigned to the merchant (0.0 to 100.0).
cvv_retry_count Quantitative (Discrete) Ratio Number of failed CVV security code verification attempts during checkout.
customer_age Quantitative (Discrete) Ratio Age of the cardholder in years.
card_age_months Quantitative (Discrete) Ratio Age of the credit card account in months.
prior_disputes Quantitative (Discrete) Ratio Historical count of chargebacks or billing disputes filed by the customer.
time_of_day_hour Quantitative (Discrete) Interval Hour of the day the transaction occurred (0 to 23).
ip_country_ mismatch Qualitative Nominal Flag indicating whether IP geolocation country differs from card issuing country (TRUE/FALSE).
billing_shipping_ mismatch Qualitative Nominal Flag indicating whether billing address differs from shipping delivery address (TRUE/FALSE).
is_ai_generated_ scam_attempt Qualitative Nominal Security flag indicating suspected synthetic, AI-driven scam activity (TRUE/FALSE).
is_foreign_ transaction Qualitative Nominal Flag indicating whether transaction originated outside the home country (TRUE/FALSE).
is_new_merchant Qualitative Nominal Flag indicating whether the customer has never transacted at this merchant before (TRUE/FALSE).
used_vpn Qualitative Nominal Flag indicating whether transaction IP routing utilized a Virtual Private Network (TRUE/FALSE).
auth_method Qualitative Nominal Payment verification protocol utilized (Biometric, 3D Secure, OTP, PIN, None).

Data Cleaning and Organization

We begin our data cleaning by dropping incomplete observations and converting all text and Boolean indicators (TRUE/FALSE) into numeric binary variables (\(1\)/\(0\)). We then split the dataset into quantitative and qualitative data frames based on variable type, converting categorical features into factors and reshaping the numeric features into a long format for visualization.

Visualizations

Figure 1

These stacked histograms show how continuous variables are distributed using a logarithmic Y-axis scale to compare legitimate and fraudulent transactions. Broad variables like customer age and time of day spread evenly across demographics and hours, whereas features like transaction amount, distance from home, and account balance skew heavily toward lower values with long right-hand tails. Even on a logarithmic count scale designed to magnify small numbers, the fraud segments remain thin, visually illustrating how limited and exceptionally rare fraudulent transactions are across the entire population.

Figure 2

This figure looks at specific counts of customer behaviors, such as repeated CVV code retries, prior dispute histories, and recent transaction counts over 24 hours. The tallest bars sit firmly at 0 or 1, with counts dropping off sharply as numbers increase. This shows that most shoppers get their security codes right on the first try and rarely file chargebacks. When a transaction shows multiple failed CVV attempts or a sudden surge in 24-hour activity, it immediately stands out as an unusual pattern. Additionally with the stacked bar graphs, we can notice how their values hardly differ from the size of their sample.

Figure 3

This set of bar charts measures non-numerical account details, including merchant category, card type, verification method, checkout channel, and device type. The bars are relatively well-distributed across categories, showing that transactions happen on a wide variety of devices, payment channels, and card types. Because these categories reflect standard customer habits rather than direct security warnings, the model uses them to establish a baseline of what everyday shopping looks like across different platforms.

Figure 4

These pie charts show the true-versus-false percentage splits for major risk flags, such as using a VPN, location mismatches, or AI scam indicators. For almost every security indicator, the navy blue slice representing “True” (flagged activity) is extremely tiny compared to the light blue “False” slice. This visually reinforces how rare true fraud and high-risk behaviors are across everyday transactions. Seeing this baseline helps us understand why detecting fraud is difficult, when normal, safe transactions account for nearly the entire pie.

Figure 5

This grid of box plots displays the middle average ranges alongside individual outlier dots for every numerical feature in the dataset. Most variables have compact boxes centered near lower values, while long lines and scattered dots point toward extreme highs. These upper outlier dots highlight unusual transactions, such as massive account balances, high merchant risk scores, or unusually far distances from home. Identifying these extreme values helps us spot single transactions that carry much higher financial risk.

Summary Statistics


===========================================================================
Statistic                      N       Mean    St. Dev.   Min       Max    
---------------------------------------------------------------------------
transaction_id               20,000 10,000.500 5,773.647   1      20,000   
amount_usd                   20,000  132.425    256.964  1.000   6,872.690 
is_foreign_transaction       20,000   0.062      0.241     0         1     
hours_since_last_txn         20,000   8.951      8.839   0.010    87.050   
txn_count_last_24h           20,000   3.192      1.780     0        12     
distance_from_home_km        20,000   22.140    22.121   0.000    216.190  
card_age_months              20,000   46.938     6.769     22       74     
customer_age                 20,000   49.669    18.494     18       81     
account_balance_usd          20,000 3,316.663  4,350.721 52.050 127,125.900
is_new_merchant              20,000   0.230      0.421     0         1     
used_vpn                     20,000   0.088      0.283     0         1     
ip_country_mismatch          20,000   0.058      0.234     0         1     
billing_shipping_mismatch    20,000   0.044      0.204     0         1     
cvv_retry_count              20,000   0.181      0.422     0         3     
velocity_score               20,000   19.808    12.366   0.000    74.400   
time_of_day_hour             20,000   11.534     6.925     0        23     
day_of_week                  20,000   2.999      2.005     0         6     
is_ai_generated_scam_attempt 20,000   0.019      0.136     0         1     
merchant_risk_score          20,000   37.400    17.061   0.000    100.000  
prior_disputes               20,000   0.281      0.529     0         4     
is_fraud                     20,000   0.017      0.129     0         1     
---------------------------------------------------------------------------

Linear Regression

1. Estimating Equations

Model 1: Simple Linear Regression

\[\text{is\_fraud}_i = \beta_0 + \beta_1 \text{merchant\_risk\_score}_i + \varepsilon_i\]

Model 2: Multiple Linear Regression (With Controls)

\[\begin{aligned} \text{is\_fraud}_i = \beta_0 &+ \beta_1 \text{amount\_usd}_i + \beta_2 \text{auth\_method}_i + \beta_3 \text{is\_foreign\_transaction}_i \\ &+ \beta_4 \text{card\_age\_months}_i + \beta_5 \text{customer\_age}_i + \beta_6 \text{is\_new\_merchant}_i \\ &+ \beta_7 \text{used\_vpn}_i + \beta_8 \text{ip\_country\_mismatch}_i + \beta_9 \text{billing\_shipping\_mismatch}_i \\ &+ \beta_{10} \text{is\_ai\_generated\_scam\_attempt}_i + \beta_{11} \text{merchant\_risk\_score}_i + \beta_{12} \text{prior\_disputes}_i \\ &+ \beta_{13} \text{cvv\_retry\_count}_i + \beta_{14} \text{velocity\_score}_i + \beta_{15} \text{time\_of\_day\_hour}_i + \varepsilon_i \end{aligned}\]

Model 3: Polynomial Linear Regression (With Quadratic Terms)

\[\begin{aligned} \text{is\_fraud}_i = \beta_0 &+ \beta_1 \text{amount\_usd}_i + \beta_2 \text{auth\_method}_i + \beta_3 \text{is\_foreign\_transaction}_i \\ &+ \beta_4 \text{card\_age\_months}_i + \beta_5 \text{customer\_age}_i + \beta_6 \text{is\_new\_merchant}_i \\ &+ \beta_7 \text{used\_vpn}_i + \beta_8 \text{ip\_country\_mismatch}_i + \beta_9 \text{billing\_shipping\_mismatch}_i \\ &+ \beta_{10} \text{is\_ai\_generated\_scam\_attempt}_i + \beta_{11} \text{merchant\_risk\_score}_i + \beta_{12} \text{prior\_disputes}_i \\ &+ \beta_{13} \text{cvv\_retry\_count}_i + \beta_{14} \text{cvv\_retry\_count}_i^2 \\ &+ \beta_{15} \text{velocity\_score}_i + \beta_{16} \text{velocity\_score}_i^2 \\ &+ \beta_{17} \text{time\_of\_day\_hour}_i + \beta_{18} \text{time\_of\_day\_hour}_i^2 \\ &+ \beta_4 \text{merchant\_risk\_score}_i^2 + \varepsilon_i \end{aligned}\]

2. Variable Definitions

Dependent Variable

  • is_fraud: Binary indicator equal to \(1\) if the transaction was fraudulent, and \(0\) otherwise.

Independent Variables & Controls

  • merchant_risk_score: Continuous risk score assigned to the merchant based on historical activity.
  • amount_usd: Continuous variable representing the total transaction value in US Dollars.
  • auth_method: Categorical variable indicating the authentication method used during checkout.
  • is_foreign_transaction: Binary indicator (\(1\) if transaction originated outside the card issuing country, \(0\) otherwise).
  • card_age_months: Continuous variable measuring the age of the payment card in months.
  • customer_age: Continuous variable representing the customer’s age in years.
  • is_new_merchant: Binary indicator (\(1\) if the customer has never purchased with this merchant before, \(0\) otherwise).
  • used_vpn: Binary indicator (\(1\) if a Virtual Private Network was detected during transaction, \(0\) otherwise).
  • ip_country_mismatch: Binary indicator (\(1\) if IP address country differs from billing country, \(0\) otherwise).
  • billing_shipping_mismatch: Binary indicator (\(1\) if billing address does not match shipping address, \(0\) otherwise).
  • is_ai_generated_scam_attempt: Binary indicator (\(1\) if AI-assisted fraud techniques were detected, \(0\) otherwise).
  • prior_disputes: Discrete integer counting historical chargebacks or dispute claims on the account.
  • cvv_retry_count: Discrete integer counting failed CVV entry attempts during checkout.
  • velocity_score: Continuous score tracking the transaction frequency/volume in a short time window.
  • time_of_day_hour: Discrete integer (\(0\) to \(23\)) representing the hour of the day of the transaction.

Quadratic Terms (Model 3)

  • I(cvv_retry_count^2), I(velocity_score^2), I(time_of_day_hour^2),
    I(merchant_risk_score^2): Squared terms included to model non-linear relationships between these features and the probability of fraud.

Model 1

Residuals:
     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
-0.067950 -0.024608 -0.015076  0.000000 -0.006766  1.010669 

Coefficients:
                       Estimate  Std. Error t value  Pr(>|t|)    
(Intercept)         -1.3520e-02  2.1866e-03 -6.1833 6.398e-10 ***
merchant_risk_score  8.1471e-04  5.3192e-05 15.3165 < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Model 2

Residuals:
     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
-0.202172 -0.037382 -0.008175  0.000000  0.014251  1.008887 

Coefficients:
                                Estimate  Std. Error t value  Pr(>|t|)    
(Intercept)                  -4.9249e-02  7.2146e-03 -6.8263 8.964e-12 ***
amount_usd                    7.9846e-06  3.3956e-06  2.3514  0.018710 *  
auth_methodBiometric         -6.6623e-03  2.7547e-03 -2.4186  0.015591 *  
auth_methodNo Authentication  3.0212e-02  3.0057e-03 10.0516 < 2.2e-16 ***
auth_methodOTP                9.7024e-04  2.4212e-03  0.4007  0.688619    
auth_methodPIN               -3.4754e-03  2.5150e-03 -1.3819  0.167023    
is_foreign_transaction        2.8670e-02  3.7424e-03  7.6608 1.932e-14 ***
card_age_months              -3.0158e-04  1.2831e-04 -2.3504  0.018761 *  
customer_age                  1.3877e-04  4.6948e-05  2.9558  0.003122 ** 
is_new_merchant               2.2778e-02  2.0638e-03 11.0368 < 2.2e-16 ***
used_vpn                      2.9266e-02  3.0656e-03  9.5464 < 2.2e-16 ***
ip_country_mismatch           5.8281e-02  3.8513e-03 15.1330 < 2.2e-16 ***
billing_shipping_mismatch     6.3315e-02  4.2509e-03 14.8945 < 2.2e-16 ***
is_ai_generated_scam_attempt  7.5437e-02  6.4036e-03 11.7805 < 2.2e-16 ***
merchant_risk_score           7.9009e-04  5.1137e-05 15.4505 < 2.2e-16 ***
prior_disputes                6.6328e-03  1.6405e-03  4.0431 5.295e-05 ***
cvv_retry_count               4.7155e-02  2.0559e-03 22.9364 < 2.2e-16 ***
velocity_score                9.7720e-04  7.0218e-05 13.9167 < 2.2e-16 ***
time_of_day_hour             -5.3475e-04  1.2540e-04 -4.2644 2.014e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Model 3

Residuals:
     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
-0.310386 -0.036212 -0.006901  0.000000  0.015457  1.011379 

Coefficients:
                                Estimate  Std. Error t value  Pr(>|t|)    
(Intercept)                  -1.5654e-02  8.1321e-03 -1.9250 0.0542414 .  
amount_usd                    7.3006e-06  3.3867e-06  2.1557 0.0311202 *  
auth_methodBiometric         -6.6470e-03  2.7461e-03 -2.4205 0.0155069 *  
auth_methodNo Authentication  3.0403e-02  2.9965e-03 10.1459 < 2.2e-16 ***
auth_methodOTP                9.7172e-04  2.4138e-03  0.4026 0.6872720    
auth_methodPIN               -3.7623e-03  2.5074e-03 -1.5005 0.1335027    
is_foreign_transaction        2.8292e-02  3.7309e-03  7.5831 3.522e-14 ***
card_age_months              -3.0199e-04  1.2791e-04 -2.3610 0.0182363 *  
customer_age                  1.4201e-04  4.6807e-05  3.0339 0.0024174 ** 
is_new_merchant               2.2718e-02  2.0578e-03 11.0400 < 2.2e-16 ***
used_vpn                      2.9459e-02  3.0564e-03  9.6385 < 2.2e-16 ***
ip_country_mismatch           5.8361e-02  3.8396e-03 15.1999 < 2.2e-16 ***
billing_shipping_mismatch     6.3191e-02  4.2381e-03 14.9103 < 2.2e-16 ***
is_ai_generated_scam_attempt  7.5747e-02  6.3843e-03 11.8645 < 2.2e-16 ***
merchant_risk_score          -4.0855e-04  1.8050e-04 -2.2635 0.0236173 *  
prior_disputes                6.7742e-03  1.6356e-03  4.1419 3.459e-05 ***
cvv_retry_count               1.3453e-02  5.2221e-03  2.5761 0.0099992 ** 
I(cvv_retry_count^2)          2.5404e-02  3.6109e-03  7.0355 2.050e-12 ***
velocity_score               -7.9695e-06  2.0358e-04 -0.0391 0.9687744    
I(velocity_score^2)           2.1362e-05  4.1295e-06  5.1730 2.325e-07 ***
time_of_day_hour             -1.6554e-03  4.8247e-04 -3.4312 0.0006022 ***
I(time_of_day_hour^2)         4.8641e-05  2.0266e-05  2.4002 0.0163974 *  
I(merchant_risk_score^2)      1.4159e-05  2.0490e-06  6.9103 4.981e-12 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Regression Table

Prediction

This confusion matrix evaluates model performance at a lowered decision threshold of 0.017, chosen specifically to capture rare fraudulent transactions. By comparing actual versus predicted values, we can see how false positives and negatives continue to occur even with the ideal threshold.

Key Limitations

  • No Long-Term Customer Historical Baseline:

We only had snapshots of individual transactions, without past spending history for each individual user.

Impact on model: The model had to guess risk based on population averages rather than comparing a purchase against a user’s unique baseline habits.

  • No Exact Time-Series / Timestamp Sequence:

What was missing: Real-world fraud detection relies on exact timestamp sequencing (e.g., transaction 1 at 2:00 PM in California, transaction 2 at 2:05 PM in London)

Impact on model: Without exact timestamps, you couldn’t calculate physical impossibility or sudden real-time spikes across minutes/hours.

Recommendations

1. Low Threshold (\(\text{Threshold} < 0.0100\)) — Step-Up Authentication (2FA Trigger)

  • Objective: Catch early-stage fraud and suspicious account behaviors before blocking legitimate customers.
  • Actionable Plan: * Automatically trigger two-factor authentication (2FA) by sending a text message verification code whenever a transaction hits this baseline risk level.
    • Require 2FA specifically when specific risk triggers are met: entering an incorrect CVV multiple times, exhibiting high transaction velocity (making purchases too fast), or when the transaction score exceeds \(0.0100\).
  • Impact: Prevents premature declines while securing moderate-risk transactions with minimal friction.

2. Baseline Threshold (\(\text{Threshold} \approx 0.0170\)) — Smart Risk Scoring & Real-Time Limits

  • Objective: Balance risk prevention with operational overhead at the dataset baseline fraud rate (~1.7%).
  • Actionable Plan: * Leverage the Smart Risk Score to dynamically restrict purchasing power.
    • Instead of a simple pass/fail grade, automatically lower temporary spending limits or restrict high-value categories for accounts crossing this threshold until identity is re-verified.
  • Impact: Protects the platform against significant monetary exposure while maintaining normal checkout flow for standard transactions.

3. High Threshold (\(\text{Threshold} \ge 0.0300\)) — Targeted Operator Verification (Saving Time & Money)

  • Objective: Maximize investigator efficiency by reserving costly manual reviews exclusively for severe threats.
  • Actionable Plan: * Route only the absolute highest-risk cases directly to human fraud investigators for manual review and phone call verification with the operator.
    • Lower-risk and moderate-risk flagged transactions are handled automatically via automated 2FA triggers and spending limits.
  • Impact: Dramatically slashes operational costs and saves investigator time by eliminating manual reviews on borderline cases, ensuring human resources are focused strictly on high-confidence fraud attempts.

Key Takeaway: Interpreting the Adjusted \(R^2\) (10.2%)

  • What It Means: An Adjusted \(R^2\) of 10.2% (0.102) indicates that our full regression specification (reg3) explains over 10.2% of the overall variance in transaction fraud status (is_fraud).
  • Why It Matters: In Linear Regression Models applied to rare event detection (~1.7% base fraud rate), total variance is limited. An Adjusted \(R^2\) exceeding 10.2% represents a strong, highly predictive model.

Improvements

  • Upgrade the Model Architecture: Linear regression is mathematically designed for continuous numbers, not true/false fraud targets.

  • We should upgrade to Logistic Regression or Tree-Based Models, which are specifically built to handle binary outcomes and improve classification accuracy.

AI Usage

Data Cleaning

  • We utilized AI in order to guide us in the data cleaning process and storing our data correctly.

Formatting

  • Additionally, we used AI to help us with formatting and making the graphs more presentable and professional.

  • Overall, we only utilized AI to debug and troubleshoot issues with our coding.

  • We also used AI to help create the prediction section with the thresholds and confusion matrix

  • Grammar and wording fixes

Conclusion: Credit Card Fraud Detection Analysis

The Core Data Challenge:

  • Real fraud is very rare (only ~1.7% of all purchases), so standard 50/50 pass-or-fail cutoff rules will miss almost all fraudulent transactions.

  • Individual warning signs don’t happen often, but when they do, they provide strong alerts that a purchase is suspicious.

Top 3 Fraud Drivers:

  • AI-Generated Scams: Synthetic scam attempts are the biggest warning sign, increasing fraud risk by 7.6%.

  • Billing & Shipping Mismatch: Shipping an order to a different address than the credit card billing location adds 6.3% to fraud risk.

  • Location Mismatches: Ordering from an IP address in a different country than the card issuer adds 5.8% to fraud risk.

Future Modeling Recommendations:

  • Standard linear regression is designed for numeric averages rather than true/false targets, leading to a baseline score (\(R^2 = 10.2\%\)).

  • We should upgrade to specialized classification models such as Logistic Regression or Random Forest, which excel at spotting non-linear fraud patterns.