Overview of the Data

In the Loan Policy Proposal project, a team and I developed a solution for a financial services company to maximize their profit on loan repayments and reduce their financial risk using logistic regression and profit analysis. We first selected relevant features and created a logistic regression model, which used borrowers’ financial history to predict loan repayments. We then conducted a profit analysis to find the optimal threshold value that maximizes the expected profit per customer. From this, we created a recommendation for the client that improves their current expected profit.

After cleaning the data, we split it into a training set (70%) and a test set (30%) and examined the class distribution. Approximately 80% of the entries had a value of 0 for NotFullyPaid, indicating that the loans were fully repaid. The remaining 20% had a value of 1, indicating the loans were not fully repaid (i.e., defaulted).

Train set:

## 
##    0    1 
## 5632 1073

Test set:

## 
##    0    1 
## 2413  460

Training Set Model Evaluation

Next, we trained and evaluated the model on the training set. Several features were statistically significant predictors of loan repayment, including the purpose of the loan. For example, taking out a loan to repay credit card debt or start a small business were particularly relevant to the model. Other significant factors included the installment amount, annual income, FICO score, and number of recent credit inquiries. These results are consistent with real-world patterns, as debt tends to be cyclical, with individuals borrowing more to manage existing debt, which can increase the risk of non-repayment.

## 
## Call:
## glm(formula = NotFullyPaid ~ ., family = binomial, data = LoansTrain)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.8239  -0.6178  -0.4991  -0.3738   2.5686  
## 
## Coefficients:
##                             Estimate Std. Error z value Pr(>|z|)    
## (Intercept)                8.044e+00  1.534e+00   5.245 1.56e-07 ***
## CreditPolicy              -3.232e-01  9.996e-02  -3.233 0.001225 ** 
## Purposecredit_card        -5.215e-01  1.307e-01  -3.990 6.61e-05 ***
## Purposedebt_consolidation -3.000e-01  9.253e-02  -3.242 0.001185 ** 
## Purposeeducational        -4.242e-02  1.858e-01  -0.228 0.819408    
## Purposehome_improvement    6.744e-02  1.512e-01   0.446 0.655593    
## Purposemajor_purchase     -2.101e-01  1.902e-01  -1.104 0.269467    
## Purposesmall_business      5.253e-01  1.401e-01   3.749 0.000178 ***
## IntRate                    2.941e+00  2.064e+00   1.425 0.154185    
## Installment                1.119e-03  2.077e-04   5.387 7.16e-08 ***
## LogAnnualInc              -3.846e-01  7.118e-02  -5.404 6.53e-08 ***
## Dti                       -5.549e-03  5.506e-03  -1.008 0.313616    
## Fico                      -8.658e-03  1.683e-03  -5.144 2.69e-07 ***
## DaysWithCrLine             1.114e-05  1.593e-05   0.699 0.484274    
## RevolBal                   3.694e-06  1.161e-06   3.181 0.001468 ** 
## RevolUtil                  9.754e-04  1.531e-03   0.637 0.524083    
## InqLast6mths               8.440e-02  1.611e-02   5.240 1.61e-07 ***
## Delinq2yrs                -6.830e-02  6.616e-02  -1.032 0.301920    
## PubRec                     2.239e-01  1.204e-01   1.859 0.062969 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 5896.6  on 6704  degrees of freedom
## Residual deviance: 5517.7  on 6686  degrees of freedom
## AIC: 5555.7
## 
## Number of Fisher Scoring iterations: 5

Examining the confusion matrix for the training set, the columns represent predicted values, while the rows represent actual values. Most of the loans that were fully repaid (the majority class) were correctly predicted as such. However, many loans that were not fully repaid were incorrectly predicted as repaid, indicating a high false negative rate. Only 33 instances were correctly predicted as not fully paid, while 29 were falsely flagged as such. This suggests that the logistic regression model tends to over-predict the majority class, likely due to the significant class imbalance in the dataset.

##    
##     FALSE TRUE
##   0  5603   29
##   1  1040   33

Optimizing the Decision Threshold

Next, we tuned the decision threshold, which is the probability cutoff that the model uses to assign a prediction of the positive class (i.e., NotFullyPaid = 1). Logistic regression, by default, assigns 1 to all probabilities greater than 0.5. Here, we outputted the confusion matrix for all thresholds from 0.1 to 0.9 (using 0.10 increments and excluding the 0.5 default) to observe which improves the model the most. Because our goal as a lender is to identify risk, we prioritized recall—capturing as many true positives as possible—even if it meant sacrificing some precision or overall accuracy.

## [1] "threshold=0.1"
##    
##     FALSE TRUE
##   0  1685 3947
##   1   125  948
## [1] "threshold=0.2"
##    
##     FALSE TRUE
##   0  4454 1178
##   1   611  462
## [1] "threshold=0.3"
##    
##     FALSE TRUE
##   0  5324  308
##   1   878  195
## [1] "threshold=0.4"
##    
##     FALSE TRUE
##   0  5543   89
##   1   992   81
## [1] "threshold=0.6"
##    
##     FALSE TRUE
##   0  5624    8
##   1  1064    9
## [1] "threshold=0.7"
##    
##     FALSE TRUE
##   0  5629    3
##   1  1071    2
## [1] "threshold=0.8"
##    
##     FALSE TRUE
##   0  5631    1
##   1  1071    2
## [1] "threshold=0.9"
##    
##     FALSE
##   0  5632
##   1  1073

Based on the output, it appears that the threshold of 0.1 and 0.2 yielded the highest sensitivity. We then explored values ranging from 0.1 to 0.2 (using 0.01 increments) to identify a more precise threshold.

## [1] "threshold=0.12"
##    
##     FALSE TRUE
##   0  2385 3247
##   1   219  854
## [1] "threshold=0.13"
##    
##     FALSE TRUE
##   0  2726 2906
##   1   278  795
## [1] "threshold=0.14"
##    
##     FALSE TRUE
##   0  3062 2570
##   1   335  738
## [1] "threshold=0.15"
##    
##     FALSE TRUE
##   0  3380 2252
##   1   384  689
## [1] "threshold=0.16"
##    
##     FALSE TRUE
##   0  3636 1996
##   1   420  653
## [1] "threshold=0.17"
##    
##     FALSE TRUE
##   0  3874 1758
##   1   476  597
## [1] "threshold=0.18"
##    
##     FALSE TRUE
##   0  4107 1525
##   1   532  541
## [1] "threshold=0.19"
##    
##     FALSE TRUE
##   0  4297 1335
##   1   578  495
## [1] "threshold=0.21"
##    
##     FALSE TRUE
##   0  4598 1034
##   1   638  435
## [1] "threshold=0.22"
##    
##     FALSE TRUE
##   0  4729  903
##   1   671  402
## [1] "threshold=0.23"
##    
##     FALSE TRUE
##   0  4845  787
##   1   711  362

From the values above, the threshold of 0.16 achieves a high true positive rate of 0.56 while still keeping precision reasonable at 0.25. We believe this threshold strikes a good balance between precision and recall, staying fairly conservative in catching risk without rejecting too many good borrowers. We applied this threshold to the test set and plotted the Receiver Operating Characteristic (ROC) Curve.

Based on the ROC curve, we observe that lowering the decision threshold increases the true positive rate, meaning the model identifies more actual loan defaults. However, setting the threshold too low, such as 0, results in the model predicting nearly all borrowers as high risk, which leads to a high false positive rate and fails to distinguish between risky and reliable borrowers. A threshold of 0.16 offers a better balance on the test set, capturing more defaults while still allowing for the approval of creditworthy borrowers. Next, we plotted the Area Under the Curve (AUC).

## [1] 0.7

The AUC of 0.7 indicates moderate performance, meaning the model has a 70% chance of correctly ranking a defaulting borrower as riskier than a non-defaulting one. While the model is generally effective at identifying risk, performance may improve by trying alternative models (such as decision forests) or approaches (class balancing, normalization).

Baseline Performance Metrics

To evaluate the effectiveness of our lending decisions using the baseline model, we calculated the total amount lent, profit, loss, and net profit per loan. These will also serve as performance metrics for comparison when we adjust the decision threshold to improve model performance. The baseline model assumes that the company lends to all borrowers, earning a 1% profit on loans that are fully repaid and incurring a 5% loss on those that default. We applied this to the test set to compare how the models perform on unseen data. We can see that the baseline yields a net profit per loan of $3.93.

Total Amount Lent ($):

## [1] "28,730,000"

Profit on Loans ($):

## [1] "241,300"

Loss ($):

## [1] "-230,000"

Net Profit per Loan ($):

## [1] 3.93

Final Performance Metrics

Finally, we assess the performance of the improved model, which lends only to borrowers predicted to repay their loans in order to minimize risk. Unlike the baseline strategy, which lent to all 2,873 borrowers, this model-driven approach approves just 1,797 individuals classified as “False” (i.e., low risk). Although fewer loans are issued, the strategy significantly reduces losses from high-risk borrowers. As a result, the final net profit per customer rises to $37.90—a substantial improvement over the baseline.

## [1] "threshold=0.16"
##    
##     FALSE TRUE
##   0  1611  802
##   1   186  274

Total Amount Lent ($):

## [1] "17,970,000"

Profit on Loans ($):

## [1] "161,100"

Loss ($):

## [1] "-93,000"

Net Profit per Loan ($):

## [1] 37.9

Conclusion

The results show that applying a predictive “Loan or No Loan” policy yields a net profit of $37.90 per customer, compared to just $3.91 under the baseline strategy of lending to all borrowers. This represents nearly a tenfold increase in profitability. These findings highlight the power of predictive modeling in guiding lending decisions—by tuning the model threshold and leveraging data, businesses can significantly reduce risk while maximizing returns.

Additionally, if the company prefers to continue lending to all customers, a hybrid approach could be adopted: high-risk borrowers could receive smaller loans (e.g., $5,000 instead of $10,000). This compromise still improves profitability while maintaining broader access to credit.