Use it when the outcome has two possible values: default or no default, pass or fail, disease or no disease.

• We want a probability between 0 and 1, not an unrestricted numerical prediction.
• Logistic regression connects predictors to that probability through log odds.
• A probability can later become a classification by choosing a threshold.

By the end: fit a logistic model in R, interpret its coefficient, and explain why a predicted probability is different from a yes-or-no decision.

ISLR2’s Default contains 10,000 simulated observations: default, balance, income, and student status. Only 333 (3.33%) defaulted; there are no missing values. We will use balance to keep the method easy to follow.

For a default probability \(p\), define:

\[ \mathrm{odds}=\frac{p}{1-p}, \qquad \mathrm{logit}(p)=\log\left(\frac{p}{1-p}\right). \]

At \(p=0.20\), the odds are \(0.20/0.80=0.25\): one default for every four non-defaults in the probability model.

Logistic regression makes the log odds linear in the predictors:

\[ \mathrm{logit}(p)=\beta_0+\beta_1 B, \qquad B=\frac{\mathrm{balance}}{500}. \]

One unit of \(B\) means an extra $500 of balance.

The inverse-logit function converts the linear predictor back to a probability:

\[ \eta=\beta_0+\beta_1B, \qquad p=\frac{1}{1+e^{-\eta}}. \]

This S-shaped function keeps predictions between 0 and 1. R estimates the coefficients by maximizing the Bernoulli log likelihood:

\[ \ell(\beta)=\sum_i\left[y_i\log(p_i)+(1-y_i)\log(1-p_i)\right]. \]

Here, \(y_i=1\) for a default and \(y_i=0\) otherwise. Maximizing this quantity rewards probabilities that agree with the observed outcomes.

We hold out 3,000 observations for checking predictions and fit on the other 7,000. The split is stratified by default status and fixed by a seed.

fit <- glm(y ~ balance500, data = train, family = binomial())
coef(fit)

## (Intercept)  balance500 
##  -10.435607    2.690351

test_probability <- predict(fit, newdata = test, type = "response")

• glm() fits a generalized linear model.
• binomial() selects a binary-outcome model with the default logit link.
• type = "response" returns probabilities; the default prediction scale is log odds.

The fitted slope is 2.690. Each extra $500 multiplies the estimated odds of default by \(e^{2.690}\), about 14.74. It does not multiply the probability by that amount.

At balances of $1,000, $1,500, and $2,000, predicted default probabilities are 0.6%, 8.6%, and 58.1%.

Hover to compare balances. Drag to zoom and double-click to reset. Notice that the same balance increase has a different probability effect depending on where it starts on the curve.

Choose a threshold \(t\): predict default when \(\hat p\geq t\). A threshold is a decision choice, separate from estimating the model.

• Recall: the share of actual defaults that we identify.
• Precision: the share of predicted defaults that actually defaulted.
• Lowering the threshold catches more defaults and creates more false alarms.

The ROC curve compares thresholds. AUC = 0.960: the model ranks a randomly chosen default above a randomly chosen non-default about 96.0% of the time.

Accuracy alone can mislead: predicting no default for everyone already gives 96.7% accuracy here, while finding none of the defaults.

• Logistic regression models log odds and converts them to probabilities in \([0,1]\).
• Exponentiating a coefficient gives an odds ratio, not a probability ratio.
• glm(..., family = binomial()) fits the model; predict(..., type = "response") gives probabilities.
• Check predictions on held-out data and choose thresholds based on the consequences of errors.

In our example, higher balances predict greater default risk. That is an association in simulated data, not proof of a causal effect or a ready-to-use credit policy.