2025-10-18
Logistic regression is a classification algorithm used to predict the probabilities of classes given an input.
In an example of only 2 classes possible, then it is called binomial logistic regression.
When there are multiple possible classes, it is called multinomial logistic regression.
Logistic Regression uses the logistic function, which produces a ‘S’ s haped curve from 0, to 1. This function is good for classification because the middle point is 0.5, which makes it easy to classify whether a value is above or below that boundary.
logistic_function = function(x){
1 / (1 + exp(-1 * (x - 0)))
}
x = seq(-10, 10, by = 0.3)
y = logistic_function(x)
data = data.frame(x, y)
ggplot(data, aes(x = x, y = y)) +
geom_line(color = 'cornflowerblue', size = 1.5) +
labs(title = "Logistic Function Graph")
This logistic (or sigmoid) function is defined as: \[ f(x) = \frac{L}{1 + e^{-k(x - x_0)}} \] Logistic regression models the probability of some event happening. Therefore, if we had some linear model that we wanted to attach probabilities to, we use log odds:
\[ \text{odds} = \frac{p}{1 - p}, \quad \log(\text{odds}) = \beta_0 + \beta_1 x \] Using this, we can map linear combinations of features (predictors) onto a probability of 0 to 1. If we try solving for ‘p’ for the odds equation, we get a familiar equation! \[ p = \frac{1}{1 + e^{-(\beta_0 + \beta_1 x)}} \]
As an example, how would you create an algorithm to classify whether someone passed their exam? We can do this, by making a model dependent on the number of hours they studied, and whether they passed or failed.
What would a multinomial logistic regression look like then? In order to calculate the probabilities of multiple classes, you would have to calculate the probability of multiple log odds. \[ {\small \log \frac{P(Y=A)}{P(Y=C)} = \beta_{0A} + \beta_{1A} \cdot x, \quad \log \frac{P(Y=B)}{P(Y=C)} = \beta_{0B} + \beta_{1B} \cdot x } \] Solve for each class’ probability: \[ {\small P(Y=A) = \frac{e^{\beta_{0A} + \beta_{1A} x}}{e^{\beta_{0A} + \beta_{1A} x} + e^{\beta_{0B} + \beta_{1B} x} + 1}, \quad P(Y=B) = \frac{e^{\beta_{0B} + \beta_{1B} x}}{e^{\beta_{0A} + \beta_{1A} x} + e^{\beta_{0B} + \beta_{1B} x} + 1}, \newline P(Y=C) = \frac{1}{e^{\beta_{0A} + \beta_{1A} x} + e^{\beta_{0B} + \beta_{1B} x} + 1} } \] Then you softmax the values so they add up to 1 (which gives you a percentage). \[ {\small P(Y=j) = \frac{e^{\beta_{0j} + \beta_{1j} x}}{\sum_{k} e^{\beta_{0k} + \beta_{1k} x}} } \]
Here is an extension of the previous graph.
Here, the graph is split into 3 graphs representing a prediction of a students grade based on the amount of hours they study.
R
Logistic Regression