Logistic Regression

1 Logistic Regression

1.1 Binary Classification

Unlike the models we discussed previously, logistic regression is used for classification tasks. Recall that the goal in classification tasks is to find a function that maps an observation to its associated class or label. A learning algorithm must use pairs of feature vectors and their corresponding labels to induce the values of the mapping function’s parameters that produce the best classifier, as measured by a particular performance metric. In binary classification, the classifier must assign instances to one of the two classes.

1.1.0.1 Examples

Examples of binary classification include predicting whether or not a patient has a particular disease, whether or not an audio sample contains human speech, or whether or not the Duke men’s basketball team will lose in the first round of the NCAA tournament. In multiclass classification, the classifier must assign one of several labels to each instance. In multilabel classification, the classifier must assign a subset of the labels to each instance. In this chapter, we will work through several classification problems using logistic regression, discuss performance measures for the classification task, and apply some of the feature extraction techniques you learned in the previous chapter.

===================================

1.2 Logits

Logits are defined as the ratio of the probability to its complement, or the ratio of favorable to unfavorable cases. If the probability of an event is a half, the odds are one-to-one or even.

\[ odds_i = \frac{\pi_i}{1-\pi_i} \]

calculating the logit or log-odds

\[ Log(odds_i) = log left( \frac{\pi_i}{1-\pi_i} \right) \]

2 Model Performace

Binary classification performance metrics A variety of metrics exist to evaluate the performance of binary classifiers against trusted labels. The most common metrics are accuracy, precision, recall, F1 measure, and ROC AUC score. All of these measures depend on the concepts of true positives, true negatives, false positives, and false negatives. Positive and negative refer to the classes.

True and false denote whether the predicted class is the same as the true class. For our SMS spam classifier, a true positive prediction is when the classifier correctly predicts that a message is spam. A true negative prediction is when the classifier correctly predicts that a message is ham. A prediction that a ham message is spam is a false positive prediction, and a spam message incorrectly classified as ham is a false negative prediction.

3 Summary

3.0.1 Summary

  • Logistic regression, also called a logit model, is used to model dichotomous response variables.
  • In the logit model the log odds of the outcome is modeled as a linear combination of the predictor variables.