For this assignment, I plan to use the penguin predictions dataset to learn how the probability threshold of a binary classification model affects its predictions and performance. I will first examine the actual class distribution and calculate the null error rate to understand the baseline performance of the model.
I will then compare probability thresholds of 0.2, 0.5, and 0.8. For each threshold, I plan to create a confusion matrix containing true positives, false positives, true negatives, and false negatives. I will use these results to calculate accuracy, precision, recall, and F1 score.
Finally, I will compare the results across the three thresholds to understand the tradeoff between predicting more or fewer positive cases. Some challenges I expect are understanding the confusion matrix values, calculating the performance metrics correctly, and explaining why a lower or higher threshold may be better depending on the real-world situation.