The objective of this assignment is to use the penguin predictions dataset to learn how different probability thresholds change a model’s behavior and performance metrics.
I plan on viewing the data in the RStudio interface. As suggested, I’ll use the ggplot2 package to display the actual class distribution of the sexes. I’ll calcuate the null error rate as our baseline metrics and share its importance to giving us a starting point in our re-calculations. Secondly, I’ll use the different thresholds (0.2, 0.5, 0.8) to calculate three separate confusion matrices. I believe that might be the most challenging part of this assignment, but the overview in class made this less scary to tackle. With the results of the confusion matrices, I can calculate accuracy, precision, recall, and F1 score, then present their results in a clear table. At the end, I can see how the different thresholds perform, and reflect on when to a 0.2 threshold and a 0.8 threshold are preferable, because their use cases can vary.