• A p-value is a statistical measure that helps us understand the significance of our results.
• It tells us how likely it is to observe our data, or more extreme, if the null hypothesis is true.

• The null hypothesis (\(H_0\)) is a default assumption that there is no effect or difference.
• Assumes there’s no relationship between the features (independent variables) and the target (dependent variable).
• P-values are calculated under the assumption that \(H_0\) is true.

• A low p-value (\(< 0.05\)) suggests that the observed data is unlikely under \(H_0\), providing evidence against \(H_0\).
• A high p-value (\(\geq 0.05\)) suggests that the observed data is consistent with \(H_0\).

• A p-value does not tell us the probability that \(H_0\) is true.
• A p-value does not measure the effect size or importance of the results.

The p-value is calculated based on the test statistic, which follows a specific distribution under the null hypothesis \(H_0\). For a given test statistic \(T\), the p-value is defined as:

• For a two-tailed test: \(p = P(T \leq t | H_0) + P(T \geq t | H_0)\)
• For a one-tailed test: \(p = P(T \leq t | H_0)\) or \(p = P(T \geq t | H_0)\)

where \(t\) is the observed value of the test statistic.

ggplot(data, aes(x=value, fill=group)) +
  geom_histogram(position="identity", alpha=0.5, bins=30) +
  theme_minimal() +
  labs(title="Overlap in Data Distributions", x="Value", y="Count")

The significance level, denoted as \(\alpha\), is a threshold used to decide whether the p-value indicates a statistically significant result. Commonly, \(\alpha = 0.05\).

• If \(p \leq \alpha\), we reject the null hypothesis \(H_0\), suggesting the observed data is unlikely under \(H_0\).
• If \(p > \alpha\), we fail to reject \(H_0\), indicating insufficient evidence to conclude a significant effect or difference.

This decision rule is foundational in hypothesis testing, guiding us in making conclusions based on our data and statistical analysis.

• Interpreting P-Values: A low p-value (\(< 0.05\)) indicates data unlikely under \(H_0\), suggesting evidence against \(H_0\).
• Misconceptions: A p-value does not indicate the probability that \(H_0\) is true nor the magnitude of an effect.