2024-10-31

Overview

P-Value is the measure of a obtaining a certain out come when assuming the NULL hypothesis is true.

The NULL hypothesis is the idea that the subject being studied does not exist. An example of this would be hours of sleep does not have an effect on student exam scores.

P-Tests are used in gathering evidence towards a study. Typically it is the first step in getting funding for projects in modern times where that the value is < 0.05.

P-Value Equation

We calculate the p-value based on the test statistic \(T\) and the distributions under the null hypothesis:

\[ p = P(T \geq t | H_0) \]

  • \(p\): Probability of observing a test statistic as extreme as \(t\) under the null hypothesis \(H_0\).

  • \(T\): Test statistic, varying by hypothesis test type.

  • \(H_0\): Null hypothesis.

  • Small p-value: Strong evidence against \(H_0\), suggesting \(H_0\) may be rejected.

  • Large p-value: Weak evidence against \(H_0\), suggesting \(H_0\) may not be rejected.

Slide with Plot 1

Slide with R Output

set.seed(100)
p_values <- runif(1000,0,1)
p_data <- data.frame(p_value = p_values)

# Hide graph using invisible
invisible(ggplot(p_data, aes(x = p_value)) +
  geom_histogram(binwidth = 0.05, fill = "lightblue", color = "black") +
  geom_vline(xintercept = 0.05, color = "red", linetype = "dashed") +
  labs(x = "p_value", y = "Frequency", title = "Distribution of p-values") +
  theme_minimal())

Slide with plot 2

P-Hacking

There exists unethical studies in P-Value tests. P-Hacking is the changing of data so that a P-Value of < 0.05 can be found. This mostly occurs when people are trying to receive money for a study but can’t get a P-Value of less than 0.05. P-Hacking can consist of many different things such as data trimming or confidence fluctuation, but ultimately leads to nothing as the Null hypothesis could not be rejected in the first place.

Slide with plotly

P-Hacking Equation Modifications

\[ p = \min \left( P(T \geq t | H_0) \right) \]

  • Testing multiple hypotheses: Trying out different hypotheses on the same dataset.
  • Selective reporting: Only reporting outcomes that show significance.
  • Adjusting variables: Modifying variables or the dataset to reach \(p < 0.05\).

Risks of p-hacking

P-hacking increases the risk of Type I errors (false positives), as the probability of observing a significant result due purely to chance rises with each additional test or adjustment.

Conclusion

In conclusion P-Tests are still a good way of determining if a study should be proceeded with.

Despite some unethical people using P-Hacking to falsify studies, this is considered a small possibility in studies when other tests modifications can be done.

P-Tests still stand as one of the best preliminary measurements of a particular case if all variables can be considered.

Soruces