The P value is the probability of obtaining the outputted results, and results more significant than it. A metric for determining the statistical significance of the test results.

Other important metrics to include are the alpha value, a threshold for determining significance, the standard deviation of the test results, how far the average point is away from the mean, and a z-value, how many standard deviations a point is from the mean.

For a test hypothesis to be satisfied, the null hypothesis rejected, the results must be deemed statistically significant. A p-value is usually found by parsing through a z-table, finding associated z-values with their p-value counterparts.

x = data point, xbar = sample mean, n = sample size, sigma = population sd, mu = population mean

Standard Deviation of a test:

\[ {s} = \sqrt{\frac{1}{n-1} \sum_{i=1}^{n} (x - \bar{x})^2} \]

z-score of a test:

\[ z = \frac{\bar{x} - \mu}{\sigma / \sqrt{n}} \]

z = test statistic

Z = z-value of the critical region

P(Z > |z|) = The area under the curve to the left or to the right of the z-value, depending on which tail the test is being done.

p-value of a two-tailed test:

\[ p = 2 \times P(Z > |z|) \]

If p is less than alpha value (standardly 0.05), that means the test results are significant. The p-value in a 2 tailed test is multiplied by 2 because it finds the area under the region both above and below the +/- z values.

The figure is on the next slide:

Green Lines = One standard deviation away from the mean.

Red Lines = Critical regions for this test. For standard alpha value, 0.05, or 5%, z-value is +/- 1.96 (split between both tails for a two tailed significance test).

The z-value, associated with the P-value of the test results, in order for the test to be statistically significant, must fall in the critical region between the lower red line and the min or the higher red line and the max.

Given (blue line) test statistic (z-value), the test is statistically significant because the statistic falls in the critical region (beyond the red lines), which corresponds to a P-value less than a = 0.05.

Here is the R Code that Created the Figure

library(ggplot2)
mean_val <- 0
sd_val <- 1
x_vals <- seq(-4, 4, 0.01)
test_statistic = 2.5
critical_z <- qnorm(1 - 0.05/2)
df <- data.frame(x = x_vals,
                 y = dnorm(x_vals, mean = mean_val, sd = sd_val))
ggplot(df, aes(x, y)) +
  labs(x = "Possible Test Results", y = "Probability Density") +
  geom_line(linewidth = .5) +
  geom_vline(xintercept = c(-critical_z, critical_z), col = "red") +
  geom_vline(xintercept = c(mean_val - sd_val, mean_val + sd_val),
             col = "green") +
  geom_vline(xintercept = 2.5, col = "blue")

Cirtical Regions of the Normal Distribution

The more opaque regions of the graph are the critical regions, where the test results would be statistically significant.

Plot is on the next slide:

A t-test a little different than a z-test because it’s a significance test that determines if there is a difference between the means of two groups. This context introduces degrees of freedom, the number of values that are free to vary. In a t-test, this is usually df = n-1, because the mean needs to stay the same. This graph demonstrates how degrees of freedom change the distribution.

Generally, the higher the degrees of freedom, the closer the graph gets to the normal distribution.

x_values <- seq(-4, 4, length.out = 100)
values_of_df <- c(1,5,20,60)
y1 <- dt(x, df = values_of_df[1])
y2 <- dt(x, df = values_of_df[2])
y3 <- dt(x, df = values_of_df[3])
y4 <- dt(x, df = values_of_df[4])
ggplot() +
  geom_line(aes(x = x, y = y1, color = "1"), 
            linewidth = 1) +
  geom_line(aes(x = x, y = y2, color = "5"), 
            linewidth = 1) +
  geom_line(aes(x = x, y = y3, color = "20"),
            linewidth = 1) +
  geom_line(aes(x = x, y = y4, color = "60"),
            linewidth = 1) +
  labs(title = "T-Distributions with Varying Degrees of Freedom",
       x = "t-value",
       y = "Probability Density",
       colour = "Degrees of Freedom")