Power calculation

Dejan Pljevljakusic
Sunday, October 09, 2016

Overview

This application is shiny-processed version of manipulate example, which investigates power as the various inputs changes, from Brian Caffo's book Statistical inference for data science (pp. 94). This shiny application is result of a project assignment of Coursera Developing Data Products course.

Example

A respiratory disturbance index (RDI) of more than 30 events/hour, say, is considered evidence of severe sleep disordered breathing. Suppose that in a sample of 100 overweight subjects with other risk factors for sleep disordered breathing at a sleep clinic, the mean RDI was 32 events/hour with a standard deviation of 10 events/hour.

We might want to test the hypothesis that

\( H_0 \): \( \mu \) = \( \mu_0 \)

versus the hypothesis

\( H_a \): \( \mu \) > \( \mu_0 \)

where \( \mu_0 \) is the population mean RDI. Clearly, somehow we must figure out a way to decide between these hypotheses using the observed data, particularly the sample mean.

Normally, we will perform hypothesis testing by forcing the probability of a Type I error (\( \alpha \)) to be small.

Power

Power is the probability of rejecting the null hypothesis when it is false. Ergo, power is a good thing. A type II error is failing to reject the null hypothesis when it's false; the probability of a type II error is usually called \( \beta \). Note Power = 1 - \( \beta \).

alt text

Let's go through an example of calculating power. Consider example involving RDI. Let's plug in the specific numbers for the example where: \( \mu_a \) = 32, \( \mu_0 \) = 30, n = 16, \( \sigma \) = 4.

\( P\left(\frac{\bar{X} - 30}{s/\sqrt{n}} > t_{1-\alpha,1-n};\mu = \mu_a \right) \)

mu0 <-  30
mua <-  32
sigma <-  4
n <-  16
alpha <-  0.05
z = qnorm(1 - alpha)

pnorm(mu0 + z * sigma/sqrt(n), mean = mua, sd = sigma/sqrt(n), lower.tail = FALSE)

[1] 0.63876

If we try to present this situation by investigating Gaussian curves of the population and sample we will get some kind of graph like this shown below.

plot of chunk figure

Instructions

This shiny application gives you an opportunity to play with all variables included in power calculation, except with population mean, which has been fixed to 30. At the start, default parameters are set where: \( \mu_0 \) = 30, \( \mu_a \) = 32, n = 16, \( \sigma \) = 4.

Critical value \( \alpha \) is set on 5%, so that if we get a sample mean that's larger than a specific threshold we can reject the null hypothesis. That's the black line. We set this line such that the probability, if the red density is true, the null hypothesis is true this area, the probability of getting statistics larger than it is 5%.

This is how the application looks like

alt text

Now power is nothing other than the probability of getting larger than this line which is calibrated to have this area under the red curve is 5%. The probability that we reject if in fact the blue curve is true. That's the power!

Below the graph, there is calculated power value. You can see that if you move means further away from each other that power is getting larger. In the same manner, if you shrink curve by decreasing variance, power is increasing. Furthermore, power is getting larger if you extent your sample size or if you decrease critical value for rejecting null hypothesis. We can assume power of 0.8 as optimal for making valid statistical conclusion.

alt text