One-Sample t-Test

A company that manufactures light bulbs claims that a particular type of light bulb will last 850 hours on average with standard deviation of 50. A consumer protection group thinks that the manufacturer has overestimated the lifespan of their light bulbs by about 40 hours. How many light bulbs does the consumer protection group have to test in order to prove their point with reasonable confidence?


     One-sample t test power calculation 

              n = 18.44624
              d = 0.8
      sig.level = 0.05
          power = 0.9
    alternative = two.sided

Null hypothesis: H0 = 850. Alternative hypothesis: Ha= 810. The significance level is the probability of a Type I error, that is, the probability of rejecting H0 when it is actually true. Default: 0.05 level. The power of the test against Ha is the probability of that the test rejects H0. Here: 0.90 level. The result tells us that we need a sample size at least 19 light bulbs to reject H0 under the alternative hypothesis Ha to have a power of 0.9. What then is the power for sample size of 15? We can see that the power is about 0.821 for a sample size of 15.


     One-sample t test power calculation 

              n = 15
              d = 0.8
      sig.level = 0.05
          power = 0.8213105
    alternative = two.sided

Paired Sample t-Test

A company markets an eight-week long weight loss program and claims that at the end of the program on average a participant will have lost 5 pounds. On the other hand, you have studied the program and you believe that their program is scientifically unsound and shouldn’t work at all. With some limited funding at hand, you want test the hypothesis that the weight loss program does not help people lose weight. Your plan is to get a random sample of people and put them on the program. You will measure their weight at the beginning of the program and then measure their weight again at the end of the program. Based on some previous research, you believe that the standard deviation of the weight difference over eight weeks will be 5 pounds. You now want to know how many people you should enroll in the program to test your hypothesis.


     Paired t test power calculation 

              n = 9.93785
              d = 1
      sig.level = 0.05
          power = 0.8
    alternative = two.sided

NOTE: n is number of *pairs*

The result tells us that we should enroll at least 10 people in the program to test our hypothesis. If we wanted a lower alpha at 0.01 level and a high power at 0.90 then we would need 19 subjects:


     Paired t test power calculation 

              n = 18.30346
              d = 1
      sig.level = 0.01
          power = 0.9
    alternative = two.sided

NOTE: n is number of *pairs*

Paired Sample t-Test Plot

Two-Group Independent Sample t-Test

A clinical dietician wants to compare two different diets, A and B, for diabetic patients. She hypothesizes that diet A (Group 1) will be better than diet B (Group 2), in terms of lower blood glucose. She plans to get a random sample of diabetic patients and randomly assign them to one of the two diets. At the end of the experiment, which lasts 6 weeks, a fasting blood glucose test will be conducted on each patient. She also expects that the average difference in blood glucose measure between the two group will be about 10 mg/dl. Furthermore, she also assumes the standard deviation of blood glucose distribution for diet A to be 15 and the standard deviation for diet B to be 17. The dietician wants to know the number of subjects needed in each group assuming equal sized groups.


     Two-sample t test power calculation 

              n = 41.31968
              d = 0.6238303
      sig.level = 0.05
          power = 0.8
    alternative = two.sided

NOTE: n is number in *each* group

The calculation results indicate that the dietician needs 42 subjects for diet A and another 42 subjects for diet B in our sample in order for the effect to be detected 80% of the time. If this study costs 200 USD per subject, we have just determined that it will cost the dietician $ 16,800 to run the study, which may be out of budget.

Two-Group Independent Sample t-Test (cont’d)

Now suppose the dietician can only collect data on 60 subjects with 30 in each group. What will the statistical power for her t-test be with respect to an alpha level of 0.05? We see that the power will be reduced to 66%.


     Two-sample t test power calculation 

              n = 30
              d = 0.6238303
      sig.level = 0.05
          power = 0.6612888
    alternative = two.sided

NOTE: n is number in *each* group

Estimating Effect Size

So we have determined that the experiment won’t work as desired. Suppose that the dietician has enough money to run 30 subjects in each group in this new experiment, but she still wants a power of 80%. What can she do? If she can find a way to cut the variability of her test and increase the effect size from 0.62 to 0.74, she would still be able to find the desired effect with just 30 participants instead of 42 for each group. The study cost would then be reduced from 16,800 to 12,000 USD, thus saving 4,800 USD obtaining the same desired result thanks to power analysis.


     Two-sample t test power calculation 

              n = 30
              d = 0.7356292
      sig.level = 0.05
          power = 0.8
    alternative = two.sided

NOTE: n is number in *each* group

Effect Size Plot

Power vs. sample size plot

To look at how the effect size affects the sample size assuming a given sample power. We can simply assume the difference in means and set the standard deviation to be 1 and create a table with effect size, d, varying from .2 to 1.2.