Assignment 9: Hypothesis Testing and Power and Sample size Determination

Assignment Objectives

• Enhance understanding the logic and procedure of hypothesis testing .

• Implement the procedures for power and sample size calculation for basic hypothesis testing procedures using nuilt-in function and manual calculation.

Policies of Using AI Tools

Policy on AI Tool Use: Please adhere to the AI tool policy specified in the course syllabus. The direct copying of AI-generated content is strictly prohibited. All submitted work must reflect your own understanding; where external tools are consulted, content must be thoroughly rephrased and synthesized in your own words.

Code Inclusion Requirement: Any code included in your essay must be properly commented to explain the purpose and/or expected output of key code lines. Submitting AI-generated code without meaningful, student-added comments will not be accepted.

**Simple versus Composite Hypothesis*

Simple Hypothesis

• Simple Hypothesis Test is a hypothesis that completely specifies the population distribution. Mathematically, simple hypothesis fixes all parameters to specific values. For example, The simple hypotheses are: \(H_0: \mu = 5\) (if \(\alpha\) is known), \(H_0: \mu = 100, \sigma^2 = 25\), and \(H_1: p = 0.7\) for a Bernoulli distribution.

• Example Scenario Test if a coin is fair:

\[ \begin{aligned} H_0&: p = 0.5 \quad \text{(completely specified)} \\ H_1&: p = 0.6 \quad \text{(also completely specified)} \end{aligned} \]

Both are simple hypotheses.

Composite Hypothesis

• Composite Hypothesis is a hypothesis that does not completely specify the distribution. Mathematically, it allows a range of values for at least one parameter. For example, in one-sided: \(\mu >5\); in two-sided: \(\mu \le 5\).

• Example Scenarios

$$ \[\begin{aligned} &H_0: \mu = 5 && \text{(simple)} \\ &H_1: \mu > 5 && \text{(composite)} \end{aligned}\] \[\begin{aligned} &H_0: \mu \leq 5 && \text{(composite)} \\ &H_1: \mu > 5 && \text{(composite)} \end{aligned}\] \[\begin{aligned} &H_0: \text{data follows } N(\mu, 1), \mu = 0 && \text{(simple)} \\ &H_1: \text{data follows Poisson}(\lambda) && \text{(composite – different family)} \end{aligned}\]

$$

This assignment focuses on performing performing a test of mean (\(\mu\)) a normal population and calculating the power and sample size based on various assumptions

Question: New Cholesterol Medication

A pharmaceutical company develops “CholestFix” to reduce Low-Density Lipoprotein (LDL, fat carrier that’s low in density) cholesterol. The current standard drug lowers LDL by an average of 25 mg/dL with a standard deviation of 15 mg/dL. A clinical trial with 5 participants were recruited in the study for three months. At the end of the study, the mean reduction is 29 mg/dL. Assume that the variance of LDL reduction of new drug is the same as that of the standard drugs.

Based on the results in the clinical trial, researchers in the company believe CholestFix is more effective.

a). Perform a formal hypothesis test of the researchers’ belief regarding LDL reduction, using a significance level of \(\alpha = 0.05\).

Answer to Question 1 Part A:

The researchers in the example above are attempting to determine whether or not CholestFix is more effective than the current standard drug. That is, they are attempting to determine whether or not CholestFix lowers LDL more than the current standard drug. Given this, we know that the appropriate test for this setting is a right-tailed t-test (given the small sample size of \(n=5\)). We can also identify the following values: \(n=5, \bar{x}=29, s=15, \mu_0=100,\) and \(\alpha=0.05\). First we will use R to get the test statistic, p-value, and degrees of freedom for this setting and then use the results to perform a formal hypotheses test.

set.seed(123) #Sets seed
simulated_drug <- rnorm(5, mean=29, sd=15) #Creates simulated data set using statistics from our setting
result <- t.test(simulated_drug, mu=25, alternative="greater")
#Performs t-test
print(result) #Prints t-test

    One Sample t-test

data:  simulated_drug
t = 1.2689, df = 4, p-value = 0.1366
alternative hypothesis: true mean is greater than 25
95 percent confidence interval:
 20.30524      Inf
sample estimates:
mean of x 
 31.90355 

For our setting, our null and alternative hypotheses are:

\[ H_0: \mu \le 25 \\ H_a: \mu >25 \]

That is, our null hypothesis is that CholestFix lowers LDL less than or equal to the current standard drug and our alternative hypothesis is that CholestFix lowers LDL more than the current standard drug.

Using the output above, we know our test statistic is \(t=1.2680\).

Next we can use R to determine our critical value and compare our test statistic to our critical value.

alpha <- 0.05
df <- 4
critical_val <- qt(1-alpha, df)
print(critical_val)

[1] 2.131847

Based on the above output we know that our critical value for this setting is 2.131847.

Given that our test statistic is less than our critical value (\(t=1.2680<2.131487\)) and our p-value is greater than our \(\alpha\) (\(\text{p-value}=0.1366>0.05=\alpha\)) we do not have enough evidence to reject our \(H_0\).

In context, this means that we do not have enough evidence (at the 0.05 significance level) to reject our null hypothesis that CholestFix lowers LDL less than or equal to the current standard drug. Therefore, we do not have enough evidence to indicate that CholestFix lowers LDL more than the current standard drug.

b). Given \(n = 50, \sigma = 15, \alpha = 0.05\), and an effect size we wish to detect \(\delta = 4\) mg/dL (corresponding to a reduction from 29 mg/dL to 25 mg/dL). What is the probability that we’d detect a true improvement?

Answer to Question 1 Part B:

Given \(n = 50, \sigma = 15, \alpha = 0.05\) and an effect size of \(\delta=4\) that we wish to detect, we can use R to determine the probability that we would detect a true improvement:

power_calc <- power.t.test(n=50, #Power-test using above values
                           delta=4,
                           sd=15,
                           sig.level=0.05,
                           type="one.sample",
                           alternative="one.sided")
print(power_calc) #Prints results

     One-sample t test power calculation 

              n = 50
          delta = 4
             sd = 15
      sig.level = 0.05
          power = 0.5849737
    alternative = one.sided

Based on the above output, we know that power of this test where \(n = 50, \sigma = 15, \alpha = 0.05\) and \(\delta=4\) is \(\text{Power}=0.5849737\). Given that \(\text{Power}=P(\text{Reject } H_0 |H_a)\) we know that the probability that we would detect a true improvement is 0.5849737.

c). Determine the minimum sample size required to detect an effect size of 4 mg/dL with a power of \(1 - \beta = 0.8\) and a significance level of \(\alpha = 0.05\). Assume the standard deviation of LDL reduction is 15 mg/dL.

Answer to Question 1 Part C:

Given \(\sigma = 15, \alpha = 0.05\) and that we want a power of \(1-\beta=0.08\), we can use R to determine the minimum sample size required to detect an effect size of \(\delta=4\):

sample_size <- power.t.test(power=0.8,
                            delta=4,
                            sd=15,
                            sig.level=0.05,
                            type="one.sample",
                            alternative="one.sided")
cat("\nRequired sample size for 80% power:", ceiling(sample_size$n))

Required sample size for 80% power: 89

Based on the above output we know that, given that \(\sigma = 15, \alpha = 0.05\) and \(\delta=4\), we need a sample size of at least 89 if we want a power of \(1-\beta=0.80\). This means that we need a sample size of at least 89 to detect a true improvement of 4 80% of the time.

d). Power curve: To assess the impact of sample size on power, we can create a power function in terms of the sample size \(n\) and use the remaining information from part (b). Plot the power curve by selecting a sequence of sample sizes.

Answer to Question 1 Part D:

We can create a power curve by plotting the powers from a sequence of sample size values (\(n\) values). We are able to use the same method used in Part B and simply apply this method to a sequence of values. Here we will use a sequence of \(n\) values from 1 to 500 in order to asses the impact of sample size on power. We will also create a line that indicates when \(\text{Power}=0.8\) (the power we previously determined could be obtained from a sample size of \(n=89\)).

n_range <- seq(1, 500)
powers <- sapply(n_range, function(n) {power.t.test(n=n, #Power-test using above values
                           delta=4,
                           sd=15,
                           sig.level=0.05,
                           type="one.sample",
                           alternative="one.sided")$power
})

plot(n_range, powers, type = "l", lwd=2, col = "red",
     main = "Power Curve",
     xlab = "Sample Size",
     ylab = "Power")

abline(h=0.8, col="blue", lty=2)

Considering the power curve above, we can see that power increases rapidly from \(n=1\) to \(n=89\). However, after \(n=89\) the growth significantly slows down. This indicates that at a certain point, increasing the size of \(n\) doesn’t do a lot to increase the power of the test. Practically, this means that when we are considering increasing \(n\) to increase the power of a test, there comes a point were the small additional power of the test may not be worth the additional work and resources of increasing the sample size.

Note: For each of the questions above, write a short summary of what you observed, justify why your analysis is valid, and interpret the results.