Assignment 9: Hypothesis Testing and Power and Sample size Determination

Assignment Objectives

• Enhance understanding the logic and procedure of hypothesis testing .

• Implement the procedures for power and sample size calculation for basic hypothesis testing procedures using nuilt-in function and manual calculation.

Policies of Using AI Tools

Policy on AI Tool Use: Please adhere to the AI tool policy specified in the course syllabus. The direct copying of AI-generated content is strictly prohibited. All submitted work must reflect your own understanding; where external tools are consulted, content must be thoroughly rephrased and synthesized in your own words.

Code Inclusion Requirement: Any code included in your essay must be properly commented to explain the purpose and/or expected output of key code lines. Submitting AI-generated code without meaningful, student-added comments will not be accepted.

**Simple versus Composite Hypothesis*

Simple Hypothesis

• Simple Hypothesis Test is a hypothesis that completely specifies the population distribution. Mathematically, simple hypothesis fixes all parameters to specific values. For example, The simple hypotheses are: \(H_0: \mu = 5\) (if \(\alpha\) is known), \(H_0: \mu = 100, \sigma^2 = 25\), and \(H_1: p = 0.7\) for a Bernoulli distribution.

• Example Scenario Test if a coin is fair:

\[\begin{aligned} H_0&: p = 0.5 \quad \text{(completely specified)} \\ H_1&: p = 0.6 \quad \text{(also completely specified)} \end{aligned}\]

Both are simple hypotheses.

Composite Hypothesis

• Composite Hypothesis is a hypothesis that does not completely specify the distribution. Mathematically, it allows a range of values for at least one parameter. For example, in one-sided: \(\mu >5\); in two-sided: \(\mu \le 5\).

• Example Scenarios

\[\begin{aligned} &H_0: \mu = 5 && \text{(simple)} \\ &H_1: \mu > 5 && \text{(composite)} \end{aligned}\] \[\begin{aligned} &H_0: \mu \leq 5 && \text{(composite)} \\ &H_1: \mu > 5 && \text{(composite)} \end{aligned}\] \[\begin{aligned} &H_0: \text{data follows } N(\mu, 1), \mu = 0 && \text{(simple)} \\ &H_1: \text{data follows Poisson}(\lambda) && \text{(composite – different family)} \end{aligned}\]

This assignment focuses on performing performing a test of mean (\(\mu\)) a normal population and calculating the power and sample size based on various assumptions

Question: New Cholesterol Medication

A pharmaceutical company develops “CholestFix” to reduce Low-Density Lipoprotein (LDL, fat carrier that’s low in density) cholesterol. The current standard drug lowers LDL by an average of 25 mg/dL with a standard deviation of 15 mg/dL. A clinical trial with 5 participants were recruited in the study for three months. At the end of the study, the mean reduction is 29 mg/dL. Assume that the variance of LDL reduction of new drug is the same as that of the standard drugs.

Based on the results in the clinical trial, researchers in the company believe CholestFix is more effective.

a). Perform a formal hypothesis test of the researchers’ belief regarding LDL reduction, using a significance level of \(\alpha = 0.05\).

H0: mu = 25 (mean LDL reduction is the same as the standard drug)

Ha: mu > 25 (mean LDL reduction is greater than the standard drug)

# Given values
xbar <- 29
mu0 <- 25
sigma <- 15
n <- 5

# Test statistic
z <- (xbar - mu0) / (sigma / sqrt(n))

# Display labeled result
cat("Test statistic (z):", round(z, 3))

Test statistic (z): 0.596

# Critical value for right-tailed test
alpha <- 0.05

z_crit <- qnorm(1 - alpha)

cat("Critical value (z):", round(z_crit, 3))

Critical value (z): 1.645

# p-value for right-tailed test
p_value <- 1 - pnorm(z)
cat("p-value:", round(p_value, 4))

p-value: 0.2755

p-value and interpretation:

The p-value is 0.2755, which is greater than alpha = 0.05. This indicates that the observed mean reduction of 29 mg/dL is not significantly greater than 25 mg/dL and does not provide strong evidence that CholestFix is more effective than the standard drug.

Decision in context

Since the p-value is greater than alpha = 0.05, we fail to reject H0. There is not sufficient statistical evidence to support the claim that CholestFix reduces LDL cholesterol more than the current standard drug.

Summary

The sample mean LDL reduction observed in the clinical trial is 29 mg/dL, which is higher than the standard drug’s average reduction of 25 mg/dL. A one-sample z-test makes sense here because the population standard deviation is known and we are testing the sample mean against a specific value. Based on the results of the hypothesis test at the significance level alpha = 0.05, the difference between the sample mean and the standard value is not statistically significant. Therefore, there is not sufficient statistical evidence to support the researchers’ claim that CholestFix reduces LDL cholesterol more than the current standard drug.

b). Given \(n = 50, \sigma = 15, \alpha = 0.05\), and an effect size we wish to detect \(\delta = 4\) mg/dL (corresponding to a reduction from 29 mg/dL to 25 mg/dL). What is the probability that we’d detect a true improvement?

# Store given values for power calculation
n <- 50
sigma <- 15
delta <- 4
alpha <- 0.05

# Calculate probability of detecting true effect
power_result <- power.t.test(
  n = n,
  delta = delta,
  sd = sigma,
  sig.level = alpha,
  type = "one.sample",
  alternative = "one.sided"
)

# Display computed power
cat("Power:", round(power_result$power, 4))

Power: 0.585

Summary:

The power is about 0.585, which is not very high compared to the usual 0.80 target. This means there is only about a 58.5% chance of detecting a real improvement if it actually exists. The method used here is appropriate since it uses the values given in the problem to calculate the probability of detecting an effect. Since the power is low, the study may not be strong enough and could fail to detect a true improvement.

c). Determine the minimum sample size required to detect an effect size of 4 mg/dL with a power of \(1 - \beta = 0.8\) and a significance level of \(\alpha = 0.05\). Assume the standard deviation of LDL reduction is 15 mg/dL.

# Store given values for sample size calculation
delta <- 4
sigma <- 15
alpha <- 0.05
target_power <- 0.80

# Calculate required sample size
sample_size_result <- power.t.test(
  power = target_power,
  delta = delta,
  sd = sigma,
  sig.level = alpha,
  type = "one.sample",
  alternative = "one.sided"
)

# Display required sample size
cat("Required sample size:", ceiling(sample_size_result$n))

Required sample size: 89

Summary

The required sample size comes out to 89, which is a lot bigger than what was used before. So this tells us that the earlier study was probably too small to really detect the effect. The way this was done makes sense because it uses the values given in the problem like the effect size, standard deviation, alpha, and the target power. Since we want power = 0.80, we need a larger sample to get there. Because of that, using a smaller sample would likely not be enough to detect the improvement even if it is actually there.

d). Power curve: To assess the impact of sample size on power, we can create a power function in terms of the sample size \(n\) and use the remaining information from part (b). Plot the power curve by selecting a sequence of sample sizes.

# Store fixed values from part (b)
delta <- 4
sigma <- 15
alpha <- 0.05

# Choose sample sizes to examine
n_seq <- seq(10, 120, by = 5)

# Compute power for each sample size
power_values <- sapply(n_seq, function(n) {
  power.t.test(
    n = n,
    delta = delta,
    sd = sigma,
    sig.level = alpha,
    type = "one.sample",
    alternative = "one.sided"
  )$power
})

# Plot power curve
plot(n_seq, power_values, type = "l", lwd = 2,
     xlab = "Sample size (n)",
     ylab = "Power",
     main = "Power Curve for Detecting LDL Improvement")

# Add reference lines
abline(h = 0.80, lty = 2)
abline(v = 89, lty = 2)

Summary:

The graph shows that power increases as the sample size increases. At smaller sample sizes, the power is pretty low, but it gets closer to 0.80 as n gets bigger. This makes sense here because it keeps the same effect size, standard deviation, and significance level from part (b), and only changes n to see how the power changes. Looking at the curve, it is clear that we need a much larger sample to reach the desired power of 0.80, which matches what we found in part (c).

Note: For each of the questions above, write a short summary of what you observed, justify why your analysis is valid, and interpret the results.