Assignment 9: Hypothesis Testing and Power and Sample size Determination

Assignment Objectives

• Enhance understanding the logic and procedure of hypothesis testing .

• Implement the procedures for power and sample size calculation for basic hypothesis testing procedures using nuilt-in function and manual calculation.

Policies of Using AI Tools

Policy on AI Tool Use: Please adhere to the AI tool policy specified in the course syllabus. The direct copying of AI-generated content is strictly prohibited. All submitted work must reflect your own understanding; where external tools are consulted, content must be thoroughly rephrased and synthesized in your own words.

Code Inclusion Requirement: Any code included in your essay must be properly commented to explain the purpose and/or expected output of key code lines. Submitting AI-generated code without meaningful, student-added comments will not be accepted.

**Simple versus Composite Hypothesis*

Simple Hypothesis

• Simple Hypothesis Test is a hypothesis that completely specifies the population distribution. Mathematically, simple hypothesis fixes all parameters to specific values. For example, The simple hypotheses are: \(H_0: \mu = 5\) (if \(\alpha\) is known), \(H_0: \mu = 100, \sigma^2 = 25\), and \(H_1: p = 0.7\) for a Bernoulli distribution.

• Example Scenario Test if a coin is fair:

\[\begin{aligned} H_0&: p = 0.5 \quad \text{(completely specified)} \\ H_1&: p = 0.6 \quad \text{(also completely specified)} \end{aligned}\]

Both are simple hypotheses.

Composite Hypothesis

• Composite Hypothesis is a hypothesis that does not completely specify the distribution. Mathematically, it allows a range of values for at least one parameter. For example, in one-sided: \(\mu >5\); in two-sided: \(\mu \le 5\).

• Example Scenarios

\[\begin{aligned} &H_0: \mu = 5 && \text{(simple)} \\ &H_1: \mu > 5 && \text{(composite)} \end{aligned}\] \[\begin{aligned} &H_0: \mu \leq 5 && \text{(composite)} \\ &H_1: \mu > 5 && \text{(composite)} \end{aligned}\] \[\begin{aligned} &H_0: \text{data follows } N(\mu, 1), \mu = 0 && \text{(simple)} \\ &H_1: \text{data follows Poisson}(\lambda) && \text{(composite – different family)} \end{aligned}\]

This assignment focuses on performing performing a test of mean (\(\mu\)) a normal population and calculating the power and sample size based on various assumptions

Question: New Cholesterol Medication

A pharmaceutical company develops “CholestFix” to reduce Low-Density Lipoprotein (LDL, fat carrier that’s low in density) cholesterol. The current standard drug lowers LDL by an average of 25 mg/dL with a standard deviation of 15 mg/dL. A clinical trial with 5 participants were recruited in the study for three months. At the end of the study, the mean reduction is 29 mg/dL. Assume that the variance of LDL reduction of new drug is the same as that of the standard drugs.

Based on the results in the clinical trial, researchers in the company believe CholestFix is more effective.

a). Perform a formal hypothesis test of the researchers’ belief regarding LDL reduction, using a significance level of \(\alpha = 0.05\).

Claim: Ha: \(\mu\) > 25 mg/dL

Hypothesis: \(H_o\) : \(\mu\) \(le\) 25 mg/dL \(H_a\) : \(\mu\) \(gt\) 25 mg/dL

#Current drug standard:
    # Current Drug: Lowers LDL by average of 25 mg/dL
    # sd=15
    # n=5
# New drug
  # new drug mu = 29 mg/dL

# Company believe CholestFix is more effective

# a). Perform a formal hypothesis test of the researchers’ belief regarding LDL reduction, using a significance level of α=0.05


# One-sample t-test in R
# Note this is a random comparative sample with same descriptive of the prompt. Use Manual Calculations for more precisness 
set.seed(123)
simulated_iq <- rnorm(n=5, mean = 29, sd = 15)
result <- t.test(simulated_iq, mu = 25, 
                 alternative = "greater")

# Output
print(result)

    One Sample t-test

data:  simulated_iq
t = 1.2689, df = 4, p-value = 0.1366
alternative hypothesis: true mean is greater than 25
95 percent confidence interval:
 20.30524      Inf
sample estimates:
mean of x 
 31.90355 

###
cat("\nManual t-value:" ,
    "\nR t-value:", round(result$statistic, 3),
    "\nCritical t-value (0.05):",
    "\nR p-value:", round(result$p.value, 4))

Manual t-value: 
R t-value: 1.269 
Critical t-value (0.05): 
R p-value: 0.1366

#Manual Check 

x_bar = 29
mu_0 = 25
sd = 15
n = 5
alpha = 0.05


#calculate the t statistic

#what is tcrit

t = (29-25)/(15/sqrt(5))
t

[1] 0.5962848

#Find the t crit of a one-tailed test specifically

t_crit = qt(1 - alpha, df = n - 1)

t > t_crit

[1] FALSE

cat("t > t_crit outputs: ", t > t_crit)

t > t_crit outputs:  FALSE

#False

#p_val = pt(t, df = n - 1, lower.tail = FALSE)

cat("There is not statistically evidence that CholestFix out performs the standard drug", "( t =", t, "p>0.05).")

There is not statistically evidence that CholestFix out performs the standard drug ( t = 0.5962848 p>0.05).

The t-statistic is less than t-crit, therefore there is not statistical evidence the new drug CholesFix out performs the standard drug (0.5962848 < 2.1318468). CholesFix should not be advertised as a better performing drug.

The analysis is justified as valid

A one sample t-test with a one-tail test was used because the analysis compared an already known drug demographic mean of 25 to an unknown new drug performance of CholesFix. As the researchers claim the new drug, CholesFix, performs greater than the existing drug, this is a one tail, one direction of interest, analysis.

Although the raw means of the new drug reduction, CholesFix, is greater than the mean of the standard drug reduction, the characteristics of the CholesFix distribution, for example a how standard deviation of 15, statistically lead to higher variability in an already, small dataset (n=5) contributing to a statistically non-significant greater mean.

b). Given \(n = 50, \sigma = 15, \alpha = 0.05\), and an effect size we wish to detect \(\delta = 4\) mg/dL (corresponding to a reduction from 29 mg/dL to 25 mg/dL). What is the probability that we’d detect a true improvement?

One tailed test

n = 50

sd = 15

alpha = 0.05

delta = 4

lam = (delta) / ( sd/ sqrt(n))

power_calc <- power.t.test(n = 50, 
                          delta = 4,    # true difference between the means in H0 and Ha
                          sd = 15,      
                          sig.level = 0.05,
                          type = "one.sample",
                          alternative = "two.sided")
cat("The probability that we detect the true improvement of the new drug CholesFix compared to the standard drug is",  round(power_calc$power, 4))

The probability that we detect the true improvement of the new drug CholesFix compared to the standard drug is 0.4557

#using one-sided in alternative line instead to check
power_calc_one_alt <- power.t.test(n = 50, 
                          delta = 4,    # true difference between the means in H0 and Ha
                          sd = 15,      
                          sig.level = 0.05,
                          type = "one.sample",
                          alternative = "one.sided")

Analysis is valid

The probability we detect the true improvement is based on the power which is 0.4557. We base power on the level of significance (0.05), sample size (n=50), and distance between the means of the null and alternative hypothesis (effect size) to assign the likelihood of being able to detect an improvement by identifying the change between the comparative \(H_o\) and \(H_a\) outcomes in this case is 4. Since our power is almost 1/2 it suggests a lower level of power that may be improved through a larger sample size.

c). Determine the minimum sample size required to detect an effect size of 4 mg/dL with a power of \(1 - \beta = 0.8\) and a significance level of \(\alpha = 0.05\). Assume the standard deviation of LDL reduction is 15 mg/dL.

# The minimumal n
  #effect size = 4
  #power 0.8
  # sd = 15



sample_size = power.t.test(power = .80,
                           delta = 4,
                           sd = 15,
                           sig.level = 0.05,
                           type= "one.sample",
                           alternative = "two.sided")



c_sample=ceiling(sample_size$n)

cat("Required sample size for 80% power is ", ceiling(sample_size$n))

Required sample size for 80% power is  113

Validity Assessment

To determine the sample size required for 80% power we we consider the alpha level (\(\alpha\) = .05), difference between \(H_a\) and \(H_o\) (\(\delta = 4\)) with the designated power level, to solve for n, the sample size. Again we use a one sample test because the population parameter was known, and was being compared to an unknown, actively investigated, sample. To ensure our sample is not underpowered and has a power level of 80% we recommend a sample size of 113.

d). Power curve: To assess the impact of sample size on power, we can create a power function in terms of the sample size \(n\) and use the remaining information from part (b). Plot the power curve by selecting a sequence of sample sizes.

#DEFINE the range of sample sizes to test
n_range = seq(10, 200, by = 5)


#use sapply to calculate power for every 'n' in range
#We keep delta

power = sapply(n_range, function(n_val)
{
  
  p_out = power.t.test(n= n_val,
                       delta =4 ,
                       sd= 15,
                       sig.level= 0.05,
                       type = "one.sample",
                       alternative = "two.sided") #should this be two.sided too?

  return(p_out$power)
  
})

plot(n_range, power, type = "b", pch = 16, 
     main = "Power Curve CholestFix",
     xlab= "Sample Size (n)",
     ylab = "Power (Probability of Detecting Effect)")
# Horizontal line at 80% (the standard goal for power)
abline(h = 0.80, col = "red", lty = 2)
abline(v = 112, col = "blue", lty = 3) # The n you found in 1c

This is a graphical summary for power considering sample size options. Detailing out interest in a 0.80 power level we use see the sample size is the 113. This information graphically represents the information listed in 1c, leaving the same interpretation for validity. By analyzing the curvature of the line, we see the sample size for the 0.8 power level covers the majority of the curve, suggesting some strength in the calculated 113 sample size. We see the sample sizes above 150 do not assist the analysis as much as a smaller sample does at lower probabilities, so by eye 0.8 probability is a good financial balance between a large sample size with a stronger effect size.

Note: For each of the questions above, write a short summary of what you observed, justify why your analysis is valid, and interpret the results.