Assignment 3: ECDF and Bootstrap Sampling and Applications

Asymptotic Distribution of Sample Variance

Assume that \(\{ x_1, x_2, \cdots, x_n \} \to F(x)\) with \(\mu = E[X]\) and \(\sigma^2 = \text{var}(X)\). Denote

\[ s^2 = \frac{1}{n-1}\sum_{i=1}^n (x_i - \mu)^2 \]

If \(n\) is large,

\[ Cs^2 \to N\left(\sigma^2, \frac{\mu_4-\sigma^4}{n} \right) \]

where \(\mu_4 = E[(X_i - \mu)^4]\) is tje 4th central moment which can be estimated by

\[ \hat{\mu}_4 = \frac{1}{n}\sum_{i=1}^n(x_i-\bar{x})^4. \]

Note: This describes the asymptotic convergence of the sample variance, following from the central limit theorem (CLT). The sample size required for this approximation to hold is situation-dependent.

Question 1: Asymptotic vs Bootstrap Sampling Distributions

Write an essay summarizing the concepts of Asymptotic and Bootstrap Sampling Distributions, along with their key applications. Your discussion should be grounded in your personal understanding of the material. Any external sources including AI tools consulted must be clearly cited.

Essay Prompt: Discuss the concepts of the bootstrap sampling plan, the bootstrap sampling distribution, and the asymptotic sampling distribution in the context of statistics (e.g., sample mean and variance) computed from an independent and identically distributed (i.i.d.) sample. Your discussion should:

• Clearly outline the key assumptions required for each method.

• Explain the practical application of each distribution.

• Provide guidance on when and why one should be preferred over the other in statistical inference.

Essay:

A sampling distribution describes how a statistic such as the sample mean \(\bar{X}\) or sample variance \(S^2\) would vary across repeated samples of size \(n\) from the same population. There are two main approaches to obtaining sampling distributions: asymptotic methods and bootstrap methods.

Bootstrap Sampling Plan and Bootstrap Sampling Distribution

The bootstrap is a Monte Carlo simulation method that replaces the unknown population distribution \(F\) with the empirical distribution function \(\hat{F}_n\). The EDF assigns equal probability \(1/n\) to each observed data point and serves as a nonparametric estimate of \(F\).

The bootstrap procedure works as follows:

1. Start with the observed i.i.d. sample \(x_1, \ldots, x_n\).
2. Treat this sample as an approximation of the population.
3. Draw a new sample of size \(n\) with replacement from the observed data.
4. Compute the statistic of interest from this resample.
5. Repeat the process \(B\) times.

The collection of replicated statistics forms the bootstrap sampling distribution. This simulated distribution can be used to estimate the standard error of a statistic, examine its variability, and support inference without relying on strong distributional assumptions.

The bootstrap requires:

1. The original data are independent and identically distributed \(i.i.d.\).
2. The sample reasonably represents the population.
3. The sample size is preserved in each resample to maintain the correct level of variability.

Because the bootstrap samples from the empirical distribution rather than assuming a specific parametric form, it is considered a nonparametric method.

Asymptotic Sampling Distribution

Asymptotic sampling distributions come from large-sample theory. When the sample size increases, many statistics converge in distribution to a Normal distribution after appropriate centering and scaling. This idea is rooted in the Central Limit Theorem.

For example, if the observations are i.i.d. with finite variance, the sample mean becomes approximately Normally distributed as \(n\) grows. Similarly, under suitable moment conditions, the sample variance also has an approximate Normal distribution for large \(n\). These results allow us to use Normal approximations to perform inference.

Asymptotic methods require: 1. Independent and identically distributed observations. 2. Finite variance. 3. A sufficiently large sample size.

When to Use Each Method

Asymptotic methods are efficient when sample size is large and assumptions are satisfied. Bootstrap methods are useful when the sampling distribution is difficult to derive or when avoiding parametric assumptions.

In summary, asymptotic methods depend on theoretical convergence results, while bootstrap methods approximate the sampling distribution through repeated resampling from the empirical distribution.

Question 2: Daily Coffee Sales (in mL) at Two Different Cafe Locations

This data set represents the volume of regular brewed coffee sold per day (in milliliters) at two different cafe locations over a period of 50 days.

2850, 3200, 2900, 3100, 2950, 7800, 8100, 7900, 3300, 3050, 4000, 4200, 3150, 3400, 7700, 8200, 
3250, 4400, 3100, 4200, 4500, 4800, 4300, 8500, 8200, 8900, 8700, 3250, 3000, 4600, 4100, 8400, 
8800, 3350, 4700, 3100, 8100, 3050, 8300, 4100, 3100, 8300, 8900, 8200, 4400, 4500, 3250, 4600, 
8400, 3300, 4200, 4500, 4800, 4300, 8500

We are interested in finding the sampling distribution of sample means that will be used for various inferences about the underlying population mean.

1. Based on the given data, can the Central Limit Theorem be used to derive the asymptotic sampling distribution of the sample mean? Justify your answer.

# Coffee sales data 
x <- c(
  2850, 3200, 2900, 3100, 2950, 7800, 8100, 7900, 3300, 3050, 4000, 4200,
  3150, 3400, 7700, 8200, 3250, 4400, 3100, 4200, 4500, 4800, 4300, 8500,
  8200, 8900, 8700, 3250, 3000, 4600, 4100, 8400, 8800, 3350, 4700, 3100,
  8100, 3050, 8300, 4100, 3100, 8300, 8900, 8200, 4400, 4500, 3250, 4600,
  8400, 3300, 4200, 4500, 4800, 4300, 8500
)

n <- length(x)
xbar <- mean(x)
s <- sd(x)

data.frame(n = n, mean = xbar, sd = s)

   n mean       sd
1 55 5250 2242.952

# Visual check on raw data distribution 

par(mfrow = c(1,2))


hist(x,
     probability = TRUE,
     breaks = 15,
     main = "Histogram of Coffee Sales",
     xlab = "Daily Sales (mL)")

lines(density(x), col = "blue", lwd = 2)


qqnorm(x, main = "Normal Q-Q Plot")
qqline(x, col = "red", lwd = 2)

par(mfrow = c(1,1))

Answer: Yes, the Central Limit Theorem can be applied here. Although the histogram shows two clusters (likely representing the two cafe locations), the CLT does not require the population to be normally distributed. It only requires that the observations are independent and have finite variance. Since the sample size is larger than 50, the sampling distribution of the sample mean can be approximated by

\[ \bar{X} \approx N\left(\mu, \frac{\sigma^2}{n}\right) \] Therefore, it is reasonable to use the asymptotic normal approximation for inference about the population mean.

2. Apply the bootstrap method to estimate the sampling distribution (often called the bootstrap sampling distribution) of the sample mean. Generate a kernel density estimate from the bootstrap sample means and plot it. Then, use this bootstrap distribution to validate your conclusion from part (a). Make sure your visuals are effective in enhancing the presentation of these results.

# Number of bootstrap replications
B <- 1000

# Store bootstrap sample means
boot.mean <- NULL

# Bootstrap loop
for(i in 1:B){
  
  # Resample WITH replacement
  boot.sample <- sample(x,size = n, replace = TRUE)
  
  # Compute mean of bootstrap sample
  boot.mean[i] <- mean(boot.sample)
}


hist(boot.mean,
     probability = TRUE,
     breaks = 20,
     main = "Bootstrap Sampling Distribution of Sample Mean",
     xlab = "Bootstrap Sample Means")

lines(density(boot.mean), col = "blue", lwd = 2)

Answer: The bootstrap sampling distribution of \(\bar{X}\) was generated by repeatedly resampling the data with replacement and computing the mean each time. The histogram and kernel density curve of the bootstrap means are approximately symmetric and bell-shaped, which supports the CLT-based conclusion in part (a) that the sampling distribution of the sample mean is reasonably approximated by a Normal distribution.

3. Repeat the analysis in parts (a) and (b) for the sample variance.

Part a

s2 <- var(x)

mu4_hat <- mean((x - mean(x))^4)

asym.sd.var <- sqrt((mu4_hat - s2^2) / n)

data.frame(sample_variance = s2,
           asymptotic_sd = asym.sd.var)

  sample_variance asymptotic_sd
1         5030833      496426.1

Answer:Under large sample theory, when the fourth moment is finite, the sample variance has an approximate Normal distribution:

\[ s^2 \to N\left(\sigma^2, \frac{\mu_4-\sigma^4}{n} \right) \] Since the sample size is greater than 50 and the data have finite variability, the asymptotic Normal approximation can be considered. However, the accuracy may be affected by the shape of the distribution.

Part b

B <- 1000

boot.var <- NULL

for(i in 1:B){
  
  boot.sample <- sample(x,
                        size = n,
                        replace = TRUE)
  
  boot.var[i] <- var(boot.sample)
}

hist(boot.var,
     probability = TRUE,
     breaks = 20,
     main = "Bootstrap Sampling Distribution of Sample Variance",
     xlab = "Bootstrap Sample Variances")

lines(density(boot.var), col = "blue", lwd = 2)

Answer for Validation :The bootstrap distribution of the sample variance appears slightly skewed rather than perfectly symmetric. This suggests that the Normal approximation for the variance may be less accurate than for the mean. Therefore, while the asymptotic approximation can be used, the bootstrap distribution provides a more data-driven assessment of variability for the sample variance.