STA 506 MID-TERM EXAM

1 Question 1

1.1 (a) Deriving the Method of Moments Equations

Assume the random variable \(X\) follows a lognormal distribution

\[ X \sim LN(\mu, \sigma^2) \]

The \(k\)-th moment of the lognormal distribution is

\[ E[X^k] = \exp\left(k\mu + \frac{1}{2}k^2\sigma^2\right) \]

1.1.0.1 First Moment

Setting \(k = 1\),

\[ E[X] = \exp\left(\mu + \frac{1}{2}\sigma^2\right) \]

The corresponding sample moment is the sample mean

\[ m_1 = \bar{X} \]

Equating the population and sample moments yields

\[ \bar{X} = \exp\left(\mu + \frac{1}{2}\sigma^2\right) \]

1.1.0.2 Second Moment

Setting \(k = 2\),

\[ E[X^2] = \exp\left(2\mu + 2\sigma^2\right) \]

The second sample moment is

\[ m_2 = \frac{1}{n}\sum_{i=1}^{n}X_i^2 \]

Equating the moments gives

\[ \frac{1}{n}\sum_{i=1}^{n}X_i^2 = \exp(2\mu + 2\sigma^2) \]

1.1.0.3 System of Moment Equations

Therefore, the Method of Moments estimators for \(\mu\) and \(\sigma^2\) are obtained by solving the following system

\[ \bar{X} = \exp\left(\mu + \frac{1}{2}\sigma^2\right) \]

\[ \frac{1}{n}\sum_{i=1}^{n}X_i^2 = \exp(2\mu + 2\sigma^2) \]

1.2 (b) Closed-form Expressions for the MME Estimators

From part (a), the Method of Moments equations are

\[ \bar{X} = \exp\left(\mu + \frac{1}{2}\sigma^2\right) \]

\[ m_2 = \frac{1}{n}\sum_{i=1}^{n}X_i^2 = \exp(2\mu + 2\sigma^2) \]

Taking natural logarithms,

\[ \ln(\bar{X}) = \mu + \frac{1}{2}\sigma^2 \]

\[ \ln(m_2) = 2\mu + 2\sigma^2 \]

Multiplying the first equation by 2 and subtracting from the second yields the estimator for \(\sigma^2\):

\[ \tilde{\sigma}^2 = \ln(m_2) - 2\ln(\bar{X}) \]

Substituting this result into the first equation and solving for \(\mu\) gives

\[ \tilde{\mu} = \ln(\bar{X}) - \frac{1}{2}\tilde{\sigma}^2 \]

Substituting \(\tilde{\sigma}^2 = \ln(m_2) - 2\ln(\bar{X})\),

\[ \tilde{\mu} = \ln(\bar{X}) - \frac{1}{2}(\ln(m_2) - 2\ln(\bar{X})) \]

which simplifies to

\[ \tilde{\mu} = 2\ln(\bar{X}) - \frac{1}{2}\ln(m_2) \]

1.2.0.1 Method of Moments Estimator for the Lognormal Mean

The mean of a lognormal distribution is

\[ \mu_{LN} = E[X] = \exp\left(\mu + \frac{1}{2}\sigma^2\right) \]

Applying the plug-in principle,

\[ \tilde{\mu}_{LN} = \exp\left(\tilde{\mu} + \frac{1}{2}\tilde{\sigma}^2\right) \]

From the first moment equation,

\[ \bar{X} = \exp\left(\mu + \frac{1}{2}\sigma^2\right) \]

therefore

\[ \tilde{\mu}_{LN} = \bar{X} \]

1.2.1 Final Method of Moments Estimators

\[ \tilde{\sigma}^2 = \ln(m_2) - 2\ln(\bar{X}) \]

\[ \tilde{\mu} = 2\ln(\bar{X}) - \frac{1}{2}\ln(m_2) \]

\[ \tilde{\mu}_{LN} = \bar{X} \]

1.3 (c) R Function for the Method of Moments Estimators

Using the results derived in part (b), the estimators are

\[ \tilde{\sigma}^2 = \ln(m_2) - 2\ln(\bar{X}) \]

\[ \tilde{\mu} = 2\ln(\bar{X}) - \frac{1}{2}\ln(m_2) \]

\[ \tilde{\mu}_{LN} = \bar{X} \]

The following function computes these estimators for our given dataset

mme_lognormal <- function(X) {

  # Compute the sample size
  n <- length(X)

  # Compute the sample mean (first sample moment)
  x_bar <- mean(X)

  # Compute the second sample moment
  m2 <- mean(X^2)

  # Compute the MME for sigma^2 using the formula derived in Question 1(b)
  sigma2_hat <- log(m2) - 2*log(x_bar)

  # Compute the MME for mu using the simplified closed-form expression
  mu_hat <- 2*log(x_bar) - 0.5*log(m2)

  # Compute the MME for the lognormal population mean
  # From the moment equation this simplifies to the sample mean
  muLN_hat <- x_bar

  # Return the three estimators as a named list
  return(list(
    mu_hat = mu_hat,
    sigma2_hat = sigma2_hat,
    muLN_hat = muLN_hat
  ))
}


wells <- c(
52.5, 57.7, 68.6, 70.5, 71.8, 72.4, 74.6, 75.1, 75.4, 76.1,
76.3, 77.2, 77.8, 78.1, 78.6, 79.2, 80.1, 80.4, 80.9, 81.3,
81.7, 82.2, 82.6, 83.1, 83.5, 84.2, 84.6, 85.1, 85.7, 86.3,
86.8, 87.4, 88.2, 88.9, 89.4, 90.1, 90.7, 91.2, 92.0, 92.6,
93.4, 94.1, 95.3, 96.8, 97.5, 98.9, 100.3, 101.7, 103.2, 105.4
)


mme_lognormal(wells)

$mu_hat
[1] 4.424698

$sigma2_hat
[1] 0.01580615

$muLN_hat
[1] 84.15

2 Question 2: Finding the MLE of \(\mu\) and \(\sigma^2\)

Assume that \(\{x_1,x_2,\dots,x_n\}\) follow a lognormal distribution \(LN(\mu,\sigma^2)\) with log-likelihood

\[ \ell(\mu,\sigma^2) = -\frac{n}{2}\ln(2\pi) - n\ln\sigma - \sum_{i=1}^{n}\ln x_i - \frac{1}{2\sigma^2}\sum_{i=1}^{n}(\ln x_i-\mu)^2 \]

To simplify differentiation, let \(\beta = \sigma^2\). The log-likelihood becomes

\[ \ell(\mu,\beta) = -\frac{n}{2}\ln(2\pi) - \frac{n}{2}\ln\beta - \sum_{i=1}^{n}\ln x_i - \frac{1}{2\beta}\sum_{i=1}^{n}(\ln x_i-\mu)^2 \]

2.1 (a) Derivation of the Score Functions

The score functions are obtained by taking partial derivatives of the log-likelihood.

2.1.1 Score function for \(\mu\)

\[ \frac{\partial \ell}{\partial \mu} = \frac{1}{\beta} \sum_{i=1}^{n}(\ln x_i-\mu) \]

Setting the score equation equal to zero,

\[ \frac{1}{\beta}\sum_{i=1}^{n}(\ln x_i-\mu)=0 \]

2.1.2 Score function for \(\beta = \sigma^2\)

\[ \frac{\partial \ell}{\partial \beta} = -\frac{n}{2\beta} + \frac{1}{2\beta^2} \sum_{i=1}^{n}(\ln x_i-\mu)^2 \]

Setting the score equation equal to zero,

\[ -\frac{n}{2\beta} + \frac{1}{2\beta^2} \sum_{i=1}^{n}(\ln x_i-\mu)^2 =0 \]

2.2 (b) Closed-form Maximum Likelihood Estimators

Solving the first score equation,

Multiplying both sides by \(\beta\),

\[ \beta \left(\frac{1}{\beta}\sum_{i=1}^{n}(\ln x_i-\mu)\right) = \beta \cdot 0 \]

which simplifies to

\[ \sum_{i=1}^{n}(\ln x_i-\mu)=0 \]

\[ \sum_{i=1}^{n}\ln x_i-n\mu=0 \]

\[ \hat{\mu}=\frac{1}{n}\sum_{i=1}^{n}\ln x_i \]

Solving the second score equation,

\[ -\frac{n}{2\beta} + \frac{1}{2\beta^2} \sum_{i=1}^{n}(\ln x_i-\mu)^2 =0 \]

Multiplying by \(2\beta^2\) gives

\[ -n\beta+\sum_{i=1}^{n}(\ln x_i-\mu)^2=0 \]

Solving for \(\beta\),

\[ \beta=\frac{1}{n}\sum_{i=1}^{n}(\ln x_i-\mu)^2 \]

Substituting \(\beta=\sigma^2\) and \(\mu=\hat{\mu}\),

\[ \hat{\sigma}^2= \frac{1}{n} \sum_{i=1}^{n}(\ln x_i-\hat{\mu})^2 \]

2.2.1 Final Maximum Likelihood Estimators

\[ \hat{\mu}=\frac{1}{n}\sum_{i=1}^{n}\ln x_i \]

\[ \hat{\sigma}^2= \frac{1}{n} \sum_{i=1}^{n}(\ln x_i-\hat{\mu})^2 \]

2.3 (c) Variance of the Lognormal Distribution

Recall that the lognormal population mean is

\[ \mu_{LN}=E[X]=\exp\left(\mu+\frac{1}{2}\sigma^2\right) \]

Using the relationship between the first and second moments,

\[ Var(X)=E[X^2]-(E[X])^2 \]

From the \(k\)-th moment formula of the lognormal distribution,

\[ E[X^k]=\exp\left(k\mu+\frac{1}{2}k^2\sigma^2\right) \]

Setting \(k=2\) gives

\[ E[X^2]=\exp(2\mu+2\sigma^2) \]

Substituting into the variance formula,

\[ Var(X) = \exp(2\mu+2\sigma^2) - \left(\exp\left(\mu+\frac{1}{2}\sigma^2\right)\right)^2 \]

Simplifying,

\[ \sigma^2_{LN} = \exp(2\mu+2\sigma^2) - \exp(2\mu+\sigma^2) \]

Factoring yields

\[ \sigma^2_{LN} = (\exp(\sigma^2)-1)\exp(2\mu+\sigma^2) \]

3 Question 3: Sampling Distribution of Lognormal Sample Mean \(\hat{\mu}_{LN}\)

3.1 (a) Function to compute \(\hat{\mu}\), \(\hat{\sigma}^2\), and \(\hat{\mu}_{LN}\)

mle_lognormal <- function(x){

  # Take the natural logarithm of the data since ln(X) ~ N(mu, sigma^2)
  logx <- log(x)

  # Compute the MLE of mu, which is the sample mean of the log-transformed data
  mu_hat <- mean(logx)

  # Compute the MLE of sigma^2
  sigma2_hat <- mean((logx - mu_hat)^2)

  # Compute the MLE of the lognormal population mean 
  muLN_hat <- exp(mu_hat + 0.5 * sigma2_hat)

  return(list(mu_hat = mu_hat,
              sigma2_hat = sigma2_hat,
              muLN_hat = muLN_hat))
}

3.2 (b) Generate 1000 bootstrap samples and compute bootstrap MLEs

# Enter the dataset of 50 observations
data <- c(
17.3,24.8,8.2,31.5,14.1,42.7,11.9,55.3,21.4,9.7,
36.2,18.6,63.1,13.2,28.9,47.5,19.8,7.5,33.4,52.1,
16.0,25.9,38.7,10.3,44.2,22.5,58.6,12.8,30.3,48.9,
20.1,35.4,15.7,60.2,26.7,41.3,18.1,53.8,23.9,46.2,
29.4,37.6,14.9,50.5,32.8,19.3,56.7,11.2,39.5,27.1
)

# Set the random seed results are the same each time the code is run
set.seed(123)

B <- 1000

bootstrap_muLN <- numeric(B)

for(b in 1:B){

  # Draw a bootstrap sample of size n with replacement from the dataset
  sample_b <- sample(data, replace = TRUE)

  # Apply the MLE function to the bootstrap sample
  result <- mle_lognormal(sample_b)

  # Store the bootstrap estimate of the lognormal mean
  bootstrap_muLN[b] <- result$muLN_hat
}

3.3 (c) and (d) Kernel Density Estimate of Bootstrap MLEs and the asymptotic normal density using plug-in estimates

# Compute the kernel density estimate of the bootstrap estimates
boot_density <- density(bootstrap_muLN)

# Plot the bootstrap sampling distribution using the KDE
plot(boot_density,
     main="Bootstrap Sampling Distribution of mu_LN",
     xlab="Estimated lognormal mean")


## (d) Add asymptotic normal density using plug-in estimates

n <- length(data)

# Use the MLE function on the original dataset to obtain the estimates
result <- mle_lognormal(data)

# Extract the MLEs of mu, sigma^2, and the lognormal mean
mu_hat <- result$mu_hat
sigma2_hat <- result$sigma2_hat
muLN_hat <- result$muLN_hat

# Compute the plug-in estimate of the lognormal population variance
sigma2_LN_hat <- (exp(sigma2_hat)-1)*exp(2*mu_hat + sigma2_hat)

# Compute the standard deviation of the asymptotic sampling distribution
sd_CLT <- sqrt(sigma2_LN_hat/n)

# Add the asymptotic normal density curve to the existing bootstrap KDE plot
curve(dnorm(x, mean=muLN_hat, sd=sd_CLT),
      add=TRUE,
      col="red",
      lwd=2)

# Add a legend identifying the two curves
legend("bottomleft",
       legend=c("Bootstrap KDE","CLT Normal Approximation"),
       col=c("black","red"),
       lwd=2)

3.4 (e) Comparison

The bootstrap kernel density estimate provides an empirical approximation of the sampling distribution of the MLE of the lognormal mean based on repeated resampling from the observed dataset. The asymptotic normal distribution obtained from the Central Limit Theorem represents the theoretical large-sample approximation of the sampling distribution of the estimator.

From the plot, the bootstrap density and the asymptotic normal curve are centered around similar values, indicating that both methods estimate a similar mean for the sampling distribution. The overall shapes of the two curves are also fairly similar, although the bootstrap distribution appears slightly more peaked and exhibits mild asymmetry compared to the normal approximation. In contrast, the CLT approximation is perfectly symmetric.

Overall, the two distributions are reasonably close, suggesting that the normal approximation provided by the Central Limit Theorem offers a good approximation to the sampling distribution of the estimator for this sample size, although the bootstrap distribution captures slight deviations from normality.