Point estimation uses sample data to estimate an unknown population parameter with a single value.
Common Examples:
A point estimator \(\hat{\theta}\) is a function of the sample data used to estimate a population parameter \(\theta\).
For a random sample \(X_1, X_2, \ldots, X_n\):
\[\hat{\theta} = g(X_1, X_2, \ldots, X_n)\]
Example: The sample mean as an estimator of \(\mu\):
\[\bar{X} = \frac{1}{n}\sum_{i=1}^{n}X_i\]
This is an unbiased estimator because \(E[\bar{X}] = \mu\).
Unbiasedness: \(E[\hat{\theta}] = \theta\)
Efficiency: Minimum variance among unbiased estimators
Consistency: As \(n \to \infty\), \(\hat{\theta} \to \theta\)
The Method of Moments matches sample moments to population moments.
For the \(k\)-th population moment: \(\mu_k = E[X^k]\)
The \(k\)-th sample moment: \(m_k = \frac{1}{n}\sum_{i=1}^{n}X_i^k\)
Example: For a normal distribution \(N(\mu, \sigma^2)\):
\[\hat{\mu}_{MOM} = m_1 = \bar{X} = \frac{1}{n}\sum_{i=1}^{n}X_i\]
\[\hat{\sigma}^2_{MOM} = m_2 - m_1^2 = \frac{1}{n}\sum_{i=1}^{n}(X_i - \bar{X})^2\]
The MLE chooses the parameter value that maximizes the likelihood of observing the data.
Likelihood function: \(L(\theta | X_1, \ldots, X_n) = \prod_{i=1}^{n}f(X_i | \theta)\)
We maximize: \(\ell(\theta) = \log L(\theta)\)
Example: For data from \(N(\mu, \sigma^2)\) with known \(\sigma^2\):
\[\hat{\mu}_{MLE} = \arg\max_{\mu} \left[-\frac{n}{2}\log(2\pi\sigma^2) - \frac{1}{2\sigma^2}\sum_{i=1}^{n}(X_i - \mu)^2\right]\]
Solution: \(\hat{\mu}_{MLE} = \bar{X}\)
population_mean <- 170
population_sd <- 10
sample_sizes <- c(10, 30, 100, 500)
estimates <- data.frame()
for (n in sample_sizes) {
for (i in 1:100) {
sample_data <- rnorm(n, mean = population_mean,
sd = population_sd)
estimates <- rbind(estimates,
data.frame(n = as.factor(n),
estimate = mean(sample_data)))
}
}
Mean estimate: 0.1514 (True value: 0.15)