March 3, 2026

What is a Point Estimate?

A Point Estimate is a statistic that attempts to estimate an often unknown parameter.

  • Statistic: A numerical value that is computed from and describes a sample.
  • Parameter: A fixed numerical value that describes and entire population.

Examples:

  • The sample variance \(s^2\) is a point estimator for the population variance \(\sigma^2\).
  • The sample mean \(\bar{x}\) is a point estimator for the population mean \(\mu\).
Reference(s): [1]

3 Properties of Point Estimators

  1. Bias: Let \(X=\{X_1, X_2, X_3,\dots , X_n\}\) be a random sample. We can define an estimator \(T\) as a function of the sample \(h(X)\) or \(T=h(\{X_1, X_2, X_3,\dots , X_n\})\). The bias of \(T\) is defined as \(E(T)-\theta\) where \(\theta\) is the parameter being estimated. A point estimator is unbiased if \(E(T)=\theta\).
  2. Consistency: An estimator is defined as consistent if it convergences to the true value of a parameter as the sample size increases.
  3. Efficiency: Let \(T_1\) and \(T_2\) be unbiased point estimators. \(T_1\) is more efficient than \(T_2\) if the variance of \(T_1\) is less than the variance of \(T_2\) or \(Var(T_1)<Var(T_2)\).
Reference(s): [3] [4]

Finding a Point Estimate

The 3 main methods to find a point estimate (oversimplified):

  1. Maximum Likelihood Estimation (MLE): Assume a distribution for the population. Create a likelihood function (the joint probability of the observed data as a function of the parameter). Find the parameter value that maximizes this function.
  2. Method of Moments (MoM): Assume a distribution for the population. Set the theoretical moments (formulas for mean, variance, etc.) equal to the sample moments (calculated from your data) and solve for the unknown parameter.
  3. Bayesian Estimation: Start with a prior distribution for the parameter based on past belief/data. Assume a distribution for the population data. Combine them using Bayes’ Theorem to create a posterior distribution and use its center (mean or mode) as the estimate.
Reference(s): [2]

Method of Moments Example (1/4)

For this experiment, we will treat the entire quakes dataset as our Full Population (\(N=1000\)). We then draw a Random Sample consisting of 15% of the data (\(n=150\)). Using only this sample, we will calculate Point Estimates for the population mean (First Moment) and variance (Second Moment) to see how accurately they represent the total dataset. Below is a histogram of the 15% sample to visualize its distribution.

Method of Moments Example (2/4)

Since we are assuming a normal distribution, the moment calculations become very simple. The sample first moment, the mean, can be calculated as \(\bar{X}=\frac{1}{n}\sum_{i=1}^{n} X_i\). The sample second moment, the variance, can be calculated as \(\hat{\sigma^2} =\frac{1}{n}\sum_{i=1}^{n} (X_i-\bar{X})^2\).

# Estimators
mu_hat  <- mean(sample_data$mag)
var_hat <- sum((sample_data$mag - mu_hat)^2) / nrow(sample_data)

# Results
print(paste("Estimated Mean:", round(mu_hat, 4)))
## [1] "Estimated Mean: 4.58"
print(paste("Estimated Variance:", round(var_hat, 4)))
## [1] "Estimated Variance: 0.156"
Reference(s): [5]

Method of Moments Example (3/4)

We will now compare the estimated mean to the population mean. 

## [1] "Difference of Roots of Variances (SD): -0.0078"

We can see that the estimated mean (red line) is extremely close to the “population” mean (dotted line). The distance between the population and sample standard deviation is 0.0078, which is relatively small as well.

Method of Moments (4/4)

To show that the sample we have taken is truly random, a 3d scatter is displayed.

References