This lesson begins our study of statistical inference which is drawing a conclusion about our population from our sample data. Inference has two main components: estimation and hypothesis testing. We start with estimation and will continue with hypothesis testing next week.
Consider a population parameter such as the population mean \(\mu\) or the population proportion \(p\). A point estimator of a population parameter is a function of the sample information that produces a single number called a point estimate.
For example, the sample mean \(\bar X\) is a point estimator of the population mean \(\mu\) and the value that \(\bar X\) assumes for a given set of data is called the point estimate \(\bar x\).
There is no single mechanism that exists for determining the best point estimator in all circumstances. What is available instead is a set of criteria under which paricular estimators can be evaluated. The sample median also gives a point estimate of the population mean \(\mu\). However, we will see later on that the sample median is not the best estimator for \(\mu\) compared to the sample mean.
The first property that an estimator should possess is unbiasedness.
A point estimator \(\hat \theta\) is said to be an unbiased estimator of \(\theta\) if its expected value is equal to that parameter, that is, if \[E(\hat \theta) = \theta\]
Note that unbiasedness does not mean that a particular value of \(\hat \theta\) must be exactly equal to \(\theta\). Sometimes \(\hat \theta\) will overestimate and other times underestimate \(\theta\), but if the sampling procedure is repeated many times then, on average, the value obtained for an unbiased estimator will be equal to the population parameter.
We have already seen from their sampling distributions that \(\bar X\) is an unbiased estimator of \(\mu\) and \(\hat p\) is an unbiased estimator of \(p\).
Example 1: Suppose we take a random sample of size 2 from a population with mean \(\mu\) and variance \(\sigma^2\). Let \(X_1\) and \(X_2\) denote the two observations in our sample. Show that the following point estimators of \(\mu\) are unbiased.
\[Y = \frac{1}{2}X_1 + \frac{1}{2}X_2\] \[Z = \frac{1}{3}X_1 + \frac{2}{3}X_2\]
Click For Answer\[E(Y)=E(\frac{1}{2}X_1+\frac{1}{2}X_2)=\frac{1}{2}E(X_1)+\frac{1}{2}E(X_2)=\frac{1}{2}\mu+\frac{1}{2}\mu=\mu\] \[E(Z)=E(\frac{1}{3}X_1+\frac{2}{3}X_2)=\frac{1}{3}E(X_1)+\frac{2}{3}E(X_2)=\frac{1}{3}\mu+\frac{2}{3}\mu=\mu\]
Unbiasedness alone is not the only desirable characteristic of an estimator. There may be several unbiased estimators for a population parameter.
If there are several unbiased estimators of a parameter, then the unbiased estimator with the smallest variance is preferred and is called the most efficient estimator. Let \(\hat \theta_1\) and \(\hat \theta_2\) be two unbiased estimators of \(\theta\) based on the same number of sample observations. Then \(\hat \theta_1\) is said to be more efficient than \(\hat \theta_2\) if the variance of \(\hat \theta_1\) is smaller than the variance of \(\hat \theta_2\).
Example 2: Suppose we take a random sample of size 2 from a population with mean \(\mu\) and variance \(\sigma^2\). Let \(X_1\) and \(X_2\) denote the two observations in our sample. Which of the following point estimators of \(\mu\) is most efficient?
\[Y = \frac{1}{2}X_1 + \frac{1}{2}X_2\] \[Z = \frac{1}{3}X_1 + \frac{2}{3}X_2\]
Click For Answer\(Y\) is more efficient since it has the smaller variance.
\[\text{Var}(Y)=\text{Var}(\frac{1}{2}X_1+\frac{1}{2}X_2)=(\frac{1}{2})^2\text{Var}(X_1)+(\frac{1}{2})^2\text{Var}(X_2)=\frac{1}{4}\sigma^2+\frac{1}{4}\sigma^2=\frac{1}{2}\sigma^2\] \[\text{Var}(Z)=\text{Var}(\frac{1}{3}X_1+\frac{2}{3}X_2)=(\frac{1}{3})^2\text{Var}(X_1)+(\frac{2}{3})^2\text{Var}(X_2)=\frac{1}{9}\sigma^2+\frac{4}{9}\sigma^2=\frac{5}{9}\sigma^2\]