Lance R. English

Standard Error

Standard error is a measure of dispersion defined as \(\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}\). What does this mean in plain words?

If you sample the same population multiple times, with the same sample size each time, the standard deviation of all the sample means should be equal to the standard deviation of the population divided by the square root of the sample size.

Thus, standard error doesn’t measure the dispersion of the observations in a single sample, it measures the dispersion of sample means from multiple samples.

This brings up a couple of questions.

First, what practical information can we get from standard error? Since the central limit theorem tells us that, as our sample size increases, the mean of sample means approaches the population mean, then standard error measurs how much our sample means vary from the population mean.

Second, what does standard error mean if you only have one sample, like is often the case in the real world? How can you even calculate it?

In this real world example above, we estimate standard error using the equation \(s_{\bar{x}}=\frac{s}{\sqrt{n}}\).

Simulation

The simulation below aims to demonstrate the central limit theorem and shows that \(s_{\bar{x}}=\frac{s}{\sqrt{n}}\) is a good estimate of standard error. We start by creating a normally distributed population with \(N=1,000,000\), \(\mu=7\), and \(\sigma=1\). We then sample this population 1,000 times with sample size n=100.

Code

#Standard Error Simulation
pop = rnorm(n=1000000,mean=7,sd=1)
sam = matrix(nrow=1000,ncol=100)

#Create 1000 samples of n=100
for(i in 1:1000) {
  sam[i,] <- sample(x=pop,size=100,replace=FALSE)
}

#Summary of samples
sam.mean = apply(sam,MARGIN=1,mean)
sam.sd = apply(sam,MARGIN=1,sd)
sam.se = sam.sd/sqrt(100)

#Is the population mean contained within standard error of each sample?
count=0
for(i in 1:1000) {
  low.bound = sam.mean[i]-sam.se[i]
  high.bound = sam.mean[i]+sam.se[i]
  if(high.bound>mean(pop)) {
    if(mean(pop)>low.bound) {
      count=count+1
    }
  }
}

#Histogram
library(ggplot2)
## Warning: package 'ggplot2' was built under R version 3.3.3
sample.mean.hist = qplot(
  sam.mean,
  geom = "histogram",
  binwidth = mean(sam.se)/5,
  xlim = c(mean(pop)-(4*mean(sam.se)),mean(pop)+(4*mean(sam.se))),
  xlab = "Sample Means",
  ylab = "Frequency",
  fill = I("black"),
  col = I("white")
)
sample.mean.hist = sample.mean.hist + geom_vline(xintercept = mean(pop)-mean(sam.se), color = "red") + 
  geom_vline(xintercept = mean(pop)+mean(sam.se), color = "red")

What do we get?

Population summary:

Population mean : 6.9996335
Population sd : 0.9987954
True standard error: 0.0998795

Average sample standard error (our estimated stanard error): 0.0993659

The population mean falls within the estimated standard error of the sample 66.7 percent of the time.

The red lines on the histogram below represent \(\pm\) estimated standard error. As you can see, our histogram looks remarkably normal with 66.7% of the values falling within one estimated standard error of the population mean, which is pretty close to the 68.44% you’d expect from any normal distribution.