The normal distribution

Gaussian distribution with mean \(\mu\) and variance \(\sigma^2\)

\[ (2 \pi \sigma^2)^{-1/2}\exp{-(x-\mu)^2 / 2 \sigma^2} \]

Expected value

\[ E[X] = \mu \] and \[ Var(X) = \sigma^2 \]

In simbols

\[ X \sim N(\mu, \sigma^2) \]

When \(\mu = 0\) and \(\sigma = 1\) the Normal distribution is called standard

The standard normal distribution with reference lines

x <- seq(-4, 4, length=100)
hx <- dnorm(x)
plot(x, hx, type="l", xlab="x value", ylab="Density", 
     main="Standard Normal Distribution")
abline(v = seq(-3,3), untf = FALSE, col = "red")

The standard normal can be thought of as just standard deviation units. For example if you go 1 to the right you’ve gone 1 standard deviation unit. The statistician usually reverts normal distributions to the standard normal by virtue of talking about standard deviations from the mean.

So if you want to know the probability that a non-standard normal, is between \(\mu - \sigma\) and \(\mu + \sigma\) - where \(\mu\) and \(\sigma\) are from it’s distribution. That would be the area between 1 standard deviation from the mean for the non-standard normal.

That is exactly the same probability area for the between minus 1 and plus one on the standard normal. So basically all standard normal distributions looks identical, the only thing that changes is the units along the axis. And then if you revert, when you’re talking about normal probabilities, to talking about normal probabilities to talking about standard deviations from the mean, all the probabilities and everything revert back to the standard normal calculations.

The area between -1 standard deviation and +1 standard deviation is \(68%\).

The area between -2 standard deviation to +2 standard deviation is \(95%\), so the area of the two tails is the residual \(5%\).

The area between -3 standard deviation to +3 standard deviation is \(99%\)

Facts about the normal density

If \(X \sim N(\mu, \sigma^2)\) then

\[ Z = \frac{X - \mu}{\sigma} \sim N(0, 1) \]

If \(Z\) is standard normal

\[ X = \mu + \sigma Z \sim N(\mu, \sigma^2) \]

More facts about the normal density

  1. Approximately \(68%\), \(95%\) and \(99%\) of the normal density lies within \(1\), \(2\) and \(3\) standard deviations from the mean, respectively

  2. \(-1.28\), \(-1.645\), \(-1.96\) and \(-2.33\) are the \(10^{th}\), \(5^{th}\), \(2.5^{th}\) and \(1^{st}\) percentiles of the standard normal distribution respectively

  3. By symmetry, \(1.28\), \(1.645\), \(1.96\) and \(2.33\) are the \(90^{th}\), \(95^{th}\), \(97.5^{th}\) and \(99^{th}\) percentiles of the standard normal distribution respectively

If the \(10%\) of the probability of a standard normal lies above \(1.28\), on a non-standard normal \(N(\mu, \sigma^2)\) the \(10%\) of the probability lies above \(\mu + 1.28\sigma\). And as above so below, the 10% of the probability of a non-standard normal lies below \(\mu - 1.28\sigma\).

One of the more important quantiles on a standard normal is \(1.96\), \(-1.96\) is the point that such that \(2,5%\) of the mass of the normal distribution lies below it. And \(+1.96\) is the point such that \(2.5%\) of the mass lies above it. This would mean that \(97.5%\) lies above \(-1.96\) and \(97.5%\) lies below \(1.96\), and so that \(95%\) lies in between.

If we are dealing with a potentially non-standard normal is \(\mu - 1.96\sigma\) and \(\mu + 1.96\sigma\)

Question

What is the \(95^{th}\) percentile of a \(N(\mu, \sigma^2)\) distribution?

Quick answer in R is:

qnorm(.95, mean = mu, sd = sd)

with the correct \(\mu\) and \(\sigma\) values plugged into that R code

But there’s another way to do it that is easy, because we have our standard normal quantiles memorized.

We know that for a standard normal at \(1.645\) the \(95%\) of the normal density lies below that point and \(5%\) above it. So for a non-standard normal the quantile has to be:

\[ \mu + 1.96\sigma \]

Another question

What is the probability that a \(N(\mu, \sigma^2)\) RV is larger than \(x\)?

Quick answer in R is:

pnorm(x, mean = mu, sd = sd, lower.tail = F)

with the correct \(\mu\) and \(\sigma\) values plugged into that R code, and remembering to use lower.tail = F, to tell R that we want the upper tail instead rather than the lower tail.

Another way to that is to conver \(X\) into how many standard deviations from the mean it is.

To do that, we just calculate the quantity:

\[ \frac{X - \mu}{\sigma} \]

And this number is simply \(X\) expressed in how many standard deviations from the mean it is. So for example, if this works out to be around 2 standard deviations from the mean, we know that should be about \(2.5%\)

A specific example

Assume that the number of daily ad clicks for a company is (approximately) normally distributed with a mean of \(1020\) and a standard deviation of \(50\). What’s the probability of getting more than \(1160\) clicks in a day?

So the quantity

\[ \frac{X - \mu}{\sigma} \]

Is equal to

\[ \frac{1160 - 1020}{50} \]

that is finally equal to

## [1] 2.8

so roughly \(\approx 3\) standard deviations from the mean. So we know that this probability has to be pretty low because is pretty far out into the tail of the normal.

We can also easily obtain that with R:

pnorm(1160, mean = 1020, sd = 50, lower.tail = F)
## [1] 0.00255513

So we get a really low probability

If we use the number of standard deviations from the mean in a standard normal distributin, the quantity calculated just above, we obtain the same number:

pnorm(2.8, mean = 0, sd = 1, lower.tail = F)
## [1] 0.00255513