4.5 - Normal distribution

Definition

  • A random variable \(Y\) is said to have a \(N(\mu,\sigma^2)\) distribution if, for \(\sigma>0\) and \(-\infty < \mu < \infty\), the pdf of \(Y\) is:

\[ f(y) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(y-\mu)^2}{2\sigma^2}}, -\infty < y < \infty\]

Normal in R

  • dnorm(y, mean, sd): evaluates \(f(y)\)
  • pnorm(y, mean, sd): evaluates \(F(y)\) (which does not have closed form)
  • qnorm(p, mean, sd): finds the \(p^{th}\) percentile
  • rnorm(N, mean, sd): simulates N random normal random variables

Plots of pdf

Our old friend, the bell curve:

library(tidyverse)
(ggplot() 
  + geom_function(fun = \(y) dnorm(y, mean = 0, sd = 1), 
                  linewidth = 1, 
                  aes(col='1')
                  ) 
  + geom_function(fun = \(y) dnorm(y, mean = 0, sd = 2), 
                  linewidth = 1,
                  aes(col='2')
                  )
  + geom_function(fun = \(y) dnorm(y, mean = 0, sd = 3), 
                  linewidth = 1,
                  aes(col='3')
                  )
  +xlim(-8,8) 
  + labs(x = 'y',y = 'f(y)', color=expression(sigma))
  + theme_classic()
)

Standard normal

  • An important special case of the normal is the \(N(0,1)\), known as the standard normal distribution.
  • Letting \(Z = (Y-\mu)/\sigma\), which measures the number of standard deviations \(Y\) is from the mean, we have:

\[ f(z) = \frac{1}{\sqrt{2\pi}} e^{-\frac{z^2}{2}},\ -\infty < z < \infty\]

  • Historically important for finding normal probabilities using normal tables
    • E.g., \(P(Y\leq 3) = P\left(\frac{Y-\mu}{\sigma} \leq \frac{3-\mu}{\sigma}\right) = P\left(Z \leq \frac{3-\mu}{\sigma}\right)\)
  • Also useful for simulating any \(Y\sim N(\mu, \sigma^2)\) by simulating \(Z\sim N(0,1)\) and using relationship \(Y = \sigma Z + \mu\).

The pdf does integrate to 1

\[\int_{-\infty}^\infty f(y) dy = \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(y-\mu)^2}{2\sigma^2}} dy = 1\]

First step: change of variable, let \(z = (y-\mu)/\sigma\implies dz = \frac{1}{\sigma}dy\).

\[\int_{-\infty}^\infty f(y) dy = \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}e^{-z^2/2} dz\]

So it suffices to show that the standard normal integrates to 1.

2 approaches:

  1. Show \(\left(\int_{-\infty}^\infty f(z) dz\right)^2 = 1\) using double integration and polar coordinate transformation;
  2. Use the symmetry of the standard normal: \(\int_{-\infty}^\infty f(z) dz = 2 \int_0^\infty f(z) dz\), change of variable and use Gamma properties.

Mean

If \(Y\sim N(\mu,\sigma^2)\) then \(E(Y) = \mu\).

Proof:

\[ E(Y) = \int_{-\infty}^{\infty} y \cdot \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left( -\frac{(y-\mu)^2}{2\sigma^2} \right) dy \]

Let \(z = (y-\mu)/\sigma \implies dz = \frac{dy}{\sigma}\)

\[ \implies E(Y)= \int_{-\infty}^{\infty} (z\sigma+\mu) \, \frac{1}{\sqrt{2\pi}} \exp\!\left( -\frac{z^2}{2} \right) dz \]

\[ = \sigma\int_{-\infty}^{\infty} z\cdot\, \frac{1}{\sqrt{2\pi}} \exp\!\left( -\frac{z^2}{2} \right) dz + \mu \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}} \exp\!\left( -\frac{z^2}{2} \right) dz \]

\[ =-\frac{\sigma}{\sqrt{2\pi}}\exp\left(-z^2/2\right)\Big|_{-\infty}^\infty + \mu\cdot 1 \]

\[ = 0 + \mu = \mu \]

Variance

If \(Y\sim N(\mu,\sigma^2)\) then \(Var(Y) = \sigma^2\).

Proof: practice!

MGF

\(Y\sim N(\mu,\sigma^2) \iff M_Y(t) = e^{\mu t + \frac{\sigma^2 t^2}{2}}\)

Proof outline:

  • Start with definition;
  • FOIL the \((y-\mu)^2\) part of the pdf;
  • Note that \(e^{ty} = (e^t)^y\), group with like terms in the pdf;
  • Complete the square, factor, re-parameterize the normal pdf.