4.4 - Gamma distribution

Definition

A random variable \(Y\) is said to have a gamma distribution with shape parameter \(\alpha >0\) and rate parameter \(\lambda>0\), and we say \(Y\sim GAM(\alpha, \lambda)\), if and only if the density function of \(Y\) is:

\[f(y) = \left\{\begin{array} {ll} \frac{\lambda^\alpha}{\Gamma(\alpha)} y^{\alpha-1}e^{-y\lambda} & y > 0 \\ 0 & otherwise\\ \end{array}\right.\]

This definition is more conducive to relating the Gamma to Poisson processes.

Like the exponential, the gamma is often parameterized with scale parameter \(\beta = 1/\lambda\) in which case \(Y\sim GAM(\alpha,\beta)\) if and only if:

\[f(y) = \left\{\begin{array} {ll} \frac{1}{\Gamma(\alpha)\beta^\alpha} y^{\alpha-1}e^{-y/\beta} & y > 0 \\ 0 & otherwise\\ \end{array}\right.\]

This is the definition used in Wackerly et al.

Gamma in `R`

dgamma(y, shape, rate = 1, scale = 1/rate): evaluates \(f(y)\)
pgamma(y, shape, rate = 1, scale = 1/rate): evaluates \(F(y)\)
qgamma(p, shape, rate = 1, scale = 1/rate): evaluates \(F^{-1}(p)\)
rgamma(N, shape, rate = 1, scale = 1/rate): simulates N realizations of \(GAM(\alpha,\lambda)\) data

Plots of pdf

The gamma distribution is right-skewed, but unlike the exponential (which has an asymptote at \(y=0\)) the gamma may have a “hump” controlled by the shape parameter \(\alpha\):

library(tidyverse)
(ggplot() 
  + geom_function(fun = \(y) dgamma(y, shape = 1, rate = 1), linewidth = 1, aes(col='1')) 
  + geom_function(fun = \(y) dgamma(y, shape = 2, rate = 1), linewidth = 1,aes(col='2'))
  + geom_function(fun = \(y) dgamma(y, shape = 3, rate = 1), linewidth = 1,aes(col='3'))
  + xlim(c(0, 8)) 
  + theme_classic(base_size = 16)
  + labs(x='y',y='f(y)',title='Gamma pdfs with common rate parameter')
  + scale_color_discrete(name=expression(alpha))
)

The Gamma function

The gamma function \(\Gamma(\alpha)\) is defined as follows, for \(\alpha > 0\):

\[\Gamma(\alpha) = \int_0^\infty t^{\alpha-1}e^{-t} dt\]

Properties:

\(\Gamma(1) = 1\)
\(\Gamma(\alpha) = (\alpha-1) \Gamma(\alpha-1)\) (recursive property)
- Corollary: if \(\alpha \in \{2,3,4,...\}\), \(\Gamma(\alpha) = (\alpha-1)!\)
\(\Gamma(1/2) = \sqrt{\pi}\)

Proof of Property 1

\[\Gamma(1) = \int_0^\infty t^{1-1}e^{-t}dt = \int_0^\infty e^{-t}dt\]

\[ = -e^{-t}\Big|_0^\infty = 0--1 = 1\]

Proof of Property 2

\[\Gamma(\alpha) = \int_0^\infty t^{\alpha-1}e^{-t} dt= \int_0^\infty u\, dv = uv\big|_0^\infty-\int_0^\infty v\, du\]

\(u = t^{\alpha-1}\implies du = (\alpha-1)t^{\alpha-2} dt\)
\(dv = e^{-t}\ dt\implies v = -e^{-t}\)

\[\implies \Gamma(\alpha) = -t^{\alpha-1}e^{-t}\Big|_0^\infty+\int_0^\infty(\alpha-1)t^{\alpha-2}e^{-t}\ dt\]

\[=-\frac{t^{\alpha-1}}{e^{t}}\Big|_0^\infty+(\alpha-1)\int_0^\infty t^{(\alpha-1)-1}e^{-t}\ dt\]

\[ = \left(-0--\frac{0}{1}\right) + (\alpha-1)\Gamma(\alpha-1)\]

Proof of Property 3

\[\Gamma(1/2) = \int_0^\infty t^{1/2-1} e^{-t}\ dt=\int_0^\infty t^{-1/2} e^{-t}\ dt\] \[t = u^2, dt = 2u\, du\]

\[\implies \Gamma(1/2) = \int_0^\infty u^{-1}e^{-u^2}\cdot 2u\, du= 2\int_0^\infty e^{-u^2}\, du\]

\[\Gamma(1/2)^2 = \left(2\int_0^\infty e^{-u^2}\ du\right)\left(2\int_0^\infty e^{-v^2}\ dv\right) = 4\int_0^\infty \int_0^\infty e^{-(u^2+v^2)}\, du\, dv\]

Polar coordinate transformation review

\[\int \int f(u,v)\, du\, dv = \int \int f(r \cos(\theta), r\sin(\theta))\, r\, dr\, d\theta\]

\[ 0 < u < \infty, 0 < v < \infty \implies 0 < r < \infty, 0 \leq \theta \leq \frac{\pi}{2}\]

Applying the polar coordinate transformation

\[\Gamma(1/2)^2 = 4\int_0^\infty \int_0^\infty e^{-(u^2+v^2)}\, du\, dv = 4\int_0^{\pi/2} \int_0^\infty e^{-\left(r^2 \cos^2(\theta) + r^2 \sin^2(\theta)\right)}\, r\,dr\,d\theta\]

\[= 4\int_0^{\pi/2} \int_0^\infty e^{-r^2 \left(\cos^2(\theta)+\sin^2(\theta)\right)}\, r\,dr\,d\theta= 4\int_0^{\pi/2} \int_0^\infty e^{-r^2 }\, r\,dr\,d\theta\]

\[= 2\int_0^{\pi/2} - e^{-r^2} \Big|_0^\infty\,d\theta= 2\int_0^{\pi/2}\, d\theta\]

\[ = 2\theta\Big|_0^{\pi/2} = \pi\]

\[ \therefore\, \Gamma(1/2) = \sqrt{\pi}\]

Gamma kernel

The “\(y\) only” portion of the gamma pdf is called a kernel:

\[k(y) = y^{\alpha-1}e^{-\lambda y}\]

We will use properties of the \(\Gamma(\alpha)\) function to show the kernel integrates to \(\Gamma(\alpha)/\lambda^\alpha\):

\[\int_0^\infty y^{\alpha-1}e^{-\lambda y} dy = \frac{\Gamma(\alpha)}{\lambda^\alpha}\]

Proof:

Let \(u = \lambda y \implies du = \lambda\ dy\). Then:

\[\int_0^\infty y^{\alpha-1}e^{-\lambda y} dy = \int_0^\infty \left(\frac{u}{\lambda}\right)^{\alpha-1}e^{-u}\frac{1}{\lambda} dy\]

\[ = \frac{1}{\lambda^a}\int_0^\infty u^{\alpha-1} e^{-u}du = \frac{\Gamma(\alpha)}{\lambda^\alpha}\]

Normalizing constant

\(\frac{\lambda^\alpha}{\Gamma(\alpha)}\) is called a normalizing constant, as it is the constant that, when multiplied by the kernel, results in an integral = 1:

\[\int_0^\infty y^{\alpha-1}e^{-\lambda y} dy = \frac{\Gamma(\alpha)}{\lambda^\alpha} \implies \int_0^\infty \frac{\lambda^\alpha}{\Gamma(\alpha)} y^{\alpha-1}e^{-\lambda y}\,dy =1\]

Use this kernel/normalizing constant relationship often!! (And not just for the gamma!)

Mean

If \(Y\sim GAM(\alpha, \lambda)\) then \(E(Y) = \frac{\alpha}{\lambda}\).

Proof:

\[E(Y) = \int_0^\infty y\cdot \frac{\lambda^\alpha}{\Gamma(\alpha)} y^{\alpha-1}e^{-\lambda y}\,dy = \frac{\lambda^\alpha}{\Gamma(\alpha)}\int_0^\infty y^{\alpha}e^{-\lambda y}\,dy\]

\[= \frac{\lambda^\alpha}{\Gamma(\alpha)}\int_0^\infty y^{(\alpha+1)-1}e^{-\lambda y}\,dy\]

Note \(y^{(\alpha+1)-1}e^{-\lambda y}\) is the kernel of a \(GAM(\alpha^*=\alpha+1, \lambda)\) density!

\[\implies E(Y) = \frac{\lambda^\alpha}{\Gamma(\alpha)} \frac{\Gamma(\alpha+1)}{\lambda^{\alpha+1}} = \frac{\lambda^\alpha}{\Gamma(\alpha)} \frac{\alpha\Gamma(\alpha)}{\lambda^{\alpha+1}}=\frac{\alpha}{\lambda}\]

Other properties

\(\lambda\) parameterization

\(f(y) = \frac{\lambda^\alpha}{\Gamma(\alpha)} y^{\alpha-1}e^{-y\lambda}, y>0\)
\(E(Y) = \frac{\alpha}{\lambda}\)
\(Var(Y) = \frac{\alpha}{\lambda^2}\)
\(M_Y(t) =\left(\frac{\lambda}{\lambda-t}\right)^\alpha, t<\lambda\)

\(\beta = 1/\lambda\) parameterization

\(f(y) = \frac{1}{\Gamma(\alpha)\beta^\alpha} y^{\alpha-1}e^{-y/\beta}, y>0\)
\(E(Y) = \alpha\beta\)
\(Var(Y) = \alpha \beta^2\)
\(M_Y(t) =\left(\frac{1}{1-\beta t}\right)^\alpha, t<1/\beta\)

Special cases

\(Y\sim GAM(1,\lambda) \implies Y \sim EXP(\lambda)\)
\(Y\sim GAM(\alpha=p/2, \lambda = 1/2) \implies Y \sim \chi^2_p\)

Poisson process revisited

Recall Poisson process, a scenario where arrivals occur randomly along continuous space or time.

Let \(\lambda\) represent the average number of arrivals per space/time interval.
Let \(X\) be the number of occurrences in space/time interval of length \(z\). Then \(X\sim POI(z\lambda)\).

Define \(Y\) as the time until the \(r^{th}\) occurrence.
- \(Y\) is small with high probability:
  - when \(\lambda\) is large
  - when \(r\) is small
- \(Y\) is large with high probability:
  - when \(\lambda\) is small
  - when \(r\) is large
Then \(Y\sim GAM(r, \lambda)\)!