Probability Distribution

Discrete Distribution

Bernoulli distribution

A Bernoulli trial is a simple random experiment with two outcomes: success \((1)\) or failure \((0)\). The success probability is \(p\), so the failure probability is \(q=1-p\). A Bernoulli random variable \(X\) describes this: \[\begin{eqnarray*} X = \left\{ \begin{array}{ll} 1 & \mbox{success - probability $p$} \\ 0 & \mbox{failure - probability $q=1-p$} \end{array} \right. \end{eqnarray*}\]

The probability mass function (pmf) is given by: \[ p_X (x) = p^x (1-p)^{1-x} \qquad x=0,1 \] Note that if \(x =1\), \(p_X (1) = p^1 q^0 = p\) and if \(x=0\), \(p_X (0) = p^0 q^1 = q\).

The culmulative distribution fuction (cdf) is given by: \[ F_X(x) = \left\{ \begin{array}{ll} 0 \qquad & \mbox{if } x <0 \\ 1-p \qquad & \mbox{if } 0 \leq x <1 \\ 1 \qquad & \mbox{if } x \geq 1. \end{array} \right. \]

We also have the expectation and variance: \[ E[X]= p,\qquad \mbox{var}(X) = pq. \]

Binomial Distritbtion

Consider \(n\) independent Bernoulli trials, each with success probability \(p\). Let \(X\) be the total number of successes. Then \(X\) has a Binomial distribution, written

\[ X \sim {\rm Bin}(n,p), \; \; \mbox{or } X \sim {\rm B}(n,p) \]

and for \(k = 0, 1, \ldots, n\),

\[ P(k) = P(X = k) = \binom{n}{k} p^{k}(1-p)^{n-k} . \] For the binomial random variable \(E[X] = n p\) and \(\mbox{var}(X) = n p(1-p)\). ## Geometric Distribution

A Geometric distribution models the number of trials needed to get the first success in a series of Bernoulli trials. The probability mass function (pmf) is given by:

\[ p_X(x) = (1-p)^{x-1} p \quad \text{for} \quad x = 1, 2, 3, \dots \]

where \(p\) is the probability of success in each trial.

The cumulative distribution function (cdf) is given by:

\[ F_X(x) = 1 - (1-p)^x \quad \text{for} \quad x = 1, 2, 3, \dots \]

The expectation and variance are:

\[ E[X] = \frac{1}{p}, \quad \text{var}(X) = \frac{1-p}{p^2}. \]

## Negative Binomial Distribution

The Negative Binomial distribution is the number of trials needed to achieve a specified number of successes. The probability mass function (pmf) is given by:

\[ p_X(x) = \binom{x-1}{r-1} p^r (1-p)^{x-r} \quad \text{for} \quad x = r, r+1, r+2, \dots \]

where \(p\) is the probability of success and \(r\) is the number of successes.

The cumulative distribution function (cdf) is given by:

\[ F_X(x) = 1 - \sum_{k=r}^{x} \binom{k-1}{r-1} p^r (1-p)^{k-r}. \]

The expectation and variance are:

\[ E[X] = \frac{r}{p}, \quad \text{var}(X) = \frac{r(1-p)}{p^2}. \]

## Discrete Uniform Distribution

A Discrete Uniform distribution is a distribution where all outcomes are equally likely. The probability mass function (pmf) is given by:

\[ p_X(x) = \frac{1}{n} \quad \text{for} \quad x = 1, 2, \dots, n \]

where \(n\) is the number of possible outcomes.

The cumulative distribution function (cdf) is given by:

\[ F_X(x) = \frac{x}{n} \quad \text{for} \quad x = 1, 2, \dots, n \]

The expectation and variance are:

\[ E[X] = \frac{n+1}{2}, \quad \text{var}(X) = \frac{n^2 - 1}{12}. \]

## Poisson Distribution

A Poisson distribution models the number of events occurring in a fixed interval of time or space, given a known average rate of occurrence. The probability mass function (pmf) is given by:

\[ p_X(x) = \frac{\lambda^x e^{-\lambda}}{x!} \quad \text{for} \quad x = 0, 1, 2, \dots \]

where \(\lambda\) is the average rate of occurrence.

The cumulative distribution function (cdf) is:

\[ F_X(x) = P(X \leq x) = \sum_{k=0}^{x} \frac{\lambda^k e^{-\lambda}}{k!}. \]

The expectation and variance are:

\[ E[X] = \lambda, \quad \text{var}(X) = \lambda. \]

## Hypergeometric Distribution

A Hypergeometric distribution models the number of successes in a sample of size \(n\) drawn without replacement from a population of size \(N\) containing \(K\) successes. The probability mass function (pmf) is given by:

\[ p_X(x) = \frac{\binom{K}{x} \binom{N-K}{n-x}}{\binom{N}{n}} \quad \text{for} \quad x = \max(0, n-(N-K)), \dots, \min(n, K) \]

The cumulative distribution function (cdf) is:

\[ F_X(x) = P(X \leq x) = \sum_{k=\max(0, n-(N-K))}^{x} \frac{\binom{K}{k} \binom{N-K}{n-k}}{\binom{N}{n}}. \]

The expectation and variance are:

\[ E[X] = \frac{nK}{N}, \quad \text{var}(X) = \frac{nK(N-K)(N-n)}{N^2(N-1)}. \]

# Continuous Distribution ## Uniform Distribution

A Uniform distribution is a continuous distribution where all outcomes are equally likely within a specified range. The probability density function (pdf) is given by:

\[ f_X(x) = \frac{1}{b-a} \quad \text{for} \quad a \leq x \leq b \]

where \(a\) and \(b\) are the lower and upper bounds of the distribution.

The cumulative distribution function (cdf) is:

\[ F_X(x) = \frac{x - a}{b - a} \quad \text{for} \quad a \leq x \leq b \]

The expectation and variance are:

\[ E[X] = \frac{a + b}{2}, \quad \text{var}(X) = \frac{(b - a)^2}{12}. \]

## Exponential Distribution

The Exponential distribution models the time between events in a Poisson process. The probability density function (pdf) is given by:

\[ f_X(x) = \lambda e^{-\lambda x} \quad \text{for} \quad x \geq 0 \]

where \(\lambda\) is the rate parameter.

The cumulative distribution function (cdf) is:

\[ F_X(x) = 1 - e^{-\lambda x} \quad \text{for} \quad x \geq 0 \]

The expectation and variance are:

\[ E[X] = \frac{1}{\lambda}, \quad \text{var}(X) = \frac{1}{\lambda^2}. \]

## Gamma Distribution

The Gamma distribution is a two-parameter family of continuous probability distributions. The probability density function (pdf) is given by:

\[ f_X(x) = \frac{x^{k-1} e^{-\frac{x}{\theta}}}{\Gamma(k) \theta^k} \quad \text{for} \quad x \geq 0 \]

where \(k\) is the shape parameter, \(\theta\) is the scale parameter, and \(\Gamma(k)\) is the Gamma function.

The cumulative distribution function (cdf) is:

\[ F_X(x) = \int_0^x \frac{t^{k-1} e^{-\frac{t}{\theta}}}{\Gamma(k) \theta^k} dt \]

The expectation and variance are:

\[ E[X] = k\theta, \quad \text{var}(X) = k\theta^2. \]

## Beta Distribution

The Beta distribution is a continuous probability distribution defined on the interval \([0, 1]\). The probability density function (pdf) is given by:

\[ f_X(x) = \frac{x^{\alpha-1} (1-x)^{\beta-1}}{B(\alpha, \beta)} \quad \text{for} \quad 0 \leq x \leq 1 \]

where \(\alpha\) and \(\beta\) are shape parameters, and \(B(\alpha, \beta)\) is the Beta function.

The cumulative distribution function (cdf) is:

\[ F_X(x) = \int_0^x \frac{t^{\alpha-1} (1-t)^{\beta-1}}{B(\alpha, \beta)} dt \]

The expectation and variance are:

\[ E[X] = \frac{\alpha}{\alpha + \beta}, \quad \text{var}(X) = \frac{\alpha \beta}{(\alpha + \beta)^2 (\alpha + \beta + 1)}. \]

## Normal Distribution

The Normal distribution is a continuous probability distribution characterized by its bell-shaped curve. The probability density function (pdf) is given by:

\[ f_X(x) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} \quad \text{for} \quad -\infty < x < \infty \]

where \(\mu\) is the mean and \(\sigma\) is the standard deviation.

The cumulative distribution function (cdf) is:

\[ F_X(x) = \frac{1}{2} \left[ 1 + \text{erf} \left( \frac{x - \mu}{\sqrt{2} \sigma} \right) \right] \]

where \(\text{erf}\) is the error function.

The expectation and variance are:

\[ E[X] = \mu, \quad \text{var}(X) = \sigma^2. \]