0.1 Motivation

Even though I had learned the beta distribution from UIC’s Bayesian methods course and tutored it, such as setting up it as the prior distribution in conjugate distribution context. But it was easy to forget because of its dried content and too abstract. Here I try to combine the rigid theory (UC coursework’s content) and intuitive thought. By that way, I was able to ‘permenently stamp’ the concept to my brain.

  • The Beta distribution is a probability distribution on/of probabilities

  • The beta distribution describes a family of continuous probability distributions that are nonzero only on the interval (0 1).

  • For example, we can use it to model the probabilities: the Click-Through Rate of the advertisement, the batting averages, the 5-year survival chance for women with breast cancer, and so on.

0.2 Definition

A continuous random variable \(X_B \sim Beta(\alpha, \beta)\) has Beta distribution if its probability density function (PDF) is

\[ f_{X_B} (x; \alpha, \beta) = \frac{1}{B(α,β)} x^{\alpha − 1} (1−x)^{\beta − 1}, \ \ \text{for} \ 0 < x < 1. \]

where \(B(\cdot)\) is the Beta function and shape parameters \(\alpha, \beta > 0\).

0.3 Intuitive interpretation

PDF Probability as a …
Binomial \(f(x) = {n \choose x} p^x (1-p)^{n-x}\) parameter
\(\rightarrow\) the function of \(x\)
Beta \(f(p) = \frac{1}{B(α,β)} p^{\alpha − 1} (1−p)^{\beta − 1}\) random variable
\(\rightarrow\) the function of \(p\)

  • The beta distribution intuitively comes into play when we look at it in terms of numerator—\(x/p\) to the power of something multiplied by \(1-x/1-p\) to the power of something—from the lens of the binomial distribution.

  • The difference between the binomial and the beta is that the above models the number of successes (\(x\)), while the below models the probability (\(p\)) of success. In other words, the probability is a parameter in binomial; In the Beta, the probability is a random variable.

  • In this context, the shape parameters \(\alpha\) and \(\beta\) or \(\alpha-1\) as the number of successes and \(\beta-1\) as the number of failures

  • We can explore the beauty of beta distribution via the the calculator for Beta distribution—Dr. Bognar at the University of Iowa built it.

  • Beta distribution is very flexible: bell-curve (The PDF of a beta distribution is approximately normal if \(\alpha + \beta\) is large enough and \(\alpha\) & \(\beta\) are approximately equal), U-shaped (when \(\alpha\) < 1, \(\beta\) < 1) and even straight line. Here’s an graph excerpt from wikipedia.
    The very flexible of Beta distribution

0.4 Beta function

The beta function is

\[ B(x,y) = \int_0^1 t^{x−1} (1−t)^{y−1} dt = \frac{\Gamma(x) \Gamma(y)}{\Gamma(x+y)}, \]

where \(\Gamma(\cdot)\) is the Gamma function.

0.5 Gamma function

The Gamma function \(\Gamma\) is an extension of the factorial function, with its argument shifted down by 1, to real and complex numbers.

For positive integer \(n\): \[ \Gamma (n) = (n−1)! = 1 \times 2 \times 3 \times ... \times (n−1) \]

The gamma function is defined for all complex numbers except the non-positive integers by the integral:

\[ \Gamma (t) = \int_0^{\infty} x^{t-1} e^{-x} dx \]

Simplify the Beta function with the Gamma Function \(\Rightarrow\) we saw the PDF of Beta written in terms of the Gamma function. The Beta function is the ratio of the product of the Gamma function of each parameter divided by the Gamma function of the sum of the parameters (proof refered the further reading topic).

0.6 Main facts

\[ E[X_B] = \mu = \frac{\alpha}{\alpha + \beta}; \ \ V[X_B] = \sigma^2 = \frac{\alpha\beta}{(\alpha + \beta)^2(\alpha + \beta + 1)} \]

The standard uniform distribution \(\text{Unif} \ (0,1)\) is a special case of the beta distribution \(Beta \ (1,1)\), when \(\alpha = \beta = 1\).

The mode is \(\omega = \frac{\alpha − 1}{\alpha + \beta − 2}\) for \(\alpha, \beta > 1\).
The concentration is \(\kappa = \alpha + \beta\).
Definitions of \(\mu, \omega\) and \(\kappa\) can be inverted:

\[ \alpha = \mu\kappa, \beta = (1 − \mu)\kappa \]

\[ \alpha = \omega(\kappa−2)+1, \beta = (1 − \omega)(\kappa−2)+1, \ \kappa > 2. \]

Parameter \(\kappa\) is a measure of number of observations needed to change our previous belief about \(\mu\).
If \(\kappa\) is small we need only a few new observations.

Example. Concentration \(\kappa = 8\) around \(\mu = 0.5\) corresponds to \(\alpha = \mu \kappa = 4\) and \(\beta = (1 − \mu) \kappa = 4\).

Parameterization in terms of mean value and standard deviation is:

\[ \alpha = \mu [\frac{\mu (1 - \mu)}{\sigma^2} - 1]; \ \ \beta = (1 - \mu)[\frac{\mu (1 - \mu)}{\sigma^2} - 1] \]

Standard deviation is typically smaller than standard deviation of uniform distribution on \([0,1]\), i.e. \(0.28867\).

Examples.
1. For \(\mu = 0.5\), \(\sigma = 0.28867\) the shape parameters are \(\alpha = 1\), \(\beta = 1\).
2. Find shape parameters of beta distribution with \(\mu = 0.5\), \(\sigma = 0.1\).

The standard uniform distribution \(Unif \ (0,1)\) is a special case of the beta distribution \(Beta \ (1,1)\), when \(\alpha = \beta = 1\).

0.8 Plots

I plan to build an shiny app to plot beta distrition on the specification of shape parameter (in the process).

0.9 Further reading