Testing Bayesian Concepts in R: using the Beta-Bernoulli Conjugate Priors to compute the Posterior Probabilities
Sandipan Dey
10 August 2016
In this article, the Beta-Bernoulli Conjugate Priors will be used to compute the Posterior Probabilities with Coin Tossing Experiment.
First, a uniform \(\beta(\alpha,\beta)\) prior is chosen with \(\alpha=1,\;\beta=1\) to model the unknown probability of success \(\Theta\) variable for the coin (assuming that all the probabilities in \([0,1]\) are equally likely), so that \(\Theta \sim \beta(1,1)\).
Then a few trials of the coin tossing are conducted to collect the data and update our prior belief about \(\Theta\) from the likelihood, which can be modeled as i.i.d. Bernoulli random variables, \(Y_i \sim B(\alpha, \beta),\; \forall{i}\).
The posterior probability distribution is also a Beta distribution as shown in the figure below from the videos of professor Herbert Lee.
Then the recursive Bayesian updates and the prior and posterior hyper-parameters and the means are updated with each trial. Also, the frequentist’s MLE and 95% confidence interval are computed, along with the Bayesian 95% credible interval.
The following animation shows the results of simulation of a random coin tossing experiment with 20 trials, starting with the uniform prior \(\beta(1,1)\).
- The next animation shows the same results starting with an improper prior \(\beta(0,0)\).
- The next animation shows the same results starting with the Jeffreys prior \(\beta(\frac{1}{2},\frac{1}{2}) \propto \theta^{-\frac{1}{2}}(1-\theta)^{-\frac{1}{2}}\).
