One of the first conditional probability paradoxes was provided by Bertrand. It is called the Box Paradox. A cabinet has three drawers. In the first drawer there are two gold balls, in the second drawer there are two silver balls, and in the third drawer there is one silver and one gold ball. A drawer is picked at random and a ball chosen at random from the two balls in the drawer. Given that a gold ball was drawn, what is the probability that the drawer with the two gold balls was chosen?
Let the following definitions hold:
As the drawers are picked at random, and there is nothing identifying about the drawers, we may assume \(P(GG) = P(GS) = P(SS) = \frac{1}{3}\).
As the ball from the drawer is picked at random, it is clear that the following conditional probabilities must hold true: \[ \begin{aligned} P(g|GG) &= 1\\ P(g|SS) &= 0\\ P(g|GS) &= 0.5 \end{aligned} \]
The question can now be restated as calculating the probability \(P(GG|g)\). To solve this, we can use Bayes rule: \[ \begin{aligned} P(GG|g) &= \frac{P(g|GG)P(GG)}{P(g|GG)P(GG) + P(g|SS)P(SS) + P(g|GS)P(GS)}\\ &= \frac{1\cdot\frac{1}{3}}{1\cdot\frac{1}{3} + 0\cdot\frac{1}{3} + \frac{1}{2}\cdot\frac{1}{3}}\\ &= \left(\frac{\frac{1}{3}}{\frac{1}{2}}\right)\\ &= \mathbb{\frac{2}{3}} \end{aligned} \]
By selecting a gold ball, the possibility of the drawer being the two-silver one is eliminated. Therefore, there are only two drawers in play. So it would seem that the probability should be 50%. However, by eliminating the two-silver drawer, the universe of possibilities had contracted to one with only four balls, three of which are gold and only one is silver. We have removed one gold ball. Therefore there are only three balls left, and two of them are gold! Moreover, since our new universe only has the choice between a two-gold and one-gold drawer, and a gold ball was picked, it has twice the probability of coming from a drawer with two such balls than with only one.
Although, if you think about it, this is probably just a verbal restatement of the Bayes Theorem calculations done above (e.g. note the 0 in the denominator for the \(SS\) case).