Bayes’ theorem is a rule of probability theory that can be used to calculate the probability of an event given prior knowledge and new evidence. It is as follows:
\(P(A|B) = \frac{P(B|A)P(A)}{P(B)}\)
Notice that given \(P(B|A)\) (and of course \(P(B)\) and \(P(A)\)), you can deduce \(P(A|B)\), which may not be information you have access to otherwise.
Example: You are running a Semiconductor wafer factory, where you have a yield of 98%. You have a primary test to see if a wafer is defective, but it is 94% accurate, so it flags a defective chip as defective 94% of the time. Further, it accidentally flags a good chip as defective 5% of the time.
\(P(D) = 0.02\), \(P(G) = 0.98\)
\(P(F|D) = 0.94\), \(P(F|G) = 0.05\)
\(P(F) = (P(F|D) \times P(D)) + (P(F|G) \times P(G)) = 0.0678\)
\(P(D|F) = \frac{P(F|D) \times P(D)}{P(F)} = \frac{0.94 \times 0.02}{0.0678} \approx 0.2773\)
This means about 72% of flagged defective dies are actually false positives. Thus, you should not throw out every chip flagged defective. You should use a more expensive and accurate secondary test to narrow it down further.
This chart compares the prior probabilities to the deduced ones (after a wafer is flagged).
Bayes’ Theorem is widely used in Engineering adjacent fields. It is implicitly used every day in factories. Apart from Quality Control, it can also be used for decision making on imperfect sensors, and civil engineering.
This naturally leads to an inference engine called Bayesian Updating:
Suppose a coin’s probability of heads \(\theta\) follows a \(\text{Beta}(\alpha, \beta)\) prior. After observing \(k\) heads in \(n\) flips (a Binomial likelihood), Bayes’ theorem gives:
\[P(\theta \mid k) = \frac{P(k \mid \theta)P(\theta)}{P(k)} \;\propto\; \theta^{k}(1-\theta)^{n-k} \times \theta^{\alpha-1}(1-\theta)^{\beta-1}\]
\[\theta \mid k \;\sim\; \text{Beta}(\alpha + k,\; \beta + n - k)\]
Starting from a mild prior of \(\text{Beta}(2, 2)\) and observing \(k = 7\) heads in \(n = 10\) flips gives a final of \(\text{Beta}(9, 5)\).
This allows us to have in inference engine where we start with a prior belief, observe data, then land on a final belief.
The code below generates the prior and final density plot:
x <- seq(0, 1, length.out = 200)
b_df <- data.frame(
x = rep(x, 2),
density = c(dbeta(x, 2, 2), dbeta(x, 9, 5)),
dist = rep(c("Prior: Beta(2, 2)", "Final: Beta(9, 5)"),
each = length(x))
)
ggplot(b_df, aes(x = x, y = density, color = dist)) +
labs(x = expression(theta), y = "Density",
title = "Coin Bias Belief: Updating") +
scale_color_manual(values = c("#8C1D40", "#FFC627")) +
theme_minimal(base_size = 13) +
theme(legend.title = element_blank(), legend.position = "bottom")
Hover over the plot to view exact values. You can see how the final probability \(P(D \mid F)\) changes as the primary test’s sensitivity improves, for several levels of specificity.