2025-03-16

Introduction to Bayes’ Theorem

Bayes’ Theorem is a fundamental principle in probability theory that allows us to update our beliefs based on new evidence.

The formula is:

\[ P(A|B) = \frac{P(B|A) P(A)}{P(B)} \]

Where: - \(P(A|B)\) = Posterior probability (probability of event A given that B has occurred) - \(P(B|A)\) = Likelihood (probability of event B occurring given A is true) - \(P(A)\) = Prior probability (initial belief about A before observing B) - \(P(B)\) = Marginal probability (total probability of B occurring)

Real-World Applications of Bayes’ Theorem

Bayes’ Theorem is widely used across various fields:

  • Medical Testing: Helps determine the probability of having a disease given a positive test result, accounting for false positives and false negatives.
  • Spam Filtering: Email classifiers use Bayes’ Theorem to calculate the likelihood that an email is spam based on its content and past data.
  • Machine Learning & AI: Forms the basis for probabilistic models, including Naïve Bayes classifiers, which are used for text classification, recommendation systems, and more.
  • Risk Assessment: Used in finance and insurance to update risk probabilities based on new information.

Example Calculation

A disease affects 1% of a population. A test is:

  • 90% accurate for true positives
  • 95% accurate for true negatives

Question: What is the probability that someone who tests positive actually has the disease?

Using Bayes’ Theorem:

\[ P(Disease | Positive) = \frac{P(Positive | Disease) P(Disease)}{P(Positive)} \]

Substituting values:

\[ P(Disease | Positive) = \frac{(0.90 \times 0.01)}{(0.90 \times 0.01) + (0.05 \times 0.99)} \]

\[ = \frac{0.009}{0.009 + 0.0495} = \frac{0.009}{0.0585} \approx 0.154 \]

So: the probability that a person who tests positive truly has the disease is 15.4%.

Visualizing Bayes’ Theorem

library(ggplot2)
data <- data.frame(
  Category = c("True Positives", "False Positives"),
  Probability = c(0.009, 0.0495)
)
ggplot(data, aes(x=Category, y=Probability, fill=Category)) +
  geom_bar(stat="identity") +
  theme_minimal() +
  ggtitle("Bayes' Theorem Visualization")

3D Visualization of Bayes Theorem

The axes represent: - X-axis: Prior Probability \(P(A)\) - Y-axis: Likelihood \(P(B|A)\) - Z-axis: Posterior Probability \(P(A|B)\)

Another Bayes Theorem Example

This graph illustrates the probability of actual outcomes based on test results: - X-axis: Test result (Positive or Negative) - Y-axis: Probability of outcome - Colors: Distinguish between individuals who actually have the disease and those who do not.

Computing Bayes’ Theorem

# The probability definitions
p_disease <- 0.01
p_positive_given_disease <- 0.90
p_positive_given_no_disease <- 0.05
p_no_disease <- 1 - p_disease

# Computing Bayes' Theorem
p_positive <- (p_positive_given_disease * p_disease) + 
  (p_positive_given_no_disease * p_no_disease)
p_disease_given_positive <- 
  (p_positive_given_disease * p_disease) / p_positive

# The result
p_disease_given_positive
## [1] 0.1538462

Conclusion

  • Bayes’ Theorem is a powerful tool for updating probabilities.
  • Used in medical testing, spam filtering, and AI.
  • It is also extensively used in machine learning for Bayesian inference and Naïve Bayes classifiers.