Introduction

  • Bayes’ Theorem gives a rule for inverting conditional probabilities where the probability of a cause can be found given its effect.
  • Allows us to find the probability that X happens given that Y happened.
  • Created by Thomas Bayes in 1763
  • Applications:
    • Machine Learning
    • Medicine
    • Bayesian Inference
    • Genetics

The Formula

\[ P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)} \] Explanation:

  • \(P(A|B)\) is the posterior (probability of A given B)

  • \(P(B|A)\) is the likelihood (probability of B given A)

  • \(P(A)\) is the prior probability, probabilty of A

  • \(P(B)\) is the marginal probability, probabilty of B

Medical Testing:

A disease affects 22% of the population. A test is 68% accurate.

Question: If you test positive, what’s the probability you actually have the disease?

Given:

  • \(P(\text{Disease}) = 0.22\)

  • \(P(\text{Positive}|\text{Disease}) = 0.68\)

  • \(P(\text{Negative}|\text{No Disease}) = 0.68\)

Find: \(P(\text{Disease}|\text{Positive})\)

Medical Testing: Solution

prior <- 0.22
sensitivity <- 0.68
false_positive <- 0.32

posterior <- (sensitivity * prior) / 
  (sensitivity * prior + false_positive * (1 - prior))

cat("Probability of disease given positive test:", 
    round(posterior, 4), "\n")
## Probability of disease given positive test: 0.3747
  • With an 68% accurate test, a positive result means about 37% chance of having the disease

Medical Testing Graph

Code for 3D Surface

prior_vals <- seq(0.01, 0.99, length.out = 50)
sensitivity_vals <- seq(0.01, 0.99, length.out = 50)

false_pos_rate <- 0.32

posterior_matrix <- matrix(NA, nrow = length(prior_vals), 
                          ncol = length(sensitivity_vals))

for(i in 1:length(prior_vals)) {
  for(j in 1:length(sensitivity_vals)) {
    p_disease <- prior_vals[i]
    p_pos_given_disease <- sensitivity_vals[j]
    
    posterior_matrix[i, j] <- (p_pos_given_disease * p_disease) / 
      (p_pos_given_disease * p_disease + false_pos_rate * (1 - p_disease))
  }
}

plot_ly(x = ~sensitivity_vals, y = ~prior_vals, z = ~posterior_matrix) %>%
  add_surface(colorscale = "Viridis") %>%
  layout(title = "Posterior Probability P(Disease|Positive Test)",
         scene = list(
           xaxis = list(title = "Sensitivity P(+|Disease)"),
           yaxis = list(title = "Prior P(Disease)"),
           zaxis = list(title = "Posterior P(Disease|+)")
         ))

3D Probability Visualization

Conveyor Belt Analysis

A robot says a part coming off a conveyor belt is defective. What is the probability it actually is?

Given:

  • 35% of parts are defective: \(P(\text{Defective}) = 0.35\)
  • True Positive: \(P(\text{Robot Defective}|\text{Defective}) = 0.80\)
  • False Positive: \(P(\text{Robot Defective}|\text{Not Defective}) = 0.08\)

Find: \(P(\text{Defective}|\text{Robot Defective})\)

p_defective <- 0.35
p_good <- 0.65
p_flag_defective <- 0.80
p_flag_good <- 0.08
p_defective_given_flag <- (p_flag_defective * p_defective) / 
  (p_flag_defective * p_defective + p_flag_good * p_good)
cat("P(Defective | Robot Defective):", round(p_defective_given_flag, 4) * 100, "%")
## P(Defective | Robot Defective): 84.34 %

Conveyor Belt Analysis Graph

Conclusion

  • Bayes’ Theorem allows us to invert conditional probabilities

  • Finds the probability of X given that Y happened

  • Useful in a wide variety of applications including medicine, computer science, and business

  • Even highly accurate tests can produce misleading results when testing for low probability occurences