2024-03-22

Bayesian Statistics

  • Definition:
    • A framework for drawing statistical conclusions based on the ideas of Bayesian probability is known as Bayesian statistics.

Bayes’ Theorem

  • Bayes’ Theorem:
    • illustrates in mathematical terms how previous convictions or odds are revised in light of fresh information.

    • Formula: \[ P(A|B) = \frac{P(B|A) \times P(A)}{P(B)} \]

Prior, Likelihood, and Posterior

  • Prior Probability:
    • Represents one’s initial opinion of the likelihood of an event before any supporting evidence is seen.
  • Likelihood:
    • Explains the likelihood of seeing the data for a particular parameter value.
  • Posterior Probability:
    • Represents the updated probability of an event after considering the observed data.

Bayesian Inference

  • Process:
    1. Specify the prior distribution.
    2. Update the prior using Bayes’ Theorem to obtain the posterior distribution.
    3. Make inferences based on the posterior distribution.

Example

  • Problem Statement:
    • A factory produces Matches, and we want to estimate the probability of a match being defective.
  • Prior Distribution:
    • We select a previous distribution that is consistent with our prior view that 10% of matches are faulty.

Likelihood Function

  • Definition:
    • The probability of observing the data given various parameter values is described by the likelihood function.
  • Example:
    • If we observe 3 defective matches out of 20, the likelihood function would be a binomial distribution with parameters n=20 and p representing the probability of defectiveness.

Posterior Distribution

  • Update Using Bayes’ Theorem:
    • The posterior distribution can be obtained by combining the likelihood function and the previous distribution.
  • Example:
    • We use the observed data to revise our previous estimate of the likelihood of defectiveness.

Bayesian Decision Making

  • Utility Theory:
    • Utility theory is used in Bayesian decision making to help us make judgments based on posterior distributions and their implications.
  • Example:
    • Taking into account the expenses of both needless inspections and faulty matches, we may opt for the course of action that maximizes predicted usefulness.

Slide with R Code 1

library(plotly)
## Warning: package 'plotly' was built under R version 4.3.3
## Loading required package: ggplot2
## 
## Attaching package: 'plotly'
## The following object is masked from 'package:ggplot2':
## 
##     last_plot
## The following object is masked from 'package:stats':
## 
##     filter
## The following object is masked from 'package:graphics':
## 
##     layout

Slide with R Code 2

prior_probs <- seq(0.01, 0.1, length.out = 20)

test_accuracies <- seq(0.8, 1, length.out = 20)

grid <- expand.grid(P_D = prior_probs, P_T_given_D = test_accuracies)
posterior_probs <- (grid$P_T_given_D * grid$P_D) / (grid$P_T_given_D * grid$P_D + 0.05 * (1 - grid$P_D))

plot <- plot_ly(grid, x = ~P_D, y = ~P_T_given_D, z = ~posterior_probs, type = "scatter3d", mode = "markers", 
                marker = list(size = 5))

plot <- plot %>% layout(scene = list(xaxis = list(title = "Prior Probability of Having Disease"),
                                     yaxis = list(title = "Test Accuracy"),
                                     zaxis = list(title = "Posterior Probability of Having Disease")))

Slide for Plotly

plot

Slide for ggplot 1

ggplot(data.frame(prior_prob = prior_probs, posterior_prob = posterior_probs), aes(x = prior_prob, y = posterior_prob)) +geom_line() +labs(title = "Posterior Probability vs Prior Probability",x = "Prior Probability of Having Disease", y = "Posterior Probability of Having Disease Given Positive Test Result")

Slide for ggplot 2

ggplot(data.frame(test_accuracy = test_accuracies, posterior_prob = posterior_probs), aes(x = test_accuracy, y = posterior_prob)) + geom_line() + labs(title = "Posterior Probability vs Test Accuracy", x = "Test Accuracy", y = "Posterior Probability of Having Disease Given Positive Test Result")