Probability
September 20, 2023
Probability Basics
Definition: A random trial is a process or experiment that has two or more possible outcomes whose occurrence cannot be predicted with certainty.
Definition: An event is any potential subset of all the possible outcomes of a random trial.
Definition: The probability of an event is the proportion of times the event would occur if we repeated a random trial over and over again under the same conditions. Probability ranges between zero and one.
Random sampling as a random trial
Instead of events, we have values of random variables.
Parasitic wasps (yuck!): Two categorical variables - Parasitized or not; sex of laid egg (M or F)
Conditional Probabilities
Definition: The conditional probability of an event is the probability of that event occurring given that another event has already occurred.
Definition: The conditional probability of an event B given that A occurred is \[\mathrm{Pr[B \ | \ A]} = \frac{\mathrm{Pr[A \ and \ B]}}{\mathrm{Pr[A]}}\]
Definition: General multiplication rule \[\mathrm{Pr[A \ and \ B]} = \mathrm{Pr[A]}\times\mathrm{Pr[B \ | \ A]}\]
Bayes Rule
Definition: The conditional probability of an event B given that A occurred (and A given that B occurred) is \[\mathrm{Pr[B \ | \ A]} = \frac{\mathrm{Pr[A \ and \ B]}}{\mathrm{Pr[A]}}, \ \ \ \ \ \ \mathrm{Pr[A \ | \ B]} = \frac{\mathrm{Pr[A \ and \ B]}}{\mathrm{Pr[B]}}\]
Definition: General multiplication rule \[\begin{align*}
\mathrm{Pr[A \ and \ B]} & = \mathrm{Pr[B \ | \ A]}\times\mathrm{Pr[A]} \\
& = \mathrm{Pr[A \ | \ B]}\times\mathrm{Pr[B]}
\end{align*}
\]
Definition: Bayes Rule \[\mathrm{Pr[B \ | \ A]} = \frac{\mathrm{Pr[A \ | \ B]}\times \mathrm{Pr[B]}}{\mathrm{Pr[A]}}\]
Mutually exclusive vs. independence
Commonly confused!
Definition: Two events are mutually exclusive if they cannot both occur at the same time. \[\mathrm{Pr[A \ and \ B]} = 0\]
Definition: Two events are independent if the occurrence of one does not inform us about the probability that the second will occur. \[\mathrm{Pr[B \ | \ A]} = \mathrm{Pr[B]}\]
Mutually exclusive vs. independence
These two conditions simplify the general additive and multiplicative rules:
If two events are mutually exclusive, then \[\mathrm{Pr[A \ or \ B]} = \mathrm{Pr[A]} + \mathrm{Pr[B]}\]
If two events are independent, then \[\mathrm{Pr[A \ and \ B]} = \mathrm{Pr[A]} \times \mathrm{Pr[B]}\]
Visualizing dependency
Independent events
Dependent events
Mosaic plots are awesome!
Totally, dude…
Definition: The probability of an event not occurring is one minus the probability that it occurs. \[\mathrm{Pr[{\it not}\ A]} = 1-\mbox{Pr[A]}\]
Definition: The law of total probability is given by \[\begin{align*}
\mathrm{Pr[A]} & = \sum_{B\ \mathrm{in} \ \mathcal{M}}\mathrm{Pr[A \ and \ B]} \\
& = \sum_{B\ \mathrm{in} \ \mathcal{M}} \mathrm{Pr[B]}\ \mathrm{Pr[A\ | \ B]},
\end{align*}\] where \(\mathcal{M}\) is a set of mutually exclusive events such that \[\sum_{B\ \mathrm{in} \ \mathcal{M}}\mathrm{Pr[B]} = 1\]
Law of total probability and mosaic plots
![alt text]()
Visualizing probability - Probability trees
Probability distributions
Definition: A probability distribution is a list of the probabilities of all mutually exclusive outcomes of a random trial.
Compare to:
Definition: A probability distribution (or relative frequency distribution) is a list of the probabilities of all values of a random variable in a sample or population.
Discrete probability distributions
How is this different? same?
Continuous probability distributions
Probability densities ![alt text]()
Tips for Solving Probability Problems
- Write out the probability that you are being asked to find. Is it a conditional probability? AND? OR?
- Identify probabilities that you are given (again, are these conditionals? ANDs? ORs?)
- Draw a probability tree (if appropriate)
Medical testing
“It’s easy to think of medical tests as black and white. If the test is positive, you have the disease; if it’s negative, you don’t. Even good clinicians sometimes fall into that trap.”
- Harriet Hall
Bayes Theorem to the rescue!!!
Example 5.9: Detection of Down Syndrome
![alt text]()
Question: What is Pr[DS | +]?
Discuss: What probabilities are given to us here?
Example 5.9: Detection of Down Syndrome
result
status - + Sum
DS 40 60 100
no DS 94905 4995 99900
Sum 94945 5055 100000
Sensitivity: Pr[+ | DS] = 60/100 = 0.6 [True positive rate]
Specificity: Pr[- | no DS] = 94905/99900 = 0.95 [True negative rate]
False positive rate: Pr[+ | no DS] = 1-0.95 = 0.05
False negative rate: Pr[- | DS] = 1-0.6 = 0.4
Example 5.9: Detection of Down Syndrome
Discuss: Graphically, what is:
Pr[DS]
Pr[+ | DS]
Pr[+ | no DS]
Pr[DS | +]
Example 5.9: Detection of Down Syndrome
Example 5.9: Detection of Down Syndrome
Example 5.9: Detection of Down Syndrome (Visual Check)
Example 5.9: Detection of Down Syndrome (Calculate)
Discuss: Pr[DS | +] = ???
Example 5.9: Detection of Down Syndrome (Calculate)
Discuss: Pr[DS | +] = ???
Example 5.9: Detection of Down Syndrome (Calculate)
Discuss: Pr[DS | +] = ???
Effect of disease rarity
Effect of disease rarity
Question: What happens to Pr[DS | +] when the proportion of the population with the disease goes to zero?
Example 5.9: Detection of Down Syndrome (Calculate)
