A phenomenon is Random if individual outcomes are uncertain, but there is nonetheless a regular distribution of outcomes in a large number of repetitions.

The Probability of any outcome of a random phenomenon can be defined as the proportion of times the outcome would occur in a very long series of repetitions.

The probability is defined as a proportion, and it always takes values between \(0\) and \(1\) (inclusively). It may also be displayed as a percentage between \(0\%\) and \(100\%\).

• \(0\%: \text{event is impossible}\)
• \(100\%: \text{event is certain}\)

Consider the event: "Rolling a 1 of a die"

If all outcomes are equally likely,

\[{\text{relative frequency of an event}} = \frac{\text{number of outcomes in the event}}{\text{total number of outcomes}}\]

Let \(\hat{p_n}\) be the proportion of outcomes that are \(1\) after the \(n\) rolls. As the number of rolls \((n)\) increases, \(\hat{p_n}\) (the relative frequency of rolls) will converge to the probability of rolling a \(1,\space p = 1/6.\) The figure shows the convergence for \(100,000\) die rolls. The tendency of \(\hat{p_n}\) to stabilize around \(p\), i.e. the tendency of the relative frequency to stabilize around the true probability, is described by the Law of Large Numbers.

Law of Large Numbers
As more observations are collected, the observed proportion \(\hat{p_n}\) of occurrences with a particular outcome after \(n\) trials converges to the true probability \(p\) of that outcome.

The figure shows the fraction of die rolls that are \(1\) at each stage in a simulation. The relative frequency tends to get closer to the probability \(1/6 \approx 0.167\) as the number of rolls increases.

Two events or outcomes are called "disjoint or mutually exclusive" if they cannot both happen in the same trial.

When rolling a die, the outcomes \(1\) and \(2\) are disjoint, and we compute the probability that one of these outcomes will occur by adding their separate probabilities: \[P(1 \text{ or } 2)=P(1)+P(2)=1/6+1/6=1/3\]

What about the probability of rolling a 1, 2, 3, 4, 5, or 6?

\[ \begin{array}{ll} P(1 \text{ or } 2 \text{ or } 3 \text{ or } 4 \text{ or } 5 \text{ or }6) = P(1)+P(2)+P(3)+P(4)+P(5)+P(6) \\ =1/6+1/6+1/6+1/6+1/6+1/6 =1 \end{array} \]

Addition Rule of Disjoint Outcomes

If \(A_1,...,A_k\) represent \(k\) disjoint outcomes, then the probability that one of them occurs is given by: \[P(A_1\text{ or }A_2 \text{ or ... or }A_k)=P(A_1)+P(A_2)+...+P(A_k)\]

Consider a standard deck of cards.

\[ \text {4 suits} \left\{ \begin{array}{ll} \text{hearts: } \color{red}{\heartsuit} \\ \text{hearts: } \color{red}{\diamondsuit} \\ \text{hearts: } \spadesuit \\ \text{hearts: } \clubsuit \end{array} \right. \] \[\text{13 cards in each suit: } Ace, 2, 3, 4, 5, 6, 7, 8, 9, 10, Jack, Queen, King\] One card is dealt from a well shuffled deck.

\[ \begin{align} P(\text{the card is an ace or a king}) &= P(\text{it's an ace})+P(\text {it's a king}) \\ & = 4/52+4/52 \\ & = 8/52 \\ & = 2/13 \end{align} \]

\[ \begin{align} & P(\text{the card is an ace or a heart}) \\ &= P(\text{it's an ace})+P(\text {it's a heart})-P(\text{it's an ace & heart}) \\ & = 4/52+13/52 - \underbrace{1/52}_{\text {adjustment made to avoid double-counting of the ace of hearts}} \\ & = 16/52 \\ & = 4/13 \end{align} \]

\[ \begin{align} P(\text{the card is an ace or a king}) &= P(\text{it's an ace})+P(\text {it's a king}) \\ & = 4/52+4/52 \\ & = 2/13 \end{align} \]

## (polygon[GRID.polygon.1], polygon[GRID.polygon.2], polygon[GRID.polygon.3], polygon[GRID.polygon.4], text[GRID.text.5], text[GRID.text.6], text[GRID.text.7], text[GRID.text.8])

\[ \begin{align} & P(\text{the card is an ace or a heart}) \\ & = P(\text{it's an ace})+P(\text {it's a heart})-P(\text{it's an ace AND heart}) \\ & = 4/52+13/52 - 1/52 = 16/52 \end{align} \]

## (polygon[GRID.polygon.9], polygon[GRID.polygon.10], polygon[GRID.polygon.11], polygon[GRID.polygon.12], text[GRID.text.13], text[GRID.text.14], text[GRID.text.15], text[GRID.text.16], text[GRID.text.17])

\[ \bbox[yellow,5px] {\color{black}{P(A \space or \space B) = P(A) + P(B) - P(A \space and \space B)}} \] where \(P(A \text{and} B)\) is the probability that both events occur.

If \(A\) and \(B\) are mutually exclusive, \(P(A \space and \space B) = 0\)

Therefore,

\[ P(A \space or \space B) = P(A) + P(B)\]

The complement of event \(A\) is denoted \(A^c\), and \(A^c\) represents all outcomes not in \(A\). \(A\) and \(A^c\) are mathematically related:

\[ \begin{align} & P(A) + P(A^c) = 1 \\ or, \space & P(A^c) = 1 - P(A) \end{align} \]

Example: if an event has chance \(40\%\), then the chance that it doesn't happen is \(60\%\).

\[ \begin{align} P(email) &=0.73 \\ P(text) &= 0.62 \\ P(\text {email & text}) &= 0.49 \\ P(\text {only email}) &= 0.73 - 0.49 = 0.24 \\ P(\text{only text}) &= 0.62 - 0.49 = 0.13 \\ P(\text{neither email nor text}) &= 1 - (0.24 + 0.49 + 0.13) = 0.14 \end{align} \]

(polygon[GRID.polygon.18], polygon[GRID.polygon.19], polygon[GRID.polygon.20], polygon[GRID.polygon.21], text[GRID.text.22], text[GRID.text.23], text[GRID.text.24], text[GRID.text.25], text[GRID.text.26]) 

Example 1: If a card is randomly drawn from a well-shuffled deck, what is the probability that it is the ace of hearts? [Note: Ace and Hearts are two independent events.]

\[ \begin{align} P(Ace \text{ and } Hearts) &= P(Ace) \times P(Hearts) \\ &= (4/52) \times (13/52) = 1/52 \end{align} \]

Example 2: About \(9\%\) of people are left-handed. Suppose \(5\) people are selected at random from the US population.
(a) What is the probability that all are right-handed?
(b) What is the probability that all are left-handed?
(c) What is the probability that not all of them are right-handed?

\[ \begin{align} &(a) \space P\text{(All are RH)} = (1-0.09)^5 = 0.624 \\ &(b) \space P\text{(All are LH)} = (0.09)^5 = 0.0000059 \\ &(c) \space P\text{(not all RH)} = 1- P(\text {all RH}) = 1-0.624 = 0.376 \end{align} \]

The conditional probability of the outcome of interest \(A\) given condition \(B\) is computed as the following:

\[P(A|B) = \frac{P(A \text{ and } B)}{P(B)}\] College enrollment and parents' educational attainment

\[ \begin{array} {l|cc|r} & \text{parents: degree} & \text{parents: no degree} & \text{total} \\ \hline \text {teen: college} & 231 & 214 & 445 \\ \text {teen: no college} & 49 & 298 & 347 \\ \hline \text {total} & 280 & 512 & 792 \end{array} \]

If a probability is based on a single variable, it is a marginal probability. The probability of outcomes for two or more variables or processes is called a joint probability.

College enrollment and parents' educational attainment

\[ \begin{array} {l|cc|c} & \text{parents: degree} & \text{parents: no degree} & \text{marginal} \\ \hline \text {teen: college} & \color{red}{0.29} & \color{red}{0.27} & \color{blue}{0.56} \\ \text {teen: no college} & \color{red}{0.06} & \color{red}{0.38} & \color{blue}{0.44} \\ \hline \text {marginal} & \color{blue}{0.35} & \color{blue}{0.65} & 1.00 \end{array} \]

\[ \begin{align} &\color{blue}{\text{Marginal Probability: }} P(\text{teen: college})=\frac{445}{792}=0.56 \\ &\color{red}{\text{Joint Probability: }} P(\text {teen: college and parents: no degree})=\frac{214}{792}=0.27 \end{align} \]

College enrollment and parents' educational attainment

\[ \begin{array} {l|cc|r} & \text{parents: degree} & \text{parents: no degree} & \text{total} \\ \hline \text {teen: college} & 231 & 214 & 445 \\ \text {teen: no college} & 49 & 298 & 347 \\ \hline \text {total} & 280 & 512 & 792 \end{array} \]

\[ \begin{align} P(\text {teen college | parents degree}) &= \frac{231/792}{280/792} = 0.825 \\ P(\text {teen college | parents no degree}) &= \frac{214/792}{512/792} = 0.418 \\ P(\text {teen no college | parents degree}) &= \frac{49/792}{280/792} = 0.175 \\ P(\text {teen no college | parents no degree}) &= \frac{298/792}{512/792} = 0.582 \end{align} \]

If \(A\) and \(B\) represent two outcomes or events, then

\[ \bbox[yellow,5px] {\color{black}{P(A \space and \space B) = P(A|B) \times P(B)}} \]

Verify whether one of the following equations holds:

\[ \begin{align} P(A|B) &= P(A) \tag 1 \\ P(A \space and \space B) &=P (A) \times P(B) \tag 2 \end{align} \] Check if the equality holds in the following equation:

\[ \begin{align} P(\text{teen college | parent degree})&\stackrel{?}{=} P(\text {teen college}) \\ 0.825 &\ne 0.560 \end{align} \] Because both sides are not equal, teenager college attendance and parent degree are not independent.

If \(A\) and \(B\) are mutually exclusive events, then they cannot occur at the same time. If asked to determine if events \(A\) and \(B\) are mutually exclusive, verify one of the following equations holds:

\[ \begin{align} P(\text{A and B})&= 0 \tag 1 \\ P(\text{A or B}) &= P(A)+P(B) \tag 2 \end{align} \]

If the equation that is checked holds true, \(A\) and \(B\) are mutually exclusive. If the equation does not hold, then \(A\) and \(B\) are not mutually exclusive.

Find \(P(A|B)\)

From general multiplication rule, we can write:

\[ \begin{align} P(A \space and \space B) &= P(A|B) \times P(B) \\ \Rightarrow P(A|B) &= \frac{P(A \space and \space B)}{P(B)} \\ &= \frac{P(A \space and \space B)}{P(A \text { and } B)+P(A' \text{ and } B)} \\ &= \frac{P(A)\times P(B|A)}{P(A)\times P(B|A) + P(A')\times P(B|A')} \\ &= \frac{(0.8 \times 0.01)}{(0.8 \times 0.01)+(0.2 \times 0.02)} \\ &= 0.67 \end{align} \]

Let's say \(1\%\) of the population has a rare disease.

Error rates:

• False negative: among people who have the disease, \(0.5\%\) test \(-\)
  
• False positive: among people who doesn't have the disease, \(0.8\%\) test \(+\)

A person is picked at random and tested. Given that the test result is \(+\), what is the probability that the person has the disease?

Find \(P(D|+)\)

From general multiplication rule, we can write:

\[ \begin{align} P(D \space and \space +) &= P(D|+) \times P(+) \\ \Rightarrow P(D|+) &= \frac{P(D \space and \space +)}{P(+)} \\ &= \frac{P(D \space and \space +)}{P(D \text { and } +)+P(ND \text{ and } +)} \\ &= \frac{P(D)\times P(+|D)}{P(D)\times P(+|D) + P(ND)\times P(+|ND)} \\ &= \frac{(0.01 \times 0.995)}{(0.01 \times 0.995)+(0.99 \times 0.008)} \\ &= 0.56 \end{align} \]

Consider the following conditional probabilities for event 1 and event 2: \[P(\text {outcome of } A_1 \text { of event 1 | outcome B of event 2})\]

Bayes' Theorem states that this conditional probability can be identified as the following fraction:

\[\frac {P(B|A_1)P(A_1)}{P(B|A_1)P(A_1)+P(B|A_2)P(A_2)+...+P(B|A_k)P(A_k)}\]

A poker hand (5 cards) is dealt from a well shuffled deck. What is the chance that there is at least one ace in the hand?

\[ \begin{align} &P(\text{at least one ace}) \\ &=1-P(\text{no aces}) \\ &=1-(48/52) \times (47/51) \times (46/50) \times (45/49) \times (44/48) \\ &=34.11\% \end{align} \]