Review of Basic Principles

Addition Law - General Case

\[P(A\text{ or }B)=P(A)+P(B)-P(A\text{ and }B)\] Example Draw a single card from a 52 card deck. What is the probability of a heart or an ace?
\[P(\text{Heart or Ace})=P(\text{Heart})+P(\text{Ace})-P(\text{Heart and Ace})\]

\[P(\text{Heart or Ace})=\frac{13}{52}+\frac{4}{52}-\frac{1}{52}=\frac{16}{52}=\frac{4}{13}\]

The subtraction of the third term is necessary because the first two terms would have counted the ace of hearts twice.

Addition Law - Special Case

If the two events are mutually exclsuive (disjoint), the third term may be omitted since its value is zero. \[P(A\text{ or }B)=P(A)+P(B)\] Example Draw a single card from a 52 card deck. What is the probability of a heart or a spade?
\[P(\text{Heart or Ace})=P(\text{Heart})+P(\text{Spade})\]

\[P(\text{Heart or Spade})=\frac{13}{52}+\frac{13}{52}=\frac{26}{52}=\frac{1}{2}\] The simpler version is applicable because no card can be both a heart and a spade. We say that these two events are disjoint or mutually exclusive.

The Complements Law

If A is an event, either it occcurs or does not occur whenever the experiment we are thinking of occurs. If we think of events as subsets of a sample space, then every outcome is in the subset A or not in the subset A. The set of all outcomes not in the event A is called the complement of A, sometimes denoted \(A^{c}\). No outcome is both in A and in \(A^{c}\) (mutually exclusive). Also, every outcome is in one of these (exhaustive). In set theoretic terms \(A \cup A^{c} = \text{Sample Space}\).

The practical consequence is that \[P(\text{A}) + P(\text{not A}) = 1.\]

Example What is the probability of not getting heart when you draw a single card from a 52 card deck?

\[P(\text{not Heart}) = 1 - P(\text{Heart}) = 1 -\frac{13}{52} = \frac{3}{4}\]

Multiplication Law - General Case

\[P(\text{A and B}) = P(\text{A})*P(\text{B given that A has occured}).\] The phrasing in the second expression is generally replaced with a single vertical bar, ‘|’ between the two event names. The law is then written as \[P(\text{A and B}) = P(\text{A})*P(\text{B|A}).\]

Example What is the probability of drawing two hearts if you draw two cards from a 52 card deck without replacement. There are two events involved, getting a heart on the first draw (H1) and getting a heart on the second draw (H2).

\[P(\text{H1 and H2}) = P(\text{H1})*P(\text{H1|H2})\]

\[P(\text{H1 and H2}) = \frac{13}{52}*\frac{12}{51}=\frac{3}{51}=\frac{1}{17}\]

Multiplication Law - Special Case

\[P(\text{A and B}) = P(\text{A})*P(\text{B}).\] If the probability that B will occur does not depend on whether A has occurred or not, we say that the two events are independent. In this case, we use the simple probability rather than the conditional probability.

Example What is the probability of drawing two hearts if you draw two cards from a 52 card deck with replacement. There are two events involved, getting a heart on the first draw (H1) and getting a heart on the second draw (H2). However, when we include the stipulation “with replacement,” the condition of the deck on the second draw is the same as it was on the first draw.

\[P(\text{H1 and H2}) = \frac{13}{52}*\frac{13}{52}=\frac{1}{4} *\frac{1}{4}=\frac{1}{16}\]

Bayes Law

In the description of the multiplication law above, we have assumed that A occurs first and may (or may not) influence the probability that a second event, B occurs. We imagine that an observer, having observed A, obtains an improved estimate of the probability that B will occur. In some circumstances, it makes sense to reverse our thinking and become more like a detective. If B has been observed, what is the probability that A occurred?

Write two equivalent expressions for \(P(\text{A and B}).\) \[P(\text{A and B}) = P(\text{A})*P(\text{B|A}) = P(\text{B}) *P(\text{A|B}).\]

Now solve the equation relating the second and third equations for \(P(\text{A|B})\).

\[P(\text{A})*P(\text{B|A}) = P(\text{B}) *P(\text{A|B})\]

\[P(\text{A|B}) = \frac{P(\text{A})*P(\text{B|A})}{P(\text{B})}\]

In many cases, we don’t know the probability of B directly. However we can construct it. If B occured, either A occurred and then B occured or; A did not occur and then B occurred anyway. These two possibilities are mutually exclusive and exhaustive. Therefore we can write

\[P(\text{B}) = P(\text{A}) * P(\text{B|A})+P(\text{ not A}) *P(\text{B| not A})\]

Example Return to the problem of drawing two hearts from a 52 card deck without replacement. Suppose the second card was a heart. What is the probability that the first card was also a heart?

\[P(\text{H1|H2}) = \frac{P(\text{H1})*P(\text{H2|H1})}{P(\text{H2})}\]

Now rewrite the denominator

\[P(\text{H1|H2}) = \frac{P(\text{H1})*P(\text{H2|H1})}{P(\text{H1})*P(\text{H2|H1})+P(\text{not H1})*P(\text{H2 | not H1})}\]

Now let’s insert the numerical values.

\[P(\text{H1|H2}) = \frac{\frac{13}{52}*\frac{12}{51}}{\frac{13}{52}*\frac{12}{51}+\frac{39}{52}*\frac{13}{51}}\]