Essential of Probability

assignment week 10

1 Introduction

Probability theory forms the foundation of statistical analysis and decision-making under uncertainty. By providing a structured approach to evaluate and quantify uncertainty, probability allows us to make informed predictions and assess risks. This chapter introduces the fundamental concepts of probability, including sample spaces, events, complementary events, independent and dependent events, as well as joint probability. Additionally, it covers key principles such as mutually exclusive and exhaustive events, and the binomial experiment and distribution. Understanding these core concepts is crucial for developing a solid foundation in statistics, enabling students to approach more advanced statistical methods with confidence and clarity.

Through this chapter, readers will gain an understanding of how probability is applied to real-world situations and how it serves as a tool to analyze and interpret data accurately.

2 Fundamental Concept

library(knitr)
include_url("https://www.youtube.com/embed/ynjHKBCiGXY")

Through this chapter, readers will gain an understanding of how probability is applied to real-world situations and how it serves as a tool to analyze and interpret data accurately. Where:

𝑛(A) = number of favorable outcomes

𝑛(S) = number of outcomes in the sample space

sample_space <- expand.grid(c("H","T"), c("H","T"))
colnames(sample_space) <- c("Flip1", "Flip2")
sample_space

prob <- rep(0.25, 4)
data.frame(Outcome = apply(sample_space,1,paste0,collapse=""), Probability = prob)

# Probability of each outcome
prob <- rep(0.25, 4)
prob_df <- data.frame(
  Outcome = apply(sample_space, 1, paste0, collapse=""),
  Probability = prob
)
prob_df

# Complement rule table
data.frame(
  Event = c("TT", "Not TT"),
  Probability = c("TT", "not_TT")
)

3 Independent and Dependent

library(knitr)
include_url("https://www.youtube.com/embed/LS-_ihDKr2M")

Independent events are situations in which the outcome of one event does not influence the outcome or probability of another event. An example is rolling a die and flipping a coin—rolling a 6 does not change the probability of getting heads, which remains 0.5. For independent events, the probability of both events occurring together is found by multiplying their individual probabilities.

Dependent events, however, occur when the outcome of one event affects the probability of another. This commonly happens when selections are made without replacement. For example, selecting marbles from a box changes probabilities after each draw. If a green marble is drawn first, the probability distribution for the next draw changes because the total number of marbles has been reduced. For dependent events, the probability of A and B occurring is equal to the probability of A multiplied by the probability of B after A has happened.

3.1 Independent Events

\[ P(A \text{ and } B) = P(A)\times P(B) \]

P(A) = the probability of event A occurring
P(B) = the probability of event B occurring
Used when one event does not affect the probability of the other

Example: Roll 5 and get heads:
\[ P = \frac{1}{6}\times\frac{1}{2} = \frac{1}{12} \]

3.2 Dependent Events

\[ P(A \text{ and } B) = P(A)\times P(B|A) \]

P(A) = the probability of event A occurring
P(B | A) = the probability of event B after event A has occurred
Used when the first event changes the probability of the next event

Example: Green then blue marble:
\[ P = \frac{7}{10}\times\frac{3}{9} = \frac{7}{30} \]

4 Union of Events

library(knitr)
include_url("https://www.youtube.com/embed/vqKAbhCqSTc")

A sample space, which represents all possible outcomes of a statistical experiment. For one six-sided die, the sample space contains six outcomes, and for two dice, it contains 36 ordered pairs.It then reviews simple probability, defined as the number of favorable outcomes divided by the total number of outcomes in the sample space. Several examples demonstrate this idea, such as the probability of rolling two 4s, two even numbers, at least one 2, or two 6s.

The video also highlights common mistakes, including misusing the independent events formula when events are not independent. Instead, examining the sample space allows the correct determination of intersections and unions of events.The union of events is introduced through the formula P(A ∪ B) = P(A) + P(B) – P(A ∩ B). A Venn diagram helps visualize why the intersection must be subtracted to avoid double-counting.The video concludes by applying the formula to calculate the probability of rolling two even numbers or at least one 2, yielding a result of 15/36 or 0.4167.

4.1 Probability of rolling two 4s

This calculation determines the probability that both dice show the number 4 (i.e., the ordered pair (4,4)). \[ P(\text{two 4s}) = \frac{\text{number of favorable outcomes}}{\text{total number of outcomes}} = \frac{1}{36} \] \[ P(\text{two 4s}) = \frac{1}{36} \approx 0.02778 \]

4.2 Probability of rolling two even numbers

This calculation determines the probability that both dice land on an even number (2, 4, or 6). \[ P(\text{two even numbers}) = \frac{\text{number of outcomes with two even numbers}} {\text{total outcomes}} = \frac{9}{36} \]

\[ P(\text{two even numbers}) = \frac{9}{36} = 0.25 \]

4.3 Probability of rolling at least one 2

The probability that at least one die shows a 2. \[ P(\text{at least one 2}) = \frac{\text{number of outcomes with at least one 2}} {\text{total outcomes}} = \frac{11}{36} \]

\[ P(\text{at least one 2}) = \frac{11}{36} \]

4.4 Probability of rolling two 6s (independent events)

The probability that both dice show a 6. \[ P(A \text{ and } B) = P(A) \times P(B) = \frac{1}{6} \times \frac{1}{6} = \frac{1}{36} \]

\[ P(\text{two 6s}) = \frac{1}{36} \]

4.5 Probability of rolling two even numbers AND at least one 2 (intersection)

The probability that both dice are even and at least one die shows a 2. \[ P(A \cap B) = \frac{\text{number of outcomes in both events}} {\text{total outcomes}} = \frac{5}{36} \] \[ P(\text{two even numbers and at least one 2}) = \frac{5}{36} \]

4.6 Probability of rolling two even numbers OR at least one 2 (union of events)

The probability that either both dice are even, or at least one die shows a 2. \[ P(A \cup B) = P(A) + P(B) - P(A \cap B) = \frac{9}{36} + \frac{11}{36} - \frac{5}{36} = \frac{15}{36} \]

\[ P(\text{two even numbers or at least one 2}) = \frac{15}{36} = 0.4167 \]

5 Exclusive and Exhaustive

library(knitr)
include_url("https://www.youtube.com/embed/f7agTv9nA5k")

Mutually exclusive events and exhaustive events using the sample space for rolling two six-sided dice. The sample space contains all possible outcomes, and for two dice, there are 36 outcomes.

5.1 Mutually Exclusive Events

Mutually exclusive events are events that do not have any outcomes in common. In a Venn diagram, events A and B do not overlap. Two events are said to be disjoint or mutually exclusive if and only if the probability of events A and B occurring together is equal to zero. In other words, the events cannot occur at the same time.

Using the sample space, consider two events A and B:

Event A: rolling at least one 5

Event B: the sum of the two dice is less than 4

Event A contains 11 outcomes, so its probability is : \[ P(A) = \frac{11}{36} \]

Event B contains 3 outcomes, so its probability is : \[ P(B) = \frac{3}{36} \]

By checking the sample space, we see that these two events have no common outcomes. Therefore, they are mutually exclusive, and the probability of A and B happening together is 0.

5.2 Exhaustive Events

Exhaustive events are events that collectively contain all outcomes in the sample space. In a Venn diagram, this means that the highlighted regions should cover the entire box representing the sample space. The events do not need to be 50–50; they can have any ratio as long as they include all outcomes. Using the same sample space:

Event A: rolling at least one 6

Event B: the sum of the dice is less than 11 Event A has 11 outcomes (11/36), and event B has 33 outcomes (33/36). Together, these events cover the entire sample space, so they are exhaustive.

For exhaustive events, the union 𝑃 ( 𝐴 ∪ 𝐵 ) P(A∪B) will always equal 1, because all outcomes are considered. Using the formula: \[ P(A \cup B) = P(A) + P(B) - P(A \cap B) \]

Event A and B overlap in 8 outcomes (8/36). Plugging in:

\[ \frac{11}{36} + \frac{33}{36} - \frac{8}{36} = \frac{36}{36} = 1 \]

This confirms that the events are exhaustive.

5.3 Mutually Exclusive & Exhaustive at the Same Time

Some events can be both mutually exclusive and exhaustive simultaneously. Example:

Event A: the sum of the dice is even

Event B: the sum of the dice is odd

There are 18 outcomes where the sum is even (18/36), and 18 outcomes where it is odd (18/36). These events do not overlap, so they are mutually exclusive. Together they cover all 36 outcomes, so they are exhaustive.

Using the union formula:

\[ \frac{18}{36} + \frac{18}{36} - 0 = \frac{36}{36} = 1 \] This confirms that events A and B are both mutually exclusive and exhaustive.

6 Binomial Experiment

library(knitr)
include_url("https://www.youtube.com/embed/nRuQAtajJYk")

A binomial probability distribution refers to the probability of obtaining a success or failure in an experiment that is repeated multiple times. The prefix bi- indicates two — in this case, two outcomes: success or failure.

Binomial Setting Requirements

A scenario is considered a binomial setting if it satisfies four conditions:

Fixed number of trials 𝑛
Two possible outcomes (success or failure).
Constant probability of success 𝑝
Independent trials — outcome of one trial does not affect others. An experiment satisfying these four conditions is called a binomial experiment.

6.1 Example 1 : Fixed Number of Trials (Coin Flip)

If you flip a fair coin three times, what is the probability of getting exactly one head, and is this a binomial experiment?

There are three possible outcomes with exactly one head:

𝐻 𝑇 𝑇

𝑇 𝐻 𝑇

𝑇 𝑇 𝐻

Each outcome has probability: 0.5 = 0.125

Since there are 3 such outcomes: P() = 3 = 0.375

All four binomial conditions are satisfied, so this is a binomial experiment.

6.2 Example 2 :Two possible outcomes (Drawing Marbles With Replacement)

We have 10 marbles: 3 pink, 2 green, 5 blue. We draw 5 marbles with replacement, and we want the probability of getting exactly 2 green marbles.

Check binomial conditions -Fixed number of trials: 𝑛 = 5

-Success = drawing a green marble

-Failure = not green

-Probability of success stays constant:

We have 2 green marbles out of 10 total marbles.

\[ p = \frac{2}{10} = 0.2 \]

\[ 1 - p = 0.8 \]

Because the draws are with replacement, the trials are independent.

Thus, this is a .

Counting the Outcomes There are 10 different ways to arrange 2 successes (greens) and 3 failures (not green). Each outcome has probability:

\[ 0.2^2 \times 0.8^3 = 0.02048 \]

Thus:

\[ P(\text{exactly 2 green marbles}) = 10 \times 0.02048 = 0.2048 \]

6.3 Binomial Formula

The binomial formula is:

\[ P(k) = \binom{n}{k} p^k (1-p)^{\,n-k} \]

Where:

\(k\) = number of successes
\(n\) = number of trials
\(p\) = probability of success
\(\binom{n}{k} = \text{n choose k}\)

Using the Binomial Formula for This Problem

Given:

\(n = 5\)
\(k = 2\)
\(p = 0.2\)

Plug into the formula:

\[ P(2) = \binom{5}{2} (0.2)^2 (1 - 0.2)^{5-2} \]

\[ P(2) = 10 \times 0.04 \times 0.512 = 0.2048 \]

7 Binomial Distribution

library(knitr)
include_url("https://www.youtube.com/embed/Y2-vSWFmgyI")

This video discusses binomial distribution starting from a review of the binomial formula.

The binomial formula is:

\[ P(k) = \binom{n}{k} p^{k} (1 - p)^{\,n-k} \]

with:

𝑘 = number of successes

𝑛 = number of trials

𝑝 = probability of success

Example: Tossing a coin 2 times If we flip a coin 2 times and are successful = get heads, then:

\(k = 0, 1, 2\) (cannot have more than 2 successes)
\(n = 2\)
\(p = 0.5\)

After plugging these values into the binomial formula, we get the probability:

\(P(0) = 0.25\)
\(P(1) = 0.50\)
\(P(2) = 0.25\)

The bar chart shows that the probabilities of 0, 1, and 2 successes are
0.25, 0.5, and 0.25, respectively.

7.1 Changes in the shape of the distribution as \(n\) increases

If the value \(n\) is increased to 10, the shape of the binomial distribution begins to resemble a normal distribution.

Binomial parameters:

Mean:
\[ \mu = np \]
Variance:
\[ \sigma^{2} = np(1 - p) \]
Standard deviation:
\[ \sigma = \sqrt{np(1 - p)} \]

In the example \(n = 10\), it can be seen that the distribution is centered around 5, in accordance with \(\mu = np\).

The effect of changes in the \(p\) value

Within \(n = 10\):

If \(p = 0.5\), the distribution is symmetrical (similar to normal).
If \(p\) is shifted from 0.5 (smaller or larger):
- the distribution becomes skewed
- \(p < 0.5\) → skewed to the right
- \(p > 0.5\) → skewed to the left

Extreme example:

\(p = 0.1\): small chance of success → data is piled up on the left side (lots of 0 successes).
\(p = 0.8\): high chance of success → data is piled up on the right side.

Data always clusters around the mean:

\[ \mu = np \]

7.2 Normal Approximation

The binomial distribution can be approximated by the normal distribution if two conditions are met:

\[ np \ge 10 \]

\[ n(1 - p) \ge 10 \]

(In some references the limit is 5, so you have to check the guidelines from your lecturer or book.)

7.3 Final Recap

The \(p\)-value controls the shape of the distribution:
- \(p = 0.5\) → symmetrical
- \(p < 0.5\) → skewed right
- \(p > 0.5\) → skewed left
As \(n\) increases, the binomial distribution approaches a normal distribution.
Binomial parameters:

\[ \mu = np \]

\[ \sigma^2 = np(1 - p) \]

\[ \sigma = \sqrt{np(1 - p)} \]

8 Reference

[1] DSCienceLabs, Essentials of Probability, in Introductory Statistics with R,
Bookdown. Available: https://bookdown.org/dsciencelabs/intro_statistics/06-Essentials_of_Probability.html, 2024.

[2] Simple Learning Pro, “Essentials of Probability – Basic Definitions,” YouTube.
Available: https://youtu.be/ynjHKBCiGXY?si=yaWVv2Sq7lqpWCtc, 2023.

[3] Simple Learning Pro, “Independent and Dependent Events,” YouTube.
Available: https://youtu.be/LS-_ihDKr2M?si=BL4AJc1-WO58pWk_, 2023.

[4] Simple Learning Pro, “Union of Events,” YouTube.
Available: https://youtu.be/vqKAbhCqSTc?si=YmTyDsqXMw0jTe7u, 2023.

[5] Simple Learning Pro, “Mutually Exclusive and Exhaustive Events,” YouTube.
Available: https://youtu.be/f7agTv9nA5k?si=1IA30WNrm3nz9OSV, 2023.

[6] Simple Learning Pro, “Binomial Experiments,” YouTube.
Available: https://youtu.be/nRuQAtajJYk?si=B10qkaW8-PbLnlUz, 2023.

[7] Simple Learning Pro, “The Binomial Distribution,” YouTube.
Available: https://youtu.be/Y2-vSWFmgyI?si=q_a6uQ0-6naQS4Ox, 2023.