Introduction to Probability

This document was composed from Dr. Snopkowski’s ANTH 504 Week 5 lecture and Danielle Navarro’s 2021 Learning statistics with R Chapter 9.

• Audio: 230209_001 

• Time: 18:30

HOW ARE PROBABILITY AND STATISTICS DIFFERENT?

Probability theory is the “doctrine of chances”

• What are the chances of a fair coin coming up heads 10 times in a row?

• If I roll two six-sided dice, how likely is it that I'll roll two sixes?

• How likely is it that five cards drawn from a perfectly shuffled deck will be all hearts?

Probabilistic questions start with a known model of the world. P(heads) = 0.5

Statistical questions work the other way around. We do not know the truth about the world, all we have is the data, and from the data, we need to learn the truth of the world.

• If my friend flips a coin 10 times and gets 10 heads, is it a fair coin?
    `HHHHHHHHHH`

• If five cards off the top of the deck are all hearts, how likely is it that the deck was shuffled?

DISCRETE PROBABILITY

• Basic probability theory related to categorical data.

• We have a bag of red and blue balls. 6 blue and 4 red. What is the probability of picking a blue ball?

• The probability of an event is the proportion of times the event occurs when we repeat the experiment an infinite number of times, independently and under the same conditions.

Pr(A) – denotes the probability of event A happening.

MONTE CARLO SIMULATIONS

• We can use R to help us perform these experiments (picking a ball at random) 
balls <- rep(c("red", "blue"), times=c(4,6))
balls
##  [1] "red"  "red"  "red"  "red"  "blue" "blue" "blue" "blue" "blue" "blue"

rep() just means to repeat something. The first argument is the thing you want to repeat, and the second argument is the number of times you want them to repeat. We have 4 red and 6 blue balls We now have our bin, or really, this is an array or vector.

• `sample()` has an argument that permits us to pick more than 1 element from our sample (bag, urn). 
sample(balls, 1)
## [1] "red"

sample() will take a sample from the vector of the number of times specified. In this senerio it asks for 1 sample.

• To get a probability, we want to repeat this infinitely. Instead, we will repeat this experiment a large number of times. This is a Monte Carlo simulation
B <- 10000
events <- replicate(B, sample(balls,1))

We asked it to pick balls a bunch of different times. B is the number of times to pick. replicate() allows you to do some commands over and over and over again. Use used B for the first argument, which tells the function how many times to repeat. The second argument gives the command. The command is sample(), which I described just a moment ago. We sample it 1,000 times. This should give us close to 0.6 blue and 0.4 red, which are the odds of balls defined above.

head(events, n=10)
##  [1] "blue" "red"  "red"  "blue" "blue" "blue" "red"  "red"  "red"  "blue"
tab <- table(events)
prop.table(tab)
## events
##  blue   red 
## 0.599 0.401

If you are using code that you want to replicate in the future. This will replicate that randomness. ### SETTING THE RANDOM SEED • Throughout the class, we use random numbers (and in R, a random number generator).

• If you want to obtain the results as shown in the book (or replicate your results), you can set the number random generator to a specific number. 

• But you probably want to avoid using the same seed every time. 

• A popular way to set a seed is: year – month – day (2023-2-7 = 2014)
set.seed(2014)
B <- 10000
events <- replicate(B, sample(balls,1))
table(events)
## events
## blue  red 
## 6029 3971

FREQUENTIST VS. BAYESIAN VIEW

• The frequentist view defines probability as a long run frequency.

The frequentist view is what we will focus on in this class and is the traditional view for statistics.

• Imagine flipping a coin over and over again. It is the monte carlo simulation.

This view is a long-run frequency. When you flip a coin, you know that the probability is 0.5 because if you flip over and over, 50% of the time, you would get heads. This is the Monte Carlo simulation.

Benefits:

• It is objective: the probability of an event is necessarily grounded in the world.

• Unambiguous: any 2 people will come up with the same result.

If we have a data set and we analyze it using a chi squared data statistic, we will always have the same answer.

Cons:

• Infinite sequences don’t actually exist in the world

If you flipped a coin a large number of times, the coin may become warn out. This sounds like a 3rd variable problem. ?

• Has a narrow scope (the weather for a given place only actually exists once)

You can look at that same day over and over again. The weather will only happen once in that moment in time.

BAYESIAN VIEW

• The subjectivist view

slightly more

• Define the probability of an event as the degree of belief that an intelligent and rational agent assigns to that truth of that event.

Intelligent and rational people can have different ideas.

Benefits:

• Allows you to assign probabilities to any event you want

• Also allows you to test if there’s evidence for the null hypothesis

In frequentist, you can’t prove the null right. Only prove it wrong. Bayesian allows to test if the null is correct.

Cons:

• Objective – different people may have different beliefs.

Different outcomes for different people.

INDEPENDENCE

The way you calculate the probability is dependent on whether the observations are independent or not.

• Two events are independent if the outcome of one does not affect the other.

This is a psychological bias.

• Can you think of an example? 
  + Flipping a coin
  + Roulette wheels color history does not affect future color history. What it lands on is independent of the past. 
  + The odds of the next child being a boy after you already have 2 boys is still 50/50. The fact that you already have 2 boys is independent of the next child.

The odds that your next child is a male.

odds_of_sex <- 1/2

Non-independent = conditional probabilities

• Non-independent occurs when the outcome of one thing does depend on the other. An example is card games, where the likelihood of getting a card depends on what other cards have been shown. 

• Our balls are another example (if selection occurs without replacement).

Once you have pulled a ball, you can’t pull that ball again.

CONDITIONAL PROBABILITIES

• When events are not independent, conditional probabilities are used.

What is the chance that something will happen, given that fact that something else has already happened?

• What is the probability that a second dealt card is a King, given that the first card was a King.

Pr(Card 2 is a king | Card 1 is a king) This is read as: What is the probability that Card 2 is a king, given that Card 1 was a king.

card_one_is_king <- 4/52
card_two_is_king <- 3/51

The second king is the conditional. The line | is indicates that a conditional will follow. The given

• If 2 events are independent, we have: Pr(A | B) = Pr(A) If A is your second child sex and B represents your first child sex, then the probability of your second child being male is the odds of anyone child being male.

ADDITION AND MULTIPLICATION RULES

• If we want to know the probability of two events, A and B, occurring, we can use the multiplication rule: Pr(A & B) = Pr(A) * Pr(B|A) Pr(A & B) = Pr(B) * Pr(A|B)

The probability that Card 2 is a king, given that Card 1 was a king.

card_one_is_king * card_two_is_king
## [1] 0.004524887

What is the probability of getting blackjack – or ,more specifically, an Ace followed by a facecard?

Pr(A & B & C) = Pr(A) * Pr(B | A) * Pr(C | A & B)

ace <- 4/54
facecard <- (4*3)/51
blackjack <- ace * facecard
blackjack
## [1] 0.01742919

If the events are independent, our multiplication rule becomes simpler: Pr(A & B & C) = Pr(A) * Pr(B) * Pr(C) The odds that someone would have 3 boys in a row.

odds_of_sex*odds_of_sex*odds_of_sex
## [1] 0.125

ADDITION RULE

• The addition rule tells us that: Pr(A | B) = Pr(A) + Pr(B) - Pr(A & B) When there is an or you need to subtract out the probability of both.

In the diagram, there is crossover between A and B. This crossover need to be subtracted out in or statements. ##### What is the probability of getting an ace or a black card? The occurrences of one and the other are added. However, the odds of getting both is not included in the statement or. There are 2 black aces.This needs to be subtracted.

black_card <- 26/52
ace <- 4/52
black_aces <- 2/52
ace_or_blk <- black_card + ace - black_aces
ace_or_blk
## [1] 0.5384615

“COMPLEMENT” RULE

What is the probability of NOT getting a blackjack?

What is the probability of not getting an ace or a black card?

1-ace_or_blk
## [1] 0.4615385