Learning objectives

At the end of this review, you will be able to

1. Recall basic notions of probability
   • Coding skill: estimating probabilities through simulated repetitions of an experiment.
2. Understand notions of conditional probability.
   • Coding skill: implementing control structures through use of if statements and simulating populations based on conditional probabilities.
3. Link random variables to concepts of probability.
   • Coding skill: generating random samples from a given distribution and using Monte Carlo methods to estimate distribution parameters.
4. Analyse basic properties of some common random variables.
   • Coding skill: explore of the relationships between prevalence, sensitivity, specificity and positive predictive value through a real-world example.

1A: Introduction

People often colloquially refer to probability…

• “What are the chances the Red Sox will win this weekend?”
• “What’s the chance of rain tomorrow?”
• “What is the chance that a patient responds to a new therapy?”

Formalizing concepts and terminology around probability is essential for improved understanding.

1B: Random experiments

• A random experiment is an action or process that leads to one of several possible outcomes.
  • Rolling two dice simultaneously.
  • Studying treatment response of individuals enrolled in a study.
• An outcome is the observable result after conducting the experiment.
  • The sum of the faces on two dice that have been rolled.
    
  • The number of patients in a study who respond ‘well’ to therapy.
• An event is a collection of outcomes.
  • The sum after rolling two dice is 7.
  • 22 of 30 patients in a study have a good response to a therapy.

1C: Formalising our notion of probability (somewhat)

Probability is used to assign a level of uncertainty to the outcomes of phenomena that either happen randomly (e.g. rolling dice) or appear random because of a lack of understanding about exactly how the phenomenon occurs (e.g., patient response to medical treatment).

The probability of an outcome is the proportion of times the outcome would occur if the random phenomenon could be observed an infinite number of times.

• If a fair coin is flipped an infinite number of times, heads would be obtained 50% of the time.

1D: Disjoint events

Two events or outcomes are called disjoint or mutually exclusive if they cannot both happen at the same time.

If \(A\) and \(B\) represent two disjoint events, then the probability that either occurs is \[P(A \cup B) = P(A \text{ or } B) = P(A) + P(B).\]

• \(A:\) rolling a number less than three.

• \(B:\) rolling an even number more than two.

• \(D:\) rolling a two or a three

1E: Addition rule for disjoint outcomes and inclusion-exclusion principle.

If there are \(k\) disjoint events \(A_1, A_2, \ldots, A_k\), then the probability that at least one of these outcomes will occur is

Let \(A\):= drawing a diamond and \(B\):= drawing a face card. What is \(P(A \cup B)?\)

1F: Complement of an event.

The complement of an event \(D\), denoted by \(D^C\) represents all outcomes of the experiment that are NOT in \(D\).

\(D\) and \(D^C\) are related in the following probabilistic sense:

1H: Independence

Lab 1: Basic concepts of probability

Notes for review may be found here.

Question: Suppose that a biased coin is tossed 5 times; the coin is weighted such that the probability of obtaining a heads is 0.6. What is the probability of obtaining exactly 3 heads?

Exercise 1 of 2: sample()

#define parameters
prob.heads =  ## FILL THIS IN!! 
number.tosses =   ## FILL THIS IN!! 
#simulate the coin tosses
outcomes = sample(c(0, 1), 
                  size = number.tosses,
                  prob = c(1 - prob.heads, prob.heads), replace = TRUE)
#view the results
table(outcomes)
#store the results as a single number
total.heads = sum(outcomes)
total.heads

Exercise 2 of 2: for

#define parameters
prob.heads = 0.6
number.tosses = 5
number.replicates = 50
#create empty vector to store outcomes
outcomes = vector("numeric", number.replicates)
#set the seed for a pseudo-random sample
set.seed(20220621)
#simulate the coin tosses
for(k in 1:number.replicates){
  
  outcomes.replicate = sample(c(0, 1), size = number.tosses,
                      prob = c(1 - prob.heads, prob.heads), replace = TRUE)
  
  outcomes[k] = sum(outcomes.replicate)
}

2: Conditional probability and independence

• \(P(A|B)\): conditional probability of an event \(A\), given a second event \(B\), is the probability of \(A\) happening, knowing that the event \(B\) has happened.

  ---

  Consider height in the US population.

• What is the probability that a randomly selected male in the population is taller than 6 feet, 4 inches?

• Suppose you learn that the individual is a professional basketball player.

• Does this change the probability that the individual is taller than 6 feet, 4 inches?.

• \(A:\) randomly selected male in the population is taller than 6 feet, 4 inches, and \(B\): randomly selected male is a professional basketball player.

• Marginal probabilities: \(P(A)\) or \(P(B)\); joint probability: \(P(A \cap B)\). We want \(P(A|B)\)

  ---

• Example: Toss a fair coin three times. Let \(A\) be the event that exactly two heads occur, and \(B\) the event that at least two heads occur.

• \(P(A|B)\) is is the probability of having exactly two heads among the outcomes that have at least two heads.

• Sample space after conditioning on \(B: \{HHH, HHT, HTH, THH\}\).

• Outcomes associated with event \(A: \{HHT, HTH, THH\}\).

• \(P(A|B) = 3/4.\)

  ---

• We must ensure the event we are conditioning on is not an impossible event, i.e., \(P(B) \neq 0\). We define conditional probability of \(A\) conditioned on \(B\) as follows: \[P(A|B) = \frac{P(A \cap B)}{P(B)}.\]

  ---

• Link with independence: if \(P(A|B) = P(A)\), then \(A\) and \(B\) are independent. Knowing whether \(B\) happened or not offers no information about whether \(A\) occurred.

Lab 2A: Bayes’ theorem through an example: Motivating problem

Breast cancer screening

• Screening is looking for signs of disease, such as breast cancer, before a person has symptoms. The goal of screening tests is to find cancer at an early stage when it can be treated and may be cured.

• Scientists are trying to better understand which people are more likely to get certain types of cancer.

• For example, they look at the person’s age, their family history, and certain exposures during their lifetime. This information helps doctors recommend who should be screened for cancer, which screening tests should be used, and how often the tests should be done.

• Mammography is the most common screening test. An important clinical question is: What are the chances that a woman with an abnormal mammogram has breast cancer?

• For clinicians, the question above may be reformulated: what is the conditional probability that a woman has breast cancer, given that her mammogram is abnormal?

• This conditional probability is called the positive predictive value (PPV) of a mammogram.

• More concisely, if \[A = \{\text{a woman has breast cancer}\},\] and \[B = \{\text{a mammogram is positive}\},\] the PPV of a mammogram is \(P(A|B)\).

• Learning objective: use our understanding of conditional probability and learn more about Bayes’ theorem to calculate the PPV of a mammogram.

Lab 2B: Bayes’ theorem through an example: Formulating the problem

• The characteristics of a mammogram (and other diagnostic tests) are given with the reverse conditional probabilities.

• The probability that the mammogram correctly returns a positive result if a woman has breast cancer (i.e., \(P(B|A)\)),

• The probability that the mammogram correctly returns a negative result if a woman does not have breast cancer (i.e., \(P(B^C|A^C)\)).

• We will assume there is (usually historical) evidence that tells us what the marginal probability of disease (i.e., \(P(A)\)) is.

• The problem: Given the probabilities \(P(B|A)\) and \(P(B^C|A^C)\), as well as the marginal probability of disease \(P(A)\), how can the positive predictive value \(P(A|B)\) be calculated?

• There are several possible strategies for approaching this type of problem; we will focus on using a purely algebraic approach using Bayes’ Theorem.

  ---

• The simplest form of Bayes’ theorem states \[P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}.\]

• Bayes’ Theorem is seldom stated in its simplest form, because in many problems, \(P(B)\) is not given directly, but is calculated using the general multiplication formula for probabilities: \[P(B) = P(B \cap A) + P(B \cap A^C) = P(B|A)P(A) + P(B|A^C)P(A^C).\]

• Allowing us to rewrite Bayes’ theorem as \[P(A|B) = \frac{P(B|A) \times P(A)}{P(B|A)P(A) + P(B|A^C)P(A^C)}.\]

Lab 2C: Bayes’ theorem through an example: Defining events in diagnostic testing

• Want to find \(PPV = P(A|B)\)

• Bayes’ theorem: Assuming we know prevalence = \(P(A)\), we can express \[PPV = P(A|B) = \frac{P(B|A)P(A)}{P(B|A)P(A) + P(B|A^C)P(A^C)}\]

• In terms of the characteristic quantities defined above, we can write

Lab 2D: Bayes’ theorem through an example: Calculating PPV using Bayes’ theorem

Exercise 1 of 1

Using R as a calculator: one advantage R has over a standard hand calculator is the ability to easily store values as named variables. This can make numerical computations like the calculation of positive predictive value much less error-prone. Using a short script like the following to do the computation, rather than directly entering numbers into a calculator, is more efficient. We focus on the relationships between prevalence, sensitivity, specificity, positive predictive value, and negative predictive value.

#define variables
prevalence = 0.10
sensitivity = 0.98
specificity = 0.95
#calculate ppv
ppv.num = prevalence*sensitivity
ppv.den = ppv.num + (1 - specificity)*(1 - prevalence)
ppv = ppv.num/ppv.den
ppv

Question: The script above can also be re-run with different starting values. How do we obtain \(PPV\) for starting prevalence values \(P(A) = \{0.01, 0.05, 0.1, 0.25\}\) which corresponds to 1%, 5%, 10% and 25% prevalences of the diseases (breast cancer for our example)? We want to graphically compare how \(PPV\) changes as we vary \(P(A)\).

Click here for answer

#define variables
prevalence = c(0.01, 0.05, 0.1, 0.2)
sensitivity = rep(0.98, 4)
specificity = rep(0.95, 4)
#calculate ppv
ppv.num = prevalence*sensitivity
ppv.den = ppv.num + (1 - specificity)*(1 - prevalence)
ppv = ppv.num/ppv.den
plot(x = prevalence, y = ppv, type = "o", pch = 20, xlab = "Prevalence P(A)", ylab = "Positive predictive value (PPV)")

3A: Distributions of random variables: Introduction

A random variable is a function that maps each event in a sample space to a number.

• A discrete random variable takes on a finite number of values.

Suppose \(X\) is the number of heads in 3 tosses of a fair coin.

• \(X\) can take the values \(\{0, 1, 2, 3\}\).

3B: Distribution of a discrete random variable

The distribution of a discrete random variable is the collection of its values and the probabilities associated with those values.

To each value \(x\) that the random variable \(X\) can take, we assign a probability \(P(X = x)\).

For example, consider the following probability distribution:

We note the following

1. \(0 \leq P(X = x_i) \leq 1\) for \(x_i \in \{0, 1, 2, 3\}\)
2. \(\sum_{x_i \in \{0, 1, 2, 3\}} P(X = x_i) = 1\).

We examine the bar-graph which summarises the distribution above.

Distributions of random variables that arise in science can be more complex. We will learn more about this soon!

3C: Expectation and variance

If \(X\) has outcomes \(x_1\), …, \(x_k\) with probabilities \(P(X=x_1)\), …, \(P(X=x_k)\), the expected value of \(X\) is the sum of each outcome multiplied by its corresponding probability: \[E(X) = x_1 P(X=x_1) + \cdots + x_k P(X=x_k) = \sum_{i=1}^{k}x_iP(X=x_i)\] The Greek letter \(\mu\) may be used in place of the notation \(E(X)\) and is sometimes written \(\mu_X\).

The variance of \(X\), denoted by \(\text{Var}(X)\) or \(\sigma^2\), is \[\begin{align*} \text{Var}(X) &= (x_1-\mu)^2 P(X=x_1) + \cdots+ (x_k-\mu)^2 P(X=x_k) \\ &= \sum_{j=1}^{k} (x_j - \mu)^2 P(X=x_j) \end{align*}\] The standard deviation of \(X\), written as \(\text{SD}(X)\) or \(\sigma\), is the square root of the variance. It is sometimes written \(\sigma_X\).

and compute \(\mu_X\) and \(\sigma^2_X\).

3D: Binomial random variables

One specific type of discrete random variable is a binomial random variable.

\(X\) is a binomial random variable if it represents the number of successes in \(n\) independent replications of an experiment where

• Each replicate has two possible outcomes: either success or failure
• The probability of success \(p\) in each replicate is constant

A binomial random variable takes on values \(0, 1, 2, \dots, n\).

\[P(X = x)=\binom{n}{x} p^x (1-p)^{n-x},\: x= 0, 1, 2, \dots, n\]

Parameters of the distribution:

• \(n\) = number of trials

• \(p\) = probability of success

Shorthand notation: \(X\sim \text{Bin}(n,p).\) The number of heads in 3 tosses of a fair coin is a binomial random variable with parameters \(n = 3\) and \(p = 0.5\).

Mean and SD for a binomial random variable

For a binomial distribution with parameters \(n\) and \(p\), it can be shown that:

• Mean = \(np\)

• Standard Deviation = \(\sqrt{np(1-p)}\)

3E: Continuous random variables

A discrete random variable takes on a finite number of values.

• Number of heads in a set of coin tosses

• Number of people who have had chicken pox in a random sample

A continuous random variable can take on any real value in an interval.

• Height in a population

• Blood pressure in a population

A general distinction to keep in mind: discrete random variables are counted, but continuous random variables are measured.

3F: Probability density functions

Recall: for a discrete random variable \(X\), we assign probabilities to each of the values that \(X\) can take. This assignment yields a probability mass function (pmf): input a value \(x\) and get \(P(X = x)\) as output.

In comparison, continuous random variables almost never take an exact prescribed value \(c\), but there is a positive probability that its value will lie in particular intervals which can be arbitrarily small.

Continuous random variables usually admit probability density functions (PDF) which share two important features

• The total area under the density curve is 1.

• The probability that a variable has a value within a specified interval is the area under the curve over that interval.

When working with continuous random variables, probability is found for intervals of values rather than individual values.

• The probability that a continuous r.v. \(X\) takes on any single individual value is 0. That is, \(P(X = x) = 0\).

• Thus, \(P(a < X < b)\) is equivalent to \(P(a \leq X \leq b)\).

3G: Normal distribution

• Among the many distributions seen in practice, one is by far the most common: the normal distribution, which has the shape of a symmetric, unimodal bell curve.

• The bell-shaped curves can differ in center and spread; the model can be adjusted using mean and standard deviation.

• Changing the mean shifts the bell curve to the left or the right, while changing the standard deviation stretches or constricts the curve.

• Density of \(N(\mu, \sigma)\): the mean and standard deviation parameters provide enough information to characterise the density function: \[f(x) = \frac{\exp{\left[-\frac{(x-\mu)}{\sigma}\right]^2}}{\sqrt{2\pi\sigma^2}}, \ x \in \mathrm{R}.\]

3H: Calculating normal probabilities using R

• Question: What is the percentile rank for a student who scores an 1800 on the SAT for a year in which the scores are N(1500,300)? \(100 \times P(X \leq 1800)\) where \(X \sim N(1500, 300)\)

  pnorm(1800, mean = 1500, sd = 300)

  ## [1] 0.8413447

  Note: must multiply output by 100 to obtain `meaningful’ percentile rank.

• Question: What score on the SAT would put a student in the 99th percentile? Want to find \(x\) such that \(P(X \leq x) = 0.99\) where \(X \sim N(1500, 300)\).

  qnorm(0.99, mean = 1500, sd = 300)

  ## [1] 2197.904

• Question: What is the probability that someone scores more than 1200 but less than 1600 on the SAT?? \(P(1200 \leq X \leq 1600)\) where \(X \sim N(1500, 300)\).

  pnorm(1600, mean = 1500, sd = 300) - pnorm(1200, mean = 1500, sd = 300) 

  ## [1] 0.4719034

Lab 3: Distributions of random variables

Exercise 1 of 2: _binom()

Exercise 2 of 2: _norm()

\(\infty\)