1.0.1 Random Variable

A random variable is continuous if it can take any value within an interval on the real number line.
Examples include: height, time, temperature, age, pressure, and velocity.

Key characteristics:

• The variable takes values in an interval such as \((a, b)\) or even \((-\infty, +\infty)\).

• The probability of any single point is always zero:

  \[P(X = x) = 0\]

• Probabilities are meaningful only over intervals:

  \[ P(a \le X \le b) = \int_{a}^{b} f(x)\, dx \]

1.1 Probability Density Function

A function \(f(x)\) is a valid Probability Density Function (PDF) if it satisfies:

Non-negativity

\[ f(x) \ge 0 \quad \forall x \]

Total Area Equals

\[ \int_{-\infty}^{\infty} f(x)\, dx = 1 \] ### Interpretation

• Larger values of \(f(x)\) indicate higher probability density around that value.
• However, \(f(x)\) is not a probability; probabilities come from the area under the curve.

library(ggplot2)
library(dplyr)

set.seed(123)

# PARAMETERS
p <- 0.3
n <- 30
nsim <- 10000

# SIMULATION
successes <- rbinom(nsim, n, p)
prop <- successes / n

df <- data.frame(prop)

# THEORETICAL VALUES
mu <- p
se <- sqrt(p * (1 - p) / n)

# COOLER PROFESSIONAL PLOT
ggplot(df, aes(x = prop)) +
  geom_histogram(aes(y = ..density..),
                 bins = 35,
                 fill = "#4F81BD",
                 color = "white",
                 alpha = 0.85) +
  geom_density(color = "#C0504D", size = 1.2) +
  geom_vline(xintercept = mu, color = "#9BBB59", size = 1.3, linetype = "solid") +
  geom_vline(xintercept = mu + se, color = "#8064A2", size = 1, linetype = "dashed") +
  geom_vline(xintercept = mu - se, color = "#8064A2", size = 1, linetype = "dashed") +
  annotate("text",
           x = mu, y = 12,
           label = "Mean = p",
           color = "#9BBB59", size = 4, fontface = "bold", vjust = -1) +
  annotate("text",
           x = mu + se, y = 10,
           label = "+1 SE",
           color = "#8064A2", size = 4) +
  annotate("text",
           x = mu - se, y = 10,
           label = "-1 SE",
           color = "#8064A2", size = 4) +
  labs(
    title = "Sampling Distribution of the Sample Proportion",
    subtitle = paste0("True Proportion p = ", p, ", Sample Size n = ", n,
                      ", Simulations = ", nsim),
    x = "Sample Proportion  (\\hat{p})",
    y = "Density",
    caption = "Visualization of the sampling distribution based on repeated sampling"
  ) +
  theme_minimal(base_size = 14) +
  theme(
    plot.title = element_text(face = "bold", size = 18),
    plot.subtitle = element_text(color = "gray40"),
    panel.grid.minor = element_blank()
  )

2.1 Learning Objectives

By the end of this chapter

• Understand what a sample is and how it differs from a population.
• Distinguish sample distributions from sampling distributions.
• Explain the sampling distribution of the sample mean.
• Identify differences between population and sampling distributions.
• Apply basic reasoning about sampling distributions to real data.
• Solve related practice problems.

Review of Samples

A sample is a subset of individuals selected from a population.

If the population is:

• All students at a university
  
• All manufactured items in a factory
  
• All possible rolls of a fair die

then a sample is a smaller collection drawn to represent the whole.

A sample statistic such as the sample mean is:

\[ \bar{X} = \frac{1}{n}\sum_{i=1}^{n} X_i \]

But different samples produce different values of \(\bar{X}\).
This variability leads to sampling distributions.

2.2 Sample Distribution vs Sampling Distribution

• Sample Distribution Describes the values within one sample.

Example:
If your sample of test scores is:

\[ \{70, 75, 80, 90, 85\} \]

That list itself forms the sample distribution.

• Sampling Distribution Describes the distribution of a statistic across many repeated samples of the same size.

If we repeatedly draw samples of size \(n = 5\) and compute each sample mean:

\[ \bar{X}_1, \bar{X}_2, \bar{X}_3, \dots \]

The distribution of those means is the sampling distribution of the sample mean.

Sampling Distribution of the Sample Mean

The sample mean has predictable behavior:

• Mean of sampling distribution:

\[ \mu_{\bar{X}} = \mu \]

• Standard Error (SE) of the mean:

\[ \text{SE}_{\bar{X}} = \frac{\sigma}{\sqrt{n}} \]

This formula shows an important concept:

• Larger sample sizes → smaller standard error
  
• Larger samples produce more stable, less variable sample means

Even if the population is skewed, the sampling distribution of the mean becomes more normal when \(n\) increases (Central Limit Theorem).

2.3 Population Distribution vs Sampling Distribution

The key difference:
Sampling distributions describe how statistics vary across repeated samples, not how individuals vary.

2.4 Uses of Sampling Distributions

Sampling distributions allow us to:

• Construct confidence intervals
• Perform hypothesis tests
• Evaluate probability statements involving sample statistics
• Understand sampling variability
• Predict how accurate a sample is for estimating a population

This is why sampling distributions are central to statistical inference.

Practice Question

Question
A population has mean \(\mu = 50\) and standard deviation \(\sigma = 12\).
If samples of size \(n = 36\) are repeatedly drawn, what is the standard error?

\[ \text{SE}_{\bar{X}} = \frac{\sigma}{\sqrt{n}} = \frac{12}{\sqrt{36}} = \frac{12}{6} = 2 \]

Answer:
The standard error of the sample mean is 2.

Practice Question

Question
If the population distribution is strongly skewed, what happens to the sampling distribution of the sample mean as \(n\) increases?

Answer:
By the Central Limit Theorem:

• The sampling distribution becomes more normal
• The standard error becomes smaller
• Estimates become more stable

In this chapter, you learned the foundational ideas behind sampling distributions:

• Samples vary, so statistics vary.
• The sampling distribution describes that variation.
• The mean of the sampling distribution equals the population mean.
• The standard error shrinks with larger sample size.
• The Central Limit Theorem ensures normality for large \(n\).

Sampling distributions form the backbone of inferential statistics and prepare you for deeper topics such as confidence intervals and hypothesis testing.

2.5 Summary

• A sample is one collection from the population.
  
• A sample distribution is the distribution of values in that one sample.
  
• A sampling distribution is the distribution of a statistic across repeated samples.
  
• The mean of the sampling distribution equals the population mean.
  
• The standard error quantifies variability of sample means.

\[ \text{SE}_{\bar{X}} = \frac{\sigma}{\sqrt{n}} \]

3.1 Learning Objectives

• Understand when the t-distribution is used instead of the normal distribution.
• Construct a confidence interval for a population mean when population standard deviation is unknown.
• Interpret confidence intervals correctly in context.
• Identify how sample size affects uncertainty.

3.2 Review: When σ is Unknown

In many real situations, the population standard deviation \(\sigma\) is not known.
When this happens, we replace it with the sample standard deviation \(s\).

This adds extra uncertainty, so instead of the normal z-distribution we use the t-distribution, which is wider and has heavier tails.

The t-Distribution

The t-distribution depends on the degrees of freedom:

\[ df = n - 1 \]

As the sample size increases, the t-distribution approaches the normal distribution.
This is why large samples often allow us to use the z-based CI.

Confidence Interval Formula

When σ is unknown, the CI for the population mean \(\mu\) is:

\[ \bar{x} \pm t_{\alpha/2,\,n-1}\left( \frac{s}{\sqrt{n}} \right) \]

Where:

• \(\bar{x}\) = sample mean
  
• \(s\) = sample standard deviation
  
• \(n\) = sample size
  
• \(t_{\alpha/2, n-1}\) = critical value from t-distribution
  
• \(\frac{s}{\sqrt{n}}\) = standard error

Example

A sample of 16 students has:

• Mean sleep time: \(\bar{x} = 6.4\) hours
  
• Standard deviation: \(s = 0.8\) hours

Construct a 95% CI for the population mean.

Step 1: Degrees of freedom \[ df = 16 - 1 = 15 \]

Step 2: Critical value For 95% CI and 15 df:

\[ t_{0.025,15} \approx 2.131 \]

Step 3: Standard error \[ SE = \frac{s}{\sqrt{n}} = \frac{0.8}{\sqrt{16}} = 0.2 \]

Step 4: CI \[ 6.4 \pm 2.131(0.2) \]

Margin of error:

\[ ME = 0.4262 \]

Final CI: \[ (5.97,\; 6.83) \]

Interpreting the Interval A 95% CI means:

If we repeatedly took samples of size 16 and constructed confidence intervals, about 95% of them would contain the true mean.

It does not mean there is a 95% probability the true mean lies in this specific interval.

3.3 Effect of Sample Size

• Larger \(n\) → smaller standard error
  
• Smaller standard error → narrower CI
  
• Wider CIs occur with small \(n\) or noisy data (large \(s\))

The t-distribution becomes closer to the normal distribution as sample size grows.

Summary

• Use t-distribution when σ is unknown.
  
• Confidence interval formula uses sample standard deviation.
  
• Degrees of freedom = \(n - 1\).
  
• Larger sample sizes produce more precise estimates.

3.4 Practice Problem

A random sample of 10 adults has:

• \(\bar{x} = 72.3\) bpm
  
• \(s = 5.6\) bpm

Construct a 90% CI for the population mean heart rate.

Practice Problem

A researcher measures cortisol levels in 25 patients:

• \(\bar{x} = 14.5\) μg/dL
  
• \(s = 3.1\) μg/dL

Compute a 99% confidence interval for the population mean.

Chapter Summary

4.1 Learning Objective

By the end of this chapter you should be able to:

• Describe what a sampling distribution is.
  
• Distinguish between population distribution, a single sample distribution, and a sampling distribution.
  
• Understand sampling distribution of a sample proportion and of the sample mean.
  
• Use formulas for the mean and standard error of sample proportions and sample means.
  
• Appreciate how larger sample size stabilizes estimates and allows normal approximation.

4.2 Sampling Distribution

4.3 Proportions

Sample Proportion vs Population Proportion

Key facts about \(\hat p\):

• The expected value of \(\hat p\) equals the true population proportion \(p\):
  \[ E[\hat p] = p \]

• The standard error (standard deviation) of \(\hat p\) is:
  \[ \sigma_{\hat p} = \sqrt{ \frac{p(1-p)}{n} } \]

Thus \(\hat p\) is an unbiased estimator of \(p\), and larger \(n\) means smaller variability (more precise estimate).

4.4 Sampling Distribution of the Sample Proportion

Under conditions of sufficiently large sample size (commonly, \(n p \ge 5\) and \(n (1-p) \ge 5\)), the sampling distribution of \(\hat p\) can be approximated by a Normal distribution:

\[ \hat p \approx N\Bigl(p,\; \frac{p(1-p)}{n}\Bigr) \]

This allows us to apply the tools of normal-based probability to proportions (e.g. confidence intervals, hypothesis tests).

set.seed(123)

# Population proportion
p <- 0.6
n <- 50             # sample size
Nsim <- 5000        # number of simulations

# Simulate sampling distribution
phat <- replicate(Nsim, mean(rbinom(n, 1, p)))

# Plot histogram
hist(phat,
     breaks = 30,
     main = "Sampling Distribution of Sample Proportion (p-hat)",
     xlab = "Sample Proportion (p-hat)",
     probability = TRUE,
     col = "skyblue",
     border = "white",
     cex.main = 2.5,   # bigger title
     cex.lab = 2,      # bigger axis labels
     cex.axis = 1.8)   # bigger axis numbers

# Add theoretical Normal curve
x_vals <- seq(min(phat), max(phat), length = 200)
theoretical <- dnorm(x_vals, mean = p, sd = sqrt(p*(1-p)/n))
lines(x_vals, theoretical, lwd = 4, col = "red")

4.5 Central Limit Theorem

Chapter Summary

In this chapter we have seen that:

• A sampling distribution describes the variability of a statistic (mean, proportion, etc.) across repeated samples.
  

• For the sample proportion > \(\hat p\):
  \[ E[\hat p] = p,\qquad \sigma_{\hat p} = \sqrt{\frac{p(1-p)}{n}} \]

  When \(n\) is large enough, \(\hat p\) is approximately normal.
  

• The same logic — via the CLT — applies to the sample mean: larger \(n\) yields more stable estimates, and normal approximation becomes valid under mild conditions.
  

• Sampling distributions (of proportions or means) are the theoretical foundation for inferential statistics — confidence intervals, hypothesis testing, margin of error — because they quantify the expected variability due to sampling.

5.1 Learning Objectives

By the end of this chapter, you will be able to:

• Define and interpret sample spaces.
  
• Apply simple probability rules.
  
• Use the binomial formula to compute event probabilities.
  
• Distinguish between population proportion \(p\) and sample proportion \(\hat{p}\).
  
• Understand the sampling distribution of a sample proportion.
  
• Apply the Central Limit Theorem for proportions when conditions are met.

5.2 Simple Probability and Sample Spaces

A sample space \(S\) is the set of all possible outcomes of a random experiment.
If all outcomes are equally likely, the probability of an event \(A\) is:

\[ P(A) = \frac{\text{number of outcomes in } A}{\text{number of outcomes in } S} \]

Example:
A dice is rolled.

\[ S = \{1,2,3,4,5,6\}, \quad A = \{2,4,6\} \] \[ P(A) = \frac{3}{6} = 0.5 \]

Simple probability describes the likelihood of one event without considering combinations, intersections, or conditional structures.

5.3 Review: The Binomial Distribution Formula

A binomial experiment consists of: 1. A fixed number of trials \(n\)
2. Only two outcomes (success/failure)
3. Constant probability of success \(p\)
4. Independent trials

The probability of getting exactly \(k\) successes:

\[ P(X=k) = {n \choose k} p^{k}(1-p)^{n-k} \]

where:

\[ {n \choose k} = \frac{n!}{k!(n-k)!} \]

The binomial distribution models count-based, discrete outcomes.

5.4 Review : Sampling Distribution of the Sample Proportion

For a population with true proportion \(p\):

\[ \hat{p} = \frac{X}{n} \] where \(X \sim \text{Binomial}(n,p)\).

Thus, \[ E(\hat{p}) = p \]

and the standard error: \[ SE_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}} \]

When \(np \ge 10\) and \(n(1-p) \ge 10\),
the sampling distribution of \(\hat{p}\) is approximately normal: \[ \hat{p} \sim N\left( p,\; \sqrt{\frac{p(1-p)}{n}} \right) \]

Visualization: Sampling Distribution of the Sample Proportion

library(ggplot2)

set.seed(123)

# Parameters for simulation
p <- 0.3
n <- 40
nsim <- 10000

# Simulate binomial outcomes
successes <- rbinom(nsim, n, p)
prop <- successes / n

df <- data.frame(prop = prop)

ggplot(df, aes(x = prop)) +
  geom_histogram(aes(y = ..density..), bins = 30,
                 color = "black", fill = "skyblue", alpha = 0.6) +
  geom_density(color = "red", size = 1) +
  labs(
    title = "Sampling Distribution of Sample Proportion",
    x = "Sample Proportion (\\hat{p})",
    y = "Density"
  ) +
  theme_minimal()

Interpretation

The histogram represents the sampling distribution of the sample proportion
\(\hat{p} = \frac{X}{n}\) for a binomial process with:

• true population proportion \(p = 0.3\)
• sample size \(n = 40\)
• number of simulations \(10{,}000\)

Key points from the visualization:

• The histogram approximates the probability distribution of \(\hat{p}\).
  
• The red curve is a kernel density estimate, showing the smooth shape of the distribution.
• The distribution is approximately normal, which reflects the Central Limit Theorem (CLT) since \(np = 12 > 10\) and \(n(1-p) = 28 > 10\).
• The center of the distribution is close to the true value \(p = 0.3\), showing that \(\hat{p}\) is an unbiased estimator of \(p\).
• The spread of the distribution reflects the standard error of \(\hat{p}\):

\[ SE_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}} = \sqrt{\frac{0.3 \cdot 0.7}{40}} \approx 0.072 \]

This means most sample proportions fall within:

\[ p \pm 2SE_{\hat{p}} \approx 0.3 \pm 0.14 \]

Reference

[1] YouTube. (Simple Learning Pro.). ZyUzRVa6hCM [Introduction to the probability of Continuos Variable]. YouTube. https://youtu.be/ZyUzRVa6hCM

[2] YouTube. (Simple Learning Pro.). 7S7j75d3GM4 Sampling Distribution. YouTube. https://youtu.be/7S7j75d3GM4

[3] YouTube. (Simple Learning Pro.). ivd8wEHnMCg Central Limit Theorem. YouTube. https://youtu.be/ivd8wEHnMCg

[4] YouTube. (Simple Learning Pro.). q2e4mK0FTbw [Sample Proportion]. YouTube. https://youtu.be/q2e4mK0FTbw

[5] YouTube. (Simple Learning Pro.). c0mFEL_SWzE [Review Sampling Distribution]. YouTube. https://youtu.be/c0mFEL_SWzE

[6] Diez, D. M., Barr, C. D., & Çetinkaya-Rundel, M. (2024). OpenIntro Statistics (5th ed.). OpenIntro. https://www.openintro.org/book/os/

[7] Blitzstein, J., & Hwang, J. (2024). Introduction to Probability (2nd ed.). CRC Press. Free draft: https://projects.iq.harvard.edu/stat110/home

[8] VanderPlas, J. (2022). A Whirlwind Tour of Data Science. O’Reilly Media. https://github.com/jakevdp/WhirlwindTourOfDataScience

[9] Severance, C. (n.d.). Python for everybody. https://www.py4e.com/book

[10] Downey, A. B. (2023). Think Stats: Exploratory Data Analysis in Python (2nd ed.). Green Tea Press. https://greenteapress.com/wp/think-stats-2e/

[11] Shafer, D. S., & Zhang, Z. (2012). Introductory statistics. Saylor Foundation.