2025-10-19

The Power of Sample Size

The Idea: Bigger n -> Steadier Averages

  • Individual data points can jump around from each other, the averages smooth these flucuations
  • The sample mean: \[ \bar{X} = \frac{1}{n}\sum_{i=1}^{n} X_i \] where each \(X_i\) is one observation
  • As the sample size \(n\) increases, variability of \(\bar{X}\) shrinks.
  • The mean \(\bar{X}\) “homes in” on the true population mean \(\mu\).
  • The collection of all possible sample means forms the
    sampling distribution of the mean,
    which shows how the mean varies from sample to sample.

The Math Behind Stability

  • Repeated samples of size \(n\) make the sample mean \(\bar{X}\) vary less than individual data.
  • Variance of the mean:
    \[ Var(\bar{X}) = \frac{\sigma^2}{n} \]
  • Standard error (spread of sample means):
    \[ SE = \frac{\sigma}{\sqrt{n}} \]
  • As \(n\) increases, \(SE\) decreases — larger samples give more precise estimates.

Simulation Setup

  • To see how sample size affects stability, we’ll simulate data from a skewed population.
  • Use an Exponential distribution (mean = 1, variance = 1).
  • For each sample size \(n \in \{5, 30, 100\}\):
    • Draw many random samples.
    • Compute each sample’s mean \(\bar{X}\).
  • Compare how the spread of sample means changes as \(n\) increases.
## # A tibble: 3 × 3
##       n emp_mean emp_sd
##   <dbl>    <dbl>  <dbl>
## 1     5    1.00  0.445 
## 2    30    1.00  0.185 
## 3   100    0.999 0.0988

Sampling Distributions Tighten with n

  • Each curve shows the distribution of sample means \(\bar{X}\) for a different sample size.
  • Population: Exponential(1) (skewed, mean = 1).
  • Sample sizes \(n = 5, 30, 100\).
  • As \(n\) increases:
    • The curves narrow (less variability).
    • The shapes become more symmetric and bell-shaped.
  • The dashed line marks the true mean \(\mu = 1\).
    dens_df <- sample_means |>
        group_by(n) |>
        reframe({
            d <- density(mean)
            tibble(x = d$x, y =d$y)
        })
    
    ggplot(dens_df, aes(x, y, color = factor(n), group = n)) +
        geom_line(linewidth = 1.1) +
        geom_vline(xintercept = 1, linetype = 3, linewidth = 0.7) +
        scale_color_manual(values = c("5"="#5B8FD9","30"="#58B88A","100"="#C76C5B"),
                     name = "Sample size n") +
        labs(title = "Sample Means Tighten as n Increases",
           subtitle = "Population: Exponential(1); dashed = true mean (μ = 1)",
           x = expression(bar(X)), y = "Density") +
      theme_minimal(base_size = 14) +
      theme(legend.position = "top", panel.grid.minor = element_blank())

Variability Shrinks as Sample Size Grows

  • Each box shows the distribution of sample means \(\bar{X}\) for a given sample size.
  • Population: Exponential(1) (mean = 1).
  • As \(n\) increases:
    • The boxes get narrower, showing less spread.
    • The medians stay close to the true mean \(\mu = 1\).
  • The dashed line marks the true population mean.
  • Bigger samples → more precise estimates of the mean.

3D View: Sample Size vs Mean Distribution

  • This 3D plot shows how the distribution of sample means changes with sample size.
  • Axes:
    • x: sample size \(n\)
    • y: sample mean \(\bar{X}\)
    • z: estimated density (frequency)
  • As \(n\) increases:
    • The ridge narrows and steepens near the true mean \(\mu = 1\).
    • The density becomes more concentrated — the means vary less.
  • Visually confirms that larger n → less spread in the sampling distribution.
## Warning: 'layout' objects don't have these attributes: 'scence'
## Valid attributes include:
## '_deprecated', 'activeshape', 'annotations', 'autosize', 'autotypenumbers', 'calendar', 'clickmode', 'coloraxis', 'colorscale', 'colorway', 'computed', 'datarevision', 'dragmode', 'editrevision', 'editType', 'font', 'geo', 'grid', 'height', 'hidesources', 'hoverdistance', 'hoverlabel', 'hovermode', 'images', 'legend', 'mapbox', 'margin', 'meta', 'metasrc', 'modebar', 'newshape', 'paper_bgcolor', 'plot_bgcolor', 'polar', 'scene', 'selectdirection', 'selectionrevision', 'separators', 'shapes', 'showlegend', 'sliders', 'smith', 'spikedistance', 'template', 'ternary', 'title', 'transition', 'uirevision', 'uniformtext', 'updatemenus', 'width', 'xaxis', 'yaxis', 'boxmode', 'barmode', 'bargap', 'mapType'

Key Takeaways

  • The sample mean \(\bar{X}\) becomes more stable as \(n\) increases.
  • Variability of the mean:
    \[ Var(\bar{X}) = \frac{\sigma^2}{n} \]
  • Standard error (spread of sample means):
    \[ SE = \frac{\sigma}{\sqrt{n}} \]
  • Bigger \(n\) → smaller \(SE\) → more precise estimates.
  • This relationship drives confidence intervals, hypothesis tests, and the CLT.