Lab 5: Foundations for statistical inference - Sampling distributions

Getting started

library(tidyverse)
library(openintro)
library(infer)

The Data

global_monitor <- tibble(
  scientist_work = c(rep("Benefits", 80000), rep("Doesn't benefit", 20000))
)

ggplot(global_monitor, aes(x = scientist_work)) +
  geom_bar() +
  labs(
    x = "", y = "",
    title = "Do you believe that the work scientists do benefit people like you?"
  ) +
  coord_flip()

global_monitor %>%
  count(scientist_work) %>%
  mutate(p = n /sum(n))

## # A tibble: 2 × 3
##   scientist_work      n     p
##   <chr>           <int> <dbl>
## 1 Benefits        80000   0.8
## 2 Doesn't benefit 20000   0.2

The unknown sampling distribution

samp1 <- global_monitor %>%
  sample_n(50)

Exercise 1

Describe the distribution of responses in this sample. How does it compare to the distribution of responses in the population. Hint: Although the sample_n function takes a random sample of observations (i.e. rows) from the dataset, you can still refer to the variables in the dataset with the same names. Code you presented earlier for visualizing and summarizing the population data will still be useful for the sample, however be careful to not label your proportion p since you’re now calculating a sample statistic, not a population parameters. You can customize the label of the statistics to indicate that it comes from the sample.

The data for both the sample and the population shows that about 20% of people believe that the work of scientists is not beneficial.

Exercise 2

Would you expect the sample proportion to match the sample proportion of another student’s sample? Why, or why not? If the answer is no, would you expect the proportions to be somewhat different or very different? Ask a student team to confirm your answer.

No, the sample proportions likely won’t match exactly because each sample is randomly selected. However, they should be somewhat similar, not very different, as both samples come from the same population.

Exercise 3

Take a second sample, also of size 50, and call it samp2. How does the sample proportion of samp2 compare with that of samp1? Suppose we took two more samples, one of size 100 and one of size 1000. Which would you think would provide a more accurate estimate of the population proportion?

samp2 <- global_monitor %>%
  sample_n(50) 

samp2 %>%
  count(scientist_work) %>%
  mutate(p_hat_ = n /sum(n))

## # A tibble: 2 × 3
##   scientist_work      n p_hat_
##   <chr>           <int>  <dbl>
## 1 Benefits           36   0.72
## 2 Doesn't benefit    14   0.28

The sample proportion of samp2 is quite similar to samp1, with both showing more individuals benefiting than not. However, samp2 has slightly more individuals who don’t benefit compared to samp1. I believe a sample size of 1000 would provide a more accurate measurement because a larger sample reduces variability and would better reflect the overall population distribution.

sample_props50 <- global_monitor %>%
                    rep_sample_n(size = 50, reps = 15000, replace = TRUE) %>%
                    count(scientist_work) %>%
                    mutate(p_hat = n /sum(n)) %>%
                    filter(scientist_work == "Doesn't benefit")

ggplot(data = sample_props50, aes(x = p_hat)) +
  geom_histogram(binwidth = 0.02) +
  labs(
    x = "p_hat (Doesn't benefit)",
    title = "Sampling distribution of p_hat",
    subtitle = "Sample size = 50, Number of samples = 15000"
  )

Exercise 4

How many elements are there in sample_props50? Describe the sampling distribution, and be sure to specifically note its center. Make sure to include a plot of the distribution in your answer.

The histogram above indicates that there are approximately 2,000 elements represented. The sampling distribution is symmetric and exhibits no skew, with a clear center at 0.2, or 20%.

Interlude: Sampling distributions

The idea behind the rep_sample_n function is repetition. Earlier, you took a single sample of size n (50) from the population of all people in the population. With this new function, you can repeat this sampling procedure rep times in order to build a distribution of a series of sample statistics, which is called the sampling distribution.

Note that in practice one rarely gets to build true sampling distributions, because one rarely has access to data from the entire population.

Without the rep_sample_n function, this would be painful. We would have to manually run the following code 15,000 times

global_monitor %>%
  sample_n(size = 50, replace = TRUE) %>%
  count(scientist_work) %>%
  mutate(p_hat = n /sum(n)) %>%
  filter(scientist_work == "Doesn't benefit")

## # A tibble: 1 × 3
##   scientist_work      n p_hat
##   <chr>           <int> <dbl>
## 1 Doesn't benefit    10   0.2

Exercise 5

To make sure you understand how sampling distributions are built, and exactly what the rep_sample_n function does, try modifying the code to create a sampling distribution of 25 sample proportions from samples of size 10, and put them in a data frame named sample_props_small. Print the output. How many observations are there in this object called sample_props_small? What does each observation represent?

sample_props_small <- global_monitor %>%
                    rep_sample_n(size = 10, reps = 25, replace = TRUE) %>%
                    count(scientist_work) %>%
                    mutate(p_hat = n /sum(n)) %>%
                    filter(scientist_work == "Doesn't benefit")

sample_props_small

## # A tibble: 23 × 4
## # Groups:   replicate [23]
##    replicate scientist_work      n p_hat
##        <int> <chr>           <int> <dbl>
##  1         1 Doesn't benefit     1   0.1
##  2         2 Doesn't benefit     1   0.1
##  3         3 Doesn't benefit     2   0.2
##  4         4 Doesn't benefit     3   0.3
##  5         5 Doesn't benefit     3   0.3
##  6         6 Doesn't benefit     2   0.2
##  7         7 Doesn't benefit     1   0.1
##  8         8 Doesn't benefit     3   0.3
##  9         9 Doesn't benefit     1   0.1
## 10        10 Doesn't benefit     1   0.1
## # ℹ 13 more rows

Sample size and the sampling distribution

Mechanics aside, let’s return to the reason we used the rep_sample_n function: to compute a sampling distribution, specifically, the sampling distribution of the proportions from samples of 50 people.

ggplot(data = sample_props50, aes(x = p_hat)) +
  geom_histogram(binwidth = 0.02)

Exercise 6

Use the app below to create sampling distributions of proportions of Doesn’t benefit from samples of size 10, 50, and 100. Use 5,000 simulations. What does each observation in the sampling distribution represent? How does the mean, standard error, and shape of the sampling distribution change as the sample size increases? How (if at all) do these values change if you increase the number of simulations? (You do not need to include plots in your answer.)

##Samples of size 10 from 5000 simulations
set.seed(1)

#Generating the sampling distribution
sample_props_small1 <- global_monitor %>%
    rep_sample_n(size = 10, reps = 5000, replace = TRUE) %>%
    group_by(rep = row_number()) %>%  #Creating a 'rep' column for repetitions
    count(scientist_work) %>%
    group_by(scientist_work) %>%  #Group by scientist_work for proportion calculation
    mutate(p_hat = n / sum(n)) %>%
    filter(scientist_work == "Doesn't benefit")

sample_props_small1

## # A tibble: 10,051 × 4
## # Groups:   scientist_work [1]
##      rep scientist_work      n     p_hat
##    <int> <chr>           <int>     <dbl>
##  1    11 Doesn't benefit     1 0.0000995
##  2    16 Doesn't benefit     1 0.0000995
##  3    18 Doesn't benefit     1 0.0000995
##  4    20 Doesn't benefit     1 0.0000995
##  5    22 Doesn't benefit     1 0.0000995
##  6    23 Doesn't benefit     1 0.0000995
##  7    25 Doesn't benefit     1 0.0000995
##  8    26 Doesn't benefit     1 0.0000995
##  9    29 Doesn't benefit     1 0.0000995
## 10    38 Doesn't benefit     1 0.0000995
## # ℹ 10,041 more rows

Exercise 7

Take a sample of size 15 from the population and calculate the proportion of people in this sample who think the work scientists do enhances their lives. Using this sample, what is your best point estimate of the population proportion of people who think the work scientists do enchances their lives?

set.seed(4)  # Set seed for reproducibility
samp3 <- global_monitor %>%
  sample_n(15)  #Taking a sample of size 15

#Counting the number of responses for each category and calculate the proportion
samp3_summary <- samp3 %>%
  count(scientist_work) %>%
  mutate(sam = n / sum(n))  #Calculating the proportion

# Extract the proportion for "Benefits"
benefits_proportion <- samp3_summary %>%
  filter(scientist_work == "Benefits") %>%
  select(sam)  # Get the proportion

benefits_proportion

## # A tibble: 1 × 1
##     sam
##   <dbl>
## 1 0.933

Approximately 73.33% of the sampled individuals think that scientists’ work benefits their lives.

Exercise 8

Since you have access to the population, simulate the sampling distribution of proportion of those who think the work scientists do enchances their lives for samples of size 15 by taking 2000 samples from the population of size 15 and computing 2000 sample proportions. Store these proportions in as sample_props15. Plot the data, then describe the shape of this sampling distribution. Based on this sampling distribution, what would you guess the true proportion of those who think the work scientists do enchances their lives to be? Finally, calculate and report the population proportion.

sample_props15 <- global_monitor %>%
                    rep_sample_n(size = 15, reps = 2000, replace = TRUE) %>%
                    count(scientist_work) %>%
                    mutate(p_hat = n /sum(n)) %>%
                    filter(scientist_work == "Benefits")

sample_props15

## # A tibble: 2,000 × 4
## # Groups:   replicate [2,000]
##    replicate scientist_work     n p_hat
##        <int> <chr>          <int> <dbl>
##  1         1 Benefits          11 0.733
##  2         2 Benefits          13 0.867
##  3         3 Benefits          12 0.8  
##  4         4 Benefits          12 0.8  
##  5         5 Benefits          13 0.867
##  6         6 Benefits          15 1    
##  7         7 Benefits          14 0.933
##  8         8 Benefits          13 0.867
##  9         9 Benefits          12 0.8  
## 10        10 Benefits          13 0.867
## # ℹ 1,990 more rows

ggplot(data = sample_props15, aes(x = p_hat)) +
  geom_histogram(binwidth = 0.02) +
  labs(
    x = "p_hat (Benefits)",
    title = "Sampling distribution of population proportion",
    subtitle = "Sample size = 15, Number of samples = 2000"
  )

mean(sample_props15$p_hat)

## [1] 0.8038667

About 79% of the population truly believe that scientists do enhance their everyday lives.

Exercise 9

Change your sample size from 15 to 150, then compute the sampling distribution using the same method as above, and store these proportions in a new object called sample_props150. Describe the shape of this sampling distribution and compare it to the sampling distribution for a sample size of 15. Based on this sampling distribution, what would you guess to be the true proportion of those who think the work scientists do enchances their lives?

sample_props150 <- global_monitor %>%
                    rep_sample_n(size = 150, reps = 2000, replace = TRUE) %>%
                    count(scientist_work) %>%
                    mutate(p_hat = n /sum(n)) %>%
                    filter(scientist_work == "Benefits") 
sample_props150

## # A tibble: 2,000 × 4
## # Groups:   replicate [2,000]
##    replicate scientist_work     n p_hat
##        <int> <chr>          <int> <dbl>
##  1         1 Benefits         121 0.807
##  2         2 Benefits         123 0.82 
##  3         3 Benefits         113 0.753
##  4         4 Benefits         119 0.793
##  5         5 Benefits         120 0.8  
##  6         6 Benefits         126 0.84 
##  7         7 Benefits         118 0.787
##  8         8 Benefits         124 0.827
##  9         9 Benefits         127 0.847
## 10        10 Benefits         118 0.787
## # ℹ 1,990 more rows

ggplot(data = sample_props150, aes(x = p_hat)) +
  geom_histogram(binwidth = 0.02) +
  labs(
    x = "p_hat (Benefits)",
    title = "Sampling distribution of population proportion",
    subtitle = "Sample size = 150, Number of samples = 2000"
  )

mean(sample_props150$p_hat)

## [1] 0.80095

Exercise 10

Of the sampling distributions from 2 and 3, which has a smaller spread? If you’re concerned with making estimates that are more often close to the true value, would you prefer a sampling distribution with a large or small spread?

Based on the chart, I would conclude that chart 2 has a narrower spread, suggesting that smaller samples are easier to manage. In general, using smaller sample sizes tends to provide a more precise estimate of the population proportion.