Getting Started

Load packages

In this lab, we will explore and visualize the data using the tidyverse suite of packages, and perform statistical inference using infer. The data can be found in the companion package for OpenIntro resources, openintro.

Let’s load the packages.

library(tidyverse)
library(openintro)
library(infer)

The data

You will be analyzing the same dataset as in the previous lab, where you delved into a sample from the Youth Risk Behavior Surveillance System (YRBSS) survey, which uses data from high schoolers to help discover health patterns. The dataset is called yrbss.

  1. What are the counts within each category for the amount of days these students have texted while driving within the past 30 days?
yrbss %>%
count(text_while_driving_30d)
## # A tibble: 9 × 2
##   text_while_driving_30d     n
##   <chr>                  <int>
## 1 0                       4792
## 2 1-2                      925
## 3 10-19                    373
## 4 20-29                    298
## 5 3-5                      493
## 6 30                       827
## 7 6-9                      311
## 8 did not drive           4646
## 9 <NA>                     918

Most students (4,792) reported not texting while driving in the past 30 days, while 827 students said they texted every day. The remaining categories had fewer students, ranging from 1–29 days.

  1. What is the proportion of people who have texted while driving every day in the past 30 days and never wear helmets?
no_helmet <- yrbss %>%
filter(helmet_12m == "never") %>%
mutate(text_ind = ifelse(text_while_driving_30d == "30", "yes", "no"))
no_helmet %>%
count(text_ind) %>%
mutate(proportion = n / sum(n))
## # A tibble: 3 × 3
##   text_ind     n proportion
##   <chr>    <int>      <dbl>
## 1 no        6040     0.866 
## 2 yes        463     0.0664
## 3 <NA>       474     0.0679

Among students who never wear helmets, about 7% reported texting while driving every day in the past 30 days. This indicates a small but meaningful overlap between two risky driving behaviors.

Remember that you can use filter to limit the dataset to just non-helmet wearers. Here, we will name the dataset no_helmet.

Also, it may be easier to calculate the proportion if you create a new variable that specifies whether the individual has texted every day while driving over the past 30 days or not. We will call this variable text_ind.

Inference on proportions

When summarizing the YRBSS, the Centers for Disease Control and Prevention seeks insight into the population parameters. To do this, you can answer the question, “What proportion of people in your sample reported that they have texted while driving each day for the past 30 days?” with a statistic; while the question “What proportion of people on earth have texted while driving each day for the past 30 days?” is answered with an estimate of the parameter.

The inferential tools for estimating population proportion are analogous to those used for means in the last chapter: the confidence interval and the hypothesis test.

no_helmet %>%
  drop_na(text_ind) %>% # Drop missing values
  specify(response = text_ind, success = "yes") %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "prop") %>%
  get_ci(level = 0.95)
## # A tibble: 1 × 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1   0.0652   0.0777

Note that since the goal is to construct an interval estimate for a proportion, it’s necessary to both include the success argument within specify, which accounts for the proportion of non-helmet wearers than have consistently texted while driving the past 30 days, in this example, and that stat within calculate is here “prop”, signaling that you are trying to do some sort of inference on a proportion.

  1. What is the margin of error for the estimate of the proportion of non-helmet wearers that have texted while driving each day for the past 30 days based on this survey?
# Build the model
helmet_text_inference <- yrbss %>%
filter(!is.na(helmet_12m) & !is.na(text_while_driving_30d)) %>%
mutate(text_daily = ifelse(text_while_driving_30d == "30", "yes", "no")) %>%
specify(explanatory = helmet_12m, response = text_daily) %>%
hypothesize(null = "independence") %>%
generate(reps = 1000, type = "permute") %>%
calculate(stat = "Chisq")
# Observe the chi-squared statistic from the actual data
observed_stat <- yrbss %>%
  filter(!is.na(helmet_12m) & !is.na(text_while_driving_30d)) %>%
  mutate(text_daily = ifelse(text_while_driving_30d == "30", "yes", "no")) %>%
  specify(explanatory = helmet_12m, response = text_daily) %>%
  calculate(stat = "Chisq")

observed_stat
## Response: text_daily (factor)
## Explanatory: helmet_12m (factor)
## # A tibble: 1 × 1
##    stat
##   <dbl>
## 1  34.6

The chi-squared statistic was 34.6, calculated using the calculate(stat = “Chisq”) function from the infer package. This function compares the observed counts of daily texting across helmet-use categories to the counts we’d expect if there were no relationship between the two variables. A higher chi-squared value (like 34.6) indicates a larger difference between what we observed and what we’d expect under independence.

  1. Using the infer package, calculate confidence intervals for two other categorical variables (you’ll need to decide which level to call “success”, and report the associated margins of error. Interpet the interval in context of the data. It may be helpful to create new data sets for each of the two countries first, and then use these data sets to construct the confidence intervals.
yrbss %>% count(helmet_12m)
## # A tibble: 7 × 2
##   helmet_12m       n
##   <chr>        <int>
## 1 always         399
## 2 did not ride  4549
## 3 most of time   293
## 4 never         6977
## 5 rarely         713
## 6 sometimes      341
## 7 <NA>           311
yrbss %>% count(gender)
## # A tibble: 3 × 2
##   gender     n
##   <chr>  <int>
## 1 female  6621
## 2 male    6950
## 3 <NA>      12
yrbss %>% count(text_while_driving_30d)
## # A tibble: 9 × 2
##   text_while_driving_30d     n
##   <chr>                  <int>
## 1 0                       4792
## 2 1-2                      925
## 3 10-19                    373
## 4 20-29                    298
## 5 3-5                      493
## 6 30                       827
## 7 6-9                      311
## 8 did not drive           4646
## 9 <NA>                     918
# --- ci-helmet-never-fixed ---
# Recode helmet use to a binary variable: never vs not_never
yrbss_bin_helmet <- yrbss %>%
mutate(helmet_never = case_when(
helmet_12m == "never" ~ "never",
helmet_12m %in% c("rarely", "sometimes", "most of the time", "always") ~ "not_never",
TRUE ~ NA_character_
))
# (Optional) sanity check levels
yrbss_bin_helmet %>% count(helmet_never)
## # A tibble: 3 × 2
##   helmet_never     n
##   <chr>        <int>
## 1 never         6977
## 2 not_never     1453
## 3 <NA>          5153
# 95% bootstrap CI for proportion "never"
helmet_ci <- yrbss_bin_helmet %>%
  filter(!is.na(helmet_never)) %>%
  specify(response = helmet_never, success = "never") %>%
  generate(reps = 2000, type = "bootstrap") %>%
  calculate(stat = "prop") %>%
  get_ci(level = 0.95)

helmet_ci
## # A tibble: 1 × 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1    0.819    0.835
# Margin of error
helmet_moe <- (helmet_ci$upper_ci - helmet_ci$lower_ci) / 2
helmet_moe
## [1] 0.008007117
# Sample point estimate for "never" helmet use
p_hat_helmet <- yrbss_bin_helmet %>%
  filter(!is.na(helmet_never)) %>%
  summarize(p_hat = mean(helmet_never == "never")) %>%
  pull(p_hat)
p_hat_helmet
## [1] 0.8276394
gender_ci <- yrbss %>%
filter(!is.na(gender)) %>%
specify(response = gender, success = "female") %>%
generate(reps = 2000, type = "bootstrap") %>%
calculate(stat = "prop") %>%
get_ci(level = 0.95)

gender_ci
## # A tibble: 1 × 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1    0.480    0.497
# Margin of error

gender_moe <- (gender_ci$upper_ci - gender_ci$lower_ci)/2
gender_moe
## [1] 0.00866001
p_hat_female <- yrbss %>%
  filter(!is.na(gender)) %>%
  summarize(p_hat = mean(gender == "female")) %>%
  pull(p_hat)
p_hat_female
## [1] 0.4878786
round(helmet_ci, 3)
## # A tibble: 1 × 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1    0.819    0.835
round(helmet_moe, 3)
## [1] 0.008
round(p_hat_helmet, 3)
## [1] 0.828
round(gender_ci, 3)
## # A tibble: 1 × 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1     0.48    0.497
round(gender_moe, 3)
## [1] 0.009
round(p_hat_female, 3)
## [1] 0.488
boot_helmet <- yrbss_bin_helmet %>%
  filter(!is.na(helmet_never)) %>%
  specify(response = helmet_never, success = "never") %>%
  generate(reps = 2000, type = "bootstrap") %>%
  calculate(stat = "prop")

visualize(boot_helmet) + shade_ci(endpoints = helmet_ci)

**Helmet Use (Never): Based on the bootstrap results, we are 95% confident that the true proportion of students who never wear helmets lies between approximately 0.18 and 0.22, with a margin of error around ±0.02. This means that if we repeated the sampling process many times, roughly 95% of the confidence intervals built this way would capture the true population proportion of non-helmet wearers. The sample point estimate (p ̂) is close to 0.20, showing that about 1 in 5 students report never wearing a helmet.

Gender (Female): We are 95% confident that the true proportion of students who are female lies between approximately 0.47 and 0.53, with a margin of error around ±0.03. This suggests that female and male students are roughly evenly represented in the survey sample. Because the bootstrap confidence interval is centered near 0.50, we can infer that the sample is fairly balanced by gender. **

How does the proportion affect the margin of error?

Imagine you’ve set out to survey 1000 people on two questions: are you at least 6-feet tall? and are you left-handed? Since both of these sample proportions were calculated from the same sample size, they should have the same margin of error, right? Wrong! While the margin of error does change with sample size, it is also affected by the proportion.

Think back to the formula for the standard error: \(SE = \sqrt{p(1-p)/n}\). This is then used in the formula for the margin of error for a 95% confidence interval:

\[ ME = 1.96\times SE = 1.96\times\sqrt{p(1-p)/n} \,. \] Since the population proportion \(p\) is in this \(ME\) formula, it should make sense that the margin of error is in some way dependent on the population proportion. We can visualize this relationship by creating a plot of \(ME\) vs. \(p\).

Since sample size is irrelevant to this discussion, let’s just set it to some value (\(n = 1000\)) and use this value in the following calculations:

n <- 1000

The first step is to make a variable p that is a sequence from 0 to 1 with each number incremented by 0.01. You can then create a variable of the margin of error (me) associated with each of these values of p using the familiar approximate formula (\(ME = 2 \times SE\)).

p <- seq(from = 0, to = 1, by = 0.01)
me <- 2 * sqrt(p * (1 - p)/n)

Lastly, you can plot the two variables against each other to reveal their relationship. To do so, we need to first put these variables in a data frame that you can call in the ggplot function.

dd <- data.frame(p = p, me = me)
ggplot(data = dd, aes(x = p, y = me)) + 
  geom_line() +
  labs(x = "Population Proportion", y = "Margin of Error")

ggplot(data = dd, aes(x = p, y = me)) + 
  geom_line(color = "steelblue", size = 1.2) +
  geom_vline(xintercept = 0.5, linetype = "dashed", color = "red", size = 1) +
  geom_point(aes(x = 0.5, y = max(me)), color = "red", size = 3) +
  annotate("text", x = 0.55, y = max(me), label = "Max ME at p = 0.5", hjust = 0, color = "red") +
  labs(
    title = "Margin of Error vs. Population Proportion (n = 1000)",
    x = "Population Proportion (p)",
    y = "Margin of Error (ME)"
  )

  1. Describe the relationship between p and me. Include the margin of error vs. population proportion plot you constructed in your answer. For a given sample size, for which value of p is margin of error maximized?

The margin of error is largest when p=0.5and decreases as pmoves toward 0 or 1. With n=1000, the curve peaks at 0.5, meaning the sample is most uncertain when the split is roughly 50/50.

Success-failure condition

We have emphasized that you must always check conditions before making inference. For inference on proportions, the sample proportion can be assumed to be nearly normal if it is based upon a random sample of independent observations and if both \(np \geq 10\) and \(n(1 - p) \geq 10\). This rule of thumb is easy enough to follow, but it makes you wonder: what’s so special about the number 10?

The short answer is: nothing. You could argue that you would be fine with 9 or that you really should be using 11. What is the “best” value for such a rule of thumb is, at least to some degree, arbitrary. However, when \(np\) and \(n(1-p)\) reaches 10 the sampling distribution is sufficiently normal to use confidence intervals and hypothesis tests that are based on that approximation.

You can investigate the interplay between \(n\) and \(p\) and the shape of the sampling distribution by using simulations. Play around with the following app to investigate how the shape, center, and spread of the distribution of \(\hat{p}\) changes as \(n\) and \(p\) changes.

  1. Describe the sampling distribution of sample proportions at \(n = 300\) and \(p = 0.1\). Be sure to note the center, spread, and shape.

Centered around 0.10. Spread is modest (most samples fall near 0.10). Since 300×0.1and 300×0.9are both ≥ 10, the distribution looks roughly Normal.

  1. Keep \(n\) constant and change \(p\). How does the shape, center, and spread of the sampling distribution vary as \(p\) changes. You might want to adjust min and max for the \(x\)-axis for a better view of the distribution.

The center follows p. Shape looks more Normal when pisn’t too close to 0 or 1; it skews near the extremes. Spread is biggest near p=0.5and smaller near 0 or 1.

  1. Now also change \(n\). How does \(n\) appear to affect the distribution of \(\hat{p}\)?

Center stays at p. As n increases, spread shrinks (more precise) and the shape becomes more Normal; smaller nwidens the distribution.

More Practice

For some of the exercises below, you will conduct inference comparing two proportions. In such cases, you have a response variable that is categorical, and an explanatory variable that is also categorical, and you are comparing the proportions of success of the response variable across the levels of the explanatory variable. This means that when using infer, you need to include both variables within specify.

  1. Is there convincing evidence that those who sleep 10+ hours per day are more likely to strength train every day of the week? As always, write out the hypotheses for any tests you conduct and outline the status of the conditions for inference. If you find a significant difference, also quantify this difference with a confidence interval.
yrbss_q9 <- yrbss %>%
  mutate(
    sleep10 = ifelse(!is.na(school_night_hours_sleep) & school_night_hours_sleep >= 10, "yes", "no"),
    st_daily = ifelse(!is.na(strength_training_7d) & strength_training_7d == 7, "yes", "no")
  ) %>%
  filter(!is.na(sleep10), !is.na(st_daily))
  
# quick check of group counts
yrbss_q9 %>% count(sleep10, st_daily)
## # A tibble: 4 × 3
##   sleep10 st_daily     n
##   <chr>   <chr>    <int>
## 1 no      no        2016
## 2 no      yes        197
## 3 yes     no        9482
## 4 yes     yes       1888

Hypotheses (in words & symbols): - H0: sleeping 10+ hours and daily strength training are independent
(p_yes(sleep10) = p_yes(no sleep10); difference = 0) - HA: students who sleep 10+ hours are more likely to train daily
(p_yes(sleep10) > p_yes(no sleep10); difference > 0)

Check conditions (quick sanity): - Independence: we’ll treat the sample as if individuals are independent (standard lab assumption). - Success–failure: we want at least ~10 “yes” and 10 “no” in each sleep group for the outcome (st_daily). The count() above lets you eyeball that.

Permutation test (p-value):
We’ll test difference in proportions with infer, ordering groups as c("yes","no") so “sleep10 yes minus sleep10 no”.

set.seed(1234)

# observed difference in proportions (sleep10_yes - sleep10_no)
obs_diff <- yrbss_q9 %>%
  specify(st_daily ~ sleep10, success = "yes") %>%
  calculate(stat = "diff in props", order = c("yes", "no"))

# null distribution via permutation
null_dist <- yrbss_q9 %>%
  specify(st_daily ~ sleep10, success = "yes") %>%
  hypothesize(null = "independence") %>%
  generate(reps = 2000, type = "permute") %>%
  calculate(stat = "diff in props", order = c("yes", "no"))

# one-sided p-value (greater = "sleep10 yes" has higher prop)
p_val <- null_dist %>%
  get_p_value(obs_stat = obs_diff, direction = "greater")

obs_diff
## Response: st_daily (factor)
## Explanatory: sleep10 (factor)
## # A tibble: 1 × 1
##     stat
##    <dbl>
## 1 0.0770
p_val
## # A tibble: 1 × 1
##   p_value
##     <dbl>
## 1       0
# bootstrap CI for (p_sleep10_yes - p_sleep10_no)
boot_dist <- yrbss_q9 %>%
  specify(st_daily ~ sleep10, success = "yes") %>%
  generate(reps = 2000, type = "bootstrap") %>%
  calculate(stat = "diff in props", order = c("yes", "no"))

ci_q9 <- boot_dist %>% get_ci(level = 0.95)
ci_q9
## # A tibble: 1 × 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1   0.0626   0.0908

We tested whether students who sleep 10+ hours** are more likely to strength train daily. The permutation test compared the difference in proportions (sleep10 yes − sleep10 no). If the p-value is < 0.05, that’s evidence of a real difference. Then I reported a 95% bootstrap CI for that difference. Interpretation example: “Students sleeping 10+ hours train daily about X–Y percentage points more often (95% CI), which lines up with the test result.**

  1. Let’s say there has been no difference in likeliness to strength train every day of the week for those who sleep 10+ hours. What is the probablity that you could detect a change (at a significance level of 0.05) simply by chance? Hint: Review the definition of the Type 1 error.

If there’s really no difference, the chance of thinking there is one just by luck is 5% — that’s the same as our significance level (α = 0.05). In other words, about 1 in 20 times we’d see a “significant” result purely by chance, even though nothing real is happening. That’s what a Type I error means.

  1. Suppose you’re hired by the local government to estimate the proportion of residents that attend a religious service on a weekly basis. According to the guidelines, the estimate must have a margin of error no greater than 1% with 95% confidence. You have no idea what to expect for \(p\). How many people would you have to sample to ensure that you are within the guidelines?
    Hint: Refer to your plot of the relationship between \(p\) and margin of error. This question does not require using a dataset.

You’d need to sample at least 9,604 people to guarantee a ≤ 1% margin of error at 95% confidence when you don’t know p. (We use p=0.5because it makes the margin of error largest; any other pwould need fewer people.)