Inference for Categorical Data

Getting Started

Load packages

In this lab, we will explore and visualize the data using the tidyverse suite of packages, and perform statistical inference using infer. The data can be found in the companion package for OpenIntro resources, openintro.

Let’s load the packages.

library(tidyverse)
library(openintro)
library(infer)

The data

You will be analyzing the same dataset as in the previous lab, where you delved into a sample from the Youth Risk Behavior Surveillance System (YRBSS) survey, which uses data from high schoolers to help discover health patterns. The dataset is called yrbss.

  1. What are the counts within each category for the amount of days these students have texted while driving within the past 30 days?

Table below:

t(table(yrbss$text_while_driving_30d))
##       
##           0  1-2 10-19 20-29  3-5   30  6-9 did not drive
##   [1,] 4792  925   373   298  493  827  311          4646
  1. What is the proportion of people who have texted while driving every day in the past 30 days and never wear helmets?

3.4%

yrbss_30never <- yrbss %>%
  filter(text_while_driving_30d == "30", helmet_12m == "never")
nrow(yrbss_30never)/nrow(yrbss)
## [1] 0.03408673

Remember that you can use filter to limit the dataset to just non-helmet wearers. Here, we will name the dataset no_helmet.

data('yrbss', package='openintro')
no_helmet <- yrbss %>%
  filter(helmet_12m == "never")

Also, it may be easier to calculate the proportion if you create a new variable that specifies whether the individual has texted every day while driving over the past 30 days or not. We will call this variable text_ind.

no_helmet <- no_helmet %>%
  mutate(text_ind = ifelse(text_while_driving_30d == "30", "yes", "no"))

Inference on proportions

When summarizing the YRBSS, the Centers for Disease Control and Prevention seeks insight into the population parameters. To do this, you can answer the question, “What proportion of people in your sample reported that they have texted while driving each day for the past 30 days?” with a statistic; while the question “What proportion of people on earth have texted while driving each day for the past 30 days?” is answered with an estimate of the parameter.

The inferential tools for estimating population proportion are analogous to those used for means in the last chapter: the confidence interval and the hypothesis test.

set.seed(8586)
no_helmet %>%
  specify(response = text_ind, success = "yes") %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "prop") %>%
  get_ci(level = 0.95)
## # A tibble: 1 × 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1   0.0650   0.0775

Note that since the goal is to construct an interval estimate for a proportion, it’s necessary to both include the success argument within specify, which accounts for the proportion of non-helmet wearers than have consistently texted while driving the past 30 days, in this example, and that stat within calculate is here “prop”, signaling that you are trying to do some sort of inference on a proportion.

  1. What is the margin of error for the estimate of the proportion of non-helmet wearers that have texted while driving each day for the past 30 days based on this survey?

0.625%

lower_ci <- 0.0650
upper_ci <- 0.0775
moe <- (upper_ci - lower_ci)/2
moe
## [1] 0.00625
  1. Using the infer package, calculate confidence intervals for two other categorical variables (you’ll need to decide which level to call “success”, and report the associated margins of error. Interpet the interval in context of the data. It may be helpful to create new data sets for each of the two countries first, and then use these data sets to construct the confidence intervals.

Part 1 - The confidence interval for the proportion of the population that sleeps 9 or more hours for seed 1239 is (5.78%, 6.66%) with a margin of error of 0.44%.

# Calculate confidence interval for Sleep
sleeps_9 <- yrbss %>%
  mutate(sleeps9 = ifelse(school_night_hours_sleep >= "9", "yes", "no"))

set.seed(1239)
sleeps_9 %>%
  specify(response = sleeps9, success = "yes") %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "prop") %>%
  get_ci(level = 0.95)
## # A tibble: 1 × 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1   0.0578   0.0666
# Calculate margin of error for Sleep
lower_ci_sleep <- 0.0578
upper_ci_sleep <- 0.0666
moe_sleep <- (upper_ci_sleep - lower_ci_sleep)/2
moe_sleep
## [1] 0.0044

Part 2 - The confidence interval for the proportion of the population that strength trains 5 or more days per week for seed 1468 is (30.88%, 32.47%) with a margin of error of 0.795%.

# Calculate confidence interval for Strength Training
trains_5 <- yrbss %>%
  mutate(trains5 = ifelse(strength_training_7d >= "5", "yes", "no"))

set.seed(1468)
trains_5 %>%
  specify(response = trains5, success = "yes") %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "prop") %>%
  get_ci(level = 0.95)
## # A tibble: 1 × 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1    0.309    0.325
# Calculate margin of error for Strength Training
lower_ci_train <- 0.3088
upper_ci_train <- 0.3247
moe_train <- (upper_ci_train - lower_ci_train)/2
moe_train
## [1] 0.00795

How does the proportion affect the margin of error?

Imagine you’ve set out to survey 1000 people on two questions: are you at least 6-feet tall? and are you left-handed? Since both of these sample proportions were calculated from the same sample size, they should have the same margin of error, right? Wrong! While the margin of error does change with sample size, it is also affected by the proportion.

Think back to the formula for the standard error: \(SE = \sqrt{p(1-p)/n}\). This is then used in the formula for the margin of error for a 95% confidence interval:

\[ ME = 1.96\times SE = 1.96\times\sqrt{p(1-p)/n} \,. \] Since the population proportion \(p\) is in this \(ME\) formula, it should make sense that the margin of error is in some way dependent on the population proportion. We can visualize this relationship by creating a plot of \(ME\) vs. \(p\).

Since sample size is irrelevant to this discussion, let’s just set it to some value (\(n = 1000\)) and use this value in the following calculations:

n <- 1000

The first step is to make a variable p that is a sequence from 0 to 1 with each number incremented by 0.01. You can then create a variable of the margin of error (me) associated with each of these values of p using the familiar approximate formula (\(ME = 2 \times SE\)).

p <- seq(from = 0, to = 1, by = 0.01)
me <- 2 * sqrt(p * (1 - p)/n)

Lastly, you can plot the two variables against each other to reveal their relationship. To do so, we need to first put these variables in a data frame that you can call in the ggplot function.

dd <- data.frame(p = p, me = me)
ggplot(data = dd, aes(x = p, y = me)) + 
  geom_line() +
  labs(x = "Population Proportion", y = "Margin of Error")

  1. Describe the relationship between p and me. Include the margin of error vs. population proportion plot you constructed in your answer. For a given sample size, for which value of p is margin of error maximized?

The relationship between proportion and margin of error is similar to the relationship between the width of a rectangle to the area of the rectangle when the width plus the length of the rectangle is equal to 1. Margin of error and the area of the rectangle are maximized when p = 0.5 and when width is equal to length for the rectangle. The margin of error vs. population proportion plot is above. The margin of error is maximized when the proportion is 50%.

Success-failure condition

We have emphasized that you must always check conditions before making inference. For inference on proportions, the sample proportion can be assumed to be nearly normal if it is based upon a random sample of independent observations and if both \(np \geq 10\) and \(n(1 - p) \geq 10\). This rule of thumb is easy enough to follow, but it makes you wonder: what’s so special about the number 10?

The short answer is: nothing. You could argue that you would be fine with 9 or that you really should be using 11. What is the “best” value for such a rule of thumb is, at least to some degree, arbitrary. However, when \(np\) and \(n(1-p)\) reaches 10 the sampling distribution is sufficiently normal to use confidence intervals and hypothesis tests that are based on that approximation.

You can investigate the interplay between \(n\) and \(p\) and the shape of the sampling distribution by using simulations. Play around with the following app to investigate how the shape, center, and spread of the distribution of \(\hat{p}\) changes as \(n\) and \(p\) changes.

  1. Describe the sampling distribution of sample proportions at \(n = 300\) and \(p = 0.1\). Be sure to note the center, spread, and shape.

The sampling distribution is centered at 10%. It spreads about 5% to either side and is shaped like a thin bell curve.

  1. Keep \(n\) constant and change \(p\). How does the shape, center, and spread of the sampling distribution vary as \(p\) changes. You might want to adjust min and max for the \(x\)-axis for a better view of the distribution.

As you vary the proportion the sampling distribution center moves to match the proportion. As the proportion approaches 0.5 the spread approaches 10%. As the proportion approaches 0.1 or 0.9 the spread approaches 5%. At a proportion of 0 or 1 the spread is 0%. The shape is most normal at a proportion of 0.5 and thins in either direction.

  1. Now also change \(n\). How does \(n\) appear to affect the distribution of \(\hat{p}\)?

As the sample size decreases the spread increases. At a proportion of .05 the spread increased from 5% with a sample size of 300 to 30% with a sample size of 30.

More Practice

For some of the exercises below, you will conduct inference comparing two proportions. In such cases, you have a response variable that is categorical, and an explanatory variable that is also categorical, and you are comparing the proportions of success of the response variable across the levels of the explanatory variable. This means that when using infer, you need to include both variables within specify.

  1. Is there convincing evidence that those who sleep 10+ hours per day are more likely to strength train every day of the week? As always, write out the hypotheses for any tests you conduct and outline the status of the conditions for inference. If you find a significant difference, also quantify this difference with a confidence interval.

The sample proportion that trains 7 days a week is 15.35%. The sample proportion that trains 7 days a week who slept 10+ hours a school night is 26.58%. The null hypothesis is that there is no significant difference between the proportion that train 7 days a week in the population as a whole versus the population that sleeps 10+ hours a school night. The 95% confidence interval for the null hypothesis is (16.11%, 17.45%) and the sample proportion for those that sleep 10+ hours on a school night is 26.58% which is outside of that confidence interval so we reject the null hypothesis that there is no significant difference. The 15.35% is not in the confidence interval becasue that number did not exclude missing values.

# Here's a table for intuition
# unfortunately I can't reference the cells individually
reftable <- t(table(yrbss$school_night_hours_sleep,yrbss$strength_training_7d))
reftable
##    
##      <5 10+   5   6   7   8   9
##   0 380 100 500 822 946 664 172
##   1  77  17 138 223 303 179  59
##   2  99  31 163 294 367 260  77
##   3  88  31 175 323 428 316  88
##   4  65  18  97 216 325 253  71
##   5  72  23 135 276 390 331  85
##   6  23   8  44  99 153 140  33
##   7 154  84 211 372 521 525 175
# These numbers aren't exclusive of records with missing values so they don't quite line up
# Number that sleep 10+ hours and trains 7 days
s10_t7 <- nrow(filter(yrbss, school_night_hours_sleep == "10+", strength_training_7d == "7"))

# Number that sleep 10+ hours
s10 <- nrow(filter(yrbss, school_night_hours_sleep == "10+"))

# Number that trains 7 days that don't sleep 10+ hours
t7_sother <- nrow(filter(yrbss, school_night_hours_sleep != "10+", strength_training_7d == "7"))

# Total number that don't sleep 10+ hours
all_sother <- nrow(filter(yrbss, school_night_hours_sleep != "10+"))

# Number that trains 7 days that don't sleep 10+ hours
t7 <- nrow(filter(yrbss, strength_training_7d == "7"))

# Total number
all <- nrow(yrbss)
# Sample proportion that train 7 days a week and don't sleep 10+ hours
# Which is what I want to compare so that our samples don't over lap but I don't use here
sp_t7_sother <- t7_sother / all_sother
sp_t7_sother
## [1] 0.1629087
# Sample proportion that train 7 days a week who slept 10+ hours
sp_t7_s10 <- s10_t7 / s10
sp_t7_s10
## [1] 0.2658228
# Sample proportion that train 7 days a week
sp_t7 <- t7 / all
sp_t7
## [1] 0.1535007
# Get the response variable
trains7dpw <- yrbss %>%
  mutate(trains7 = ifelse(strength_training_7d >= "7", "yes", "no"))

# Calculate the confidence interval
set.seed(3251)
trains7dpw %>%
  specify(response = trains7, success = "yes") %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "prop") %>%
  get_ci(level = 0.95)
## # A tibble: 1 × 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1    0.161    0.175

This is how I thought the problem should be calculated, but I could not get response and explanatory variables to work together.

# Get the explanatory variable as a factor
sleeps_trains <- yrbss %>%
  mutate(sleepfactor = as.factor(school_night_hours_sleep))

# Get the response variable
sleeps_trains <- sleeps_trains %>%
  mutate(trains7 = ifelse(strength_training_7d >= "10+", "yes", "no"))

# Calculate the confidence interval
set.seed(2793)
sleeps_trains %>%
  specify(response = trains7, explanatory = sleepfactor, success = "yes") %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "prop") %>%
  get_ci(level = 0.95)
  1. Let’s say there has been no difference in likeliness to strength train every day of the week for those who sleep 10+ hours. What is the probablity that you could detect a change (at a significance level of 0.05) simply by chance? Hint: Review the definition of the Type 1 error.

A Type 1 error means rejecting the null hypothesis when it’s actually true. The probability of making a type 1 error is 1 minus the confidence level. So for a confidence level of 95% you have a 5% chance of making a type 1 error.

If I try to answer it a different way by calculating the margin of error I get a margin of error of 4.1%. This is calculated with 316 records that sleep 10+ hours a night of which 84 also strength train 7 days a week, for a p_hat of 26.58%; a 1 tail 95% confidence level z-score of 1.65; and a sample size of 316. This method is probably plausible-sounding junk.

# How I arrived at 316 recrods that sleep 10+ hours a night
#filter(yrbss, school_night_hours_sleep == "10+")

# How I arrived at 84 records that sleep 10+ hours a night and strength train 7 days a week
#filter(yrbss, school_night_hours_sleep == "10+", strength_training_7d == "7")
  1. Suppose you’re hired by the local government to estimate the proportion of residents that attend a religious service on a weekly basis. According to the guidelines, the estimate must have a margin of error no greater than 1% with 95% confidence. You have no idea what to expect for \(p\). How many people would you have to sample to ensure that you are within the guidelines?
    Hint: Refer to your plot of the relationship between \(p\) and margin of error. This question does not require using a dataset.

The scenario producing the highest margin of error is a proportion of the population who go to church of 50%. Recreating the Population Proportion versus Margin of Error plot from question 5 with a sample size of 10,000 instead of 1,000 shows a margin of error of assumedly less than 0.0100. I assume there are more than 100,000 people in the population so the sample size is less than 10% of the population. Backing into what the z-score would be if 10,000 were the correct answer I get 2 which is less than the z-score for a 99% confidence with one tail of 2.33.

margin of error = z x sqrt(p x (1-p)/n)

0.999% = 2.33 x aqrt(.5 x .5 / 13,600)

If I calculate it with the above formula assuming the z-score is 2.33 for 99% confidence with one tail, then a sample size of 13,600 puts us below a margin of error of 1% assuming a population proportion of 50%. Interestingly that’s very close to the number of records in our data sample (13,583).

n <- 10000
p <- seq(from = 0, to = 1, by = 0.01)
me <- 2 * sqrt(p * (1 - p)/n)

dd <- data.frame(p = p, me = me)
ggplot(data = dd, aes(x = p, y = me)) + 
  geom_line() +
  labs(x = "Population Proportion", y = "Margin of Error")