Lab 08: Inference for categorical data

library(tidyverse)
library(openintro)
library(infer)

We will be using the dataset yrbss for this project.

glimpse(yrbss)

## Rows: 13,583
## Columns: 13
## $ age                      <int> 14, 14, 15, 15, 15, 15, 15, 14, 15, 15, 15, 1~
## $ gender                   <chr> "female", "female", "female", "female", "fema~
## $ grade                    <chr> "9", "9", "9", "9", "9", "9", "9", "9", "9", ~
## $ hispanic                 <chr> "not", "not", "hispanic", "not", "not", "not"~
## $ race                     <chr> "Black or African American", "Black or Africa~
## $ height                   <dbl> NA, NA, 1.73, 1.60, 1.50, 1.57, 1.65, 1.88, 1~
## $ weight                   <dbl> NA, NA, 84.37, 55.79, 46.72, 67.13, 131.54, 7~
## $ helmet_12m               <chr> "never", "never", "never", "never", "did not ~
## $ text_while_driving_30d   <chr> "0", NA, "30", "0", "did not drive", "did not~
## $ physically_active_7d     <int> 4, 2, 7, 0, 2, 1, 4, 4, 5, 0, 0, 0, 4, 7, 7, ~
## $ hours_tv_per_school_day  <chr> "5+", "5+", "5+", "2", "3", "5+", "5+", "5+",~
## $ strength_training_7d     <int> 0, 0, 0, 0, 1, 0, 2, 0, 3, 0, 3, 0, 0, 7, 7, ~
## $ school_night_hours_sleep <chr> "8", "6", "<5", "6", "9", "8", "9", "6", "<5"~

Exercise 1

What are the counts within each category for the amount of days these students have texted while driving within the past 30 days?

Ans. The counts have been displayed below:

count_tWD<-yrbss%>%
  count(text_while_driving_30d)
count_tWD

## # A tibble: 9 x 2
##   text_while_driving_30d     n
##   <chr>                  <int>
## 1 0                       4792
## 2 1-2                      925
## 3 10-19                    373
## 4 20-29                    298
## 5 3-5                      493
## 6 30                       827
## 7 6-9                      311
## 8 did not drive           4646
## 9 <NA>                     918

Exercise 2

What is the proportion of people who have texted while driving every day in the past 30 days and never wear helmets?

Ans.

no_helmet<-yrbss%>%
  filter(helmet_12m=="never")
no_helmet<-no_helmet%>%
  mutate(text_ind=ifelse(text_while_driving_30d=="30","yes","no"))
no_helmet%>%
  count(text_ind)

## # A tibble: 3 x 2
##   text_ind     n
##   <chr>    <int>
## 1 no        6040
## 2 yes        463
## 3 <NA>       474

Through the above results, we can see that 6 percentage of the total sample data have admitted to texting while driving in the past 30 days.

Exercise 3

What is the margin of error for the estimate of the proportion of non-helmet wearers that have texted while driving each day for the past 30 days based on this survey?

Ans. The margin of error for the estimate can be generated using the binomial model as follows: First we estimate the confidence interval for a confidence level of 95%,

no_helmet %>%
  specify(response = text_ind, success = "yes") %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "prop") %>%
  get_ci(level = 0.95)

## Warning: Removed 474 rows containing missing values.

## Warning: You have given `type = "bootstrap"`, but `type` is expected to be
## `"draw"`. This workflow is untested and the results may not mean what you think
## they mean.

## # A tibble: 1 x 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1   0.0647   0.0775

and then we calculate the estimated margin of error:

n=6977
z=1.96 #for 95% CI
p<-seq(from=0, to=1, by=0.01) #from StackOverflow
me<-2*sqrt(p*(1-p)/n)
sum(me)

## [1] 0.9392815

The margin of error is therefore given as 0.939

Exercise 4

Using the infer package, calculate confidence intervals for two other categorical variables (you’ll need to decide which level to call “success”, and report the associated margins of error. Interpret the interval in context of the data. It may be helpful to create new data sets for each of the two countries first, and then use these data sets to construct the confidence intervals.

#Finding out if the students slept well:
sleepwell<-yrbss%>%
  mutate(sleptwell=ifelse(school_night_hours_sleep>8,"yes","no"))
sleepwell%>%
  specify(response=sleptwell, success="yes")%>%
  generate(reps=1000, type="bootstrap")%>%
  calculate(stat="prop")%>%
  get_ci(level=0.95)

## Warning: Removed 1248 rows containing missing values.

## Warning: You have given `type = "bootstrap"`, but `type` is expected to be
## `"draw"`. This workflow is untested and the results may not mean what you think
## they mean.

## # A tibble: 1 x 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1   0.0576   0.0666

We can thus say with a 95% certainty that the proportion of students who slept more than 8 hours ranges from between 0.0577 and 0.0660.

#For this one we are going to find the Confidence Interval on whether the student is an Hispanic not not: here Hispanic will be considered a success
isHispanic<-yrbss%>%
  mutate(Hispanic=ifelse(hispanic=="hispanic","yes","no"))
isHispanic%>%
  specify(response=Hispanic, success="yes")%>%
  generate(reps=1000, type="bootstrap")%>%
  calculate(stat="prop")%>%
  get_ci(level=0.95)

## Warning: Removed 231 rows containing missing values.

## Warning: You have given `type = "bootstrap"`, but `type` is expected to be
## `"draw"`. This workflow is untested and the results may not mean what you think
## they mean.

## # A tibble: 1 x 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1    0.249    0.264

We can thus assert with 95% cofidence that the proportion of hispanic students in the dataset is between the range of 0.2496 to 0.2640.

###Excercise 5 Describe the relationship between p and me. Include the margin of error vs. population proportion plot you constructed in your answer. For a given sample size, for which value of p is margin of error maximized?

n <- 1000
p <- seq(from = 0, to = 1, by = 0.01)
me <- 2 * sqrt(p * (1 - p)/n)
dd <- data.frame(p = p, me = me)
ggplot(data = dd, aes(x = p, y = me)) + 
  geom_line() +
  labs(x = "Population Proportion", y = "Margin of Error")

The margin of error seems to increase as the population proportion increases, until it reaches 50%, then it starts to dip back to zero.

Exercise 6

Describe the sampling distribution of sample proportions at n=300 and p=0.1. Be sure to note the center, spread, and shape.

Ans. The sampling distribuition appears to be normal. The center appears to be near 0.1, and the spread is between 0.09 and 0.175.

Exercise 7

Keep n constant and change p. How does the shape, center, and spread of the sampling distribution vary as p changes. You might want to adjust min and max for the x-axis for a better view of the distribution.

Ans. As one keeps n constant and changes p, the shape appears to remain the same, although the graph starts shifting towards the right. The spread appears to remain normal.

Exercise 8

Now also change n. How does n appear to affect the distribution of p^?

Ans. As n increases the distribution of the p hats decreases.

Exercise 9

Is there convincing evidence that those who sleep 10+ hours per day are more likely to strength train every day of the week? As always, write out the hypotheses for any tests you conduct and outline the status of the conditions for inference. If you find a significant difference, also quantify this difference with a confidence interval.

sleepMore<-yrbss%>%
  filter(school_night_hours_sleep=="10+")
sleepMore<-sleepMore%>%
  mutate(text_ind=ifelse(strength_training_7d=="7","yes","no"))
no_helmet%>%
  count(text_ind)

## # A tibble: 3 x 2
##   text_ind     n
##   <chr>    <int>
## 1 no        6040
## 2 yes        463
## 3 <NA>       474

# Using a confidence interval of 95%
sleepMore %>%
  specify(response = text_ind, success = "yes") %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "prop") %>%
  get_ci(level = 0.95)

## Warning: Removed 4 rows containing missing values.

## Warning: You have given `type = "bootstrap"`, but `type` is expected to be
## `"draw"`. This workflow is untested and the results may not mean what you think
## they mean.

## # A tibble: 1 x 2
##   lower_ci upper_ci
##      <dbl>    <dbl>
## 1    0.218    0.321

Through the above code segment, we see that there is no such correlation between the two factors, and the extremely low values of the confidence intervals seems to indicate low levels of correlation.

Exercise 10

Let’s say there has been no difference in likeliness to strength train every day of the week for those who sleep 10+ hours. What is the probability that you could detect a change (at a significance level of 0.05) simply by chance? Hint: Review the definition of the Type 1 error.

Ans. We would have 5% chance of detecting a change. A type 1 error is the mistaken rejection of a false positive.

Exercise 11

Suppose you’re hired by the local government to estimate the proportion of residents that attend a religious service on a weekly basis. According to the guidelines, the estimate must have a margin of error no greater than 1% with 95% confidence. You have no idea what to expect for p. How many people would you have to sample to ensure that you are within the guidelines? Hint: Refer to your plot of the relationship between p and margin of error. This question does not require using a dataset.

Ans. Assuming a value of p=0.1, I would expect to sample 3458 people of the population; since the more I increase my p, the more people I have to sample, I am confident that I can get a 95% confidence interval is I increase my p to 1.5, which leads me to sample approximately 4899 people.