Inference for Categorical Data

library(tidyverse)
library(openintro)

Exercise 1

In the first paragraph, several key findings are reported. Do these percentages appear to be sample statistics (derived from the data sample) or population parameters?

“59% of the world said that they think of themselves as religious person, 23% think of themselves as not religious whereas 13% think of themselves as convinced atheists.”

These are population parameters.

Exercise 2

The title of the report is “Global Index of Religiosity and Atheism”. To generalize the report’s findings to the global human population, what must we assume about the sampling method? Does that seem like a reasonable assumption?

We must assume that the samples are independent and random. I imagine it is hard to completely eliminate bias in a study like this but it may be close enough.

#download.file("http://www.openintro.org/stat/data/atheism.RData", destfile = "atheism.RData")
load("atheism.RData")

Exercise 3

What does each row of Table 6 correspond to? What does each row of atheism correspond to?

Each row in the table corresponds to a country. Each row in atheism corresponds to a single person/response.

Exercise 4

Using the command below, create a new dataframe called us12 that contains only the rows in atheism associated with respondents to the 2012 survey from the United States. Next, calculate the proportion of atheist responses. Does it agree with the percentage in Table 6? If not, why?

us12 <- subset(atheism, nationality == "United States" & year == "2012")

atheists_12 <- us12 %>% filter(response == "atheist")

table(us12)

## , , year = 2012
## 
##                                               response
## nationality                                    atheist non-atheist
##   Afghanistan                                        0           0
##   Argentina                                          0           0
##   Armenia                                            0           0
##   Australia                                          0           0
##   Austria                                            0           0
##   Azerbaijan                                         0           0
##   Belgium                                            0           0
##   Bosnia and Herzegovina                             0           0
##   Brazil                                             0           0
##   Bulgaria                                           0           0
##   Cameroon                                           0           0
##   Canada                                             0           0
##   China                                              0           0
##   Colombia                                           0           0
##   Czech Republic                                     0           0
##   Ecuador                                            0           0
##   Fiji                                               0           0
##   Finland                                            0           0
##   France                                             0           0
##   Georgia                                            0           0
##   Germany                                            0           0
##   Ghana                                              0           0
##   Hong Kong                                          0           0
##   Iceland                                            0           0
##   India                                              0           0
##   Iraq                                               0           0
##   Ireland                                            0           0
##   Italy                                              0           0
##   Japan                                              0           0
##   Kenya                                              0           0
##   Korea, Rep (South)                                 0           0
##   Lebanon                                            0           0
##   Lithuania                                          0           0
##   Macedonia                                          0           0
##   Malaysia                                           0           0
##   Moldova                                            0           0
##   Netherlands                                        0           0
##   Nigeria                                            0           0
##   Pakistan                                           0           0
##   Palestinian territories (West Bank and Gaza)       0           0
##   Peru                                               0           0
##   Poland                                             0           0
##   Romania                                            0           0
##   Russian Federation                                 0           0
##   Saudi Arabia                                       0           0
##   Serbia                                             0           0
##   South Africa                                       0           0
##   South Sudan                                        0           0
##   Spain                                              0           0
##   Sweden                                             0           0
##   Switzerland                                        0           0
##   Tunisia                                            0           0
##   Turkey                                             0           0
##   Ukraine                                            0           0
##   United States                                     50         952
##   Uzbekistan                                         0           0
##   Vietnam                                            0           0

50/1002, or about 4.99% of the responses were “atheist”. This agrees with table 6.

Exercise 5

Write out the conditions for inference to construct a 95% confidence interval for the proportion of atheists in the United States in 2012. Are you confident all conditions are met?

For inference on proportions, the sample proportion can be assumed to be nearly normal if it is based upon a random sample of independent observations and if both np≥10 and n(1−p)≥10.

inference(us12$response, est = "proportion", type = "ci", method = "theoretical", 
          success = "atheist")

## Single proportion -- success: atheist 
## Summary statistics:

## p_hat = 0.0499 ;  n = 1002 
## Check conditions: number of successes = 50 ; number of failures = 952 
## Standard error = 0.0069 
## 95 % Confidence interval = ( 0.0364 , 0.0634 )

Exercise 6

Based on the R output, what is the margin of error for the estimate of the proportion of the proportion of atheists in US in 2012?

.0499 +/- 0.0135, or, 1.35% margin of error at 95% confidence

Exercise 7

Using the inference function, calculate confidence intervals for the proportion of atheists in 2012 in two other countries of your choice, and report the associated margins of error. Be sure to note whether the conditions for inference are met. It may be helpful to create new data sets for each of the two countries first, and then use these data sets in the inference function to construct the confidence intervals.

Iceland12 <- subset(atheism, nationality == "Iceland" & year == "2012")
Ukraine12 <- subset(atheism, nationality == "Ukraine" & year == "2012")

inference(Iceland12$response, est = "proportion", type = "ci", method = "theoretical", 
          success = "atheist")

## Single proportion -- success: atheist 
## Summary statistics:

## p_hat = 0.0998 ;  n = 852 
## Check conditions: number of successes = 85 ; number of failures = 767 
## Standard error = 0.0103 
## 95 % Confidence interval = ( 0.0796 , 0.1199 )

inference(Ukraine12$response, est = "proportion", type = "ci", method = "theoretical", 
          success = "atheist")

## Single proportion -- success: atheist 
## Summary statistics:

## p_hat = 0.0296 ;  n = 1013 
## Check conditions: number of successes = 30 ; number of failures = 983 
## Standard error = 0.0053 
## 95 % Confidence interval = ( 0.0192 , 0.0401 )

n <- 1000
p <- seq(0, 1, 0.01)
me <- 2 * sqrt(p * (1 - p)/n)
plot(me ~ p, ylab = "Margin of Error", xlab = "Population Proportion")

Exercise 8

Describe the relationship between p and me.

Me is a non-negative function of population proportion on [0, 1]. The curve looks like the arc of a circle with a max of 0.03 at p = 0.5 and minimums of 0 at p = 0.0 and p = 1.0

p <- 0.1
n <- 1040
p_hats <- rep(0, 5000)

for(i in 1:5000){
  samp <- sample(c("atheist", "non_atheist"), n, replace = TRUE, prob = c(p, 1-p))
  p_hats[i] <- sum(samp == "atheist")/n
}

hist(p_hats, main = "p = 0.1, n = 1040", xlim = c(0, 0.18))

Exercise 9

Describe the sampling distribution of sample proportions at n=1040 and p=0.1. Be sure to note the center, spread, and shape. Hint: Remember that R has functions such as mean to calculate summary statistics.

mean(p_hats)

## [1] 0.09969

sd(p_hats)

## [1] 0.009287382

summary(p_hats)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
## 0.07019 0.09327 0.09904 0.09969 0.10577 0.12981

Exercise 10

Repeat the above simulation three more times but with modified sample sizes and proportions: for n=400 and p=0.1, n=1040 and p=0.02, and n=400 and p=0.02. Plot all four histograms together by running the par(mfrow = c(2, 2)) command before creating the histograms. You may need to expand the plot window to accommodate the larger two-by-two plot. Describe the three new sampling distributions. Based on these limited plots, how does n appear to affect the distribution of p̂ ? How does p affect the sampling distribution?

par(mfrow = c(2, 2))


p_1 <- 0.1
n_1 <- 400
p_hats_1 <- rep(0, 5000)

for(i in 1:5000){
  samp_1 <- sample(c("atheist", "non_atheist"), n_1, replace = TRUE, prob = c(p_1, 1-p_1))
  p_hats_1[i] <- sum(samp_1 == "atheist")/n_1
}

p_2 <- 0.02
n_2 <- 1040
p_hats_2 <- rep(0, 5000)

for(i in 1:5000){
  samp_2 <- sample(c("atheist", "non_atheist"), n_2, replace = TRUE, prob = c(p_2, 1-p_2))
  p_hats_2[i] <- sum(samp_2 == "atheist")/n_2
}

p_3 <- 0.02
n_3 <- 400
p_hats_3 <- rep(0, 5000)

for(i in 1:5000){
  samp_3 <- sample(c("atheist", "non_atheist"), n_3, replace = TRUE, prob = c(p_3, 1-p_3))
  p_hats_3[i] <- sum(samp_3 == "atheist")/n_3
}


hist(p_hats, main = "p = 0.1, n = 1040", xlim = c(0, 0.18))
hist(p_hats_1, main = "p = 0.1, n = 400", xlim = c(0, 0.18))
hist(p_hats_2, main = "p = 0.02, n = 1040", xlim = c(0, 0.18))
hist(p_hats_3, main = "p = 0.02, n = 400", xlim = c(0, 0.18))

sd(p_hats)

## [1] 0.009287382

sd(p_hats_1)

## [1] 0.01504642

sd(p_hats_2)

## [1] 0.004422505

sd(p_hats_3)

## [1] 0.006976835

mean(p_hats)

## [1] 0.09969

mean(p_hats_1)

## [1] 0.099759

mean(p_hats_2)

## [1] 0.01995423

mean(p_hats_3)

## [1] 0.0198785

Changing p changed the mean of the resulting sampling distribution; changing n increased the standard error.

Exercise 11

If you refer to Table 6, you’ll find that Australia has a sample proportion of 0.1 on a sample size of 1040, and that Ecuador has a sample proportion of 0.02 on 400 subjects. Let’s suppose for this exercise that these point estimates are actually the truth. Then given the shape of their respective sampling distributions, do you think it is sensible to proceed with inference and report margin of errors, as the reports does?

On the bottom of page 7, the report states that “In general the error margin for surveys of this kind is +3-5% at 95% confidence level.” This almost sounds a bit evasive; they aren’t disclosing their own margin of error but stating something like the industry standard and heavily implying that they are in compliance.

It doesn’t strike me as unethical, necessarily, if the report is roughly up to this standard; a line has to be drawn somewhere about what to include and what to summarize. But it does seem to me that something is being left out, if the observations in a particular table are not all equally reliable.

On your own The question of atheism was asked by WIN-Gallup International in a similar survey that was conducted in 2005. (We assume here that sample sizes have remained the same.) Table 4 on page 13 of the report summarizes survey results from 2005 and 2012 for 39 countries.

Answer the following two questions using the inference function. As always, write out the hypotheses for any tests you conduct and outline the status of the conditions for inference.

1. Is there convincing evidence that Spain has seen a change in its atheism index between 2005 and 2012? Hint: Create a new data set for respondents from Spain. Form confidence intervals for the true proportion of atheists in both years, and determine whether they overlap.

Sp05 <- subset(atheism, nationality == "Spain" & year == "2005")
Sp12 <- subset(atheism, nationality == "Spain" & year == "2012")

inference(Sp05$response, est = "proportion", type = "ci", method = "theoretical", 
          success = "atheist")

## Single proportion -- success: atheist 
## Summary statistics:

## p_hat = 0.1003 ;  n = 1146 
## Check conditions: number of successes = 115 ; number of failures = 1031 
## Standard error = 0.0089 
## 95 % Confidence interval = ( 0.083 , 0.1177 )

inference(Sp12$response, est = "proportion", type = "ci", method = "theoretical", 
          success = "atheist")

## Single proportion -- success: atheist 
## Summary statistics:

## p_hat = 0.09 ;  n = 1145 
## Check conditions: number of successes = 103 ; number of failures = 1042 
## Standard error = 0.0085 
## 95 % Confidence interval = ( 0.0734 , 0.1065 )

The confidence intervals overlap, so there is not compelling evidence that Spain has had a decrease in atheism between 2005 and 2012.

2. Is there convincing evidence that the United States has seen a change in its atheism index between 2005 and 2012?

us05 <- subset(atheism, nationality == "United States" & year == "2005")

inference(us05$response, est = "proportion", type = "ci", method = "theoretical", 
          success = "atheist")

## Single proportion -- success: atheist 
## Summary statistics:

## p_hat = 0.01 ;  n = 1002 
## Check conditions: number of successes = 10 ; number of failures = 992 
## Standard error = 0.0031 
## 95 % Confidence interval = ( 0.0038 , 0.0161 )

inference(us12$response, est = "proportion", type = "ci", method = "theoretical", 
          success = "atheist")

## Single proportion -- success: atheist 
## Summary statistics:

## p_hat = 0.0499 ;  n = 1002 
## Check conditions: number of successes = 50 ; number of failures = 952 
## Standard error = 0.0069 
## 95 % Confidence interval = ( 0.0364 , 0.0634 )

The 95% confidence interval for 2012 is greater than that of 2005, so we have compelling evidence that the atheism index increased during those years

If in fact there has been no change in the atheism index in the countries listed in Table 4, in how many of those countries would you expect to detect a change (at a significance level of 0.05) simply by chance? Hint: Look in the textbook index under Type 1 error.

a can be interpreted as the probability of finding significant evidence for H_A given that H₀ is true, so in this case we would expect to find a difference in 5% of the countries.

Suppose you’re hired by the local government to estimate the proportion of residents that attend a religious service on a weekly basis. According to the guidelines, the estimate must have a margin of error no greater than 1% with 95% confidence. You have no idea what to expect for p. How many people would you have to sample to ensure that you are within the guidelines? Hint: Refer to your plot of the relationship between p and margin of error. Do not use the data set to answer this question.

n <- 10000
p <- seq(0, 1, 0.01)
me <- 2 * sqrt(p * (1 - p)/n)
plot(me ~ p, ylab = "Margin of Error", xlab = "Population Proportion")

For n = 10000, the margin of error does not exceed .01 for any value of p.