Submitted by: Preetha Rajan

Email: praj016@aucklanduni.ac.nz

Complete all Exercises, and submit answers to Questions on the Coursera platform.

In August of 2012, news outlets ranging from the Washington Post to the Huffington Post ran a story about the rise of atheism in America. The source for the story was a poll that asked people, “Irrespective of whether you attend a place of worship or not, would you say you are a religious person, not a religious person or a convinced atheist?” This type of question, which asks people to classify themselves in one way or another, is common in polling and generates categorical data. In this lab we take a look at the atheism survey and explore what’s at play when making inference about population proportions using categorical data.

Getting Started

Load packages

In this lab we will explore the data using the dplyr package and visualize it using the ggplot2 package for data visualization. The data can be found in the companion package for this course, statsr.

Let’s load the packages.

library(statsr)
library(dplyr)
library(ggplot2)
library(varhandle)
## Warning: package 'varhandle' was built under R version 3.5.2
library(janitor)
## Warning: package 'janitor' was built under R version 3.5.2

The survey

The press release for the poll, conducted by WIN-Gallup International, can be accessed here.

Table 1

Table 1

Take a moment to review the report then address the following questions.

  1. How many people were interviewed for this survey?
    1. A poll conducted by WIN-Gallup International surveyed 51,000 people from 57 countries.
    2. A poll conducted by WIN-Gallup International surveyed 52,000 people from 57 countries.
    3. A poll conducted by WIN-Gallup International surveyed 51,917 people from 57 countries.
    4. A poll conducted by WIN-Gallup International surveyed 51,927 people from 57 countries.

Answer: A total of 51,927 persons were interviewed from 57 countries.

  1. Which of the following methods were used to gather information?
    1. Face to face
    2. Telephone
    3. Internet
    4. All of the above

Answer: In each country, a national probability sample of around 1000 men and women was interviewed either face to face, via telephone or online. So, the answer is all of the above.

  1. True / False: In the first paragraph, several key findings are reported. These percentages appear to be sample statistics.
    1. True
    2. False

Answer: True. Remember, this study consisted of national probability samples where individuals from various countries were interviewed either face to face, via telephone or online. We have sample statistics in terms of proportions (converted to percentages) calculated by making use of the responses of survey respondents, to survey questions regarding religious affiliations.

  1. True / False:The title of the report is “Global Index of Religiosity and Atheism”. To generalize the report’s findings to the global human population, We must assume that the sample was a random sample from the entire population in order to be able to generalize the results to the global human population. This seems to be a reasonable assumption.
    1. True
    2. False

Answer: First of all, this study is an observational study. Individuals are observed, interviewed and measures are recorded. Morever, this survey is entitled ‘Global Index of Religiosity and Atheism’ and is supposed to represent the entire adult human population on earth. This means that we need to make sure that no demographic segments of the entire adult population on earth are under-represented or over-represented. The data does seem to indicate that the respondents from various countries, who participated in the survey, were selected at random. However, there are certain interesting factors that lead me to believe that the findings of the report, don’t necessarily generalize the results to the global human population:

  1. Great Britain (an important country in Western Europe) was omitted from the survey.

  2. There are more than 150 countries in the world. WIN-Gallup surveyed only 57 of these countries, typically conducting around 1,000 internet, phone or personal interviews. These sample figures were then “scaled up” to reflect each country. Surveyed countries are extrapolated to non-surveyed nations to give global figures.

  3. “This method makes small sample results highly significant in some cases. Take China for example. Looking at the small print we discover that WIN-Gallup’s figures for China’s 1.35 Billion people rely upon an online Internet survey of only 500 Chinese people! Since almost half of those surveyed identified themselves as”atheist," WIN-Gallup concludes that globally “China has the largest population of atheists (47%).” Since China accounts for almost 20% of the world’s population, this single country accounts for nine percentage points of the “13% convinced atheists,” who inhabit the globe, according Gallup. Put differently, based on WIN-Gallup’s figures, 70% of the world’s atheists live in China! Yet this dramatic finding is based on an on-line survey of only 500 Chinese people. Surely such dramatic claims need a stronger basis. 9% of those 500 Chinese surveyed either didn’t respond or replied “I don’t know.” Moreover, we ask, how can an on-line survey be representative of China’s population where less than 40% have Internet access? What about the 50% of China’s people living in rural areas, mostly without Internet access? Such questions highlight the tenuous nature of Gallup’s claims; they are a “house of cards.” - http://churchintoronto.blogspot.com/2012/08/global-atheism-on-rise-really.html

So, the appropriate option here is False.

The data

Turn your attention to Table 6 (pages 15 and 16), which reports the sample size and response percentages for all 57 countries. While this is a useful format to summarize the data, we will base our analysis on the original data set of individual responses to the survey. Load this data set into R with the following command.

data(atheism)

Before answering questions 5 and 6, let us get a sense of the data, through R’s head and tail functions:

head(atheism)
## # A tibble: 6 x 3
##   nationality response     year
##   <fct>       <fct>       <int>
## 1 Afghanistan non-atheist  2012
## 2 Afghanistan non-atheist  2012
## 3 Afghanistan non-atheist  2012
## 4 Afghanistan non-atheist  2012
## 5 Afghanistan non-atheist  2012
## 6 Afghanistan non-atheist  2012
tail(atheism)
## # A tibble: 6 x 3
##   nationality response     year
##   <fct>       <fct>       <int>
## 1 Vietnam     non-atheist  2005
## 2 Vietnam     non-atheist  2005
## 3 Vietnam     non-atheist  2005
## 4 Vietnam     non-atheist  2005
## 5 Vietnam     non-atheist  2005
## 6 Vietnam     non-atheist  2005
  1. What does each row of Table 6 correspond to?
    1. Countries
    2. Individual Persons
    3. Religions

Here is a picture of table 6:

Table 6

Table 6

Answer: Each row of Table 6 corresponds to Countries

  1. What does each row of atheism correspond to?
    1. Countries
    2. Individual Persons
    3. Religions

Answer: Based upon the output from R (using the head and tail functions), each row of the data frame ‘atheism’ corresponds to individual persons.

To investigate the link between these two ways of organizing this data, take a look at the estimated proportion of atheists in the United States. Towards the bottom of Table 6, we see that this is 5%. We should be able to come to the same number using the atheism data.

Create a new dataframe called us12 that contains only the rows in atheism associated with respondents to the 2012 survey from the United States:

us12 <- atheism %>%
  filter(nationality == "United States" , atheism$year == "2012")

Check if the code worked correctly to ensure that the data set indeed contains data from the US and for the year 2012:

unique(us12$response)
## [1] non-atheist atheist    
## Levels: atheist non-atheist
unique(us12$nationality)
## [1] United States
## 57 Levels: Afghanistan Argentina Armenia Australia Austria ... Vietnam
unique(us12$year)
## [1] 2012
  1. Next, calculate the proportion of atheist responses in the United States in 2012, i.e. in us12. True / False: This percentage agrees with the percentage in Table~6.
    1. True
    2. False

Answer: to answer this question, we need to use varhandle’s unfactor function to be able to convert all variables to character and then, get a count using the tabyl function

us12$nationality <- unfactor(us12$nationality)
us12$response <- unfactor(us12$response)
#Create a frequency distribution table (a table of counts for the categorical variable abhlth by year)
#Using the library janitor and the tabyl function
tabyl(us12, response) %>% 
  adorn_totals(c('row', 'col'))
##     response    n   percent     Total
##      atheist   50 0.0499002   50.0499
##  non-atheist  952 0.9500998  952.9501
##        Total 1002 1.0000000 1003.0000

Based on the output from R and the image of table 6, the percentage of atheist responses in the United States in 2012 (5%) does agree with what has been presented for the US in table 6.

See table below:

Table 6

Table 6

Inference on proportions

As was hinted earlier, Table 6 provides sample statistics, that is, calculations made from the sample of 51,927 people. What we’d like, though, is insight into the population population parameters. You answer the question, “What proportion of people in your sample reported being atheists?” with a statistic; while the question “What proportion of people on earth would report being atheists” is answered with an estimate of the parameter.

The inferential tools for estimating population proportion are analogous to those used for means in the last lab: the confidence interval and the hypothesis test.

Exercise: Write out the conditions for inference to construct a 95% confidence interval for the proportion of atheists in the United States in 2012. Are you confident all conditions are met?

Answer: Table 7

We have a random sample consisting of data on 1,002 respondents from the United States. This is (for sure) less than 10% of the entire US population.

Next, note that out of a sample of 1,002, we have 50 atheists and 950 non-atheists. 50/1002 = 5% 950/1002 = 95%

If success is being an atheist, then p hat is 0.05 Number of successes = 50

If failure is being a non-atheist, then 1-p hat = 0.95 Number of failures = 952

There are indeed more than 10 successes and 10 failures.

If the conditions for inference are reasonable, we can either calculate the standard error and construct the interval by hand, or allow the inference function to do it for us.

inference(y = response, data = us12, statistic = "proportion", type = "ci", method = "theoretical", success = "atheist")
## Single categorical variable, success: atheist
## n = 1002, p-hat = 0.0499
## 95% CI: (0.0364 , 0.0634)

Note that since the goal is to construct an interval estimate for a proportion, it’s necessary to specify what constitutes a `success'', which here is a response ofatheist`.

Although formal confidence intervals and hypothesis tests don’t show up in the report, suggestions of inference appear at the bottom of page 7: “In general, the error margin for surveys of this kind is \(\pm\) 3-5% at 95% confidence.”

Exercise: Imagine that, after reading a front page story about the latest public opinion poll, a family member asks you, “What is a margin of error?” In one sentence, and ignoring the mechanics behind the calculation, how would you respond in a way that conveys the general concept?

Answer: We are often interested in population parameters. But these are difficult and impossible to collect. We use sample statistics as point estimates for the unknown population parameters of interest. These sample statistics vary from sample to sample, yielding slightly different estimates. Quantifying how sample statistics vary, provides a way to estimate the margin of error associated with our point estimate.

  1. Based on the R output, what is the margin of error for the estimate of the proportion of the proportion of atheists in US in 2012?
    1. The margin of error for the estimate of the proportion of atheists in the US in 2012 is 0.05.
    2. The margin of error for the estimate of the proportion of atheists in the US in 2012 is 0.025.
    3. The margin of error for the estimate of the proportion of atheists in the US in 2012 is 0.0135.

Answer: Recall the output from R:

inference(y = response, data = us12, statistic = "proportion", type = "ci", method = "theoretical", success = "atheist")
## Single categorical variable, success: atheist
## n = 1002, p-hat = 0.0499
## 95% CI: (0.0364 , 0.0634)

  1. p-hat - lower confidence interval limit = 0.0499 - 0.0364 = 0.0135

  2. upper confidence interval limit - p-hat = 0.0634 - 0.0499 = 0.0135

  3. width of the confidence interval = upper confidence interval limit - lower confidence interval limit = 0.0634 - 0.0364 = 0.027

Confidence Interval Interpretation: We are 95% confident that 3.6% to 6.3% of US adults in 2012, identified themselves as Atheists.

Remember that the margin of error is equal to the radius (or half the width) of the confidence interval, since we are constructing a two-sided confidence interval:

Margin of Error = 0.027/2 = 0.0135

Exercise: Using the inference function, calculate confidence intervals for the proportion of atheists in 2012 in two other countries of your choice, and report the associated margins of error. Be sure to note whether the conditions for inference are met. It may be helpful to create new data sets for each of the two countries first, and then use these data sets in the inference function to construct the confidence intervals.

#Subset the data to include India and China atheist data for 2012:
India12 <- atheism %>%
  filter(nationality == "India" , atheism$year == "2012")

China12 <- atheism %>%
  filter(nationality == "China" , atheism$year == "2012")

#As usual, unfactor all the variables that are factors to avoid any errors in the inference function:
India12$nationality <- unfactor(India12$nationality)
India12$response <- unfactor(India12$response)
China12$nationality <- unfactor(China12$nationality)
China12$response <- unfactor(China12$response)

#Use the inference function for both countries
inference(y = response, data = India12, statistic = "proportion", type = "ci", method = "theoretical", success = "atheist")
## Single categorical variable, success: atheist
## n = 1092, p-hat = 0.0302
## 95% CI: (0.0201 , 0.0404)

inference(y = response, data = China12, statistic = "proportion", type = "ci", method = "theoretical", success = "atheist")
## Single categorical variable, success: atheist
## n = 500, p-hat = 0.47
## 95% CI: (0.4263 , 0.5137)

Answer: I am selecting data from China and India in 2012. China supposedly has the world’s greatest irreligious population and the Gallup study seems to point to this fact. It will be Will be interesting to see if the trends in the data reflect what is being claimed in the WIN-Gallup study (according to this study, in 2012, 3% of India’s population identified as atheists, while 47% of China’s population identified as atheists), by first examing the range of plausible values as generated by the confidence interval within which the true population parameter (in this case a single proportion- the proportion of atheists in each country) is expected to lie.

Confidence Interval Interpretation (China): We are 95% confident that 42.6% to 51.3% of Chinese adults in 2012, identified themselves as Atheists.

Confidence Interval Interpretation (India): We are 95% confident that 2% to 4% of Indian adults in 2012, identified themselves as Atheists.

How does the proportion affect the margin of error?

Imagine you’ve set out to survey 1000 people on two questions: are you female? and are you left-handed? Since both of these sample proportions were calculated from the same sample size, they should have the same margin of error, right? Wrong! While the margin of error does change with sample size, it is also affected by the proportion.

Think back to the formula for the standard error: \(SE = \sqrt{p(1-p)/n}\). This is then used in the formula for the margin of error for a 95% confidence interval: \(ME = 1.96\times SE = 1.96\times\sqrt{p(1-p)/n}\). Since the population proportion \(p\) is in this \(ME\) formula, it should make sense that the margin of error is in some way dependent on the population proportion. We can visualize this relationship by creating a plot of \(ME\) vs. \(p\).

The first step is to make a vector p that is a sequence from \(0\) to \(1\) with each number separated by \(0.01\). We can then create a vector of the margin of error (me) associated with each of these values of p using the familiar approximate formula (\(ME = 1.96 \times SE\)). Lastly, we plot the two vectors against each other to reveal their relationship.

d <- data.frame(p <- seq(0, 1, 0.01))
n <- 1000
d <- d %>%
  mutate(me = 1.96*sqrt(p*(1 - p)/n))
ggplot(d, aes(x = p, y = me)) +
  geom_line()

  1. Which of the following is false about the relationship between \(p\) and \(ME\).
    1. The \(ME\) reaches a minimum at \(p = 0\).
    2. The \(ME\) reaches a minimum at \(p = 1\).
    3. The \(ME\) is maximized when \(p = 0.5\).
    4. The most conservative estimate when calculating a confidence interval occurs when \(p\) is set to 1.

Answer: As per the graph, the Margin of Error gradually starts to increase when the population proportion starts to increase from 0. The Margin of Error then, reaches its peak when the population proportion is at 0.5 and then starts to decline as the population parameter increases towards 1 and minimizes at p=1, similar to it being minimized at p=0, as evidenced by the dome shaped curve, depicting the relationship between the population proportion and Margin of Error.

Also, recall that the Margin of Error and sample size are interrelated. Recall, the following formula:

Formula

Formula

While calculating the confidence interval, the sample proprtion enters the formula to calculate the Standard Error. Now, it is critical to remember that a researcher is always constrained by resources. Re-sampling due to wrong choice of sample size, can prove to be an unecessary waste of resources. So,if we need a ‘best guess’ for the proportion of successes and failures, 50-50 is indeed a good guess. Setting p-hat to 0.5, gives the most conservative estimate possible. By ‘conservative’, we mean the highestpossible sample size. Keeping all the mentioned facts in mind, option d is false.

The question of atheism was asked by WIN-Gallup International in a similar survey that was conducted in 2005. We assume here that sample sizes have remained the same. Table 4 on page 13 of the report summarizes survey results from 2005 and 2012 for 39 countries.

Answer the following two questions using the inference function. As always, write out the hypotheses for any tests you conduct and outline the status of the conditions for inference.

  1. True / False: There is convincing evidence that Spain has seen a change in its atheism index between 2005 and 2012.

    Hint: Create a new data set for respondents from Spain. Then use their responses as the first input on the inference, and use year as the grouping variable.
    1. True
    2. False 
#Take a look at the year variable first in the original atheist data set to make sure that there are no additional years (only 2005 and 2012 data should be present)

unique(atheism$year)
## [1] 2012 2005
#Filter out data for the country Spain

Spain.atheism <- filter(atheism, atheism$nationality=="Spain")

#Convert the year varaible in the data set so as to avoid a warning message from the inference function - Explanatory variable was numerical, it has been converted to categorical. In order to avoid this warning, first convert your explanatory variable to a categorical variable using the as.factor() function 

#Also, the response variable has been imported as a factor variable and has to be unfactored

Spain.atheism$year <- as.character(Spain.atheism$year)
Spain.atheism$response <- unfactor(Spain.atheism$response)

#Next, use the inference function - here we have a grouping variable year (also known as the explanatory variable) and the response varaible called 'response', stating the respondent's religious affiliation - success is defined as being an atheist - we are essentially comparing the proportion of atheists
#in 2012, relative to 2005, to see if the changes are statistically significant - we do this first by the confidence interval method:
inference(y = response, x= year, data = Spain.atheism, statistic = "proportion", type = "ci", method = "theoretical", success = "atheist")
## Response variable: categorical (2 levels, success: atheist)
## Explanatory variable: categorical (2 levels) 
## n_2005 = 1146, p_hat_2005 = 0.1003
## n_2012 = 1145, p_hat_2012 = 0.09
## 95% CI (2005 - 2012): (-0.0136 , 0.0344)

Answer: The confidence interval includes the null value (which is 0). Though we are at the moment, constructing a confidence interval, we can think about a hypothesis tet as well. Recall, that if we state the problem in the context of a hypothesis test, the null hypothesis states that there is no difference between the proportion of atheists in Spain in 2005 and the proportion of atheists in Spain in 2012. Since the confidence interval includes the null value, the data does not provide convincing evidence of a change in the atheism index between 2005 and 2012. So, the option is false.

Let’s see if the results of the hypothesis test is in agreement with the inferences drawn from the confidence interval.

inference(y = response, x= year, data = Spain.atheism, statistic = "proportion", type = "ht", null=0, alternative="twosided", method = "theoretical", success = "atheist")
## Response variable: categorical (2 levels, success: atheist)
## Explanatory variable: categorical (2 levels) 
## n_2005 = 1146, p_hat_2005 = 0.1003
## n_2012 = 1145, p_hat_2012 = 0.09
## H0: p_2005 =  p_2012
## HA: p_2005 != p_2012
## z = 0.8476
## p_value = 0.3966

p-value interpreation: Given that the null hypothesis is true, there is a 39% probability that the difference in the proportion of atheists in Spain in 2012, relative to 2005, was due to chance.

Note: I am taking advantage here of the fact that the inference function will generate a warning message stating that the success-failure condition has not been met (there has to be at least 10 successes and 10 failures in both samples - for 2005 and 2012). I am also leaving it to R here to calculate the pooled sample proportion, which is the total number of successes across both samples divided by the total of the sample sizes for 2005 and 2012.

Let’s do the same for the US

  1. True / False: There is convincing evidence that the United States has seen a change in its atheism index between 2005 and 2012.
    1. True
    2. False 
#Take a look at the year variable first in the original atheist data set to make sure that there are no additional years (only 2005 and 2012 data should be present)

unique(atheism$year)
## [1] 2012 2005
#Filter out data for the country Spain

US.atheism <- filter(atheism, atheism$nationality=="United States")

#Convert the year varaible in the data set so as to avoid a warning message from the inference function - Explanatory variable was numerical, it has been converted to categorical. In order to avoid this warning, first convert your explanatory variable to a categorical variable using the as.factor() function 

#Also, the response variable has been imported as a factor variable and has to be unfactored

US.atheism$year <- as.character(US.atheism$year)
US.atheism$response <- unfactor(US.atheism$response)

#Next, use the inference function - here we have a grouping variable year (also known as the explanatory variable) and the response varaible called 'response', stating the respondent's religious affiliation - success is defined as being an atheist - we are essentially comparing the proportion of atheists
#in 2012, relative to 2005, to see if the changes are statistically significant - we do this first by the confidence interval method:
inference(y = response, x= year, data = US.atheism, statistic = "proportion", type = "ci", method = "theoretical", success = "atheist")
## Response variable: categorical (2 levels, success: atheist)
## Explanatory variable: categorical (2 levels) 
## n_2005 = 1002, p_hat_2005 = 0.01
## n_2012 = 1002, p_hat_2012 = 0.0499
## 95% CI (2005 - 2012): (-0.0547 , -0.0251)

Answer: In this case, the null value 0 is not within the calculated confidence interval. Could the data be providing us with convincing evidence that the proportion of atheists in the US in 2012, is different from that in 2005?

Also, we can interpret the confidence interval as follows: We are 95% confident that there was a 2.5% to 5.4% decline in the number of atheists in the US in 2012, relative to 2005.

Let’s check and see if the confidence interval inference and the hypothesis test inference are in agreement with each other:

inference(y = response, x= year, data = US.atheism, statistic = "proportion", type = "ht", null=0, alternative="twosided", method = "theoretical", success = "atheist")
## Response variable: categorical (2 levels, success: atheist)
## Explanatory variable: categorical (2 levels) 
## n_2005 = 1002, p_hat_2005 = 0.01
## n_2012 = 1002, p_hat_2012 = 0.0499
## H0: p_2005 =  p_2012
## HA: p_2005 != p_2012
## z = -5.2431
## p_value = < 0.0001

Clearly, given how small the p-value is and the magnitude of the z statistic, there is almost a 0% probability that the difference in proportion of atheists in the US in 2012, relative to 2005 is due to chance.

Hence, the results of the confidence interval is in agreement with the results of the hypothesis test.

  1. If in fact there has been no change in the atheism index in the countries listed in Table 4, in how many of those countries would you expect to detect a change (at a significance level of 0.05) simply by chance?

    Hint: Type 1 error.
    1. 0
    2. 1
    3. 1.95
    4. 5

Answer: Assuming a 5% level of significance, in table 4, there were 39 countries that were surveyed. If we fail to reject the null hypothesis for 39 countries, based on the level of significance and the definition of a type I error and its relationship with the level of significance, we can expect to see a change in the proportion of atheists, simply by chance for 2 counties (39(5%) = 39(0.05) = 1.95)

  1. Suppose you’re hired by the local government to estimate the proportion of residents that attend a religious service on a weekly basis. According to the guidelines, the estimate must have a margin of error no greater than 1% with 95% confidence. You have no idea what to expect for \(p\). How many people would you have to sample to ensure that you are within the guidelines?

    Hint: Refer to your plot of the relationship between \(p\) and margin of error. Do not use the data set to answer this question.
    1. 2401 people
    2. At least 2401 people
    3. 9604 people
    4. At least 9604 people

Answer: We are considering a statistical inference test that deals with a single proportion. Since we cannot rely on any previous study, let us resort to the best guess for p-hat, namely a 50% success and 50% failure, which implies setting p-hat to be equal to 0.5

Consider the following formula for Margin of Error. With this formula, the connection between Margin of Error, Sample Size and Standard Error becomes clear:

\[ME = z^*\sqrt {\frac {p(1-p)}{n}}\]

Here, since we are considering a 95% confidence interval, recall that the critical value associated with this is 1.96

So, we have:

\[0.01 = 1.96\sqrt {\frac {0.5(0.5)}{n}}\]

Squaring on both sides gets us the following:

\[0.0001(n) = 3.8416({0.5(0.5)})\]

\[0.0001(n) = 0.9604\] \[n = {\frac {0.9604}{0.0001}}\]

\[n = 9604\]

As per the calculations above, we require at least 9,604 people in order to have a margin of error of no more than 1%.

This is a product of OpenIntro that is released under a Creative Commons Attribution-ShareAlike 3.0 Unported. This lab was written for OpenIntro by Andrew Bray and Mine Çetinkaya-Rundel.