Tutorial 6 (202)
Neha Dhillon
Setting up environment
# List of packages
packages <- c("tidyverse", "infer", "fst") # add any you need here
# Install packages if they aren't installed already
new_packages <- packages[!(packages %in% installed.packages()[,"Package"])]
if(length(new_packages)) install.packages(new_packages)
# Load the packages
lapply(packages, library, character.only = TRUE)## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.4 ✔ readr 2.1.4
## ✔ forcats 1.0.0 ✔ stringr 1.5.1
## ✔ ggplot2 3.4.4 ✔ tibble 3.2.1
## ✔ lubridate 1.9.3 ✔ tidyr 1.3.0
## ✔ purrr 1.0.2
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors## [[1]]
## [1] "lubridate" "forcats" "stringr" "dplyr" "purrr" "readr"
## [7] "tidyr" "tibble" "ggplot2" "tidyverse" "stats" "graphics"
## [13] "grDevices" "utils" "datasets" "methods" "base"
##
## [[2]]
## [1] "infer" "lubridate" "forcats" "stringr" "dplyr" "purrr"
## [7] "readr" "tidyr" "tibble" "ggplot2" "tidyverse" "stats"
## [13] "graphics" "grDevices" "utils" "datasets" "methods" "base"
##
## [[3]]
## [1] "fst" "infer" "lubridate" "forcats" "stringr" "dplyr"
## [7] "purrr" "readr" "tidyr" "tibble" "ggplot2" "tidyverse"
## [13] "stats" "graphics" "grDevices" "utils" "datasets" "methods"
## [19] "base"Task 1
Based on the recoding of lrscale for “left” and “right” and omitting “moderates” (see Tutorial 4), and educ.ba from this tutorial, do the long form coding for Chi-Square. Using the steps outlined in the tutorial, first generate the tables of expected proportions and frequencies. Determine and interpret the critical value for independence. Finally, determine and interpret both the Pearson’s chi-squared statistic and the p-value. Discuss main takeaways.
##The P value is less than 2.2e-16 and close to 0. Pearson’s chi square squared statistic is 18578. These numbers are represent the outcome, which represents the educ.ba data, because the p-value and Pearson’s chi-squared statistic portray the educ.ba data because the Pearson chi squared data would examine what differs between the data in the categorical data that the educ.ba would portray The Pearson chi squared data represents its signifgence as the explanatory information. The P value remains relevent becuase it refers to the liklihood of this data resulting as an outcome in terms of the educ.ba data. The -value and Pearson’s chi-squared statistic work together to show how data is differed in comparision to additioanl data in a table, pearson chi squared data,and the liklihood of it happeneing, p value.
Task 2
Do Steps 1 to 3 using the infer package (as we did in the tutorial), for the variables sbbsntx (recoded to the corresponding 5 categories) as the response and domicil (recoded to urban and rural, setting 2 to urban instead of peri-urban) as the explanatory variable for the country of France.
See variable info here: https://ess.sikt.no/en/datafile/ffc43f48-e15a-4a1c-8813-47eda377c355/92?tab=1&elements=[%22874a86e2-a0f6-40b4-aef7-51c8eed98a7d/1%22])
Provide interpretations of the output (consider the variable info).
#Inputting France data
france_data <- read.fst("france_data.fst")
table(france_data$domicil)##
## 1 2 3 4 5 7 8
## 3584 2361 6257 5668 1162 3 3table(france_data$sbbstx)## < table of extent 0 >#Organizing France data
france_data <- france_data %>%
mutate(
sbbsntx = recode(as.character(sbbsntx),
'1' = "Agree Strongly",
'2' = "Agree",
'3' = "Neither Agree or Disagree",
'4' = "Disagree",
'5' = "Disagree Strongly",
'7' = NA_character_,
'8' = NA_character_,
'9' = NA_character_)
) %>%
filter(!is.na(sbbsntx))france_data <- france_data %>%
mutate(
domcil = recode(as.character(domicil),
'1' = "Urban",
'2' = "Urban",
'3' = "Rural",
'4' = "Rural",
'5' = "Rural",
'7' = NA_character_,
'8' = NA_character_,
'9' = NA_character_)
) %>%
filter(!is.na(domcil))france_data <- france_data %>%
filter(!is.na(domicil) & !is.na (sbbsntx))
france_data <- france_data %>%
mutate(
domicil = factor(domicil),
sbbsntx = factor(sbbsntx)
)#Step 1 Calculating the test of statistic sample
test_stat <- france_data %>%
specify(explanatory = domicil,
response = sbbsntx) %>%
hypothesize(null = "independence") %>%
calculate(stat = "chisq")
print(test_stat$stat)## X-squared
## 36.01302#Step 2 - Simulate the null distribution
null_distribution <- france_data %>%
specify(explanatory = domicil,
response = sbbsntx) %>%
hypothesize(null = "independence") %>%
generate(reps = 1000, type = "permute") %>%
calculate(stat = "chisq")#Step 3 calculating the p value of sample
p_val <- null_distribution %>%
get_pvalue(obs_stat = test_stat, direction = "two-sided")
p_val## # A tibble: 1 × 1
## p_value
## <dbl>
## 1 0.006#The P Value of 0.02 is the explanatarory variavle for French citizens who believe that social benefits cost buisinessess too much in their taxes. Since it is 0 and not -0, this p value of 0.02 is strong enough to reject the null hypothesis.
Task 3
Now do Step 4 (i.e., the null distribution visualization with confidence intervals) for the same variables and country as Task 2, and interpret the output.
null_distribution %>%
visualize() +
shade_p_value(obs_stat = test_stat, direction = "two-sided")## Warning: Chi-Square usually corresponds to right-tailed tests. Proceed with
## caution.
null_distribution## Response: sbbsntx (factor)
## Explanatory: domicil (factor)
## Null Hypothesis: independence
## # A tibble: 1,000 × 2
## replicate stat
## <int> <dbl>
## 1 1 13.7
## 2 2 20.9
## 3 3 16.7
## 4 4 9.29
## 5 5 11.7
## 6 6 9.65
## 7 7 16.7
## 8 8 17.5
## 9 9 13.4
## 10 10 25.8
## # ℹ 990 more rowsconf_int <- null_distribution %>%
get_confidence_interval(level = 0.95, type = "percentile")
null_distribution %>%
visualize() +
shade_p_value(obs_stat = test_stat, direction = "two-sided") +
shade_confidence_interval(endpoints = conf_int)## Warning: Chi-Square usually corresponds to right-tailed tests. Proceed with
## caution.
null_distribution## Response: sbbsntx (factor)
## Explanatory: domicil (factor)
## Null Hypothesis: independence
## # A tibble: 1,000 × 2
## replicate stat
## <int> <dbl>
## 1 1 13.7
## 2 2 20.9
## 3 3 16.7
## 4 4 9.29
## 5 5 11.7
## 6 6 9.65
## 7 7 16.7
## 8 8 17.5
## 9 9 13.4
## 10 10 25.8
## # ℹ 990 more rows##This output refers to the data that is tested in terms of its credibility, refering to the null distribution and null hypothesis. There has been a change in colour in the graph, in terms of its visual presentation. This shows that the “social benefits/services cost buisinessess too much in taxes/charges” beliefs have a peak at the stat of 11 and a count of 160 according to the histogram. This bell shaped data peak. The data does not fall within the null distribution becuase there is no shaded red regions.
Task 4
Conduct Steps 1 to 3 using the infer package (as we did in the tutorial), for the variables ccrdprs (leaving it on the 0-10 numeric scale) as the response (or outcome) variable and lrscale (recoded as left and right, omitting “moderates” as we did in Task 1) as the explanatory variable.
Variable info here: https://ess.sikt.no/en/datafile/ffc43f48-e15a-4a1c-8813-47eda377c355/92?tab=1&elements=[%2283b08cf2-508e-49a0-9fc8-c3cc281290ec/4%22]
Provide interpretations of the output (consider the variable info).
france_data <- read.fst("france_data.fst")
table(france_data$lrscale)##
## 0 1 2 3 4 5 6 7 8 9 10 77 88
## 945 499 1269 1996 1838 5274 1594 1749 1458 409 686 381 940table(france_data$ccrdprs)##
## 0 1 2 3 4 5 6 7 8 9 10 66 77 88 99
## 72 26 64 78 114 478 403 736 952 358 708 27 1 29 1france_data <- france_data %>%
mutate(
ccrdprs = recode(as.character(ccrdprs),
'1' = "Agree Strongly",
'2' = "Agree",
'3' = "Neither Agree or Disagree",
'4' = "Disagree",
'5' = "Disagree Strongly",
'7' = NA_character_,
'8' = NA_character_,
'9' = NA_character_)
) %>%
filter(!is.na(ccrdprs))france_data <- france_data %>%
mutate(
lrscale = recode(as.character(domicil),
'1' = "Urban",
'2' = "Urban",
'3' = "Rural",
'4' = "Rural",
'5' = "Rural",
'7' = NA_character_,
'8' = NA_character_,
'9' = NA_character_)
) %>%
filter(!is.na(lrscale))france_data <- france_data %>%
filter(!is.na(lrscale) & !is.na (ccrdprs))
france_data <- france_data %>%
mutate(
lrscale = factor(lrscale),
sbbsntx = factor(ccrdprs)
)#Step 1 Calculating the test of statistic sample
test_stat <- france_data %>%
specify(explanatory = lrscale,
response = ccrdprs) %>%
hypothesize(null = "independence") %>%
calculate(stat = "chisq")
print(test_stat$stat)## X-squared
## 27.73254#Step 2 - Simulate the null distribution
null_distribution <- france_data %>%
specify(explanatory = lrscale,
response = ccrdprs) %>%
hypothesize(null = "independence") %>%
generate(reps = 1000, type = "permute") %>% # only line we are adding, use reps = 1000 as standard but change if too computationally demanding
calculate(stat = "chisq")#Step 3 calculating the p value of sample
p_val <- null_distribution %>%
get_pvalue(obs_stat = test_stat, direction = "two-sided")
p_val## # A tibble: 1 × 1
## p_value
## <dbl>
## 1 0.002##This means that 0.008 of French citizens feel that personal responsibility will reduce climate change. 0.008 is the p value meaning that it is 0 not -0 which indicates that there is evidence that it will reject the null.
Task 5
Now do Step 4 (i.e., the null distribution visualization with confidence intervals) for the same variables and country as Task 4, and interpret the output.
null_distribution %>%
visualize() +
shade_p_value(obs_stat = test_stat, direction = "two-sided")## Warning: Chi-Square usually corresponds to right-tailed tests. Proceed with
## caution.
null_distribution## Response: ccrdprs (factor)
## Explanatory: lrscale (factor)
## Null Hypothesis: independence
## # A tibble: 1,000 × 2
## replicate stat
## <int> <dbl>
## 1 1 9.75
## 2 2 3.80
## 3 3 12.3
## 4 4 6.58
## 5 5 7.03
## 6 6 6.61
## 7 7 15.1
## 8 8 4.94
## 9 9 11.4
## 10 10 11.3
## # ℹ 990 more rowsconf_int <- null_distribution %>%
get_confidence_interval(level = 0.95, type = "percentile")
null_distribution %>%
visualize() +
shade_p_value(obs_stat = test_stat, direction = "two-sided") +
shade_confidence_interval(endpoints = conf_int)## Warning: Chi-Square usually corresponds to right-tailed tests. Proceed with
## caution.
null_distribution## Response: ccrdprs (factor)
## Explanatory: lrscale (factor)
## Null Hypothesis: independence
## # A tibble: 1,000 × 2
## replicate stat
## <int> <dbl>
## 1 1 9.75
## 2 2 3.80
## 3 3 12.3
## 4 4 6.58
## 5 5 7.03
## 6 6 6.61
## 7 7 15.1
## 8 8 4.94
## 9 9 11.4
## 10 10 11.3
## # ℹ 990 more rows##The shaded red regions indicate the null distribution is more extreme than the observed statistic. This means that the null hypothesis is not true, because the data is less than 5%. In this case, the null is rejected in regards to people in France that believe personal responsibility contributes to climate change. The data in the green shaded part represents the nulled data.