Based on the recoding of lrscale for “left” and “right” and omitting “moderates” (see Tutorial 4), and educ.ba from this tutorial, do the long form coding for Chi-Square. Using the steps outlined in the tutorial, first generate the tables of expected proportions and frequencies. Determine and interpret the critical value for independence. Finally, determine and interpret both the Pearson’s chi-squared statistic and the p-value. Discuss main takeaways.
rm(list=ls()); gc() ## used (Mb) gc trigger (Mb) limit (Mb) max used (Mb) ## Ncells 527509 28.2 1172460 62.7 NA 669417 35.8 ## Vcells 972201 7.5 8388608 64.0 16384 1851610 14.2
packages <- c(“tidyverse”, “infer”, “fst”) # add any you need here
new_packages <- packages[!(packages %in% installed.packages()[,“Package”])] if(length(new_packages)) install.packages(new_packages)
lapply(packages, library, character.only = TRUE)
##-Attaching core tidyverse packages
##✔ forcats 1.0.0 ✔ stringr 1.5.0 ##✔ ggplot2 3.4.3 ✔ tibble 3.2.1
##✔ lubridate 1.9.2 ✔ tidyr 1.3.0 ##✔ purrr 1.0.2
##-Conflicts ✖ dplyr::filter() masks stats::filter() ✖ dplyr::lag()
masks stats::lag() ℹ Use the conflicted package to force all conflicts
to become errors fst package v0.9.8
[[1]] [1] “lubridate” “forcats” “stringr” “dplyr” “purrr”
[6] “readr” “tidyr” “tibble” “ggplot2” “tidyverse” [11] “stats”
“graphics” “grDevices” “utils” “datasets” [16] “methods” “base”
[[2]] [1] “infer” “lubridate” “forcats” “stringr” “dplyr”
[6] “purrr” “readr” “tidyr” “tibble” “ggplot2”
[11] “tidyverse” “stats” “graphics” “grDevices” “utils”
[16] “datasets” “methods” “base”
[[3]] [1] “fst” “infer” “lubridate” “forcats” “stringr”
[6] “dplyr” “purrr” “readr” “tidyr” “tibble”
[11] “ggplot2” “tidyverse” “stats” “graphics” “grDevices” [16] “utils”
“datasets” “methods” “base”
ess <- read_fst(“All-ESS-Data.fst”)
france_data <- ess %>% # Filter for France filter(cntry == “FR”) %>%
# Recoding clsprty and cleaning trstplt (will use later) mutate( clsprty = case_when( clsprty == 1 ~ “Yes”, # Recode 1 to “Yes” clsprty == 2 ~ “No”, # Recode 2 to “No” clsprty %in% c(7, 8, 9) ~ NA_character_, # Handle other specific cases where you want to set it as NA TRUE ~ as.character(clsprty) # Keep other values as-is but ensure they are characters ), trstplt = ifelse(trstplt %in% c(77, 88, 99), NA, trstplt),
# Recoding gender gndr = case_when( gndr == 1 ~ “Male”, gndr == 2 ~ “Female”, gndr == 9 ~ NA_character_, TRUE ~ as.character(gndr)
# Recoding education with case_when for consistency
educ.ba = case_when(
essround < 5 & edulvla == 5 ~ "BA or more",
essround >= 5 & edulvlb > 600 ~ "BA or more",
TRUE ~ "No BA"
) %>% # Ensure all NAs are handled uniformly mutate( edulvla = ifelse(edulvla %in% c(77, 88), NA_integer_, edulvla), edulvlb = ifelse(edulvlb %in% c(5555, 7777, 8888), NA_integer_, edulvlb), educ.ba = as.character(educ.ba) # Force educ.ba to be treated as character
tab_dat <- france_data
mytab <- table(tab_dat\(clsprty, tab_dat\)educ.ba)
rsums <- rowSums(mytab)
csums <- colSums(mytab)
N <- sum(mytab)
ptab <- tcrossprod(rsums/N, csums/N) cat(“Table of Expected Proportions:”)
print(ptab) ## [,1] [,2] ## [1,] 0.1101249 0.3815067 ## [2,] 0.1138739 0.3944946 # Table of expected frequencies ftab <- N * ptab cat(“Table of Expected Frequencies:”) ## Table of Expected Frequencies: print(ftab) ## [,1] [,2] ## [1,] 2059.443 7134.554 ## [2,] 2129.555 7377.444
alpha <- 0.05 # significance level c_val <- qchisq(alpha, df = 1, lower.tail = FALSE) cat(“Critical Value:”, round(c_val, 3), “”) ## Critical Value: 3.840
After using the chisq.test function, the computed Chi-squared test statistic is greater than 3.840, the null hypothesis is rejected at the 0.05 significance level. This would therefore suggest that, within the sample, there is a statistically significant correlation between feeling near to a party and education level.
test_stat <- sum((mytab - ftab)^2 / ftab) cat(“Pearson’s X^2:”, round(test_stat, 4), “”) ## Pearson’s X^2: 166.1849
observed Chi-squared value (166.1849) significantly surpasses this barrier, as determined by computing a critical value of 3.840.
Statistical significance is demonstrated by the significant difference between our observed data and the expected results based on the independence assumption. As such, there is a statistically significant correlation between educational attainment and the sense of being near to a party. This suggests that there are differences in the chances of feeling connected to a party between educational levels.
p_val <- pchisq(test_stat, df = 1, lower.tail = F) cat(“p-value:”, round(p_val, 4), “”) ## p-value: 0 # Using chisq.test for validation: cat(“Chi-squared test result:”) ## Chi-squared test result: print(chisq.test(mytab, correct = F)) ## ## Pearson’s Chi-squared test ## ## data: mytab ## X-squared = 166.18, df = 1, p-value < 2.2e-16
The p-value that corresponds to the Chi-squared test is < 2.2e-16. Strong evidence is present against the null hypothesis, as indicated by the extremely small p-value, which is almost zero. Stated differently, the variables in the dataset have a statistically significant correlation with one another.
Do Steps 1 to 3 using the infer package (as we did in the tutorial), for the variables sbbsntx (recoded to the corresponding 5 categories) as the response and domicil (recoded to urban and rural, setting 2 to urban instead of peri-urban) as the explanatory variable for the country of France.
See variable info here: https://ess.sikt.no/en/datafile/ffc43f48-e15a-4a1c-8813-47eda377c355/92?tab=1&elements=[%22874a86e2-a0f6-40b4-aef7-51c8eed98a7d/1%22])
Provide interpretations of the output (consider the variable info).
france_data <- ess %>% # Filter for France filter(cntry == “FR”) %>%
#Recoding to the corresponding 5 categories mutate( sbbsntx =
case_when( sbbsntx == 1 ~ “Agree strongly”,
sbbsntx == 2 ~ “Agree”,
sbbsntx == 3 ~ “Neither agree nor disagree”, sbbsntx == 4 ~ “Disagree”,
sbbsntx == 5 ~ “Disagree strongly”, sbbsntx %in% c(7, 8, 9) ~
NA_character_, # Handle other specific cases where you want to set it as
NA TRUE ~ as.character(sbbsntx) # Keep other values as-is but ensure
they are characters ),
# Recoding to urban and rural domicil = case_when( domicil == 1 ~
“Urban”,
domicil == 2 ~ “Urban”,
domicil == 3 ~ “Rural”, domicil == 4 ~ “Rural”, domicil == 5 ~ “Rural”,
domicil %in% c(7, 8, 9) ~ NA_character_, # Handle other specific cases
where you want to set it as NA TRUE ~ as.character(domicil) # Keep other
values as-is but ensure they are characters ) ) test_stat <-
france_data %>% specify(explanatory = domicil, # change variable name
for explanatory variable response = sbbsntx) %>% # change variable
name for outcome of interest hypothesize(null = “independence”) %>%
calculate(stat = “Chisq”) # replace in between quotation marks
appropriate test statistic
## Warning: Removed 14999 rows containing missing values. print(test_stat$stat) ## X-squared ## 19.97186
The 19.97186 chi-squared statistic shows a significant relationship between living in an urban or rural location and the degree of agreement regarding the perceived cost of social benefits/services on enterprises in the form of taxes or charges. This implies that the degree of agreement varies considerably amongst people depending on where they live. Moreover, this discovery offers strong proof to reject the null hypothesis.
null_distribution <- france_data %>% specify(explanatory =
domicil, response = sbbsntx) %>% hypothesize(null = “independence”)
%>% generate(reps = 1000, type = “permute”) %>% # only line we are
adding, use reps = 1000 as standard but change if too computationally
demanding calculate(stat = “Chisq”) ## Warning: Removed 14999 rows
containing missing values. p_val <- null_distribution %>% #
replace name here if you assigned something other than null_distribution
above get_pvalue(obs_stat = test_stat, direction = “greater”) # would
only replace test_stat if assigned another name in Step 1 ## Warning:
Please be cautious in reporting a p-value of 0. This result is an
approximation ## based on the number of reps chosen in the
generate() step. ## ℹ See get_p_value()
(?infer::get_p_value()) for more information. p_val ## # A
tibble: 1 × 1 ## p_value ##
Frequentist hypothesis testing assumes that a p-value of 0 indicates strong evidence against the null hypothesis.
Now do Step 4 (i.e., the null distribution visualization with confidence intervals) for the same variables and country as Task 2, and interpret the output.
null_sbbsntx%>% visualize() + shade_p_value(obs_stat=sbbsntx_stat, direction= twoo sided)
conf_int <- null_distribution %>% get_confidence_interval(level = 0.95, type = “percentile”)
null_distribution %>% visualize() + shade_p_value(obs_stat = test_stat, direction = “greater”) + shade_confidence_interval(endpoints = conf_int)
null_distribution
Conduct Steps 1 to 3 using the infer package (as we did in the tutorial), for the variables ccrdprs (leaving it on the 0-10 numeric scale) as the response (or outcome) variable and lrscale (recoded as left and right, omitting “moderates” as we did in Task 1) as the explanatory variable.
Variable info here: https://ess.sikt.no/en/datafile/ffc43f48-e15a-4a1c-8813-47eda377c355/92?tab=1&elements=[%2283b08cf2-508e-49a0-9fc8-c3cc281290ec/4%22]
Provide interpretations of the output (consider the variable info).
france_data <- ess %>% # Filter for France filter(cntry == “FR”) %>%
# Recoding 'lrscale'
mutate(
lrscale = case_when(
lrscale %in% 0:3 ~ "Left", # Values 0 to 3 in 'lrscale' are categorized as "Left"
lrscale %in% 7:10 ~ "Right", # Values 7 to 10 in 'lrscale' are categorized as "Right"
lrscale %in% 4:6 ~ "Moderate", # Values 4 to 6 in 'lrscale' are categorized as "Moderate"
lrscale %in% c(77, 88, 99) ~ NA_character_ # Values 77, 88, 99 in 'lrscale' are set as NA
),
ccrdprs = ifelse(ccrdprs %in% c(66, 77, 88, 99), NA, ccrdprs), # Set values 66, 77, 88, 99 in 'ccrdprs' to NA
) test_stat <- france_data %>% specify(explanatory = lrscale, # change variable name for explanatory variable response = ccrdprs) %>% # change variable name for outcome of interest hypothesize(null = “independence”) %>% calculate(stat = “F”) ## Warning: Removed 15413 rows containing missing values. print(test_stat$stat) ## [1] 15.41869
The ANOVA value of 15.41869 indicates that the groups under comparison differ from one another. There is an obvious correlation between a person’s political stance (left or right) and their sense of personal accountability for addressing climate change. In other words, a person’s political stance essentially determines how much they believe has to be done to combat climate change.
null_distribution <- france_data %>% specify(explanatory =
lrscale, response = ccrdprs) %>% hypothesize(null = “independence”)
%>% generate(reps = 1000, type = “permute”) %>% # only line we are
adding, use reps = 1000 as standard but change if too computationally
demanding calculate(stat = “F”) ## Warning: Removed 15413 rows
containing missing values. p_val <- null_distribution %>% #
replace name here if you assigned something other than null_distribution
above get_pvalue(obs_stat = test_stat, direction = “greater”) # would
only replace test_stat if assigned another name in Step 1 ## Warning:
Please be cautious in reporting a p-value of 0. This result is an
approximation ## based on the number of reps chosen in the
generate() step. ## ℹ See get_p_value()
(?infer::get_p_value()) for more information. p_val ## # A
tibble: 1 × 1 ## p_value ##
According to frequentist hypothesis testing assumptions, there is strong evidence to reject the null hypothesis with a p-value of 0.
Now do Step 4 (i.e., the null distribution visualization with confidence intervals) for the same variables and country as Task 4, and interpret the output.
conf_int <- null_distribution %>% get_confidence_interval(level = 0.95, type = “percentile”)
null_distribution %>% visualize() + shade_p_value(obs_stat = test_stat, direction = “greater”) + shade_confidence_interval(endpoints = conf_int) ## Warning in min(diff(unique_loc)): no non-missing arguments to min; returning ## Inf