Chi-square test

This document was composed from Dr. Snopkowski’s ANTH 504 Week 5 lecture and Danielle Navarro’s 2021 Learning statistics with R Chapter 12.

Statistical Tests

For each statistical test we discuss, I want you to note:

  1. What type of variables do we need for this test?

  2. What is the null and alternative hypotheses?

  3. What test (and R code) do we need to run the statistical analysis?

  4. What are the assumptions of the test?

  • How do we check the assumptions of the test?

  • What alternative tests do we run if the assumptions are not met?

Categorical Data – Chi-Square Test

  • Sometimes we have data consisting of the frequency of cases falling into unique categories

  • Examples:

    • Number of people voting for different politicians

    • Numbers of students who pass or fail their degree in different subject areas

    • Number of patients or waiting list controls who are ‘free from diagnosis’ (or not) following a treatment

An Example: Dancing Cats and Dogs

  • Analyzing two or more categorical variables

    • The mean of a categorical variable is meaningless
      • The numeric values you attach to different categories are arbitrary
      • The mean of those numeric values will depend on how many members each category has.
    • Therefore, we analyze frequencies

  • An example

    • Can animals be trained to line-dance with different rewards?

    • Participants: 200 cats

    • Training

      • The animal was trained using either food or affection, not both

    • Dance

      • The animal either learned to line-dance or it did not

    • Outcome:

      • The number of animals (frequency) that could dance or not in each reward condition

    • We can tabulate these frequencies in a contingency tabl

A Contingency Table

Each person, item or entity contributes to only one cell of the contingency table.

Pearson’s chi-square test

  • Used to see whether there’s a relationship between two categorical variables

  • Compares the frequencies you observe in certain categories to the frequencies you might expect to get in those categories by chance.

Ho: p1 = p2 (proportions are equal)

Ha: p1 ≠ p2 (proportions are unequal

Pearson’s Chi-Square Test

  • The equation:

    • i represents the rows in the contingency table and j represents the columns.

    • The observed data are the frequencies the contingency table

  • The ‘model’ is based on ‘expected frequencies’.

    • Calculated for each of the cells in the contingency table.

    • n is the total number of observations (in this case 200)

How do we know if it’s significant?

  • Test statistic

    • Checked against a distribution with (r − 1)(c − 1) degrees of freedom. 2-1 is 1 and 2-1 is 1 and 1 * 1 is 1 so the df is 1.

      • We can use the pchisq() function or the qchisq() function in a similar way pnorm() and qnorm()

    • If significant then there is a significant association between the categorical variables in the population.

    • The test distribution is approximate so in small samples use Fisher’s exact test.

Pearson’s Chi-Square Test

Interpreting Chi-Square

  • The test statistic gives an ‘overall’ result.

  • We can break this result down using standardized residuals.

  • There are two important things about these standardized residuals:

    • Standardized residuals have a direct relationship with the test statistic (they are a standardized version of the difference between observed and expected frequencies).

    • These standardized are z-scores (e.g. if the value lies outside of the range between –1.96 and +1.96 then it is significant at p < .05).

  • Effect Size

    • The odds ratio can be used as an effect size measure
library(tidyverse)
## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
## ✔ ggplot2 3.4.0      ✔ purrr   1.0.1 
## ✔ tibble  3.1.8      ✔ dplyr   1.0.10
## ✔ tidyr   1.2.1      ✔ stringr 1.5.0 
## ✔ readr   2.1.3      ✔ forcats 0.5.2 
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
chi <- function(n) {
  x <- rnorm(n)
  sum(x^2)
}
S <- replicate(10000, chi(15))
P <- data.frame(S) %>% 
  ggplot(aes(S)) +
  geom_density()
P

Try this equation with chi(n) with n equal of different values to see how it changes. n is the degrees of freedom.

P + stat_function(fun=dchisq, args=list(df=14))

qchisq(0.95, df = 1)
## [1] 3.841459
pchisq(25.35, df = 1, lower.tail=FALSE)
## [1] 4.781524e-07

The value we found (25.35) is larger than the critical value (3.84) and the p-value is less than 0.05 (p=4.78e-7). Therefore, we reject the null in favor of the alternative.

Important Points

  • The chi-square test has two important assumptions:

    • Independence:

      • Each person, item or entity contributes to only one cell of the contingency table.

    • The expected frequencies should be greater than 5.

      • In larger contingency tables up to 20% of expected frequencies can be below 5, but there a loss of statistical power.

      • Even in larger contingency tables no expected frequencies should be below 1.

      • If you find yourself in this situation consider using Fisher’s exact test.

  • Proportionately small differences in cell frequencies can result in statistically significant associations between variables if the sample is large enough

    • Look at row and column percentages to interpret effects

How to conduct in R?

#Enter data into R:
cont_table <- data.frame(food=c(28,10), affection=c(48,114))
cont_table
##   food affection
## 1   28        48
## 2   10       114
#Run chi-square test: 
chisq.test(cont_table) 
## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  cont_table
## X-squared = 23.52, df = 1, p-value = 1.236e-06
#OR run it all on one line
chisq.test(data.frame(c(28,10),c(48,114)))
## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  data.frame(c(28, 10), c(48, 114))
## X-squared = 23.52, df = 1, p-value = 1.236e-06

What is Yates’ continuity correction? If you only have 1 degree of freedom, you need to do a correction because the overestimates the chi-square test statistic, so we subtract ½ from the numerator of our test statistic before squaring. If you want to turn it off, use the argument: correct=F

#Run chi-square test: 
chisq.test(cont_table, correct=FALSE) 
## 
##  Pearson's Chi-squared test
## 
## data:  cont_table
## X-squared = 25.356, df = 1, p-value = 4.767e-07

Effect Sizes

People typically report the odds ratio when they do a chi-square test (although it is not as useful is you have > 2x2 table)

odds_ratio

Communicating Results

  1. Start with some descriptive statistics.

  2. The description tells you what the null hypothesis being tested is

  3. A “stat” block is included

  4. The results are interpreted

Of the 200 cats trained in this experiment, 38 were trained with food as a reward, while 162 were trained with affection as the reward. A chi-square test for association with Yates continuity correction was conducted to test whether there was an association between training type and dancing. Results show a significant association between the type of training and whether or not cats would dance χ2(1, N = 200) = 25.36, p < .001. Based on the odds ratio, the odds of cats dancing were 6.65 times higher if they were trained with food than if trained with affection.

What if your expected frequencies are smaller than 5?

  • Let’s look at some new data. Is there a relationship between being a professional athlete and smoking?

  • What is the null and alternative hypothesis?

  • Here is the contingency table:

chisq.test(data.frame(c(7,1),c(2,5)))
## Warning in chisq.test(data.frame(c(7, 1), c(2, 5))): Chi-squared approximation
## may be incorrect
## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  data.frame(c(7, 1), c(2, 5))
## X-squared = 3.2254, df = 1, p-value = 0.0725
fisher.test(data.frame(c(7,1), c(2,5)))
## 
##  Fisher's Exact Test for Count Data
## 
## data:  data.frame(c(7, 1), c(2, 5))
## p-value = 0.04056
## alternative hypothesis: true odds ratio is not equal to 1
## 95 percent confidence interval:
##    0.8646648 934.0087368
## sample estimates:
## odds ratio 
##   13.59412

How would you go about analyzing this as a chi-square test?

Chart