Learning Objectives

Sources

Load Libraries

library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.1.4     ✔ readr     2.1.5
## ✔ forcats   1.0.0     ✔ stringr   1.5.1
## ✔ ggplot2   3.5.2     ✔ tibble    3.3.0
## ✔ lubridate 1.9.4     ✔ tidyr     1.3.1
## ✔ purrr     1.0.4     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(openxlsx)

Load data

setwd("E:/Biostat and Study Design/204/Lectures/Data")
NHANES_df <- openxlsx::read.xlsx('NHEFS.xlsx')

Observed and Expected Frequencies

Observed frequencies are the counts made from the data (experiments or measurements). Expected frequencies are counts calculated using probability theory.

Example: A single fair die was rolled 45 times with the following results, find the expected frequency.

Outcome 1 2 3 4 5 6
Observed Frequency O 13 6 12 9 3 2
Expected Frequency E ? ? ? ? ? ?


With 45 outcomes and 6 categories, the expected frequency for each cell is the same: 45/6 = 7.5.

Outcome 1 2 3 4 5 6
Observed Frequency O 13 6 12 9 3 2
Expected Frequency E 7.5 7.5 7.5 7.5 7.5 7.5


To measure the discrepancy between the O and E values, we use the chi-square \(\chi^2\) test statistic:

\[ {\chi}^2=\sum \frac{(O - E)^2}{E}\] where \(O\) is the expected frequency and \(E\) is the observed frequency.

The value of \(\sum(O - E)^2\) measures the magnitude of the differences between observed and expected differences, but does not provide information regarding the differences relative to what was expected. The relative magnitude of difference is found through division by the expected frequencies, as in the test statistic \(\sum {(O - E)^2}/{E}\). The theoretical distribution of \(\sum {(O - E)^2}/{E}\) is a discrete distribution the can be approximated by a chi-square distribution.

Going to back to our example, we can calculate the chi-square statistics using the formula above.

Outcome O E O-E (O-E)^2 (O-E)^2/E
1 13 7.5 5.5 30.25 4
2 6 7.5 -1.5 2.25 0.3
3 12 7.5 4.5 2.5 0.33
4 9 7.5 1.5 2.25 0.3
5 3 7.5 4.5 20.3 2.7
6 2 7.5 -5.5 30.25 4.03
Sum 11.66


Using chi-square distribution table, we can calculate the P-Value corresponding to a \(\chi^2\) score of 11.6, which is ≤ 0.05.

Alt
Source: https://stats.libretexts.org/

Contingency Tables & Chi-Square Test of Independence

A contingency table (or two-way frequency table) is a table consisting of frequency counts of categorical data corresponding to two different variables. (one variable is used to categorize rows, and a second variable is used to categorize columns). We use the term contingency table because we test for independence between the row and column variables.

In chi-square test of independence, we test the null hypothesis that in a contingency table, the row and column variables are independent. The formula for chi-square test of independence is:

\[ {\chi}^2=\sum \frac{(O - E)^2}{E}\]

where \(O\) is the expected frequency and \(E\) is the observed frequency. \(E\) is calculated using the following formula:

\[ E=\frac{(row\:total)(column\:total)}{grand\:total} \] The degrees of freedom for chi-square test of independence is calculated using the following formula:

\[ DF= (r-1)(c-1)\] Where \(r\) is the number of rows and \(c\) is the number of columns.

Alt

The following assumptions must be met for the chi-square test of independence to return valid results:

Example: Recently, a study published the success and failure of different treatments for tarsal navicular stress fracture. Using significance of level of 0.05, test the claim that the success of the treatment is independent of the type of treatment.

Treatment Success Failure
Surgery 54 12
Weight-Bearing Cast 41 51
Non-Weight-Bearing Cast for 6 Weeks 70 5
Non-Weight-Bearing Cast for Less Than 6 Weeks 17 5


\({H_0}\): Success is independent of treatment.

\({H_1}\): Success is dependent on treatment.

We confirm that each group has 5 events and non-events. We start by calculating \(E\) for each cell using the formula above. The first cell has a row total of 66, and column total 182. The grand total for the contingency table is 253.

\[ E=\frac{(row\:total)(column\:total)}{grand\:total} = \frac{(66)(182)}{(255)}=47.1\] Using the same formula, we calculate the \(E\) for the remaining cells.

Treatment Success Failure
Surgery 54 (E = 47.11) 12 (E = 18.89)
Weight-Bearing Cast 41 (E = 65.66) 51 (E = 26.33)
Non-Weight-Bearing Cast for 6 Weeks 70 (E = 52.53) 5 (E = 21.47)
Non-Weight-Bearing Cast for Less Than 6 Weeks 17 (E = 15.70) 5 (E = 6.30)


We use the \({\chi}^2\) distribution with this test statistic:

\[ {\chi}^2=\sum \frac{(O - E)^2}{E}=\frac{(54-47.48)^2}{47.48}+\:...\:+\frac{(5-6.17)^2}{6.17}=53.96\] The degrees of freedom \((r-1)(c-1)=(4-1)(2-1)=3\). Since 53.96 exceeds the limit \({\chi}^2\) limit (7.815) corresponding to 3 degrees of freedom, we know that P-value < 0.05.

Interpretation: Since P-Value < 0.05, we reject the null hypothesis and conclude that there is sufficient evidence to reject the claim that the outcome is independent of treatment.

Treatment Success Failure Success Rate (%)
Surgery 54 12 81.8
Weight-Bearing Cast 41 51 44.6
Non-Weight-Bearing Cast for 6 Weeks 70 5 93.3
Non-Weight-Bearing Cast for Less Than 6 Weeks 17 5 77.3


Success rates suggest that the best treatment is non–weight-bearing cast for 6 weeks.

Example: Using data from the NHANES study, determine if the proportion of patients treated with antidepressant medications is independent of sex.

\({H_0}\): Antidepressant treatment is independent of sex (\(p_1=p_2\)).

\({H_1}\): Antidepressant treatment is dependent on sex (\(p_1 \neq\ p_2\)).

Let’s start first by constructing the contingency table, and confirm that each group has 5 events and non-events

table_antidepressants <- table(sex=NHANES_df$sex,antidepressants=NHANES_df$nerves)
table_antidepressants
##         antidepressants
## sex        0   1
##   Female 658 172
##   Male   736  63

Next, let’s examine the proportion of subjects in each group.

table_weight_prop <- prop.table(table_antidepressants,margin = 1)
round(table_weight_prop,3) *100
##         antidepressants
## sex         0    1
##   Female 79.3 20.7
##   Male   92.1  7.9

It appears that the portion of females treated with antidepressants is higher than that of males treated with antidepressants. Next, we use the chi-square test of independence.

chisq.test(NHANES_df$nerves,NHANES_df$sex,correct = FALSE)
## 
##  Pearson's Chi-squared test
## 
## data:  NHANES_df$nerves and NHANES_df$sex
## X-squared = 54.352, df = 1, p-value = 1.676e-13

Interpretation: Since P-Value ≤ 0.05, we reject the null hypothesis, and conclude that there are sufficient evidence to reject the claim that antidepressants treatment is independent of sex in 1971.

Fisher’s Exact Test

Chi-Square test of independence has the requirement that every cell must have at least 5 events and non-events in each group for accuracy. This requirement is necessary for the \({\chi}^2\) distribution to be a suitable approximation to the exact distribution of the \({\chi}^2\) test statistic. Fisher’s exact test provides an exact P-value and does not require an approximation technique. Because the calculations are quite complex, Fisher’s exact test is often calculated using software technology.

Example: Using data from NHANES study, determine if the proportion of subjects with PICA is independent of sex.

\({H_0}\): PICA diagnosis is independent of sex (\(p_1=p_2\)).

\({H_1}\): PICA diagnosis is dependent on sex (\(p_1 \neq\ p_2\)).

Let’s start first by examining the frequency of PICA between males and females.

NHANES_df_pica <- NHANES_df %>% filter(pica!=2) #remove PICA =2 (missing values) - refer to data dictionary
table_weight <- table(sex=NHANES_df_pica$sex,PICA=NHANES_df_pica$pica)
table_weight
##         PICA
## sex        0   1
##   Female 398   6
##   Male   433   1

Next, let’s examine the proportion of subjects in each group.

table_weight_prop <- prop.table(table_weight, margin = 1)
round(table_weight_prop,3)*100
##         PICA
## sex         0    1
##   Female 98.5  1.5
##   Male   99.8  0.2

Since the frequency of males with PICO is less than 5, it is not appropriate to use chi-square test of independence and we should use Fisher’s exact test instead.

fisher.test(NHANES_df_pica$sex,NHANES_df_pica$pica)
## 
##  Fisher's Exact Test for Count Data
## 
## data:  NHANES_df_pica$sex and NHANES_df_pica$pica
## p-value = 0.06062
## alternative hypothesis: true odds ratio is not equal to 1
## 95 percent confidence interval:
##  0.003327652 1.273412380
## sample estimates:
## odds ratio 
##   0.153478

Since P-Value ≥ 0.05, we fail to reject the null hypothesis and conclude that there is no sufficient evidence to reject the claim that PICA diagnosis is independent of sex.

McNemar’s Test for Matched Pairs

McNemar’s test is used to compare the frequencies of paired samples of dichotomous data. It is similar to the paired samples t-test but for dichotomous nominal instead of interval variables. For 2 X 2 tables consisting of frequency counts that result from matched pairs, the frequency counts within each matched pair are not independent and, for such cases, we can use McNemar’s test of the null hypothesis that the frequencies from the discordant (different) categories occur in the same proportion.

Example: You are investigating a new pharmacist intervention to help improve patients adherence to treatment. You recruited a sample of patients, and you assessed their adherence to treatment before and after the intervention. Using a significance level of 0.05, determine if the pharmacist intervention influenced adherence.

\({H_0}\): Adherence is not affected by pharmacist intervention.

\({H_1}\): Adherence is affected by pharmacist intervention.

#create dataset
data_mc <- matrix(c(40, 10, 30, 20), nrow = 2,
    dimnames = list("After Intervntion" = c("Good adherence", "Poor adherence"),
                    "Before Intervntion" = c("Good adherence", "Poor adherence")))

data_mc
##                  Before Intervntion
## After Intervntion Good adherence Poor adherence
##    Good adherence             40             30
##    Poor adherence             10             20
mcnemar.test(data_mc)
## 
##  McNemar's Chi-squared test with continuity correction
## 
## data:  data_mc
## McNemar's chi-squared = 9.025, df = 1, p-value = 0.002663

Interpretation: Since P-Value < 0.05, we reject the null hypothesis, and conclude that there is sufficient evidence to support the claim that pharmacist intervention affects patients adherence.