Introduction

Welcome to the first Guided Tutorial for the statistics in R section of 3007S. This tutorial introduces you to entering and analyzing categorical data using the chi-square test. You should read the chapter corresponding to chi-square tests in the course textbook, and refer to it frequently while working on this tutorial.

NB! some code inside code chunks will have a # in front that you need to remove to run it. Be reminded that we use # inside a code chunk to make notes for ourselves or keep code that we don’t want to run inside a chunk.

Before we start on the theory and statistics, some things you need to know about running analyses in R:

Setup chunk

At the start of every analysis, it is standard practice to include a setup chunk at the start of your rmd (see below). This step up chunk is used to load all your packages when working in R.

Packages:

Packages are a new concept, thus far we have only used base R. Packages can be thought of as special “toolboxes” that we need to install and load into R so that we can have access to specific “tools” for our analysis - we call these tools functions.

For example, mean() and sum() are functions (they tell R to do something) that you may have used before.

Specific packages contain specific functions.

Installing Packages

The first time that you want to use a package in R, you need to install it before loading it.

NB! You can check to see if a package is already installed in R by using the “Packages” tab on the right-hand side of the console. Here, you can search through the packages you have installed on this version of R (click back to “Files” tab if you feel lost).

If you need to install a package (it is not in the list in the “Packages” tab), we use the install.packages(“…”) function.

Loading packages

Once we have installed the packages we want (and were missing), we need to load them into R to use. You will need to load in the packages EVERY TIME you open R.

We load ALL packages we want to use in ONE chunk (called the set-up chunk) because:

  • It is easier to re-load them if needed, as they are all in once place.
  • They are all loaded at the beginning of the analysis, so we can use them throughout.
  • You do not accidentally “lose” or “forget” a package in your analysis.

You can check to see which packages are loaded in the “Packages” tab on the right too; they will have a check mark.

Pacman for Loading Packages:

In this course, we will use the “pacman” package (note it is with a lowercase p), and the p_load() function, to load our packages. Pacman is useful because it can load all our packages in one go (see below), and will install any packages we’re missing for us. Very useful, as it simplifies coding and limits errors that can arise from installing a package multiple times!

#if we need to install pacman (check on your "Packages" tab) use: 
#install.packages("pacman")

Now that we know what goes into a set-up chunk, see below.

This is what a set-up chunk should look like, begin each rmd with this:

#Basic Set up chunk:

library(pacman) #load pacman

p_load(tidyverse, psych, readxl, haven, janitor, knitr, grid) #load packages

What is going on in this chunk?

  • First, use library(pacman) to load in pacman (check that it is installed first!)
  • Next, we want to load the tidyverse, psych, readxl, haven, janitor, knitr, and grid packages.
  • We use p_load (a function from pacman) to do this easily, in one line!

If you are working in R and you decide you want to load an additional package, return to this set-up chunk, add the packages you want to the list, and re-run the chunk (click the green arrow in the right-hand corner of the chunk).

Code Chunk Settings

A final point to learn is using the settings in a code chunk for a nicer, briefer output (space can be an issue in tutorial submissions!).

You’ll see in the right-hand corner of each code chunk is a little cog. If you click on the cog, you are given some options and settings to use for the chunk, including a drop-down menu with options.

The best ones to use are:

  • “Show output only”: from the drop-down menu. This hides your code in the html, and only shows your outputs (saves space!)
  • “Show warnings”: set this off, hides all warning messages in the html for this chunk.
  • “Show messages”: set this off, hides all messages in the html for this chunk.

1. Loading Datasets into R

Now that we have loaded our packages (using pacman and p_load), we need to load in our data to use for an analysis.

Data can come in various formats (file types), like Excel, SPSS, TXT, CSV, etc. We’ll be using Excel in this course. In R, we can either enter data manually (which is difficult to do) or import it from external files such as Excel spreadsheets (easier and more common). Meaning, if we have an Excel spreadsheet with data on it, we can load th whole thing into R!

a. Data in Excel Spreadsheets

Take a loot at the “Tut_1_example.xlsx” Excel spreadsheet that came with this guided tutorial. Open it in Excel.

This is commonly what datasets look like when we want to use them in R. NB! Take note:

  • We have a column called participant_id which has a unique identifier for each participant.
  • We use an underscore “_” instead of spaces ” ” for column names (R does not like spaces!)
  • We have saved the dataset into the same file as our Rproj and rmd - this is very important!

Now, we would like to load this Excel dataset into R to use.

Inside a code chunk, we can use the read_excel() or read_xlsx() functions (from the readxl package) to read in the dataset. Remember to give it an appropriate name!

  • First, remove the # in front of the code below so that we can run it.
  • Replace the “add_your_data_name_here” with a nice name for your dataset.
  • Run the chunk; the read_xlsx() function reads the dataset into R.
#Example: excel_data <- read_xlsx("your_dataset.xlsx")

#Practice: 
#add_your_data_name_here <- read_xlsx("Tut_1_example.xlsx")

Now that the dataset has been read in, you can click on it in the right-hand corner to open and look at it in R.

b. Data from Other Sources (Not Excel)

Sometimes, we might get data from other file types (not Excel). R can still load these and use them, we just need special packages and functions to do so. We use different functions based on the file type. An examples is given below.

For files with .csv: use the read_csv() function from base R:

#Example:
#new_data <- read_csv("name_of_ther_dataset.csv")

This is one place where using an AI agent (like ChatGPT) can be very useful! It can help you find the correct package and function you need to read in “weird” dataset types, like .txt or .csv, etc, into R.

2. Visualizing Data

Now that we have loaded our packages and our dataset, we are ready to begin with the analysis. The first step of any analysis is to visualize the data.

Visualizing data in R is crucial for understanding the distribution and relationships within categorical data typically analyzed with Chi-square tests.

a. Tables in R

Creating readable and informative tables in R can be done using several packages and functions. Here we discuss table(), kable() from the knitr package, and tabyl() from the janitor package.

i. Basic Frequency Tables - table() and kable()

The basic table() function in R allows you to create a very simple frequency table. Run the code chunk below:

#Using the dataset we have: 
example_data <- read_xlsx("Tut_1_example.xlsx")

# Creating the simple frequency table
simple_table <- table(example_data$gender, example_data$favorite_flavour)

#showing the table
simple_table 
##        
##         chocolate strawberry
##   man          20         28
##   woman        34         18

What is going on in this chunk?

  • We have a dataset called example_data (it is the same dataset we looked at before, just with a new name).
  • Using table(), we specify the two columns (using $) that we want the frequencies for: gender and favorite_flavour.
  • We assign the object a name simple_table; shows us how many men versus women like chocolate versus strawberry.
  • We see that men like strawberry the most, and women like chocolate the most!

The table above is very basic - lets use kable() from the knitr package to make it more aesthetically pleasing. We use the caption = “…” to give it a title:

kable(simple_table, caption = "Frequency Table for Gender and Flavour Preference")
Frequency Table for Gender and Flavour Preference
chocolate strawberry
man 20 28
woman 34 18

ii. Basic Frequency Tables with Proportions - tabyl()

The tabyl() function from the janitor package is useful for creating more detailed tables that include proportions. These are often useful to discuss during an analysis! It uses the format: tabyl(your_dataset, your_variable)

Tabyl() only lets us see proportions (percentages) for one variable at a time. So looking at gender first:

#to see the proportion of men versus women we have: 
tabyl(example_data, gender)

So in our sample, there are n = 48 men, making up 48% of participants. And there are n = 52 women, making up 52% of participants. We have 48 + 52 = 100 participants in total.

Next, lets look at their flavour preferences:

#to see the proportion of chocolate versus strawberry we have: 
tabyl(example_data, favorite_flavour)

So in our sample, n = 54 which is 54% of participants like chocolate the most, and n = 46 which is 46% of participants like strawberry the most!

We could go a step further and try to add both gender and flavour preference into the tabyl() function, but then it does not work out the percentages:

#the exact same table as using table() 
tabyl(example_data, gender, favorite_flavour)

b. Plotting Graphs in R: Bar plots for Chi-square

Now that we’ve taken an initial look into our dataset, our variables, and the proportions of participants, we want to move on to visualize this information in a nice way. A simple way to do this is using bar plots, which can help in understanding the distribution of categorical data.

Bar plots are useful for visualizing the frequency of categorical data. Run the code chunk below:

barplot(simple_table, 
        beside = TRUE, 
        legend = TRUE,
        col = c("lightblue", "pink"),
        xlab = "Flavours", 
        ylab = "Count",
        main = "Barplot of Gender and Preference")

#For fun: Replace the "True" with "False", re-run the code, and see what happens!

What is going on in this chunk?

  • The barplot() function tells R to plot a bar plot.
  • We want to visualize the example_data, so this is included in the function.
  • Next, beside = TRUE tells R to put the columns beside each other.
  • Next, legend = TRUE adds the legend.
  • The col = c(“…”, “…”) assigns colors to the columns
  • The xlab = “…” gives the x-axis a title
  • The ylab = “…” gives the y-lab a title
  • Finally, main = “…” gives the plot a title.

What can we see from the bar plot? We note that, like our tables have suggested, men seem to like strawberry the most, and women seem to like chocolate the most!

3. Analyzing Data in R: Chi-Square

Now that we’ve inspected our data and visualized it, we can move on to doing the actual analysis: Chi-square tests. Again, be sure to read the associated section of your textbook.

Quick Revision: Chi-square tests are used to determine whether there is a significant association between categorical variables in a dataset. They work by comparing observed frequencies (what you actually see in your data) versus expected frequencies (what you expect to see in your data), and whether there are significant differences between these.

the Chi-square distribution is a theoretical or mathematical distribution that represents the distribution of the sum of squared standard normal deviates (z scores). This distribution is used to calculate p-values in chi-square tests.

Types of Chi-Square Tests:

Goodness-of-Fit Test:

Test of Independence or Association:

a. Observed and Expected Values in Chi-Square

i. Probability Distribution

As mentioned, when dealing with categorical data, the observed frequencies are the actual counts in each category in you dataset, whereas the expected frequencies are the counts we would expect if there were no effect or association between the variables in our data.

So, the tables and bar plot we generated above show us the observed frequencies of values in our data (how many men versus women, and chocolate versus strawberry lovers, we ACTUALLy have in our data).

ii. Chi-squared Distribution in R

Taking a moment to explore the theory, in R we can generate the theoretical Chi-squared distribution with the dchisq, pchisq, qchisq, and rchisq functions, representing the density, cumulative distribution, quantile, and random generation functions, respectively.

For example, run the code below to visualize the Chi-squared distribution with 10 degrees of freedom:

#No need to memorise this code:
curve(dchisq(x, df = 10), 
      from = 0, to = 30, 
      main = "Chi-squared Distribution with 10 degrees of freedom")

NB! This figure was generated so that you could see it, out of interest - do not worry about being able to replicate this code, or interpret it!

b. One Dimensional Chi-square: Goodness of Fit (GOF) Test

We run a Goodness-of-fit test when we have only one categorical variable that we are interested in analyzing. The Goodness-of-fit test compares the observed frequency of the one categorical variable with it’s expected frequency - said another way, does our data fit the expected pattern.

In this example, we use our dataset from before BUT were are only interested in the flavor preferences (chocolate or strawberry) of people. So we sampled 100 people and asked them their flavour preference. We want to know if their preferences are random or if something about the sample makes them more likely to fall into one of these categories (liking chocolate or strawberry).

So we compare the flavour preferences we found (the observed frequencies from our data) with an expected frequency. For our example, we have n = 100 participants, so our expected frequencies would be n = 50 chocolate and n = 50 strawberry. And we know from the tables above that our observed frequencies are n = 54 chocolate and n = 46 strawberry.

So, our expected frequencies has under-estimated the amount of chocolate lovers (54 versus 50), and over-estimated the number of strawberry lovers (46 versus 50)!

Doing a Goodness-of_fit Analysis in R

To do a Goodness-of-fit analysis in R, we want to compare the expected frequency and the observed frequency of values of a categorical variable in our dataset, and follow three steps:

Step 1: Create a Frequency Table from Observed Values

First, we need to generate a frequency table of the categorical variable of interest - in our example, this is flavour presence. We use the table() function from before, and now we need to save these counts into an object:

#we only include our variable of interest: favorite_flavour
observed_flavour_counts <- table(example_data$favorite_flavour)

#let's look at the observed frequency table: 
observed_flavour_counts
## 
##  chocolate strawberry 
##         54         46

Again, we note that we have n = 54 participants who prefer chocolate, and n = 46 participants who prefer strawberry.

Step 2: Run a Chi-square test

We next can run a chi-square test (more on this later!) using the chisq.test() function. This function automatically calculates the expected frequencies for us! It assumes that all the expected frequencies are equally distributed; that is, that n = 50 participants should like chocolate, and n = 50 should like strawberry.

#we run the chi-square test, using the observed counts calcualted earilier 
chisq_GOF_test <- chisq.test(observed_flavour_counts)

#we can take a look at the result:
chisq_GOF_test
## 
##  Chi-squared test for given probabilities
## 
## data:  observed_flavour_counts
## X-squared = 0.64, df = 1, p-value = 0.4237

Interpreting this Output

  • We see from the output that the p-value = .434, which is much larger than our standard significant level of .05
  • Meaning, that the difference between the expected and observed frequencies is not significant.
  • So, while we have more chocolate lovers than expected, and less strawberry lovers than expect, these differences between the observed and expected frequencies are not significant; they are due to random chance!

If your p-value is non-significant, you would stop your analysis here. If you p-value is significant, we would next move to calculate the residuals after calculating the expected frequencies (see section below).

Step 3: View the Expected Frequencies

We can also easily view the expected frequencies used by the chisq.test() function above, see code below:

#to view the expeted frequencies 
chisq_GOF_test$expected
##  chocolate strawberry 
##         50         50

Special Case: Unusual Expected Frequencies

What if we wanted to use different expected frequencies? For example, what if we expected n = 40 (40%) of our participants to like chocolate, and n = 60 (60%) to like strawberry? We would then want to use these expected frequencies in our test, not just n = 50 (50% each) like before.

Luckily, we can do this easily by specifying these values in the chisq.test() function:

#re-run the test with our special expected frequencies 
chisq_GOF_test2 <- chisq.test(observed_flavour_counts, p = c(0.4, 0.6))

#view the result
chisq_GOF_test2
## 
##  Chi-squared test for given probabilities
## 
## data:  observed_flavour_counts
## X-squared = 8.1667, df = 1, p-value = 0.004267

Interpreting this Output

  • We see from the output that the p-value = .004, which is much smaller than our standard significant level of .05
  • Meaning, that the difference between these new expected and observed frequencies is significant!

Finally, we’ll view the special expected frequencies to make sure we specified them correctly:

chisq_GOF_test2$expected
##  chocolate strawberry 
##         40         60

Indeed, we’ve specified them correctly.

c. Two Dimensional Chi-square: Test of Association/Independence

Above we looked at a Goodness-of-fit analysis, which we use when we have one categorical variable of interest (in our example, this was flavour preference only). Another Chi-square test (and more commonly used) is the test of Independence or Association; that is, we want to test if two categorical variables are significantly associated or not.

In our example, using our dataset from before, this would be whether gender and flavour preference are associated; does a participant’s gender help us predict what flavour they might prefer?

A key concept for these chi-square tests is independence: The concept of independence refers to the lack of association between the two variables in a contingency table (see below). If knowing the value of one variable tells you nothing about the value of the other, they are considered independent!

Doing a Chi-square Test of Association/Independence

Again, we want to compare the expected frequencies and the observed frequencies of values of the categorical variables in our dataset (now for both gender and flavour preference), and again follow three steps:

Step 1: Create a Contingency Table from Observed Values

We have already done this above using the table() function, but will show it here again:

#table containing the observed counts of our two categorical variables of interest
simple_table <- table(example_data$gender, example_data$favorite_flavour)

#show the table
simple_table
##        
##         chocolate strawberry
##   man          20         28
##   woman        34         18

This table should look very familiar by now!

Step 2: Run a Chi-square test

Next, we move on to run the analysis using the chisq.test() function, which automatically calculates our expected frequencies and compares them to our observed frequencies (more on this below!)

#we run the chi-square test, using the simple_table as it contains the observed counts
chisq_test <- chisq.test(simple_table)

#we can take a look at the result:
chisq_test
## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  simple_table
## X-squared = 4.7381, df = 1, p-value = 0.0295

Interpreting this Output

  • We see from the output that the p-value = .0295, which is much larger than our standard significant level of .05
  • Meaning, that the difference between the expected and observed frequencies is significant.
  • So there is a significant association between gender and flavour preference in our participants!

Again, if your p-value was non-significant, you would stop your analysis here. If you p-value is significant (like in this case), we would move on to calculate the residuals after calculating the expected frequencies (see section below); the residuals would show us WHERE exactly the significant differences are!

Step 3: View the Expected Frequencies

We now view the expected frequencies used by the chisq.test() function above, see code below:

chisq_test$expected
##        
##         chocolate strawberry
##   man       25.92      22.08
##   woman     28.08      23.92

While these expected frequencies may look a little weird as decimals, this is normal.

Remember, expected frequencies in a contingency table are theoretical, and calculated based on the row and column totals: Expected frequency = (row total*column total)/(grand total), and sometimes this results in decimals.

NB! What is important to remember is that the expected values reflect the values we’d expect to find, on average, if there were no relationship between our variables, gender and flavor preference!

d. Computing Residuals: Finding where difference lies

As mentioned, if we find a significant p-value for our chi-squared test, we would then move on to calculate our residuals as the final part of our analysis. There are two kinds: the standardized residuals and the adjusted residuals (see below).

Residuals, in the context of a Chi-square test, are a measure of the differences between observed and expected frequencies. Said another way, residuals indicate the discrepancies between observed and expected values.

Standrdized Residuals

We start by calculating the standardized residuals. These show us if an observed value differs from its expected value; and whether the observed value is greater than or smaller than the expected value.

We say they are standerdized because they have been scaled by the size of the expected value. We can extract them using $residuals, run the code below:

#take not that we use our saved chi-square test output and $residual 
chi_residuals <- chisq_test$residuals

#and we take a look at the output
chi_residuals
##        
##         chocolate strawberry
##   man   -1.162798   1.259860
##   woman  1.117180  -1.210434

Interpreting this Output

  • If the residual is close to 0, the observed value is close to the expected value (no significant difference).
  • If the residual is positive, the observed value is GREATER than the expected value.
  • If the residual is negative, the observed value is SMALLER than the expected value.

We would not spend too much time interpreting these, and instead move on to the adjusted residuals.

Adjusted Residuals

Next, we calculate the adjusted residuals. These are more useful, as they help pinpoint which specific cells/values in the table are contributing to the significant chi-square test statistic. Adjusted residuals are like refined z-scores for each cell.

For adjusted residuals, a value near 0 means the observed value is close to the expected value, and thus, does not differ significantly; an adjusted residual value ±1.96 means the difference is large, and thus, significant! To recap Psy2015f, if you have a z-score greater than 1.96 or smaller than -1.96 then we know we have a significant value.

Again, we have code that can extract them for us:

#Adjusted residuals (more directly interpretable)
adj_residuals <- chisq_test$stdres

#view them
adj_residuals
##        
##         chocolate strawberry
##   man   -2.377517   2.377517
##   woman  2.377517  -2.377517

Again, we would look at the sign to tell if the values are significantly greater or smaller.

Looking at this output, and as an example, we see that significantly more men liked strawberry than expected (2.38), and significantly fewer women liked strawberry than expected (-2.38)!

You have reached the end of chi-squared analyses. Well done! You can now completed the Free Form Exercise.