Analysing Survey Data with R - Part I

This document contains some common tools and tricks to explore survey data. For ease of generalisation, we will be simulating our own survey data rather importing a pre-prepared data set. If you are importing your own data set however, the following packages and functions will be useful for importing sav (SPSS), csv or Excel files respectively:

library(haven) #part of the tidyverse
haven::read_sav() #import SPSS files

library(readr) #part of the tidyverse
readr::read_csv() #import csv files

library(readxl)
readxl::read_xlsx() #import xlsx files

sav files usually contain variable labels, which may be useful for when you conduct survey analysis, e.g. remindng you what a particular Q2c_14 variable stands for.

Simulating the data

To start simulating some survey data, let’s first load the tidyverse package, set seed and create the variables:

library(tidyverse)

Loading tidyverse: ggplot2
Loading tidyverse: tibble
Loading tidyverse: tidyr
Loading tidyverse: readr
Loading tidyverse: purrr
Loading tidyverse: dplyr
Conflicts with tidy packages --------------------------------
filter(): dplyr, stats
lag():    dplyr, stats

set.seed(69)
resp_id <- 1:2000
gender <- round(runif(2000,min=0,max=1)) #0 is male, 1 is female
age <- sample(c("18-24","25-34","35-54","55-64","65+"),2000,replace=TRUE,prob=c(0.15,0.20,0.3,0.15,0.2))
visited_prev <- round(runif(2000,min=0,max=1)) #Whether respondent has visited before
csat <- round(runif(2000,min=0,max=10)) #Customer Satisfaction Score
csat[which(visited_prev==0)] <- NA #NA if respondent had not visited before
df <- data.frame(resp_id,gender,age,visited_prev,csat) #assign to a data frame

Let’s have a look at what the simulated data looks like:

glimpse(df)

Observations: 2,000
Variables: 5
$ resp_id      <int> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, ...
$ gender       <dbl> 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 1, 0,...
$ age          <fctr> 55-64, 35-54, 55-64, 55-64, 18-24,...
$ visited_prev <dbl> 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1,...
$ csat         <dbl> NA, 9, NA, NA, NA, NA, 9, NA, NA, 5...

Apart from the age variable which was set to factor, the rest of the variables will need to be set to the right type/class. As a general rule for survey analysis, respondent IDs should be set as character, and categorical variables as factor. Here’s what we’ll do:

df$resp_id <- as.character(df$resp_id)
library(stringr)
df$resp_id <- str_pad(df$resp_id,3,side="left",pad="0") #Make the ID into a "proper" 3-digit ID

This is what resp_id looks like after we run the code:

glimpse(df$resp_id)

 chr [1:2000] "001" "002" "003" "004" "005" ...

The other bits that we have to do is to coerce the other categorical variables in the data into factors. This is straightforward to do in R:

df$gender <- factor(df$gender)
levels(df$gender) <-c("Male","Female") #Rename levels of factor for easy reading
df$visited_prev <- factor(df$visited_prev)
glimpse(df)

Observations: 2,000
Variables: 5
$ resp_id      <chr> "001", "002", "003", "004", "005", ...
$ gender       <fctr> Female, Female, Female, Female, Ma...
$ age          <fctr> 55-64, 35-54, 55-64, 55-64, 18-24,...
$ visited_prev <fctr> 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1...
$ csat         <dbl> NA, 9, NA, NA, NA, NA, 9, NA, NA, 5...

The data looks ready. We can now begin exploring this data.

Exploring the data

Some prefer to go straight for plots (e.g. boxplots, histogram) when in the initial stage of exploring data. However, for survey data, I find that it’s generally useful to eye-ball some contingency tables. Let’s have a look at the gender variable first:

prop.table(table(df$gender))

  Male Female 
0.5125 0.4875 

So this table tells us that 51.3% and 48.8% of the sample are male and female respectively, which sounds pretty reasonable.

The next step is to see if any age groups are over or under-represented in either of the genders. I’m going to use the package gmodels to produce these cross-tables:

library(gmodels)
CrossTable(df$age,df$gender,prop.r=FALSE,prop.t=FALSE,prop.chisq=FALSE,format="SPSS")

   Cell Contents
|-------------------------|
|                   Count |
|          Column Percent |
|-------------------------|

Total Observations in Table:  2000 

             | df$gender 
      df$age |     Male  |   Female  | Row Total | 
-------------|-----------|-----------|-----------|
       18-24 |      146  |      137  |      283  | 
             |   14.244% |   14.051% |           | 
-------------|-----------|-----------|-----------|
       25-34 |      206  |      219  |      425  | 
             |   20.098% |   22.462% |           | 
-------------|-----------|-----------|-----------|
       35-54 |      314  |      292  |      606  | 
             |   30.634% |   29.949% |           | 
-------------|-----------|-----------|-----------|
       55-64 |      150  |      137  |      287  | 
             |   14.634% |   14.051% |           | 
-------------|-----------|-----------|-----------|
         65+ |      209  |      190  |      399  | 
             |   20.390% |   19.487% |           | 
-------------|-----------|-----------|-----------|
Column Total |     1025  |      975  |     2000  | 
             |   51.250% |   48.750% |           | 
-------------|-----------|-----------|-----------|

 

What this table tells us is that there are slightly more 65+ males than females in this sample, but the age group with the highest incidence for both genders is 35-54. We can make this a little easier for us to visualise by creating some plots using the ggplot2 package.

library(ggplot2)
ggplot(df,aes(x=age))+
  geom_bar()+
  facet_grid(~gender)

Now that we’ve got a good sense of the demographic background which all seems to make sense, we can have a look at the key metric i.e. csat, or customer satisfaction.

Here are just a histogram and a boxplot to check the distribution of csat.

ggplot(df,aes(x=csat))+geom_histogram(binwidth=1)

ggplot(df,aes(x=1,y=csat))+geom_boxplot()

Since the csat variable was generated using a runif() (uniform distribution) function restricted to values between 0 and 10, we are unlikely to see a ‘realistic’ fall-out of customer satisfaction scores that we might expect from an actual survey.

Another way to get a good sense of the csat data is to use the summary() function, as below:

summary(df$csat) #get min, median, mean, max, etc. 

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
  0.000   3.000   5.000   5.024   7.000  10.000    1009 

One thing that we would be interested out of this survey data is whether customer satisfaction scores actually correlate with age and gender. First, however, we need to take note of the visted_prev variable, which represents whether the respondent has visited our business before. Again, we can use gmodel::CrossTable():

CrossTable(df$visited_prev,df$gender,prop.r=FALSE,prop.t=FALSE,prop.chisq=FALSE,format="SPSS")

   Cell Contents
|-------------------------|
|                   Count |
|          Column Percent |
|-------------------------|

Total Observations in Table:  2000 

                | df$gender 
df$visited_prev |     Male  |   Female  | Row Total | 
----------------|-----------|-----------|-----------|
              0 |      514  |      495  |     1009  | 
                |   50.146% |   50.769% |           | 
----------------|-----------|-----------|-----------|
              1 |      511  |      480  |      991  | 
                |   49.854% |   49.231% |           | 
----------------|-----------|-----------|-----------|
   Column Total |     1025  |      975  |     2000  | 
                |   51.250% |   48.750% |           | 
----------------|-----------|-----------|-----------|

 

CrossTable(df$visited_prev,df$age,prop.r=FALSE,prop.t=FALSE,prop.chisq=FALSE,format="SPSS")

   Cell Contents
|-------------------------|
|                   Count |
|          Column Percent |
|-------------------------|

Total Observations in Table:  2000 

                | df$age 
df$visited_prev |    18-24  |    25-34  |    35-54  |    55-64  |      65+  | Row Total | 
----------------|-----------|-----------|-----------|-----------|-----------|-----------|
              0 |      141  |      209  |      312  |      147  |      200  |     1009  | 
                |   49.823% |   49.176% |   51.485% |   51.220% |   50.125% |           | 
----------------|-----------|-----------|-----------|-----------|-----------|-----------|
              1 |      142  |      216  |      294  |      140  |      199  |      991  | 
                |   50.177% |   50.824% |   48.515% |   48.780% |   49.875% |           | 
----------------|-----------|-----------|-----------|-----------|-----------|-----------|
   Column Total |      283  |      425  |      606  |      287  |      399  |     2000  | 
                |   14.150% |   21.250% |   30.300% |   14.350% |   19.950% |           | 
----------------|-----------|-----------|-----------|-----------|-----------|-----------|

 

To be continued…