Lab #2: Exploratory Data Analysis

1.1: Loading data

In order to load the data into an R Markdown file so that it knits correctly, you must specify the exact pathway to the data. Remember, when we knit/render an R Markdown file, every line of code that is needed to run your code chunks has to be included in the Rmd file – it doesn’t interact with your global environment. Essentially, every time you knit, it acts as if the environment is empty and only runs the code in your Rmd file. Thus, we have to direct the R Markdown file where to find our data. It can make life easier to set a working directory or create a project for each lab so that the file pathway is short (we discussed this in Lab #1).

We are going to use the data collected from prior semesters of STAT 250 students to investigate the amount of time per week students spend either exercising or watching TV.

Modify the code below to read in the StudentSurvey_Lab2.csv data. Remember, you can test a code chunk by running the code in the chunk without knitting the whole document.

survey<-read.csv("Lab2_StudentSurvey.csv", header=TRUE)

We have now created a datafame called survey. Now we’ll want to investigate the data within our dataframe.

1.2: Using Data in R

Recall that in properly organized data, each row is an individual and each column is a variable. Once the data is loaded, you will have to “call” the variables for use in each function.

First, we need to know the names of the variables in our dataframe. Modify the code below to print out the list of variable names.

names(survey)

[1] "semester" "gender"   "age"      "class"    "exercise" "tv"      

We can also get a quick overview of the data using summary(). With the whole data set named in the function, you will get a summary for each variable in the data set. Modify the code below based on the description in the prior sentence.

summary(survey)

        semester      gender         age              class    
 Fall-2014  : 80   Female:347   Min.   :17.00   Freshman :  9  
 Fall-2015  : 80   Male  :166   1st Qu.:20.00   Junior   :278  
 Fall-2016  :103   NA's  :  1   Median :21.00   Senior   :135  
 Spring-2015: 86                Mean   :21.89   Sophomore: 86  
 Spring-2016: 93                3rd Qu.:23.00   NA's     :  6  
 Spring-2018: 72                Max.   :56.00                  
                                NA's   :3                      
    exercise            tv        
 Min.   : 0.000   Min.   : 0.000  
 1st Qu.: 2.000   1st Qu.: 2.000  
 Median : 4.000   Median : 5.000  
 Mean   : 5.108   Mean   : 6.641  
 3rd Qu.: 7.000   3rd Qu.:10.000  
 Max.   :24.000   Max.   :60.000  
 NA's   :20       NA's   :26      

1. Identify the variable type (e.g. numeric discrete) for each of the variables in our dataset.

Semester: Qualitative ordinal Gender: Qualitative nominal Age: Quantitative continuous Class: Qualitative ordinal Exercise: Quantitative discrete TV: Quantitative discrete

2. What type of summary does R provide for each of the variable types?

For age, exercise, and TV, R provides the 5 number summary. For semester, gender, and class, R provides a summary of the number of students in each subcategory.

What type of variable should age be? It would help for us to inspect the raw data of just the age variable. Recall that to call a variable inside a dataframe, we can use $ to direct first to a dataframe object, then to the variable within it. Add code within the code chunk below to use the $ option to print out the age variable to the screen.

Age should be quantitative continuous.

summary(survey$gender)

Female   Male   NA's 
   347    166      1 

3. You’ll notice that age was recorded as whole-years only. What type of variable does it make it?

It makes it quantitative discrete.

Let’s verify that R is reading the data the same way that we are. Each of the following codes can help us do this. Don’t run them yet – let’s discuss them first. Before we do, notice a new feature of R code below. In R, using a # essentially tells R to ignore anything after the #. You can use it to annotate your R code, such as below, by writing comments explaining what each line of code does. Note: # has a different meaning in R chunks than in the general Markdown document. In R chunks, it preceeds comments. In Markdown, it denotes headers.

class(survey$age)          # print the class of age (integer, string, etc.)

[1] "integer"

is.integer(survey$age)     # is age an integer?

[1] TRUE

as.integer(survey$age)     # assign, if necessary, age as an integer

  [1] 20 21 22 24 18 18 19 19 18 20 24 21 21 22 21 23 21 26 21 20 20 23 21
 [24] 21 19 19 21 20 22 19 21 22 23 21 21 21 20 20 20 20 23 20 18 27 20 31
 [47] 20 20 21 20 19 32 21 23 23 28 20 20 23 21 21 20 22 22 22 20 20 26 22
 [70] 20 21 20 19 21 30 38 38 22 22 20 NA 23 23 23 22 20 33 23 20 22 21 20
 [93] 21 19 21 35 26 19 21 20 19 20 19 26 24 23 19 20 22 22 33 20 20 22 22
[116] 19 18 21 23 21 21 19 23 29 29 25 32 20 26 21 28 18 22 23 20 19 18 20
[139] 20 19 22 19 19 24 20 22 21 21 19 19 21 26 23 25 21 20 24 20 18 22 22
[162] 20 21 26 19 19 56 32 21 21 27 20 22 20 22 22 22 22 22 22 31 36 20 44
[185] 20 22 26 22 19 21 21 20 23 21 19 23 20 20 22 19 21 30 20 20 20 23 21
[208] 23 27 21 20 22 21 21 28 22 19 30 30 28 23 22 19 22 32 20 19 21 20 19
[231] 23 20 19 21 27 20 19 19 21 23 25 19 NA 23 22 22 20 24 20 20 22 20 24
[254] 21 19 24 19 19 28 32 20 19 19 19 20 20 19 19 23 20 19 25 19 20 24 22
[277] 22 21 19 19 19 23 21 21 21 23 20 19 19 19 20 21 18 29 18 21 22 19 22
[300] 27 24 24 23 20 27 21 20 19 19 41 20 20 19 27 19 19 20 20 27 21 26 20
[323] 19 20 20 20 20 20 21 19 19 37 20 19 21 19 20 20 20 21 20 19 19 20 21
[346] 19 21 19 23 21 19 23 24 23 21 22 20 19 23 21 23 20 23 22 21 21 20 33
[369] 21 23 20 20 20 20 20 21 20 19 18 23 22 21 22 20 22 18 33 20 21 25 22
[392] 54 19 23 19 20 22 19 22 21 22 20 26 20 22 19 20 25 20 17 22 20 22 19
[415] 25 21 20 24 21 27 21 20 21 23 23 22 20 29 23 20 18 21 23 23 19 19 19
[438] 19 20 26 20 20 26 19 19 23 19 19 21 20 19 22 22 20 20 20 19 21 19 20
[461] 19 20 19 19 19 20 20 21 20 19 21 23 20 21 19 20 NA 20 20 21 21 20 20
[484] 20 20 19 24 28 23 23 33 21 22 20 23 26 21 36 22 21 23 19 21 34 25 19
[507] 18 22 22 20 20 20 19 20

The first two functions essentially serve to ask the same question: either “what class of data is this variable” or “is the variable an integer class.” If class() returns “integer” than is.integer() will return a TRUE; If class() returns anything else, than is.integer() will return a FALSE. Either will answer our question if our variable is correctly assigned as an integer or not. If not, we can run as.integer() to tell R to treat our variable as an integer. Run one of the two first codes and see how R has classed age. If necessary, then run the last line of code.

3.1: Summary Statistics

9. What is a summary statistic? How does it differ from raw data?

A summary statistic gives a summary of each variable, whereas raw data shows all of the data for each individual in the variables.

We can use R to calculate summary statistics for single quantitative variables, such as the mean and standard deviation. Modify the code below so it produces the mean and standard deviation for both the number of hours per week spent exercising and spent watching tv.

mean(survey$exercise, na.rm= TRUE) # calculates the mean for the variable exercise

[1] 5.107794

sd(survey$exercise, na.rm= TRUE) # calculates the standard deviation for exercise

[1] 4.107598

mean(survey$tv, na.rm= TRUE) # calculate the mean for tv

[1] 6.641393

sd(survey$tv, na.rm= TRUE) # calculate the standard deviation for tv

[1] 7.001062

We often also want to know the number of individuals in our sample (our sample size). If individuals are rows in a column variable, then how long that column is would be the number of individuals. Thus, the code for sample size is length() and we supply the specific variable we want to know the sample size for. Note: if you supply just the dataframe to length(), it will give you the length of the dataframe, or the number of variables in the dataframe, not the sample size of each one; so be careful when you apply this function.

Modify the code below to find the sample size for both our exercise and tv variables

length(survey$exercise)  # find the sample size of the variable exercise

[1] 514

length(survey$tv) # find the sample size of the variable tv

[1] 514

We ran code before to produce the mean and standard deviation alone – it just prints to the screen. I often will want to save the output to an object name so I can use it within the R Markdown document. When you save output to an object name it will not print out the value unless you tell R to provide the value.

mean(survey$exercise, na.rm = TRUE) # prints mean but does not save it

[1] 5.107794

mEx<-mean(survey$exercise, na.rm = TRUE) # calculates mean and saves it as mEx
mEx # prints to screen the value saved to mEx

[1] 5.107794

sdEx<-sd(survey$exercise, na.rm = TRUE) # calculates standard deviations and saves it as sdEx
sdEx # prints to screen the value saved to sdEx

[1] 4.107598

10. Use the named objects for mean and standard deviation and write out a description of each value with the output inline of the text.

Example: The mean amount of time per week spent exercising by all prior STAT 250 students is 5.1077935 hrs.

The mean amount of time per week spent exercising by all prior STAT 250 students is 5.1077935. The average spread about the mean is sdEx.

Below, insert a code chunk and calculate the same set of descriptive statistics about the variable tv. You will need to save the values as objects to use it later inline with the text.

mean(survey$tv, na.rm = TRUE) # prints mean but does not save it

[1] 6.641393

mTv<-mean(survey$tv, na.rm = TRUE) # calculates mean and saves it as mTv
mTv # prints to screen the value saved to mTv

[1] 6.641393

sdTv<-sd(survey$tv, na.rm = TRUE) # calculates standard deviations and saves it as sdTv
sdTv # prints to screen the value saved to sdTv

[1] 7.001062

11. Use the named objects for mean and standard deviation and write out a description of each value with the output inline of the text.

The mean amount of time per week spent watching TV by all prior STAT 250 students is 6.6413934. The average spread about the mean is sdTV.

Using a stored object name is useful if you are going to use the value often (to cut down on typing), but if you only need to use the value small number of times, you can also write the code directly in line. For example, the maximum number of hours of tv watched in a week in our dataset of prior STAT 250 students is NA.

12. Use in-line code directly to describe the maximum hours of exercise a prior STAT 250 student had in a week.

The maximum amount of hours spent watching TV by all prior STAT 250 students is 60.

We often write descriptive statistics using mathematical notation; we can do this in R Markdown too. Mathematical equations are surrounded by $. We can call greek letters by their full names preceded by a backslash (e.g. \mu, \sigma), use special notation as well by calling the name preceded by a backslash and then applying it to a numeric value (e.g. \bar{x}), or use roman characters directly (e.g. s). An equation would then look like, for example: \(\bar{x} = 3.9\).

13. Which set of symbols would we use for the values we calculated on exercise in its last code chunk? Why?

The mean of exercise would be calculated using \bar{x} as it is the sample mean. The standard deviation would be calculated using s.

14. Write the values for exercise referenced above, using mathematical notation.

\(\bar{x} = 5.107794\) \(s = 4.107598\)

3.2: Descriptive Graphics

There are two main types of descriptive graphics we can use for numeric data: histograms and boxplots. We’ll learn how to make both in R and customize them using arguments.

hist(survey$exercise, # creates a histogram using the exercise data from the survey
     xlab = "Number of hours spent exercising", # labels the x-axis
     ylab = "Number of students exercising for x number of hours") # labels the y-axis

min(survey$exercise, na.rm = TRUE) # calculates the minimum value in exercise

[1] 0

max(survey$exercise, na.rm = TRUE) # calculates the maximum value in exercise

[1] 24

hist(survey$exercise, xlim=c(0,24)) # creates a histogram using the min and max hours of exercise 

hist(survey$tv, # creates a histogram using the tv data from the survey
     xlab = "Number of hours spent watching TV", # labels the x-axis
     ylab = "Number of students watching x number of hours of TV") # labels the y-axis

3.2.b: Boxplots

Now let’s create a boxplot to look at the distribution of exercise. There are two ways to print the boxplot. Add labels to the x and y axis appropriate for each style.

boxplot(survey$exercise)

boxplot(survey$exercise, horizontal = TRUE)

18. What did adding horizontal = TRUE to the code do to the boxplot?

It rotated the box plot from vertical to horizontal.

Insert a code chunk to create a boxplot for tv.

boxplot(survey$tv, horizontal = TRUE)

19. Describe the distributions based on the boxplots for tv and exercise.

The distribution for exercise is right-skewed and the distribution for tv is also right-skewed

We can calculate the 5-number summary for both variables by supplying the variable name to the function summary(). Insert a code chunk below to do so.

summary(survey$exercise)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
  0.000   2.000   4.000   5.108   7.000  24.000      20 

summary(survey$tv)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
  0.000   2.000   5.000   6.641  10.000  60.000      26