Session 8

** Please click all the tabs (in sequence) to get the entire set of information in these pages. **

** To download code, see the instructions in Session 2: https://rpubs.com/hkb/DAX-Session2 **

Objectives

• Exercise: Creating / Importing a Data Set

• Anatomy of a data set

  • Rows, columns, cells
  • Variables, values
  • Long form, wide form

• Getting Familiar with dplyr package

  • The %>% notation
  • select
  • filter
  • mutate
  • rename
  • group_by
  • summarize

Creating / Importing a Data Set

some.vector = c(1,2,4)
some.vector

[1] 1 2 4

We’ll use the data.frame function which is of the form

data.frame(col1name = col1values, col2name = col2values, … )

# build an example data frame
df.eg <- data.frame(name = c("Alex", "Bob", "Charlie", "David", "Emma", "Farah", "Ganesh"), id = letters[1:7], grade = 7+sample(2,7, replace=TRUE), PE.score = sample(50,7), history.score = sample(29, 7), english.score = 50 + sample(2,7, replace=TRUE)) 

print(df.eg)

Anatomy of a data set

• Rows: Alex, Bob, Charlie, David, Emma (people)
• Columns: name, id, grade, PE.score, english.score = attributes about people
• Cells: values of those attributes

One way to think of that is that Columns are Variables, while Cells are Values. So, “name” is a Variable that takes on different values (Alex, Bob, etc.). Similarly, grade is a Variable, with values 4, 5, etc.

Now, the test scores are an interesting animal. They could be thought of as Variables (English score, etc.) and the values being the numbers.

But in some cases one might think of “Subject” as the variable, and “English” and “PE” being values. In this view, we would think of Subject as Variable, and Subject.Name and Subject.Score as additional variables, with appropriate values under each.

The existing view that we have is called a “wide form” view of the data - it has a column for each variable.

The long form view of the data is shown below, and is obtained using the “gather” function. The long form view is very helpful if you want to do some analytics or visualization against a series of variables (here, PE.score and english.score).

data_long <- gather(df.eg, Subject, Score, PE.score:english.score, factor_key=TRUE)
data_long

Digging Into dplyr

Here’s a great site: https://dplyr.tidyverse.org/

• select() picks variables based on their names
• filter() picks cases based on their values
• arrange() changes the ordering of the rows
• rename() renames existing variables
• mutate() adds new variables (usually, functions of existing variables)
• group_by() implicitly breaks the data set into groups
• summarise() reduces multiple values down to a single summary

select(data_long, name, Score)

data_long %>% select(name) %>% distinct()

filter(data_long, Score > 25)

data_long %>% filter(Score > 25)

Now suppose we want to report all grade 9 students, plus students with scores > 25. So, the conditions are Score > 25 OR grade = 9.

data_long %>% filter(Score > 25 | grade == 9)

Now suppose we want to report only grade 9 students with scores > 25. So, the conditions are Score > 25 AND grade = 9.

ARRANGE

data_long %>% arrange(desc(Score), grade)

data_long %>% rename(student.id = id)

to add a new column, we’ll use mutate.

df.eg <- df.eg %>% mutate(total.score = PE.score + history.score + english.score)
df.eg

data_long %>% group_by(name) %>% summarise(sum(Score))

data_long %>% group_by(name) %>% mutate(total.score = sum(Score))

NA