Manipulating Data Structure

Author

Giulia Gaggero, CESLab

Subsetting Data Structures with R functions

When working with data structures you might be interested in searching and extracting smaller portions of data.

Some examples:

Finding the first 10 elements in a vector of names (e.g. the ordered list of best universities in the World).
ecstract only females participants from your dataset, only the target group, etc.

To subset datastructures in R, you have to indicate the subset within square brackets [].

Let’s start with vectors:

vector1<-c(1,2,7,4) 

vector1[3] ### Extract the third element in the vector

[1] 7

vector1[c(2,3)] ### Extract the second and third element in the vector

[1] 2 7

Pay attention to the fact that the index and the value are two different things! If I write vector[3], I am looking for the third element in a vector, NOT for the elements with a value of “3”. It is only by chace that in vector1 the fourth element is also valued 4.

vector1[-3] ##extract all elements except the third one

[1] 1 2 4

vector1[-c(2,3)] ##extract all elements except the second and the third ones

[1] 1 4

vector1[c(FALSE,FALSE,TRUE,FALSE)] ### extract only the third element

[1] 7

Subsetting from matrices:

Subsetting from matrices is still based on square brackets []

matrix [indexrow,indexcol]

### Recreate a matrix with names for both rows and columns 
matrix2<-matrix(c(1:6), nrow = 3, ncol= 2)

rownames(matrix2)<- c("Subject1","Subject2","Subject3")
colnames(matrix2)<- c("Q1","Q2")

matrix2

         Q1 Q2
Subject1  1  4
Subject2  2  5
Subject3  3  6

###
matrix2[1,1] ## Extracts the element in first row and first column

[1] 1

matrix2[1,2] ### Extracts the element in first row and second column

[1] 4

matrix2[1,] ## Extracts the first row

Q1 Q2 
 1  4

matrix2[,1] ## Extracts the first column

Subject1 Subject2 Subject3 
       1        2        3

### Extraction with column/row names

matrix2[,"Q1"]

Subject1 Subject2 Subject3 
       1        2        3

matrix2["Subject3",]

Q1 Q2 
 3  6

Subsetting from DataFrames

### Recreate the data Frame ###

SubjID<-c("Subj1","Subj2","Subj3","Subj4","Subj5")
Age<-c(67,45,23,65,54)
Nationality<-factor(c("Italian","German","French","Italian", "German"))
EmpathyScore<-c(83,56,65,74,80)

myDf1<-data.frame(SubjID,Age,Nationality,EmpathyScore) ##combine the vectors
myDf1

  SubjID Age Nationality EmpathyScore
1  Subj1  67     Italian           83
2  Subj2  45      German           56
3  Subj3  23      French           65
4  Subj4  65     Italian           74
5  Subj5  54      German           80

str(myDf1) ##detect the structure of a data frame

'data.frame':   5 obs. of  4 variables:
 $ SubjID      : chr  "Subj1" "Subj2" "Subj3" "Subj4" ...
 $ Age         : num  67 45 23 65 54
 $ Nationality : Factor w/ 3 levels "French","German",..: 3 2 1 3 2
 $ EmpathyScore: num  83 56 65 74 80

dim(myDf1) ## dimensions of data frame

[1] 5 4

## If I want to change the names of my columns, I use the function colnames()
colnames(myDf1)<-c("ParticipantID","Age","Nation","EmpathyS")

I can extract elements from columns/rows exactly as in the case of matrices (remember that data.frames are matrices that accept columns of different types)

myDf1[3,]

  ParticipantID Age Nation EmpathyS
3         Subj3  23 French       65

myDf1[,1]

[1] "Subj1" "Subj2" "Subj3" "Subj4" "Subj5"

However, you can also extract/select elements in columns using the $ symbol

myDf1$ParticipantID ## extract the column ParticipantID

[1] "Subj1" "Subj2" "Subj3" "Subj4" "Subj5"

myDf1$Age ## extract the column Age

[1] 67 45 23 65 54

myDf1$Age[1] ## extract first element from the column Age

[1] 67

It is finally possible to filter a data.frame by a specific variable level (e.g. select only data from Italian Participants, etc.)

myDf1[myDf1$myDf1[myDf1$Nation == "Italian"] ,]

[1] ParticipantID Age           Nation        EmpathyS     
<0 righe> (o 0-length row.names)

myDf1[myDf1$EmpathyS > 65,]

  ParticipantID Age  Nation EmpathyS
1         Subj1  67 Italian       83
4         Subj4  65 Italian       74
5         Subj5  54  German       80

The same operation is done using the function subset()

subset(myDf1,Nation == "Italian")

  ParticipantID Age  Nation EmpathyS
1         Subj1  67 Italian       83
4         Subj4  65 Italian       74

subset(myDf1,EmpathyS > 65)

  ParticipantID Age  Nation EmpathyS
1         Subj1  67 Italian       83
4         Subj4  65 Italian       74
5         Subj5  54  German       80

Subsetting can also be done using the SQL syntaxt from the sqldf package

#install.packages("sqldf")
library(sqldf)

Warning: il pacchetto 'sqldf' è stato creato con R versione 4.3.3

Caricamento del pacchetto richiesto: gsubfn

Warning: il pacchetto 'gsubfn' è stato creato con R versione 4.3.3

Caricamento del pacchetto richiesto: proto

Warning: il pacchetto 'proto' è stato creato con R versione 4.3.3

Caricamento del pacchetto richiesto: RSQLite

Warning: il pacchetto 'RSQLite' è stato creato con R versione 4.3.3

sqldf("SELECT Age FROM myDf1") ### Use in capital letter "SELECT varname FROM dataframe.name

sqldf("SELECT ParticipantID FROM myDf1 WHERE Age > 65")  ## SELECT participant ID with Age > 65

  ParticipantID
1         Subj1

Subsetting from Lists

For extracting elements from lists, need to use double brackets [[ ]]

### recreate a list
vector2<-c("giacomo","giovanni","karolyn") ## character vector  
matrix1<-matrix(c(1:6), nrow = 2, ncol= 3) ## numeric matrix  
list1 <-list(vector2,matrix1,myDf1)    
list1

[[1]]
[1] "giacomo"  "giovanni" "karolyn" 

[[2]]
     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6

[[3]]
  ParticipantID Age  Nation EmpathyS
1         Subj1  67 Italian       83
2         Subj2  45  German       56
3         Subj3  23  French       65
4         Subj4  65 Italian       74
5         Subj5  54  German       80

Extract elements from the list:

list1[[1]]  ## extract the first element (the vector)

[1] "giacomo"  "giovanni" "karolyn"

list1[1]

[[1]]
[1] "giacomo"  "giovanni" "karolyn"

list1[[2]] ## extract the second element (the matrix)

     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6

### Let's see what happens if I give names to each element in the list

names(list1)<-c("vettore","matrice","dataFrame")  

list1[[2]]   ### [[]]extract the values of the selected object

     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6

list1[2] ### []extract the selected object ("matrice")

$matrice
     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6

list1[[2]][2,3]

[1] 6

Exercise on Filtering and Subsetting

vector1

[1] 1 2 7 4

matrix2

         Q1 Q2
Subject1  1  4
Subject2  2  5
Subject3  3  6

Extract:

the third element from vector1
the number 4 from vector 1
the second and third element from vector 1
the number 6 from matrix2
the second row from matrix2 (try to use different at least two methods)

the second column from matrix2 (try to use different at least two methods)

myDf1

  ParticipantID Age  Nation EmpathyS
1         Subj1  67 Italian       83
2         Subj2  45  German       56
3         Subj3  23  French       65
4         Subj4  65 Italian       74
5         Subj5  54  German       80

Extract the column EmpathyS from the data frame by using the symbol $ and another method
select the participants with less than 65 years old using “subset” function
select the participants with an age inferior to 65 years old AND superior to 23 years old

list1

$vettore
[1] "giacomo"  "giovanni" "karolyn" 

$matrice
     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6

$dataFrame
  ParticipantID Age  Nation EmpathyS
1         Subj1  67 Italian       83
2         Subj2  45  German       56
3         Subj3  23  French       65
4         Subj4  65 Italian       74
5         Subj5  54  German       80

Extract the second element (“giovanni”) from the vector stored within list1.

Extract the first element of the matrix stored within list1

The function “which”

From a vector, you can always filter data using “which” and creating a condition (equal, non equal, major, minor). Which is based on a boolean filter (FALSE/TRUE).

vector1

[1] 1 2 7 4

which(vector1 > 3) ### Which gives you back the position (index) of elements that are higher than 3

[1] 3 4

which(vector1 < 3) ## Ask for the position of elements minor than 3

[1] 1 2

which(vector1 != 3) ## Ask for the position of elements non equal to 3

[1] 1 2 3 4

which(vector1 == 3) ## ## Ask for the position of elements equal to 3

integer(0)

vector1[which(vector1 > 3)] ###in this way, you obtain the values...

[1] 7 4

vector1[which(vector1 < 3)] ###in this way, you obtain the values...

[1] 1 2

The which function can be used also with string vectors

myDf1$Nation ## the column nation from myDf1 is itself a string vector

[1] Italian German  French  Italian German 
Levels: French German Italian

which(myDf1$Nation == "Italian") ## extract the indexes

[1] 1 4

myDf1[which(myDf1$Nation == "Italian"),] ## Extract only the rows corresponding to the data from Participant whose Nationality is Italian

  ParticipantID Age  Nation EmpathyS
1         Subj1  67 Italian       83
4         Subj4  65 Italian       74

Exercise:

Extract only data from participants whose age is higher than 50 using which
Extract only data from these participants c(“Subj1”,“Subj2”,“Sub3”) using which

Order, Compare and combine data structures

Order vectors

The sort() function sort vectors in ascending (or descending) order.

vector1

[1] 1 2 7 4

sort(vector1)  ## The default is the ascending order

[1] 1 2 4 7

##implicitly it is written sort(vector1,decreasing = FALSE) ##

sort(vector1, decreasing = TRUE) ## Descending order

[1] 7 4 2 1

The Order() function returns the permutation that is necessary to rearrange the vector

vector1

[1] 1 2 7 4

order(vector1)

[1] 1 2 4 3

The rev() function provides the reversed version of the vector

vector1

[1] 1 2 7 4

rev(vector1)

[1] 4 7 2 1

The rev() function returns the “reversed” version of the original vector

vector1REV<-rev(vector1)

vector1

[1] 1 2 7 4

vector1REV

[1] 4 7 2 1

Compare Vectors

The Setequal() function compares two vectors (minimum two arguments) to check whether their elements are all equal or not (it returns TRUE/FALSE)

setequal(vector1, vector1REV) ### it doesn't matter the order of values

[1] TRUE

setequal(vector1,c(2,2,3,2))

[1] FALSE

The setdiff() function takes two argument Vector1, Vector2 and returns the element in 1 than are not in 2. It can be used to compare two sets.

setdiff(vector1,vector1REV)  ### NO element are different

numeric(0)

vector1

[1] 1 2 7 4

vector3<-c(0,2,7) 

## Pay attention to the fact that the order of argument matters for setdiff! ##

setdiff(vector1,vector3)  ### [1,4 are in vector1 but not in vector3]

[1] 1 4

setdiff(vector3,vector1)  ### [0 is in vector3 but not in vector1]

[1] 0

Combine vectors and matrices

vector1

[1] 1 2 7 4

vector2

[1] "giacomo"  "giovanni" "karolyn"

vector3

[1] 0 2 7

vector4<-c(TRUE, FALSE, TRUE)
vector4

[1]  TRUE FALSE  TRUE

The Union() function combines the elements in the two vectors

union(vector1,vector3)

[1] 1 2 7 4 0

## elements common to the two vectors are not repeated in the union(vector1,vector2) 

### an implicit coercion has been applied  

union(vector1,vector4) ## implicit coercion has been applied,...see how

[1] 1 2 7 4 0

vector1_4<-c(vector1,vector4) 

vector1_4

[1] 1 2 7 4 1 0 1

The Intersect() function provides only the elements common to the two vectors

intersect(vector1, vector3)

[1] 2 7

intersect(vector1, vector2) ## No common Element intersect(vector1, vector4) ### here implicit coercion has been applied

character(0)

Cbind() & Rbind() functions are used to combine columns/rows in matrices or data frames.

# Recall matrices # 

matrix1

     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6

matrix2

         Q1 Q2
Subject1  1  4
Subject2  2  5
Subject3  3  6

matrix3<-matrix(sample(c(1:100), 6), nrow = 2) ## create a new 2*3 matrix with 6 random number

matrix3

     [,1] [,2] [,3]
[1,]   75   49   23
[2,]   12   32   90

rbind(matrix1, matrix3)  ### Rbind concatenates rows

     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6
[3,]   75   49   23
[4,]   12   32   90

cbind(matrix1,matrix3)   ### Cbind concatenates columns

     [,1] [,2] [,3] [,4] [,5] [,6]
[1,]    1    3    5   75   49   23
[2,]    2    4    6   12   32   90

Pay attention that the two matrices should match for columns (if you us rbind) or rows (if you use cbind)

## cbind(matrix1, matrix2) It returns an errors because the number of rowns do not match

Exercises on functions to order, compare and combine data structures

vectorA<-c("brixen","bozen","trento","salamanca","madrid","porto")
vectorB<-c("brixen","bozen","cadige","granada","salamanca","paris")

Check:

whether the two vectors are equal (have the same elements)
which elements are only in A but not in B
which elements are only in B but not in A
the elements that are common to both vectors
create a vector with the elements from both vectors (but repeated only once)
What happens if I write sort(vectorA)? Explain the output
What happens if I write order(vectorA)? Explain the output
transform both vectorA and vectorB in matrices of size 2*3. Then concatenate them by columns or by rows.