Visualizing Social Interaction with Music Notation
James P. Curley
3rd February 2015
email: jc3181 AT columbia DOT edu
Background
In his 2006 paper Music notation: a new method for visualizing social interaction in animals and humans Ivan Chase (a professor at Stony Brook) suggested a method for visualizing social interactions occurring between individuals living in groups over time. His idea was to use the y-axis as a de facto ‘stave’ with horizontal lines representing different individuals. The x-axis would represent time. Vertical arrows would go between different horizontal lines (individuals) to represent a behavioral interaction directed from one individual to the next.
I’ve always been fond of this approach as I find it a very useful to help look for patterns in activity by individuals or within relationships. For instance, it can help show you where ‘bursts’ of social interactions occur or perhaps whether certain individuals interact only after other individuals have finished interacting.
I had a quick go at creating one of these types of graphs in ggplot2 - the result isn’t too bad, I think.
The Data
For this example, I’m using a sample dataset of 11 individual animals who all live in the same social group. The data records every antagonistic interaction between any two individuals. For example, if animal ‘A’ chased animal ‘B’. There are 999 such observations, and I’ve just assumed that the unit of time increments by 1. We could use actual time to plot the data - but for this worked example, I’m just using a continuous time from 1 to 999.
The dataset we are using is called ‘newdf’. Winners and losers are given letter id codes (“A” to “K”). The letters roughly correspond to the individual rank of the animal. This ranking is calculated based on the I&SI method described in a 1998 paper by De Vries available here.
head(newdf, 10)## Time Winner Loser
## 1 1 A H
## 2 2 A I
## 3 3 A I
## 4 4 A D
## 5 5 A H
## 6 6 A E
## 7 7 D J
## 8 8 A D
## 9 9 A E
## 10 10 A C
Here is a summary table of all interactions. Rows represent winners and columns represent losers. So, for instance, “A” beat “C” 113 times but “C” never beat “A”. Whereas “B” beat “C” 13 times and “C” beat “B” 4 times.
table(newdf$Winner, newdf$Loser)##
## A B C D E F G H I J K
## A 0 18 113 57 79 38 37 58 11 94 24
## B 0 0 13 6 13 6 10 2 3 9 13
## C 0 4 0 30 23 25 34 60 7 11 20
## D 0 0 0 0 6 6 5 11 0 8 5
## E 0 0 1 1 0 4 4 3 1 8 1
## F 2 1 2 4 2 0 7 6 2 6 4
## G 1 1 0 1 5 4 0 15 1 25 1
## H 1 0 0 0 0 0 0 0 1 1 3
## I 0 1 0 1 1 0 0 0 0 0 0
## J 2 0 1 0 1 0 1 1 0 0 2
## K 0 0 1 0 1 4 0 3 0 1 0
Whilst this table gives useful information, it doesn’t give us any temporal information. That’s the purpose of the music notation plot.
Music Notation Plotting
The first thing to do is convert the letter ids of the ‘Winner’ and ‘Loser’ variables into numeric (for plotting the y-axis):
df1 <- data.frame(id = 1:11, ids = LETTERS[1:11])
newdf$winner <- df1$id[match(newdf$Winner, df1$ids)]
newdf$loser <- df1$id[match(newdf$Loser, df1$ids)]
head(newdf,10)## Time Winner Loser winner loser
## 1 1 A H 1 8
## 2 2 A I 1 9
## 3 3 A I 1 9
## 4 4 A D 1 4
## 5 5 A H 1 8
## 6 6 A E 1 5
## 7 7 D J 4 10
## 8 8 A D 1 4
## 9 9 A E 1 5
## 10 10 A C 1 3
Secondly, from experience, ggplot graphs can look too clustered with more than 250 observations on the x-axis, so i’m going to split the graphs into four chunks of 250 observations:
#split into 4 lots of 250
newdf.a <- newdf[1:250,]
newdf.b <- newdf[251:500,]
newdf.c <- newdf[501:750,]
newdf.d <- newdf[751:999,]
It can be a struggle to pick out a color palette that provides 11 distinctive colors. I decided to use RColorBrewer ‘s ’Set1’ which has 9 colors and then to add two extra light colors.
###set colors
library(RColorBrewer)
mycolors <- brewer.pal(9, "Set1")
mycolors <- c(mycolors, "#FFE4C4", "#E0EEEE")
Next, we actually plot the data. To do this, I’m using geom_segment to plot the lines between individuals and arrow from the grid library to confer the arrow points. I don’t think these look fantastic - but they work easily with ggplot2 and are ok enough. Finally, I reverse the y-axis so the highest ranked animal (number 1 / “A”) is at the top of the chart and their interactions go downwards. This is more logical when thinking about social interactions within a dominance hierarchy.
Finally, I used the gridExtra library to stack graphs on top of each other:
library(ggplot2)
library(grid)
g1 <-
ggplot() +
geom_segment(aes(x = Time,xend = Time, y = winner, yend = loser, color=factor(winner)),
arrow=arrow(length = unit(.3, "cm")), lwd=.75, data=newdf.a) +
geom_point(aes(x = Time, y = winner, color = factor(winner)), size=3, data=newdf.a) +
scale_y_reverse(breaks=1:11)+
scale_color_manual(values = mycolors) +
theme_bw()+
theme(legend.position = "none")+
ylab("Animal rank")
g2 <-
ggplot() +
geom_segment(aes(x = Time,xend = Time, y = winner, yend = loser, color=factor(winner)),
arrow=arrow(length = unit(.3, "cm")), lwd=.75, data=newdf.b) +
geom_point(aes(x = Time, y = winner, color = factor(winner)), size=3, data=newdf.b) +
scale_y_reverse(breaks=1:11)+
scale_color_manual(values = mycolors) +
theme_bw()+
theme(legend.position = "none")+
ylab("Animal rank")
g3 <-
ggplot() +
geom_segment(aes(x = Time,xend = Time, y = winner, yend = loser, color=factor(winner)),
arrow=arrow(length = unit(.3, "cm")), lwd=.75, data=newdf.c) +
geom_point(aes(x = Time, y = winner, color = factor(winner)), size=3, data=newdf.c) +
scale_y_reverse(breaks=1:11)+
scale_color_manual(values = mycolors) +
theme_bw()+
theme(legend.position = "none")+
ylab("Animal rank")
g4 <-
ggplot() +
geom_segment(aes(x = Time,xend = Time, y = winner, yend = loser, color=factor(winner)),
arrow=arrow(length = unit(.3, "cm")), lwd=.75, data=newdf.d) +
geom_point(aes(x = Time, y = winner, color = factor(winner)), size=3, data=newdf.d) +
scale_y_reverse(breaks=1:11) +
scale_color_manual(values = mycolors) +
theme_bw() +
theme(legend.position = "none") +
ylab("Animal rank")
library(gridExtra)
grid.arrange(g1, g2, g3, g4, nrow=4)
I think this is an ok attempt at visualizing a lot of temporal information. There are obviously improvements that I could make. For instance, perhaps coloring the horizontal lines in the same color of each animal. Also making this visualization interactive so we can pull out each individual by itself, or perhaps to look at interactions that go in the upwards directions only.
If you’ve any questions or comments, or want to talk about how to analyze or visualize these type of data please get in touch.