Complete Portofolio for EDUC 7847: Social and Statistical Network Analysis
Emma Ross
December 20, 2024
packages = c("splitstackshape", "stringr", "igraph", "spdep", "magrittr", "htmlwidgets", "htmltools", "plotly","networkD3","spatialreg","ggplot2","classInt","RColorBrewer")
## Now load or install&load all
package.check <- lapply(
packages,
FUN = function(x) {
if (!require(x, character.only = TRUE)) {
install.packages(x, dependencies = TRUE)
library(x, character.only = TRUE)
}
}
)Module 1 purpose & interpretation:
In this first module we explore the Scopus database, choose a topic of interest, and perform the necessary transformations to build an initial graph of the connections of co-authors (ignoring single authors). The purpose of the analysis is to better understand the interactions academics as it relates to publishing in a specific field of interest, which, for me, is in border studies. It’s important to consider what key word limits, time range, type of publication, country of origin, document type, etc. to employ as you create this data subset in Scopus that will best serve the analysis you are doing. For me I choose the widest range of time and include as many relevant key words to make sure I have the broadest sampling possible within the border studies field. For other people, maybe they specifically want to look at how borders are being talked about in the U.S. only, then they might pose a more limited search. The purpose of the analysis here, which is mostly data cleaning, is to prepare the an adjacency list of co-authors. This is done by formatting the authors names to avoid special symbols or prefixes / suffixes, then splitting the authors column list of names to get each individual author in their own column, then looping through those columns to get down to a two column list of each co-authorship pair. This adjacency list is then used to make the graph visual.
The interpretation of making the graph is limited, which is fine considering this is just a first step in visualizing the network. It is not clean enough to understand anything from it at this point, but that will be developed in later modules. The analysis of the dimensions of the adjacency list and connections in the graph allow us to see how many unique articles and authors there are, and the number of connections (co-author pairings) we have. In the analysis we also rank the co-authorship pairings to see who are the most prolific co-author pairs, meaning the co-author pairs that have published together the most. We inspect some of the abstracts of the papers that these pairings published to see what these articles are about and begin to understand what the most important topics are in the border studies field.
1 Module: Data Formats and Transformations I (one-mode)
For this deliverable you will need one publication dataset of a topic and time frame of your interest. You can create this file from Scopus as discussed in class. Please explain the dataset used indicating the time frame, the topic and other inclusion criteria like type of publication, for example. For this deliverable make sure to describe:
1.1 The topic you selected (with a description of why you are interested in such a topic)
I have decided to search scopus based on “border” as a key word because I have become interested in border studies and how border policies effect migration and human rights protections, for this reason as well I’ve applied the key word limits of “borders”, “migration”, “mobility”, and “refugee”. I’ve been interested in migration issues for a long time but this semester I’m taking a political science course that focuses on Borders in International Relations and I’ve learned how important it is to account for border policy while looking at migration. I will use the social sciences as my subject area because most papers and authors I’d be interested in work within political science, sociology, etc. I’m open to Spanish or English language sources and feel it’s important to include Spanish in this search considering a lot of border policy work is looking at the US-Mexico border. On the other hand, there are a plethora of borders throughout the world that merit study so I am not limiting the country/ territory.
1.2 The time span (illustrate why you selected this time span)
I’m considering the time period 2010-2024 (present) in my search because a lot of migration scholars consider that modern day large migration waves came after Arab Spring in 2010, and I’m going to present day because clearly border issues are still very prevalent and evolving.
1.3 Type of publication (journal article, conference papers, book chapters or all types) mentioning also why you selected this topic.
I’ve chosen to consider Journal articles only since this is the source that would be most helpful to me in continuing to learn about border issues, additionally this is the only type of source we’ve considered in my class.
1.4 Create a co-authorship network wherein all relationships among coauthors are established.
#clean data, prepare author names that match ID, get one-mode matrix g and two-mode matrix g2
source <- read.csv("scopus_final.csv")
#dim(source)
# clean the data, keep only authors where ID is included
source <- source[source$Author.s..ID!="[No author id available]",]
#--> we see that all rows are kept, so all authors had an ID
#dim(source)
#try 2
#Let's take out only the column with author relationships:
authors<-as.data.frame(source[,1]) #take first column, all rows
colnames(authors)<-"Authors" #same column name as original data set
#Since next step is to separate all names, we need to consider how they are separated in papers with multiple co-authors, also need to account for different name suffixes:
#Addressing names and titles
authors$Authors <- gsub(" Jr.,", "",
gsub(" II.,", "",
gsub(" Jr.", "",
gsub(" M.S.", "",
gsub(" M.S.,", "",
gsub(" II.", "", authors[,1]))))))
#authors$Authors <- gsub("\\.,", ";",authors$Authors) #-> not needed because data set already formatted with ; between each name
#Lower case all the text to avoid the same author being recognized two in case of accidental lack of capitalized letters in one instance of their name.
authors$Authors <- tolower(authors$Authors)
a1<-cSplit(authors, splitCols = "Authors", sep = ";", direction = "wide", drop = FALSE) #retain the matrix form version of the adjacency list input
#Here we just drop the original first column, which is where the complete list of coauthors for each paper is held
a1<-a1[,-1]
#read it as a matrix
mat <- as.matrix(a1)
a1<-mat
edgelist1<-matrix(NA, 1, 2)#empty matrix two columns
for (i in 1:(ncol(a1)-1)) {
edgelist11 <- cbind(a1[, i], c(a1[, -c(1:i)]))
edgelist1 <- rbind(edgelist1,edgelist11)
edgelist1<-edgelist1[!is.na(edgelist1[,2]),]
#handles where it's not considered empty but it's missing
edgelist1<-edgelist1[edgelist1[,2]!="",]
}
#dim(edgelist1)
plot(graph.edgelist(edgelist1))
g<- graph.edgelist(edgelist1, directed = FALSE)
#weight exists in igraph (it's a hidden attribute)
E(g)$weight <- 1 #must step
#simplify command "collapses" it -- i.e. it takes the long list of relationships where A - B can be listed multiple times, so that it only has A - B once but the weight is the sum of how many times A - B showed up
g.c <- simplify(g)
#E(g.c)$weight
links<-as.data.frame(cbind(as_edgelist(g.c), E(g.c)$weight))
#links
#dim(links)
links$V3<-as.numeric(links$V3)
links<- links[order(links$V3, decreasing=T),]
#table(links[,1])2 Module: using IDs
# Creating adjacency list from names
#we already have this from above, but we can recall it as follows
a1<-cSplit(authors, splitCols = "Authors", sep = ";", direction = "wide", drop = FALSE) #retain the
a1<-a1[,-1]
#Adjacency list from authors' IDs, that is, split the authorsID column
df_authors_id <- as.data.frame(source[ , 3])
colnames(df_authors_id) <- "AU"
author_id_split <- cSplit(df_authors_id, splitCols = "AU", sep = ";", direction = "wide", drop = TRUE)
dim(author_id_split) ## [1] 2354 93dim(a1)## [1] 2354 932.1 With your own words, explain what problems we aimed to solve with the creation of an ID adjacency list and a names adjacency list?
Authors’ names may be misspelled or changed over time, but their ID number will not. Looking at IDs instead of names allows for better accuracy.
2.2 During your cleaning procedures, did your adjacency lists for names and ID have the same dimensions?
2.2.1 If so, what is the meaning of these dimensions?
Yes, they did! The dimensions of both are 2354, 93, meaning that there are 2,354 articles included in this data set and there is at least one article with 93 total co-authors on it.
2.2.2 Let us assume that you did not have the same dimensions, what would be the reason for this discrepancy and how would you address this issue?
If the dimensions were different that would mean that some author’s whose names showed up did not show up as IDs, or vice versa. We would address this by looking at the rows (articles) where there are not an equal number of author name / ID columns that are filled in (not NA). We would then need to investigate each row with this issue.
2.3 As part of the feature engineering procedures, we created a database with ID and Name columns.
2.3.1 What is the meaning of the number of rows of this database?
The number of rows is the number of unique individual authors we have within our dataset, which is 3,274.
2.3.2 Does this number match the number of unique actors you have in your first deliverable in the object g (i.e., the graph we created)?
No it does not, there were 7,639 connections there where as we have 3,274 now.
2.3.3 If the number does not match, what do you think is the reason for this mismatch?
It does not match because the graph in module 1 was representing co-authorship pairs, clearly it will be a larger number as each individual author can have multiple coauthor pairings with other individual authors.
2.4 Please explain the role of unlisting in this data cleaning or feature engineering process.
#unlisting the values of the adjacency matrices
df_authorid_authorname_unlisted <- data.frame(id = unlist(author_id_split), names = unlist(a1))
dim(df_authorid_authorname_unlisted)## [1] 218922 2#gets rid of where the id is missing
df_authorid_authorname_unlisted <- df_authorid_authorname_unlisted[!is.na(df_authorid_authorname_unlisted$id),]
dim(df_authorid_authorname_unlisted)## [1] 4000 2#gets rid of duplicates
df_authorid_authorname_unlisted <- df_authorid_authorname_unlisted[!duplicated(df_authorid_authorname_unlisted$id),]
dim(df_authorid_authorname_unlisted)## [1] 3274 2head(df_authorid_authorname_unlisted)## id names
## AU_011 57196458199 milan c.
## AU_012 56684310700 esposito f.
## AU_013 56135176100 castro neira y.
## AU_014 55086379400 lonergan g.
## AU_015 57188966884 baykurt b.
## AU_016 59312942500 sigala m.The unlisting is used to take multiple names or IDs that are stored in separate columns and stack them all on top of each other to get only one column of all author names and one column of all author IDs. It works well for us because the articles are in the same order in each matrix so as we unlist and stack all the names / IDs it acts to pair each name and ID together. For example, if the first row (article) has two authors, then when we unlist that row we will have author ID 1 in row 1, column 1 and author ID 2 in row 2, column 1; while we have author name 1 in row 1 column 2 and author name 2 in row 2 column 2. Thus it shows that author name 1 and author ID 1 are a pair and that author name 2 and author ID 2 are a pair. We then later get rid of NA or missing values, and then we use the remove duplicates so that we get down to a unique list of authors where each one is listed only one time. Notice how the dimensions is reduced with the removal of duplicates (Not with NA because there weren’t any).
3 Module: Data Formats and Transformations I (two-mode)
Create a publication network wherein all relationships link (co-)authors with their respective publication.
# clean the data, keep only authors and articles
source <- source[source$Author.s..ID!="[No author id available]",]
### create a author-author matrix through transformation of a two-mode edgelist#
authors <- as.data.frame(source[,3])
colnames(authors) <- "AU"
source_split <- cSplit(authors, splitCols = "AU", sep = ";", direction = "wide", drop = TRUE)
#dim(source_split)
mat_source_split <- as.matrix(source_split)
combined <- cbind(source$EID, mat_source_split)
mat_combined_eid_source_split <- as.matrix(combined)
edgelist_two_mode <- cbind(mat_combined_eid_source_split[, 1], c(mat_combined_eid_source_split[, -1]))
edgelist_two_mode <- edgelist_two_mode[!is.na(edgelist_two_mode[,2]), ]
head(edgelist_two_mode)## [,1] [,2]
## [1,] "2-s2.0-85205144697" "57196458199"
## [2,] "2-s2.0-85158062010" "56684310700"
## [3,] "2-s2.0-85153379892" "56135176100"
## [4,] "2-s2.0-85188279386" "55086379400"
## [5,] "2-s2.0-85152933522" "57188966884"
## [6,] "2-s2.0-85203064030" "59312942500"#2:1 makes it so article linked to author
g2 <- graph.edgelist(edgelist_two_mode[, 2:1], directed = TRUE)
#g2
#5624 is total number of everything
#4000 is number of connections
#articles, authors
#igraph will assume its a one mode network unless there is a type attribute
#true if name does exist at vertex level in the human column of edge list
V(g2)$type <- V(g2)$name %in% edgelist_two_mode[ , 2]
#V(g2)$name is the column called name at the vertex level
#true means human, false means article
table(V(g2)$type)##
## FALSE TRUE
## 2350 3274#we have 3,274 authors and 2,350 articles
#i<-table(V(g2)$type)[2]
#Transformations to retain actors, move from graph to matrix
mat_g2_incidence <- t(get.incidence(g2))
mat_g2_incidence_to_1 <- mat_g2_incidence%*%t(mat_g2_incidence)
summary(diag(mat_g2_incidence_to_1))## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 1.000 1.000 1.000 1.222 1.000 11.000diag(mat_g2_incidence_to_1)<-0 #set like this if looking at peer effects or spill overs, like no one is influence by theirselves
#also need to do it for visualization purposes
g <- graph.adjacency(mat_g2_incidence_to_1, mode = "undirected")
plot(g)
After this, describe the network by answering: ## How many authors are represented? 3,274 authors ## How many publications are represented in this network? 2,350 articles
table(V(g2)$type)##
## FALSE TRUE
## 2350 32743.1 And how many connections were established?
dim(edgelist_two_mode)## [1] 4000 2There are 4000 connections established.
3.4 How are points 4 and 5 similar or different? In what ways?
These are different questions because of the object of interest, in question 4 we want to know the individual authors that have the most connections with the papers, while in question 5 we want to know the individual paper that has the most connections to authors (ie was written by the most people). The directionality of the question is important.
3.5 If the number of human actors is different in this deliverable and deliverable 1, what is the cause?
It is the same.
Module 4 purpose & interpretation:
Now with a clear understanding of the dimensions of our dataset, number of unique authors and publications, most proliferate authors and publications with the highest number of authors, we move to improving on the visual representation of these networks. The purpose here is to allow ourselves and our audience to detect outliers and patterns in the publishing dynamics of the border studies field. To make the graph visually appealing and interpretable we need to use HTML code to format the colors and spacing of the nodes and connections in the network. We also add labels to the nodes. One important consideration in this module is to decide if we want to account for solo authors or not. This changes how we process the data to make the adjacency list. We make the choice to include solo authors here by determining if the number of solo authors we’d lose by not including them is significant enough to warrant inclusion, which in our case it is.
The interpretation here is in the visual understanding of the network, which gives us an idea of how much authors in this field tend to work together, and lets us see when there are more connections than you would expect in comparison to others. By the color of the nodes we can see how many publications are connected to each author, or how many authors are connected to each publication (in the two mode visual). By having both one mode and two mode visuals we can compare and see how they differ and the implications that has for understanding the publishing dynamics of the border studies field.
4 Module: Interactive Network Visualizations
Replicate the interactive network visualizations (one mode and two mode) but relying on your publication data. Decide whether you want to plot the co-author relationships starting from a one-mode network or whether you want to link authors to papers and then transform this to a one-mode network. Recall that the latter retains solo authors, in case you have those in your dataset. Whatever you decide, explain why you selected to map the co-authors or start with the two-mode form and the get the co-authors via network transformation.
4.1 One Mode replication
To create the interactive network visualization for one mode I first need to choose what approach to take. I end up choosing to start with the two-mode form and then get co-authors later because I have 1,496 papers written by single authors and I want to include all of this data in the visualization.
#re-load the data set and set to 'a'
a<-read.csv("scopus_final.csv")
#count the number of single authors to see what approach to take
single_author_count <- sum(!grepl(";", a$Authors))
cat("There are", single_author_count, " papers in the dataset that are written by a solo author.")## There are 1496 papers in the dataset that are written by a solo author.Now we create the adjacency list and connect it to the papers’ EIDs and then create a weighted graph and the visualization. Note that the solo authors are included and are represented by the nodes completely unconnected. In the visualization you can see each node’s name (Author name), as well as their Publication count, degree measure, and max centrality measure. Note that hovering over the line will tell you the number of co-publications for those co-authors.
#Procedures to decompose an adjacency list into a weighted graph
#Keeping only column with author relationships
authors<-as.data.frame(a[,1])
#rename that column
colnames(authors)<-"AU"
#edit the names to only keep A-Z (getting rid of nonenglish characters)
authors$AU <- str_remove_all(authors$AU, "[^[\\a-zA-Z ]]") #only keeps A-Z, so gets rid of non-English characters
#split the authors column by the ; which is the separator seen in the file
a1<-cSplit(authors, splitCols = "AU", sep = ";", direction = "wide", drop = TRUE)
#read it as a matrix
mat <- as.matrix(a1)
#getting the EID next to all the authors
mat <- cbind(a$EID, mat)
edgelist1<-matrix(NA, 1, 2)#empty matrix two columns
# for (i in 1:(ncol(mat)-1)) {
#this retains solo authors
#making author to paper network (2 node) (not getting connection between author A and author B)
#for (i in 1:1) {
for (i in 1:(ncol(mat)-1)) {
edgelist11 <- cbind(mat[, i], c(mat[, -c(1:i)]))
edgelist1 <- rbind(edgelist1,edgelist11)
edgelist1<-edgelist1[!is.na(edgelist1[,2]),]
edgelist1<-edgelist1[edgelist1[,2]!="",]
}
#dim(edgelist1)
g<- graph_from_data_frame(edgelist1[, 2:1], directed = FALSE)
V(g)$type <- V(g)$name %in% edgelist1[ , 2]
#table(V(g)$type)
i<-table(V(g)$type)[2]
#Transformations to retain actors
mat_g2_incidence <- t(get.incidence(g)) #rectangular matrix
#dim(mat_g2_incidence)
dta <- data.frame(id=rownames(mat_g2_incidence), count=rowSums(mat_g2_incidence))
dta <- dta[order(dta$count, decreasing=T), ] #gives # of publication each person has
mat_g2_incidence_to_1 <- mat_g2_incidence%*%t(mat_g2_incidence)
diag(mat_g2_incidence_to_1)<-0
g<-graph.adjacency(mat_g2_incidence_to_1, mode="undirected")
E(g)$weight <- 1 #must step
g.c <- simplify(g)
# E(g.c)$weight
#centrality measures
cent<-data.frame(ID=V(g.c)$name, ev=evcent(g.c)$vector, deg=degree(g.c)/max(degree(g.c)), bet=betweenness(g.c, normalized=F)/max(betweenness(g.c, normalized=F)), clo=closeness(g.c)/max(closeness(g.c)[!is.na(closeness(g.c))]))
#head(cent)
cent$max_cent <- cent$ev + cent$deg + cent$bet + cent$clo
cent<- cent[order(cent$max_cent, decreasing=T),]
#head(cent)
V(g.c)$label <- V(g.c)$name
V(g.c)$name<-1:length(V(g.c)) #(1:length(V(g)))-1
links<-as.data.frame(cbind(get.edgelist(g.c),as.numeric(E(g.c)$weight)))
links$V1<-as.numeric(as.character(links$V1))
#links
links$V2<-as.numeric(as.character(links$V2))
#str(links)
#links
links<-cbind(links[,1:2]-1, links[,3])
colnames(links)<-c("source","target", "value")
#Get pub count matching names
#most Prolific authors by name
V(g.c)$pub_count <- dta$count[match(V(g.c)$label, dta$id)]
V(g.c)$label <- paste("Name: ", V(g.c)$label, ", Pub. count = ", V(g.c)$pub_count, ", Degree = ", degree(g.c), ", Max Centrality (max = 4) = ", round(cent$max_cent[match(V(g.c)$label, cent$ID)], 3) , sep="")
#head(V(g.c)$label)
nodes <- data.frame(name=V(g.c)$label, group = V(g.c)$pub_count, size=round(betweenness(g.c,directed=F,normalized=T)/max(betweenness(g.c,directed=F,normalized=T))*115, 3)) #so size isn't tiny
###
netviz <- forceNetwork(Links = links, Nodes = nodes,
Source = 'source', Target = 'target',
NodeID = 'name',
Group = 'group', # color nodes by group calculated earlier
charge = -5, # node repulsion
linkDistance = 20,
opacity = 1,
Value = 'value',
Nodesize = 'size', # color nodes by group calculated earlier
zoom = T,
legend= TRUE,
fontSize=24,
colourScale = JS("d3.scaleOrdinal(d3.schemeCategory20)"))
HTMLaddons <-
"function(el, x) {
d3.select('body').style('background-color', 'white')
d3.selectAll('.legend text').style('fill', 'green')
d3.selectAll('.link').append('svg:title')
.text(function(d) { return 'Intensity: ' + d.value ; })
var options = x.options;
var svg = d3.select(el).select('svg')
var node = svg.selectAll('.node');
var link = svg.selectAll('link');
var mouseout = d3.selectAll('.node').on('mouseout');
function nodeSize(d) {
if (options.nodesize) {
return eval(options.radiusCalculation);
} else {
return 6;
}
}
d3.selectAll('.node').on('click', onclick)
function onclick(d) {
if (d3.select(this).on('mouseout') == mouseout) {
d3.select(this).on('mouseout', mouseout_clicked);
} else {
d3.select(this).on('mouseout', mouseout);
}
}
function mouseout_clicked(d) {
node.style('opacity', +options.opacity);
link.style('opacity', +options.opacity);
d3.select(this).select('circle').transition()
.duration(750)
.attr('r', function(d){return nodeSize(d);});
d3.select(this).select('text').transition()
.duration(1250)
.attr('x', 0)
.style('font', options.fontSize + 'px ');
}
}
"
netviz$x$links$linkDistance <- (1/links$value)*125
onRender(netviz, HTMLaddons) 4.2 Two mode replication
In the two mode visualization that follows you can see each node’s name (Author name or paper ID), as well as their Publication count, degree measure, and max centrality measure. Note that hovering over the connecting lines of the nodes shows the title of the publication and the number of citations it has.
#Procedures to decompose an adjacency list into a weighted graph
a<-read.csv("scopus_final.csv")
#Keeping only column with author relationships
authors<-as.data.frame(a[,1])#"Author.s..ID"])
colnames(authors)<-"AU"
#package for first split
# install.packages("splitstackshape")
authors$AU <- str_remove_all(authors$AU, "[^[\\a-zA-Z ]]")
#As can be seen in the file the separator of interest is :
a1<-cSplit(authors, splitCols = "AU", sep = ";", direction = "wide", drop = TRUE) #retain the matrix form version of the adjacency list input
#fix(a1)
#class(a1)
#In case package cannot be installed uncomment and keep working using this version, alternatively you can decompose the cells into columns using excel
#read it as a matrix
mat <- as.matrix(a1)
# mat
dim(mat)# the resulting column dimension is the number of times you will have to repeat the following procedure minus 1## [1] 2354 93mat <- cbind(a$EID, mat)
edgelist1<-matrix(NA, 1, 2)#empty matrix two columns
# for (i in 1:(ncol(mat)-1)) {
for (i in 1:1) {
edgelist11 <- cbind(mat[, i], c(mat[, -c(1:i)]))
edgelist1 <- rbind(edgelist1,edgelist11)
edgelist1<-edgelist1[!is.na(edgelist1[,2]),]
edgelist1<-edgelist1[edgelist1[,2]!="",]
}
#dim(edgelist1)
# install.packages("igraph")
g<- graph.data.frame(edgelist1[, 2:1], directed = FALSE)
V(g)$type <- V(g)$name %in% edgelist1[ , 2]
#table(V(g)$type)
i<-table(V(g)$type)[2]
V(g)$label<-V(g)$name
#Transformations to count number of publications
mat_g2_incidence <- t(get.incidence(g))
dta <- data.frame(id=rownames(mat_g2_incidence), count=rowSums(mat_g2_incidence))
dta <- dta[order(dta$count, decreasing=T), ]
# Centrality key actors
cent<-data.frame(bet=betweenness(g, normalized=T, directed = FALSE)/max(betweenness(g, normalized=T, directed = FALSE)),eig=evcent(g)$vector, degree=degree(g, mode="total")/max(degree(g, mode="total")))
cent$index <- rowSums(cent)
cent$name<-rownames(cent) #Ids in this case
#head(cent);tail(cent)
###
cent$bet[1:i]<-(cent$bet[1:i]/max(cent$bet[1:i]))#*10
cent$eig[1:i]<-cent$eig[1:i]/max(cent$eig[1:i])
cent$index[1:i]<-cent$index[1:i]/max(cent$index[1:i])*5
#summary(cent[1:i,])
cent$bet[(i+1):nrow(cent)]<-cent$bet[(i+1):nrow(cent)]/max(cent$bet[(i+1):nrow(cent)])#*20
cent$eig[(i+1):nrow(cent)]<-cent$eig[(i+1):nrow(cent)]/max(cent$eig[(i+1):nrow(cent)])
cent$index[(i+1):nrow(cent)]<-cent$index[(i+1):nrow(cent)]/max(cent$index[(i+1):nrow(cent)])*10
#summary(cent[(i+1):nrow(cent),])
#head(cent)
cent$max_cent <- NA
cent$max_cent[1:i] <- cent$eig[1:i] + cent$degree[1:i] + cent$bet[1:i]
cent$max_cent[(i+1):nrow(cent)] <- cent$eig[(i+1):nrow(cent)] + cent$degree[(i+1):nrow(cent)] + cent$bet[(i+1):nrow(cent)]
cent<- cent[order(cent$max_cent, decreasing=T),]
#head(cent)
# Units or elements
V(g)$pub_citat_count <- c(dta$count[match(V(g)$label[1:i], dta$id)], a$Cited.by[match(V(g)$label[(i+1):length(V(g)$name)], a$EID)])
V(g)$label <- c(paste("Name: ", V(g)$label[1:i], ", Pub. count = ", V(g)$pub_citat_count[1:i], ", Degree = ", degree(g)[1:i], ", Max Centrality (max = 3) = ", round(cent$max_cent[match(V(g)$label[1:i], cent$name)], 3) , sep=""),
paste("EID: ", V(g)$label[(i+1):length(V(g)$name)], ", Citation count = ", V(g)$pub_citat_count[(i+1):length(V(g)$name)], ", Degree = ", degree(g)[(i+1):length(V(g)$name)], ", Max Centrality (max = 3) = ", round(cent$max_cent[match(V(g)$label[(i+1):length(V(g)$name)], cent$name)], 3) , sep=""))
#head(V(g)$label)
#tail(V(g)$label)
#Gets edgelist from graph, also any other attribute at the edge level to be included in the mapping
links_p<-as.data.frame(cbind(get.edgelist(g))) #To get a place_holder transformation
#Adding other attributes
links_p$V3<-as.numeric(a$Cited.by)[match(links_p$V2, a$EID)]
links_p$V4<-a$Title[match(links_p$V2, a$EID)]
# Making edgelist for interactive visualization
V(g)$name<-1:length(V(g))-1
links<-as.data.frame(cbind(get.edgelist(g)))
links$V1<-as.numeric(as.character(links$V1))
links$V2<-as.numeric(as.character(links$V2))
links$V3 <- links_p$V3
links$V4 <- links_p$V4
#str(links)
#head(links)
colnames(links)<-c("source","target", "citation", "title")
nodes <- data.frame(name= V(g)$label, pubs_citation = V(g)$pub_citat_count, groups = ifelse(V(g)$type==1, "Author", "Article"))
#head(nodes)
{
nodes$group<-NA
nodes$group <-cut(nodes$pubs_citation, c(0,1,3,10,20,max(nodes$pubs_citation)), right=TRUE, include.lowest = FALSE)
table(is.na(nodes$group))
table(nodes$group)
head(nodes[is.na(nodes$group),],20)
nodes$group<-ifelse(is.na(nodes$group), "No Citation", ifelse(nodes$group=="(0,1]", "1 cit/pub", ifelse(nodes$group=="(1,3]", "2 or 3 cit/pub", ifelse(nodes$group=="(3,10]", "4 to 10 cit/pub", ifelse(nodes$group=="(10,20]", "11 to 20 cit/pub", "Over 20 cit/pub")))))
counts<-data.frame(table(nodes$group))
counts$labels <- paste(counts$Var1, ", N= ", counts$Freq, sep="")
nodes$groups <- counts$labels[match(nodes$group, counts$Var1)]
#head(nodes)
}
ColourScale <- 'd3.scaleOrdinal()
.domain(["1 cit/pub, N= 3174", "11 to 20 cit/pub, N= 42", "2 or 3 cit/pub, N= 464", "4 to 10 cit/pub, N= 155", "No Citation, N= 721", "Over 20 cit/pub, N= 38"])
.range(["#ff3397", "#e5ff33", "#F4BB44", "#B2BEB5", "#EE4B2B", "#FF3131"]);'
netviz<-forceNetwork(Links = links, Nodes = nodes,
Source = 'source', Target = 'target',
NodeID = 'name',
Group = "groups", # color nodes by group calculated earlier
charge = -20, # node repulsion
linkDistance = JS("function(d) { return d.linkDistance; }"),#JS("function(d){return d.value}"),
linkWidth = JS("function(d) { return Math.sqrt(d.citation)*4; }"),
opacity = 0.8,
Value = "citation",
Nodesize = 'pubs_citation',
radiusCalculation = JS("Math.sqrt(d.nodesize*30)+10"),
zoom = T,
fontSize=14,
bounded= F,
legend= TRUE,
# linkColour = ifelse(links$campus == "NEUNK", "#CCFFFF", ifelse(links$campus == "NEUNL", "#e3eaa7", "#abb2b9")),
colourScale = JS(ColourScale))
HTMLaddons <-
"function(el, x) {
d3.select('body').style('background-color', 'white')
d3.selectAll('.legend text').style('fill', 'green')
d3.selectAll('.link').append('svg:title')
.text(function(d) { return 'Number of citations : ' + d.value + ', Title: ' + d.title ; })
var options = x.options;
var svg = d3.select(el).select('svg')
var node = svg.selectAll('.node');
var link = svg.selectAll('link');
var mouseout = d3.selectAll('.node').on('mouseout');
function nodeSize(d) {
if (options.nodesize) {
return eval(options.radiusCalculation);
} else {
return 6;
}
}
d3.selectAll('.node').on('click', onclick)
function onclick(d) {
if (d3.select(this).on('mouseout') == mouseout) {
d3.select(this).on('mouseout', mouseout_clicked);
} else {
d3.select(this).on('mouseout', mouseout);
}
}
function mouseout_clicked(d) {
node.style('opacity', +options.opacity);
link.style('opacity', +options.opacity);
d3.select(this).select('circle').transition()
.duration(750)
.attr('r', function(d){return nodeSize(d);});
d3.select(this).select('text').transition()
.duration(1250)
.attr('x', 0)
.style('font', options.fontSize + 'px ');
}
}
"
netviz$x$links$value <- links$citation
netviz$x$links$title <- links$title
netviz$x$links$linkDistance <- (links$citation)*50
onRender(netviz, HTMLaddons)4.3 Finally, tell us WHY should we even care about doing all of this (or do we)?
Creating these visualizations allows us and our audience to see the network for themselves and makes it easy to spot edge case type situations. For example, in the one mode visualization you can immediately notice the cluster of nodes (I think over 80!) that are all together, which is so different from the majority of the other connections in the visualization. Also, your attention might next be attracted to the nodes with different colors, which shows you how many publications each author has. Our audience as academics and researchers is growing and growing and with that means we have more responsibility to communicate our results well and make things attractive and attention grabbing, which is what these visualizations do.
Module 5 purpose & interpretation:
In this module we use a different dataset which represents the friendships of a group of people. The purpose here is to consider how social connections factor into certain outcomes. In this case we are considering each person’s fees collected and their hourly rate. We use a statistical test called Moran’s I which allows us to determine if the differences in the outcome variables are statistically associated with the relationship of the peoples. The statistical method lets us see to what extent the network is significant as well, i.e. do my direct social connections only matter or does it also matter that my friend A is connected to person B, so by secondary association I am also connected to person B. Moran’s I is normalized to range from −1 to +1, with +1 indicating a perfect spatial autocorrelation, suggesting a 1 to 1 variation in a given outcome and the average outcomes of a units’ neighbors.
The interpretation in this module relies on comparing the Moran’s I values between outcomes, which will tell us if the networked associations among friends make more of a difference (or have a greater connection) on peoples’ hourly rates or their fees collected. By looking at these numbers we can theorize why the network would matter more or less for either outcome. We also can interpret the importance of isolates (people with no friend connections) in this module as well because we consider the Moran’s I statistic both with and without isolates. Ultimately we determine how many higher order neighbors (i.e. neighbors beyond the direct neighbor) are statistically significant to the outcomes of concern.
5 Module: Friendship Data
Please replicate the procedures used in social dependence or peer effects to answer the following questions but using a friendship dataset instead:
5.1 Is there evidence of stronger dependence in the per hour rate compared to the fees brought in 1990 amounts?
Hour rate was 0.520938960 with a significant p value and fees was 0.455321539 with a significant p value as well. Thus, there is evidence that hourly rate has stronger dependence than fees because the Moran I’s statistic is larger. This could be because there are more factors that effect fees than hourly rate. However, this difference seems to be subtle since the difference between the two values is quite small. Also, this difference in Moran’s I is smaller than that of the coworker network, this indicates that the friendship network has less differences in connections between these two characteristics.
#Friendship data loading in
idfriend <- "1dwX4kKlx-ctkU0JyH74p3r1w-jJdTAqi"
friendshiplazega <- read.csv(sprintf("https://docs.google.com/uc?id=%s&export=download", idfriend))
#idadvice <- "1dbZDumTH9dFwNStIKKEO9bhx_ftYtLam"
#advicelazega <- read.csv(sprintf("https://docs.google.com/uc?id=%s&export=download", idadvice))
# 1 is friendship, 0 is no friendship
advicelazega <- as.matrix(friendshiplazega)
#advicelazegac <- advicelazega
#str(advicelazega)
#dim(advicelazega)
#Reading attributes also provided by Lazega
idattributes <- "1e0GtrRS5PFFNdnd1e4fJcjeuBZ6deF7g"
datattrout <- read.csv(sprintf("https://docs.google.com/uc?id=%s&export=download", idattributes))
#Transforming the matrices to spatial form (row normalized) as shown in equation (21)
advicelazega <-advicelazega /rowSums(advicelazega)
summary(rowSums(advicelazega))## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 1 1 1 1 1 1 6# Replacing potential NAN to zeros as shown in equation (22)
advicelazega[is.na(advicelazega)]<-0
listwAd<-mat2listw(advicelazega)
#########
#Testing influence
#########
moran.test(datattrout$HrRATE90,listwAd, zero.policy=TRUE)##
## Moran I test under randomisation
##
## data: datattrout$HrRATE90
## weights: listwAd
## n reduced by no-neighbour observations
##
## Moran I statistic standard deviate = 8.6798, p-value < 2.2e-16
## alternative hypothesis: greater
## sample estimates:
## Moran I statistic Expectation Variance
## 0.520938960 -0.015625000 0.003821452moran.test(datattrout$FeesCollec90,listwAd, zero.policy=TRUE)##
## Moran I test under randomisation
##
## data: datattrout$FeesCollec90
## weights: listwAd
## n reduced by no-neighbour observations
##
## Moran I statistic standard deviate = 7.628, p-value = 1.192e-14
## alternative hypothesis: greater
## sample estimates:
## Moran I statistic Expectation Variance
## 0.455321539 -0.015625000 0.003811733#########
#Visualizing Local Moran’s I clusters and outliers
#########
# mt <- moran.test(datattrout$FeesCollec90, listwAd, zero.policy=TRUE)
# label_x = "Individual Fees Collected"
# label_y = "Lagged Individual Fees Collected"
# mp <- moran.plot(datattrout$FeesCollec90, listwAd, zero.policy=T,
# labels=datattrout$id, xlab = label_x, ylab = label_y)
# title(main="Moran’s Plot", cex.main=2, col.main="grey11",
# font.main=2, sub=paste("Plot includes 71 lawyers (Moran’s I = ",
# round(mt$ estimate[1], 3), ", p < .0001)", sep=""), cex.sub=1.15, col.sub="grey11", font.sub=2,)5.2 Do you have isolates in the model? Clue, you can test for the presence of isolates using: table(rowSums(friendshiplazega)==0)
Yes, there are 6 isolates in the model, meaning there are 6 people in this community that do not have any friends.
table(rowSums(advicelazega)==0)##
## FALSE TRUE
## 65 65.3 How many higher order neighbors did you find? Does this change by outcome?
Note that the friendship network we are looking at is sparse enough that by having isolates and keeping them in it is not possible to find the higher order neighbors. To address this issue we consider secondary friends (i.e. friends of a friend) as direct friends (inherently making a new friendship network).
We find only one higher order neighbors because only the first order neighbor is statistically significant with the complete confidence interval of that neighbor falling in the Moran’s I above 0 range.
The number of higher order neighbors does not change by outcome because for either hourly rate or fees in both cases only the first neighbor is significant and considered higher order.
advicelazega1 <- advicelazega%*%t(advicelazega)#increasing relationships by considering friends of friends
diag(advicelazega1)<-0
advicelazega[advicelazega1>1]<-1
#Transforming the matrices to spatial form (row normalized) as shown in equation (21)
advicelazega1 <-advicelazega1 /rowSums(advicelazega1)
# Replacing potential NAN to zeros as shown in equation (22)
advicelazega1[is.na(advicelazega1)]<-0
listwAd1<-mat2listw(advicelazega1)
#run plots to see number of higher order neighbors
keep_listNAd1 <- subset(listwAd1[[2]], subset=card(listwAd1[[2]])>= 0)
plot.spcor(sp.correlogram(keep_listNAd1, datattrout$HrRATE90, order = 3, method = "I", zero.policy=T), xlab = "Social lags", main = "Social correlogram: Autocorrelation with CIs")
plot.spcor(sp.correlogram(keep_listNAd1, datattrout$FeesCollec90, order = 3, method = "I", zero.policy=T), xlab = "Social lags", main = "Social correlogram: Autocorrelation with CIs")
5.4 After removing isolates: How do the Moran’s I estimates change? & Are there any differences in these changes by outcome?
The Moran’s I for Hour rate increased to 0.684439193 with a significant p value (.1635 change) and the Moran’s I for fees increased to 0.528467238 (.0731 change) with a significant p value as well. The Moran’s I for the fees changed less than that of the hour rate, indicating that by removing isolates the remaining network of connections has a greater dependence on hour rate than fees.
##########################################
### REMOVE UNCONNECTED:::::
#########
#How are disconnected unit influencing these results and how to remove them?
#########
sub_listNAd <- subset(listwAd[[2]], subset=card(listwAd[[2]])> 0)
#sub_listNAd
#weights
sub_listWAd <- nb2listw(sub_listNAd, glist=NULL, style="W", zero.policy=NULL)
#making this a spatial points dataframe
sub_datattrout <- subset(datattrout, subset=card(listwAd[[2]]) > 0)
moran.test(sub_datattrout$HrRATE90,sub_listWAd, zero.policy=TRUE)##
## Moran I test under randomisation
##
## data: sub_datattrout$HrRATE90
## weights: sub_listWAd
##
## Moran I statistic standard deviate = 10.907, p-value < 2.2e-16
## alternative hypothesis: greater
## sample estimates:
## Moran I statistic Expectation Variance
## 0.684439193 -0.015625000 0.004119984moran.test(sub_datattrout$FeesCollec90,sub_listWAd, zero.policy=TRUE)##
## Moran I test under randomisation
##
## data: sub_datattrout$FeesCollec90
## weights: sub_listWAd
##
## Moran I statistic standard deviate = 8.5307, p-value < 2.2e-16
## alternative hypothesis: greater
## sample estimates:
## Moran I statistic Expectation Variance
## 0.528467238 -0.015625000 0.004067992#########
#Visualizing Local Moran’s I clusters and outliers
#########
# mt <- moran.test(sub_datattrout$HrRATE90, sub_listWAd, zero.policy=TRUE)
# label_x = "Hr Rate Collected"
# label_y = "Lagged Hr Rates Collected"
# mp <- moran.plot(sub_datattrout$HrRATE90, sub_listWAd, zero.policy=T,
# labels=sub_datattrout$id, xlab = label_x, ylab = label_y)
# title(main="Moran’s Plot", cex.main=2, col.main="grey11",
# font.main=2, sub=paste("Plot includes 71 lawyers (Moran’s I = ",
# round(mt$ estimate[1], 3), ", p < .0001)", sep=""), cex.sub=1.15, col.sub="grey11", font.sub=2,)5.5 After removing isolates: How many higher order neighbors did you find? and Does this change by outcome?
We find two higher order neighbors because they are statistically significant with the complete confidence interval of that neighbor falling in the Moran’s I above 0 range. The number of higher order neighbors does not change by outcome because for either hourly rate or fees in both cases only the first and second order neighbors are significant and considered higher order.
#########
# Methodological questions with practical implications:
# How do we know that our proposed initial matrix of weights is robust enough to estimate outcome dependence?
# How many neighbors of neighbors (higher order neighboring structures) do we have to account for to establish a data driven selection of neighboring structures?
#########
plot.spcor(sp.correlogram(sub_listNAd, sub_datattrout$HrRATE90, order = 6, method = "I", zero.policy=T), xlab = "Social lags", main = "Social correlogram: Autocorrelation with CIs")
plot.spcor(sp.correlogram(sub_listNAd, sub_datattrout$FeesCollec90, order = 6, method = "I", zero.policy=T), xlab = "Social lags", main = "Social correlogram: Autocorrelation with CIs")
###Test
#moran.plot(sub_datattrout$HrRATE90, nth_order[[2]], zero.policy=T, labels=sub_datattrout$id)
###Module 6 purpose & interpretation:
In this last module we return to our publications data set. The purpose is to employ the same methods from module 5 to consider if the networked interactions of the co-authors in our data set have any influence on the publication counts of authors. This will help us understand if academics in this field find publishing success by working with multiple people, or, perhaps the connections aren’t as important and being a solo author is the way to go. We again must consider if we want to include solo authors and we also need to consider if we include isolates here, but isolates in this context would be solo authors that for every paper they publish it solo. We also visualize the relationship of publication counts of authors and co-authors.
The interpretation in this module relies on the Moran’s I value and the visualization. The Moran’s I tells us if the networked associations among authors makes a difference in those authors’ publication counts. We also determine the number of higher order neighbors that are statistically significant. The interpretation here is if your co-authors’ co-authors (co-authors’…) matter to your own number of publications.
6 Module: Going back to publication data
##Decide whether you want to focus on co-authors or want to include all authors, including solo authors. I will include solo authors because as mentioned before there are 1,496 papers written by solo authors in my data set.
6.1 What are the implications of this decision? That is, tell me how this decision changes whether you start with one-mode or two-mode data. What about isolates?
If you want to focus on co-authors only, then you can start with the one-mode adjacency list. Also, in this context the isolate means an author that published alone (i.e. doesn’t have any connections with other authors for a paper) so that means by creating the one mode adjacency list you also get rid of the isolates. On the other hand, if you want to account for solo authors, which are the isolates, you need to create the two mode adjacency list between articles and authors in the data structuring process. By way of this process the isolates will automatically be kept in.
6.3 Visualize this influence level, what may be the mechanism to explain these clusters and outliers?
We see clusters where an author has published 2 or 3 times themselves, but have had no co-authors. This could be explained by a researcher who has published alone once to publish again in the future, i.e. people who work independently continue to work independently. We also see that there are more authors / coauthors with relatively matching amounts of publications, i.e. the lines at y_i=1, y_i=2, … have more points where the y_j values are the same or very similar. This makes sense because in academia researchers with similar levels of experience tend to work together (at least from my observation), so it’s more likely that the author and co-authors have similar amounts of publications. The far right or high up outliers might be explained by a mismatch of author experiences where an established researcher agrees to work with some early career academics or advise a PhD student on a paper. Overall we see that most authors have published 4 or less articles in the Border studies field.
#Moran's I procedures for social dependence
mi <- moran.test(pd$pub, listw, zero.policy=TRUE)
mp_math <- moran.plot(pd$pub, listw, labels=as.character(pd$id), zero.policy = TRUE)
#adds a label, br breaks the line
mp_math$new_label <- paste("Author ID: ", mp_math$labels, "<br>own pub record: ", pd$pub[match(mp_math$labels, pd$id)],
"<br>Co-authors' record: ", pd$lag.pub[match(mp_math$labels, pd$id)], sep="")
mphu <-ggplot(mp_math, aes(x=jitter(x), y=jitter(wx), text=new_label)) + geom_point(shape=1, alpha=0) +
geom_hline(yintercept=mean(mp_math$wx), lty=2) +
geom_vline(xintercept=mean(mp_math$x), lty=2) + theme_minimal() +
geom_point(data=mp_math[(mp_math$wx>=mean(mp_math$wx)&mp_math$x>=mean(mp_math$x))&mp_math$is_inf==FALSE,], aes(x=x, y=wx), shape=1, alpha=.1) +
geom_point(data=mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx>=mean(mp_math$wx))&mp_math$is_inf==TRUE,], aes(x=x, y=wx), shape=9, alpha=.4) +
geom_point(data=mp_math[(mp_math$x<mean(mp_math$x)&mp_math$wx>=mean(mp_math$wx))&mp_math$is_inf==FALSE,], aes(x=x, y=wx), shape=1, alpha=.1) +
geom_point(data=mp_math[(mp_math$x<mean(mp_math$x)&mp_math$wx>=mean(mp_math$wx))&mp_math$is_inf==TRUE,], aes(x=x, y=wx), shape=9, alpha=.4) +
geom_point(data=mp_math[(mp_math$x<mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==FALSE,], aes(x=x, y=wx), shape=1, alpha=.1) +
geom_point(data=mp_math[(mp_math$x<mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,], aes(x=x, y=wx), shape=9, alpha=.4) +
geom_point(data=mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==FALSE&(mp_math$x-mp_math$wx<min(mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,]$x-mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,]$wx)),], aes(x=x, y=wx), shape=1, alpha=.3, position=position_jitter(h=0.1,w=0.1)) +
geom_point(data=mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==FALSE&(mp_math$x-mp_math$wx>=min(mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,]$x-mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,]$wx)),], aes(x=x, y=wx), shape=9, alpha=.8, colour=rgb(223, 255, 0, max=255, 255/1), position=position_jitter(h=0.1,w=0.1)) +
geom_point(data=mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,], aes(x=x, y=wx), shape=9, alpha=.8, colour=rgb(255, 0, 126, max=255, 255/1), position=position_jitter(h=0.1,w=0.1)) +
xlab("Authors' Own Publication count (y_i)") + ylab("Coauthors' Publication count (y_j)") + ggtitle("Peer effects",
subtitle = "") + labs(caption = "(PHUDCFILY)") +
theme(plot.title = element_text(color=rgb(255, 0, 126, max=255, 255/1), size=14, face="bold"),
plot.subtitle = element_text(color = "blue"))#
ggplotly(mphu, tooltip = c("new_label")) %>%
layout(title = list(text = paste0('<b>Peer effects on Publication Performance</b>',
'<br>',
'<sup style="color:black; font-weight:bold">',
paste(dim(mat_g2_incidence_to_1)[1],' authors in ', dim(mat_g2_incidence)[2],' Publications. Conservative: ', dim(mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,])[1], ' HL cases (pink), Liberal: ', dim(mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==FALSE&(mp_math$x-mp_math$wx>=round(min(mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,]$x-mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,]$wx),3)),])[1], ' HL cases (green) </sup>', sep=""),
'<span style="font-size:10px; font-weight:bold; color: rgb(255, 0, 126, max=255, 255/1)">',
paste("\nSource: Scopus. Moran's I = ", round(mi$estimate[1], 3)," P. Value ", ifelse(mi$p.value==0, "< 0.001", mi$p.value), ', Liberal threshold = ', round(min(mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,]$x-mp_math[(mp_math$x>=mean(mp_math$x)&mp_math$wx<mean(mp_math$wx))&mp_math$is_inf==TRUE,]$wx),3), '</span>', sep=""))))6.4 Test how many higher order neighbors should we account for in this framework?
We find that we should account for two higher order neighbors because they are statistically significant with the complete confidence interval of that neighbor falling in the Moran’s I above 0 range.
#higher order
plot.spcor(sp.correlogram(listw[[2]], pd$pub, order = 5, method = "I", zero.policy=TRUE), xlab = "Social lags", main = "Social correlogram: Autocorrelation with CIs")
6.5 If needed create a higher order weight matrix and test for autocorrelation.
Autocorrelation occurs when “the spatial neighbors of unit 𝑖 realize outcomes that are more similar to the outcomes of such a unit 𝑖 than what we could expect to observe under a spatial random process” (González Canché). The Moran’s I allows us to test for autocorrelation, and we see here that after making the higher order weight matrix the Moran’s I statistic is still significant, suggesting that the spatial relationships with (or in this case the networked connections of authors) is related to the author’s publication count.
nth_order <- nblag(listw[[2]], maxlag=2)#this assumes two
nth_order <- nblag_cumul(nth_order)
nth_order <- nb2listw(nth_order, style="W", zero.policy=T)
mt <- moran.test(pd$pub, nth_order, zero.policy=TRUE)
mt##
## Moran I test under randomisation
##
## data: pd$pub
## weights: nth_order
## n reduced by no-neighbour observations
##
## Moran I statistic standard deviate = 8.9353, p-value < 2.2e-16
## alternative hypothesis: greater
## sample estimates:
## Moran I statistic Expectation Variance
## 0.1896925919 -0.0004557885 0.0004528587