Descriptive Analytic Exercise 1: Visualizing and Interpreting Networks

SNA Grad Seminar, Fall 2017 Due: October 24th, 11:59 pm Name of Student:

The purpose of this lab is to develop your familiarity conducting descriptive network analysis using the statistical software package R. This assignment will make use of a data set you collect by defining a search query (a collection of your user-defined search terms) from the New York Times’s Article Search Application Programming Interface. Networks are generated from the co-occurrences between search terms included in the same search query. For example, a link exists between “apple” and “orange” if there are articles in the New York Times that contained these two terms. You will be visualizing and interpreting individual and global network properties of this network.

You will be graded primarily on the completeness and accuracy of your responses, but the clarity of the prepared report will also affect your grade. While students may work together to perform the analysis, each student must submit his or her own report and is responsible for writing the narrative in the report. You must answer all of the bolded questions.

Part 1: Collect Network Data (20 pts)

For this lab, you will search the New York Times, save that data, create networks from that data, compare the differences among networks, and demonstrate your proficiency with basic network descriptive statistics.

Loading and Installing Packages, Set Working Directory

When working with R, you should run each line of code individually, unless it is part of a function definition, so you can see the results. Generally speaking, any line of code that includes ‘{’ (the beginning of a function definition) should be run with all the other lines until you hit ‘}’.

library(magrittr)

Warning message:
In scan(file = file, what = what, sep = sep, quote = quote, dec = dec,  :
  EOF within quoted string

library(httr)
library(data.table)
library(igraph)
library(dplyr)
library(xml2)

Choose a topic for your search terms

You can decide search terms based on personal interests, research interests, or popular topical areas, among others. You have flexibility in selecting your search term list. For example, you can search for some commercial brands, celebrities, countries, universities, etc. It will be most useful if you choose a collection of words that are not all extremely common. Think about a set of words that might have interesting co-occurrences in articles within the New York Times website. For example, you might be interested in the last names of every Senator involved in a certain political debate, football teams, or cities and their co-occurrence in news articles. Generally speaking, proper nouns are best, but you might have compelling reasons to choose verbs or adjectives. You might want to throw a couple of terms in that aren’t thematically related to make sure you don’t get a totally connected component. The more interesting your network is in terms of differing centrality, distinct components, etc., the easier it will be to do the written analysis. Keep in mind that the Article Search archive is very large; many terms co-occur. You might want to consider two tenuously related subjects. The example file uses four football teams and their home senators, plus a few topical terms.

Create your text input

Create a plain text file with .txt extension in the same directory as the R Markdown Notebook used in this assignment. Make a note of the file name for use in the next code snippet. Place one search term per line, and use 15–20 terms. You’ll also likely want to add quotation marks around your search terms to ensure that you’re only receiving results for the complete term. NOTE: The function will process your terms so that they work in the URL request. You do not need to encode non-alphabetic characters.

The text file cannot include any additional information or characters and it must be a .txt file; Word or RTF documents won’t work.

Analysis

a. Provide a high level overview of the terms you included in the search query. #The terms included in the search query are famous celebrities and car brands. I choose 16 terms in total which are “Taylor Swift”, “Lady Gaga”, “Kesha”, “Tom Cruise”, “George Clooney”, “Jennifer Lawrence”, “Emma Stone”, “Dwayne Johnson”, “Ferrari”, “Lamborghini”, “Mercedes”, “BMW”, “Audi”, “Volov”, “Porsche”, “Lexus”.

b. Why did you choose this collection of terms? Were there some specific overarching question—intellectual or extracurricular curiosity—that motivated this collection of terms? #Celebrities always like fansy and luxury cars. I hope to find whether some car brands are mostly shown in article with celebrities, then I could guess celebrities are mostly favoriable with these brands.

c. How did you decide which terms to use in the search query? Were these terms you intuitively deemed important? Were they culled from a specific source or the result of some separate analysis or search query? #The terms I choose are most famous celebrities and luxury brands I found in Google. Celebrities and luxury car brands shall have certain number of records on NYT. Also, celebrities tend to choose luxury brands.

d. What are the insights you hope to glean by looking at the network of terms in terms of individual node metrics, sub-grouping of nodes, overall global network properties? #For individual node: I hope to see which car brand is mostly connected by celebrities. #For sub-grouping of nodes: I hope to see if some celebrities are connected, they might be good friends. #For overall network: I hope to see the density of network.

Working with the API to Collect Your Data

The New York Times controls access to its API by assigning each user a key. Each key has a limited number of calls that can be made within a certain time period. You can read more about the limitations of the API system here.

You will need to create your own API key to complete this assignment. Go to the New York Times developers page and request a key. You will copy that key (received via email) into the api variable below.

Our first function will gather all of the search terms and their number of hits to be placed in a table. All lines of a function should be run together.

Now we will invoke our function to put information from the API into our global environment.

total_table <- t(sapply(word_list,Get_hits_one))

[1] "Taylor-Swift"
[1] 353
[1] "Lady-Gaga"
[1] 335
[1] "Kesha"
[1] 50
[1] "Tom-Cruise"
[1] 156
[1] "George-Clooney"
[1] 160
[1] "Jennifer-Lawrence"
[1] 224
[1] "Emma-Stone"
[1] 190
[1] "Dwayne-Johnson"
[1] 128
[1] "Ferrari"
[1] 388
[1] "Lamborghini"
[1] 74
[1] "Mercedes"
[1] 853
[1] "BMW"
[1] 521
[1] "Audi"
[1] 335
[1] "Volvo"
[1] 229
[1] "Porsche"
[1] 301
[1] "Lexus"
[1] 116

total_table <- as.data.frame(total_table)
total_table$ResultsTotal <- as.numeric(as.character(total_table$ResultsTotal))

If you get zero hits for any of these terms, you should substitute that term for somethign else and rerun the lab up to this point. Next, we will define the function that will collect the article co-occurences network.

In this next step, we will call the API and collect the co-occurrence network. This may take some time. If you receive “numeric(0)” in any of your resposnes, you’ve likely hit your API key limit and will either need to wait for the calls to reset (24 hours) or request a new key. If you receive the error message “$ operator is invalid for atomic vectors,” you have also hit the API call limit. This could be due to running the script multiple times, or due to hitting too many results based on very common search terms. Request a new API, shorten your word list, and try again. Don’t forget you need to reload your word list from the first part of the Lab in order to get a different set of results! You must also rerun the functions to reassign the API value. If none of your results come back as “0,” you might want to redo your search with the appropriate words.

# Convert the pairs list into a table
pairs_list <- expand.grid(word_list,word_list) %>% filter(Var1 != Var2)

package <U+393C><U+3E31>bindrcpp<U+393C><U+3E32> was built under R version 3.4.2

pairs_list <- t(combn(word_list,2))
#Create a network table, run the Get_hits_two function using the pairs lists
network_table <- t(apply(pairs_list,1,Get_hits_two))

[1] "Taylor-Swift Lady-Gaga"
[1] 38
[1] "Taylor-Swift Kesha"
[1] 16
[1] "Taylor-Swift Tom-Cruise"
[1] 2
[1] "Taylor-Swift George-Clooney"
[1] 2
[1] "Taylor-Swift Jennifer-Lawrence"
[1] 5
[1] "Taylor-Swift Emma-Stone"
[1] 1
[1] "Taylor-Swift Dwayne-Johnson"
[1] 2
[1] "Taylor-Swift Ferrari"
[1] 1
[1] "Taylor-Swift Lamborghini"
[1] 0
[1] "Taylor-Swift Mercedes"
[1] 3
[1] "Taylor-Swift BMW"
[1] 0
[1] "Taylor-Swift Audi"
[1] 0
[1] "Taylor-Swift Volvo"
[1] 1
[1] "Taylor-Swift Porsche"
[1] 0
[1] "Taylor-Swift Lexus"
[1] 0
[1] "Lady-Gaga Kesha"
[1] 8
[1] "Lady-Gaga Tom-Cruise"
[1] 3
[1] "Lady-Gaga George-Clooney"
[1] 1
[1] "Lady-Gaga Jennifer-Lawrence"
[1] 10
[1] "Lady-Gaga Emma-Stone"
[1] 1
[1] "Lady-Gaga Dwayne-Johnson"
[1] 1
[1] "Lady-Gaga Ferrari"
[1] 1
[1] "Lady-Gaga Lamborghini"
[1] 1
[1] "Lady-Gaga Mercedes"
[1] 1
[1] "Lady-Gaga BMW"
[1] 0
[1] "Lady-Gaga Audi"
[1] 1
[1] "Lady-Gaga Volvo"
[1] 0
[1] "Lady-Gaga Porsche"
[1] 0
[1] "Lady-Gaga Lexus"
[1] 0
[1] "Kesha Tom-Cruise"
[1] 0
[1] "Kesha George-Clooney"
[1] 0
[1] "Kesha Jennifer-Lawrence"
[1] 0
[1] "Kesha Emma-Stone"
[1] 0
[1] "Kesha Dwayne-Johnson"
[1] 0
[1] "Kesha Ferrari"
[1] 0
[1] "Kesha Lamborghini"
[1] 0
[1] "Kesha Mercedes"
[1] 1
[1] "Kesha BMW"
[1] 0
[1] "Kesha Audi"
[1] 0
[1] "Kesha Volvo"
[1] 0
[1] "Kesha Porsche"
[1] 0
[1] "Kesha Lexus"
[1] 0
[1] "Tom-Cruise George-Clooney"
[1] 8
[1] "Tom-Cruise Jennifer-Lawrence"
[1] 5
[1] "Tom-Cruise Emma-Stone"
[1] 4
[1] "Tom-Cruise Dwayne-Johnson"
[1] 6
[1] "Tom-Cruise Ferrari"
[1] 0
[1] "Tom-Cruise Lamborghini"
[1] 0
[1] "Tom-Cruise Mercedes"
[1] 2
[1] "Tom-Cruise BMW"
[1] 0
[1] "Tom-Cruise Audi"
[1] 0
[1] "Tom-Cruise Volvo"
[1] 1
[1] "Tom-Cruise Porsche"
[1] 0
[1] "Tom-Cruise Lexus"
[1] 0
[1] "George-Clooney Jennifer-Lawrence"
[1] 14
[1] "George-Clooney Emma-Stone"
[1] 2
[1] "George-Clooney Dwayne-Johnson"
[1] 6
[1] "George-Clooney Ferrari"
[1] 0
[1] "George-Clooney Lamborghini"
[1] 0
[1] "George-Clooney Mercedes"
[1] 2
[1] "George-Clooney BMW"
[1] 0
[1] "George-Clooney Audi"
[1] 0
[1] "George-Clooney Volvo"
[1] 0
[1] "George-Clooney Porsche"
[1] 1
[1] "George-Clooney Lexus"
[1] 1
[1] "Jennifer-Lawrence Emma-Stone"
[1] 12
[1] "Jennifer-Lawrence Dwayne-Johnson"
[1] 7
[1] "Jennifer-Lawrence Ferrari"
[1] 0
[1] "Jennifer-Lawrence Lamborghini"
[1] 0
[1] "Jennifer-Lawrence Mercedes"
[1] 1
[1] "Jennifer-Lawrence BMW"
[1] 0
[1] "Jennifer-Lawrence Audi"
[1] 1
[1] "Jennifer-Lawrence Volvo"
[1] 2
[1] "Jennifer-Lawrence Porsche"
[1] 1
[1] "Jennifer-Lawrence Lexus"
[1] 0
[1] "Emma-Stone Dwayne-Johnson"
[1] 4
[1] "Emma-Stone Ferrari"
[1] 0
[1] "Emma-Stone Lamborghini"
[1] 0
[1] "Emma-Stone Mercedes"
[1] 0
[1] "Emma-Stone BMW"
[1] 0
[1] "Emma-Stone Audi"
[1] 0
[1] "Emma-Stone Volvo"
[1] 1
[1] "Emma-Stone Porsche"
[1] 0
[1] "Emma-Stone Lexus"
[1] 0
[1] "Dwayne-Johnson Ferrari"
[1] 0
[1] "Dwayne-Johnson Lamborghini"
[1] 0
[1] "Dwayne-Johnson Mercedes"
[1] 1
[1] "Dwayne-Johnson BMW"
[1] 0
[1] "Dwayne-Johnson Audi"
[1] 0
[1] "Dwayne-Johnson Volvo"
[1] 2
[1] "Dwayne-Johnson Porsche"
[1] 0
[1] "Dwayne-Johnson Lexus"
[1] 0
[1] "Ferrari Lamborghini"
[1] 19
[1] "Ferrari Mercedes"
[1] 182
[1] "Ferrari BMW"
[1] 16
[1] "Ferrari Audi"
[1] 14
[1] "Ferrari Volvo"
[1] 4
[1] "Ferrari Porsche"
[1] 33
[1] "Ferrari Lexus"
[1] 6
[1] "Lamborghini Mercedes"
[1] 15
[1] "Lamborghini BMW"
[1] 13
[1] "Lamborghini Audi"
[1] 17
[1] "Lamborghini Volvo"
[1] 2
[1] "Lamborghini Porsche"
[1] 22
[1] "Lamborghini Lexus"
[1] 4
[1] "Mercedes BMW"
[1] 134
[1] "Mercedes Audi"
[1] 81
[1] "Mercedes Volvo"
[1] 40
[1] "Mercedes Porsche"
[1] 44
[1] "Mercedes Lexus"
[1] 36
[1] "BMW Audi"
[1] 106
[1] "BMW Volvo"
[1] 50
[1] "BMW Porsche"
[1] 47
[1] "BMW Lexus"
[1] 40
[1] "Audi Volvo"
[1] 38
[1] "Audi Porsche"
[1] 94
[1] "Audi Lexus"
[1] 32
[1] "Volvo Porsche"
[1] 7
[1] "Volvo Lexus"
[1] 16
[1] "Porsche Lexus"
[1] 10

#Convert the network table into a dataframe
network_table <- as.data.frame(network_table)
# Read each the content of each item within the $CoOccurreences factor as characters, 
# then force those characters into the "numeric" or "double" type.
network_table$CoOccurrences <- as.numeric(as.character(network_table$CoOccurrences))
# Convert data to data.table type.
total_table <- as.data.table(total_table)
network_table <- as.data.table(network_table)
# Remove zero edges from your network
network_table <- network_table[!CoOccurrences==0] 
# Create a graph object with your data
g_valued <- graph_from_data_frame(d = network_table[,1:3,with=FALSE],directed = FALSE,vertices = total_table)

Analysis

Is the graph directed or undirected? #The graph is undirected. How many nodes and links does your network have?

numVertices <- vcount(g_valued)
numVertices

[1] 16

numEdges <- ecount(g_valued)
numEdges

[1] 71

What is the number of possible links in your network?

maxEdges <- numVertices*(numVertices-1)/2
maxEdges

[1] 120

What is the density of your network?

graphDensity <- numEdges/maxEdges # manual calculation
graphDensity

[1] 0.5916667

graphDensity1 <- graph.density(g_valued) # using the graph.density function from igraph
graphDensity1

[1] 0.5916667

Briefly describe how your choice of dataset may influence your findings. What differences would you expect if you use different search terms? Are the current search terms related to one another? If so, how? Do you think the limitation to one word might skew your answers? (i.e. if you’re interested in Hillary Clinton, but you include “Clinton” as a term, you will get stories that mention Chelsea, Bill, & even P-Funk Allstar George Clinton). #If I choose different terms like unknown singers, it will be mentioned less frequently in NYT, thus there might be less edges, the results would be less interesting. #For current search terms are related to one anther, but the weight of edge varies. #The limitation of one word would significantly reduce the accuracy of network. For example, if “Tom Cruise” was limited as “Cruise”, then even people not “Tom Cruise” would also be counted which is messed up with correct counts.

Part 2: Visualize Your Network (20 points)

Let’s start by visualizing the network that we’ve collected from the New York Times Article Search API. We’ll need to choose node colors and set a layout. You can learn more about Fruchterman Reingold layout and other layouts here.

Analysis

In a paragraph, describe the macro-level structure of your graphs based on the Fruchterman Reingold visualization. Is it a giant, connected component, are there distinct sub-components, or are there isolated components? Can you recognize common features of the subcomponents? Does this visualization give you any insight into the co-occurrence patterns of the search-terms? If yes, what? If not, why? #The network is a gaint connected components, there is no isolated components. According to visulization, the term has most connections with other terms are placed at center place such as Mercedes, while some terms has least connections with other terms are placed outside such as Kesha.

Now we’ll create a second visualization using a different layout.

Analysis

In a paragraph, compare and contrast the information given to you by the two different layouts. #Both layouts have clear seperation between car brands and celebrities. #According to graph with Fruchterman Reingold layout, it shows that at subcomponents of car brands, it forms a closed network and Audi is at center of this circle. Mercedes and Volove is like brokers which has most connection with celebrities subcomponents. Ferrari and Porsche has fewer connections with celebrities subcomponents. Lamborghini, Lexus, BMW and Audi are mainly internal connected in car subcomponents. At celebrity subcompoent side, Lady Gaga, Jennifer and George has most connection with car brands. Kesha has least connection and only connects to Lady Gaga and Taylor Swift. #According to graph with Davidson and Harel layout. For car brand subcomponent, Audi is still at center, which means Audi might has highest weight of connection with other car brands.

Part 3: Community Detection Analysis with R (20 Points)

Identifying subgroups within a network is of great interest to social network researchers, so a variety of algorithms have been developed to identify and measure subgroups. We will use some of R’s built-in tools to identify subgroups and central nodes for visual inspection.

For the remainder of the visualizations we will use the Fruchterman Reingold layout.

Cluster the nodes in your network.

cluster <- cluster_walktrap(g_valued)
# Find the number of clusters
membership(cluster)   # affiliation list

     Taylor-Swift         Lady-Gaga             Kesha        Tom-Cruise    George-Clooney Jennifer-Lawrence 
                2                 2                 2                 2                 2                 2 
       Emma-Stone    Dwayne-Johnson           Ferrari       Lamborghini          Mercedes               BMW 
                2                 2                 1                 1                 1                 1 
             Audi             Volvo           Porsche             Lexus 
                1                 1                 1                 1 

length(sizes(cluster)) # number of clusters

[1] 2

# Find the size the each cluster 
# Note that communities with one node are isolates, or have only a single tie
sizes(cluster) 

Community sizes
1 2 
8 8 

How many communities have been created? # 2 How many nodes are in each community? # 8 for each community In networks containing node attribute information, we can often gain insight into a network by looking at the nodes that get placed in the same partition. For your network, what might each cluster of nodes potentially have in common? Describe each cluster, its membership, and the relationship between nodes in the cluster. #The first cluster is all celebrities, second cluster is all car brands. The relationship between cluster 1 might be frinds, relationship between cluster 2 might be competitors.

Next we visualize the clusters by coloring nodes according to their modularity class.

What information does this layout convey? Are the clusters well-separated, or is there a great deal of overlap? Is it easier to identify the common themes among clusters in this layout rather than looking only at the graphs? #The clusters are well seperated. Both ways are easy to identify the themes of terms.

What differences are there between nodes in the same cluster and across clusters? # Nodes in same cluster belong to same theme, there is no node belong to both cluster.

Describe the brokers between any components and cliques. What are common features of these brokers? About how many brokers would you have to remove from your network to “shatter” it into two or more disconnected components? #There are three cliques for interets, first clique include all car brands, second clique inlcude all celebrities except Kesha, third clique include Kesha, Talor Swift, Lady Gaga and Mercedes. The broker between first and third clique is Mercedes. There are many connections between first and second cliques. #To “shatter components”, Taylor Swift, Lady Gaga, Mercedes can be removed, there will be two components, Kesha is one component and rest of nodes form another component.

Part 4: Centrality Visualization & Weighted Values (20 Points)

For each network, you will use centrality metrics to improve your visualization. You may need to adjust the size parameter to make your network more easily visible.

Degree Centrality

totalDegree <- degree(g_valued,mode="all")
sort(totalDegree,decreasing=TRUE)[1:5]

         Mercedes             Volvo         Lady-Gaga      Taylor-Swift Jennifer-Lawrence 
               14                12                11                10                10 

g2 <- g_valued
V(g2)$size <- totalDegree*2 #can adjust the number if nodes are too big
plot(g2, layout = L, vertex.label=NA)

Briefly explain degree centrality and why nodes are more or less central in the network. #Degree centrality indicates number of nodes it has connection to.Most of nodes has high centrality in this network, some nodes are less central such as Kesha, as Kesha was not as popular as other celebrities, thus it has fewer connections.

Weighted Degree Centrality

wd <- graph.strength(g_valued,weights = E(g_valued)$CoOccurrences)
sort(wd,decreasing=TRUE)[1:5]

Mercedes      BMW     Audi  Ferrari  Porsche 
     543      406      384      276      259 

wg2 <- g_valued
V(wg2)$size <- wd*.1 # adjust the number if nodes are too big
plot(wg2, layout = L, vertex.label=NA, edge.width=sqrt(E(g_valued)$CoOccurrences))

What does the addition of weighted degree and edge information tell you about your graph? #The addition of weighted degree and edge show the difference of edge values. Some nodes with large number of cooccurance indicate large node value. Thus Mercedes, BMW and Audi become largest node in graph.

Betweenness Centrality

b <- betweenness(g_valued,directed=TRUE)
sort(b,decreasing=TRUE)[1:5]

      Mercedes          Volvo      Lady-Gaga   Taylor-Swift George-Clooney 
     13.886111       9.026190       7.609524       4.245635       3.159524 

g4 <- g_valued
V(g4)$size <- b*1.2#can adjust the number
plot(g4, layout = L, vertex.label=NA)

Briefly explain betweenness centrality and why nodes are more or less central in the network. #Betweenness centrality indicates how many times geodestic distance pass a node. The nodes at edge typically has small betweenness centrality and nodes at center has higher betweenness centraliy as nodes at center shortens geodestic distance at two sides.

Weighted Betweenness Centrality

wbtwn <- betweenness(g_valued,weights = E(g_valued)$CoOccurrences)
sort(wbtwn,decreasing=TRUE)[1:5]

     Lady-Gaga       Mercedes George-Clooney    Lamborghini          Volvo 
     49.623810      20.638095      20.500000      14.000000       9.090476 

wBtwnG <- g_valued
V(wBtwnG)$size <- wbtwn*.5 # adjust the number if nodes are too big
plot(wBtwnG, layout = L, vertex.label=NA, edge.width=sqrt(E(g_valued)$CoOccurrences)) 

What does the addition of weighted degree and edge information tell you about your graph? While considering the weight of betweenness, Lady gaga is the winner and is double higher than Mercedes. Edge at car brand cluster are thicker than celebrity cluster as its value of cooccourance is higer than celebrity cluster.

Closeness Centrality

c <- closeness(g_valued)
sort(c,decreasing=TRUE)[1:5]

         Mercedes             Volvo         Lady-Gaga      Taylor-Swift Jennifer-Lawrence 
       0.06250000        0.05555556        0.05263158        0.05000000        0.05000000 

g5 <- g_valued
V(g5)$size <- c*500  #can adjust the number
plot(g5, layout = L, vertex.label=NA)

Briefly explain closeness centrality and why nodes are more or less central in the network. closeness centrality measures the average length of geodesic paths beween one node to the rest of nodes. In this network, as most of nodes are directly connected, thus they all have high closeness centrality and value of closeness centrality is very close.

Weighted Closeness Centrality

wClsnss <- closeness(g_valued,weights = E(g_valued)$CoOccurrences)
sort(wClsnss,decreasing=TRUE)[1:5]

     Lady-Gaga       Mercedes George-Clooney     Emma-Stone Dwayne-Johnson 
    0.02777778     0.02380952     0.02272727     0.02272727     0.02272727 

wClsnssG <- g_valued
V(wClsnssG)$size <- wClsnss*1000 # adjust the number if nodes are too big
plot(wClsnssG, layout = L, vertex.label=NA, edge.width=sqrt(E(g_valued)$CoOccurrences)) 

What does the addition of weighted degree and edge information tell you about your graph? #While considering the weight, Lady Gaga has highest closeness centrality.

Eigenvector Centrality

eigc <- eigen_centrality(g_valued,directed=TRUE)
sort(eigc$vector,decreasing=TRUE)[1:5]

         Mercedes             Volvo         Lady-Gaga Jennifer-Lawrence      Taylor-Swift 
        1.0000000         0.8851732         0.7860427         0.7835956         0.7414936 

g6 <- g_valued
V(g6)$size <- eigc$vector*50 #can adjust the number
plot(g6, layout = L, vertex.label=NA)

Briefly explain eigenvector centrality and why nodes are more or less central in the network. #Eigenvector centrality shows that a node’s centrality is a function of its neighvors centrality. If a node is sorrounded by nodes with high strength, this node will also have high strength. Nodes at center typically has higher eigenvector centrality than nodes at edge.

Analysis

Choose the visualization that you think is most interesting and briefly explain what it tells you about a central node in your network. Discuss the type of centrality, and what that node’s centrality score tells you about the search co-occurrence network. #The most interesting network is the graph with clusterings, this graph shows the central node might be Mercedes as it has most connections with other nodes.Mercedes has highest (weighted)degree centrality, betweenness, closeness and eigenvector centrality. #The weighted centrality considers the factor of co-occurrence.However, by considering this factor, Mercedes doesn’t have highest weighted betweenness and closeness centrality and ranked second.

Briefly discuss an interesting difference between types of centrality for your network. #For betweenness and closeness centrality, the highest value is always Mercedes. While considering the weights, Lady Gaga will be higher than Mercedes.

Global Network Metrics with R

Compute the network centralization scores for your network for degree, betweenness, closeness, and eigenvector centrality.

centralization.degree(g_valued,normalized = TRUE)

$res
 [1] 10 11  3  8  9 10  7  8  9  8 14  7  9 12  9  8

$centralization
[1] 0.3416667

$theoretical_max
[1] 240

# Betweenness centralization
centralization.betweenness(g_valued,normalized = TRUE)

$res
 [1]  4.2456349  7.6095238  0.0000000  0.3789683  3.1595238  2.8123016  0.2361111  0.3789683  1.9611111
[10]  0.7277778 13.8861111  0.0000000  1.5444444  9.0261905  1.9250000  1.1083333

$centralization
[1] 0.1099541

$theoretical_max
[1] 1575

# Closeness centralization 
centralization.closeness(g_valued,normalized = TRUE)

$res
 [1] 0.7500000 0.7894737 0.5555556 0.6818182 0.7142857 0.7500000 0.6521739 0.6818182 0.7142857 0.6818182
[11] 0.9375000 0.6521739 0.7142857 0.8333333 0.7142857 0.6818182

$centralization
[1] 0.4826945

$theoretical_max
[1] 7.241379

# Eigenvector centralization 
centralization.evcent(g_valued,normalized = TRUE)

$vector
 [1] 0.7414936 0.7860427 0.2689500 0.6475531 0.6903315 0.7835956 0.5513788 0.6475531 0.7078242 0.6365116
[11] 1.0000000 0.5609145 0.7118733 0.8851732 0.7026684 0.6273066

$value
[1] 9.397791

$options
$options$bmat
[1] "I"

$options$n
[1] 16

$options$which
[1] "LA"

$options$nev
[1] 1

$options$tol
[1] 0

$options$ncv
[1] 0

$options$ldv
[1] 0

$options$ishift
[1] 1

$options$maxiter
[1] 1000

$options$nb
[1] 1

$options$mode
[1] 1

$options$start
[1] 1

$options$sigma
[1] 0

$options$sigmai
[1] 0

$options$info
[1] 0

$options$iter
[1] 3

$options$nconv
[1] 1

$options$numop
[1] 16

$options$numopb
[1] 0

$options$numreo
[1] 14


$centralization
[1] 0.3607736

$theoretical_max
[1] 14

Record the centralization score of each centrality measure. #degree centralization: 0.3416667 #betweenness centralization: 0.1099541 #closeness centralization: 0.4826945 #eigenvector centralization: 0.3607736 Briefly explain what the centralization of a network is. #Centralization of network indicates the difference among nodes in terms of number of nodes they connect to.

Compare the centralization scores above with the graphs you created where the nodes are scaled by centrality. Describe the appearance of more centralized v. less centralized networks. #The centralization scores are not high, it is reasonable there are two big cliques and a small clique. To get a more centralized network, the one of formation is like star, one node has connection to many other nodes, while other nodes has low connectivity. A less centralized network is like every node has similar connectivity.

Part 5. Power Laws & Small Worlds (20)

Power Laws

Networks often demonstrate power law distributions. Plot the degree distribution of the nodes in your base graph.

deg_distr <-degree.distribution(g_valued, cumulative=T, mode="all")
deg_distr

 [1] 1.0000 1.0000 1.0000 1.0000 0.9375 0.9375 0.9375 0.9375 0.8125 0.5625 0.3125 0.1875 0.1250 0.0625
[15] 0.0625

plot(deg_distr, ylim=c(.01,10), bg="black",pch=21, xlab="Degree", ylab="Cumulative Frequency") 

Test whether it’s approximately a power law, estimate log f (k) = log a − c log k. “This says that if we have a power-law relationship, and we plot log f (k) as a function of log k, then we should see a straight line: −c will be the slope, and log a will be the y-intercept. Such a “log-log” plot thus provides a quick way to see if one’s data exhibits an approximate power-law: it is easy to see if one has an approximately straight line, and one can read off the exponent from the slope.” (E&K, Chapter 18, p.546).

power <- power.law.fit(deg_distr)
power

$continuous
[1] TRUE

$alpha
[1] 1.467411

$xmin
[1] 0.0625

$logLik
[1] -13.96778

$KS.stat
[1] 0.3322858

$KS.p
[1] 0.0584186

plot(deg_distr, log="xy", ylim=c(.01,10), bg="black",pch=21, xlab="Degree", ylab="Cumulative Frequency")

Does your network exhibit a power law distribution of degree centrality? #Yes

Small Worlds

Networks often demonstrate small world characteristics. Compute the average clustering coefficient (ACC) and the characteristic path length (CPL).

transitivity(g_valued, type = c("average"))

[1] 0.7726208

# Characteristic path length (CPL)
average.path.length(g_valued)

[1] 1.408333

Compute the ACC and CPL for 100 random networks with the same number of nodes and ties as your test network.

accSum <- 0
cplSum <- 0
for (i in 1:100){
  grph <- erdos.renyi.game(numVertices, numEdges, type = "gnm")
  accSum <- accSum + transitivity(grph, type = c("average"))
  cplSum <- cplSum + average.path.length(grph)
}
accSum/100

[1] 0.5925709

cplSum/100

[1] 1.409

Based on these data, would you conclude that the observed network demonstrates small world properties? Why or why not? #The observed network has small world properties, as it has high local clustering coefficient and small average path length to other nodes.

Wrapping up

To complete the lab, make sure output/previews have been generated for each block of code. Then click the “Publish” button on the upper right hand corner of this screen and sign up for an RPubs account. Submit the URL of the published, completed lab on Canvas.