---
title: "Charlotte Protest -- AstroTweets"
output: 
  flexdashboard::flex_dashboard:
    storyboard: true
    social: menu
    source: embed
---

```{r setup, include=FALSE}
library(flexdashboard); library(dygraphs); library(dplyr); library(xts); library(lubridate); library(shiny); library(quanteda); library(RColorBrewer); library(DT)

tweets <- read.csv("./astrotweets.csv", stringsAsFactors = F)

tweets$estTime <- ymd_hms(tweets$postedTime, tz = "EDT")

agg.time <- tweets %>% group_by(estTime, type) %>% summarise(Count = n())
agg.time$estTime <- strptime(agg.time$estTime, "%Y-%m-%d %H")

library(reshape2)
aqm <- melt(agg.time, id=c("estTime","type"), measure.vars = c("Count"), na.rm=TRUE)
count <- dcast(aqm, estTime ~ type, fun.aggregate = sum)
count$estTime <- with_tz(count$estTime, tz = "EDT")

tweet.time <- xts(
  x = count[,2:4],
  order.by = count$estTime,
  tz = "EDT"
)

myCorpus <- corpus(tweets$summary)
  docvars(myCorpus, "Organization") <- as.character(tweets$type)
  dfm <- dfm(myCorpus, groups = "Organization", ignoredFeatures=c(stopwords("english"),"t.co", "https", "rt", "amp", "http", "t.c", "u"), ngrams=c(1,2))
  dfm <- trim(dfm, minDoc = 2)
```

### The organizations' tweets seem to be event-driven, rather than spread out.

```{r}
dygraph(tweet.time,  main = "Charlotte Protest Astro-Tweets") %>%
  dyOptions(colors = RColorBrewer::brewer.pal(8, "Dark2")) %>%
  dyRangeSelector(dateWindow = c("2016-09-20 00:00:00", "2016-09-27 00:00:00")) %>%
  dyAnnotation("2016-09-20 20:00:00", text = "A", tooltip = "KLS Shot") %>%
  dyShading(from = "2016-09-21 00:00:00", to = "2016-09-21 08:00:00", color = "#FFE6E6") %>%
  dyShading(from = "2016-09-22 00:00:00", to = "2016-09-22 08:00:00", color = "#FFE6E6")
```

*** 

The shooting occurred around 4pm on Wednesday, September 20th. I've shaded in red the two nights of protest.

The ACLU's tweets occurred around 3pm the next day (Thursday, Sept 21) with most of the delayed responses being retweets of the initial tweets.

On Friday, Moms Rising's tweets were started around 6pm that night. 

Lastly, Color of Change tweets followed days after to encourage the police to release tapes regarding the incident.


### The profiles seem to indicate that most users are supporters, as they tend to include issues related to the organization's goals.

```{r}
  suppressWarnings(plot(dfm, comparison = T, scale=c(2,0.5), random.order = F, rot.per=0.1, title.size = 1,
                  colors=brewer.pal(8, "Dark2")))
```

*** 

This slide compares the profile summaries (open-ended text) of the users who tweeted on behalf of the three organizations. It seems that most users tend to share interests with the goals of each organization, leading me to suspect that the ACLU and Mom's Rising's users are advocates for each organization, not necessarily paid users. 

One exception is Color of Change, in which there's a wider variety of words and it's not clear if these are advocates or paid users.

For example, the ACLU's tweets come from individuals focused on human rights and advocacy for a variety of issues. Moreover, the list of additional words combine a variety of characterstics that can be inferred as related terms unique to the ACLU users: "progressive", "#uniteblue", "politics", "social_justice","music","photographer".

On the other hand, the Moms Rising users' most distinctive words are related to mothers: "wife", "mom", "lady", "mother", "kids". 

You can look at the individual cases on the next slide.

It's important to note that nearly one-third of the tweets do not have profile information filled out. Therefore, this analysis only covers the two-thirds of users' with a profile summary, not necessarily all users.

### Friends and Followers Distribution

```{r}
library(highcharter)
highchart() %>%
  hc_add_series_boxplot(x = tweets$followersCount, by = tweets$type, name = "Followers", outliers = F) %>%
  hc_add_series_boxplot(x = tweets$friendsCount, by = tweets$type, name = "Friends", outliers = F)
  
```

*** 

This plot shows the distribution of friends (users whom the user follows) and followers (users who follow the user) for each of the three organization's related-tweets.

The users who posted the Mom's Rising organization tend to have wider Twitter networks as they (on average) tend to have more followers and friends than users who posted tweets on behalf of ACLU and Color of Change.


### Color of Change's Tweets are disportionately from Twitter's web client.

```{r}
library("treemap"); library(dplyr);
tweets2 <- tweets %>% group_by(type, generator.displayName) %>% summarise(Count=n())

tm <- treemap(tweets2, index = c("type","generator.displayName"),
              vSize = "Count", 
              type = "index",
              draw = F)

hc_tm <- highchart() %>% 
  hc_add_series_treemap(tm, allowDrillToNode = TRUE,
                        layoutAlgorithm = "squarified",
                        name = "tmdata") %>% 
  hc_title(text = "Tweet Counts by Organization and Device Used") %>% 
  hc_tooltip(pointFormat = "{point.name}:

             Count: {point.value:,.0f}
")

hc_tm
```

*** 

This plot is a tree-map of the number of tweets by the device used to post the tweet.

The treemap is separated into three groups, each group is by the three organizations: Color of Change (pink), ACLU (orange) and MomsRising (blue).

You can click on any of the organizations to drill down to the organization-level.

The key difference is that Color of Change's tweets disproportionately come from the Twitter Web Client (i.e. through a normal browser) as compared to the other organizations.

78% of Color of Change tweets come from the Web Client while only do 45% of ACLU's and 54% of MomsRising's.

It's not clear why this is the case.


### The underlying data

```{r}
temp <- tweets[,c("type","verb","postedTime","body","preferredUsername","displayName","summary",
                  "actor.location.displayName","friendsCount","followersCount","statusesCount")]
datatable(temp, rownames = F)  
```