Tweet analysis

Pre-workshop (and some during workshop) analysis of tweets for the Flipping the Data Science course workshop at USCOTS 2015.

library(dplyr)
library(ggplot2)
library(knitr)

Attached is a tab delimited text file that contains 2,882 tweets that were extracted from Twitter. All tweets are from people who tweeted about a particular university. More details about how these were extracted will be discussed at the workshop. The columns are user ID, a unique tweet ID, the date/time of the tweet, the number of times the tweet was retweeted, and a text of the tweet. Note that this file has been de-identified.

Import or read the file into your preferred programming environment.

d = read.delim("USCOTS Flipped Data Science Workshop - Tweets.txt", 
               h = F, stringsAsFactors = FALSE)
varnames = c("userID", "tweetID", "date", "retweets", "tweet")
names(d) = varnames

Identify what message was retweeted the most?

d %>%
  filter(retweets == max(retweets))

##   userID      tweetID                           date retweets
## 1  UID29 5.944829e+17 Sat May 02 12:45:37 +0000 2015    45591
##                                                                                           tweet
## 1 rt  her royal highness the duchess of cambridge was safely delivered of a daughter at 834am

Calculate the proportion of tweets that were retweeted.

d %>%
  mutate(retweeted = ifelse(retweets > 0, "retweeted", "not retweeted")) %>%
  group_by(retweeted) %>%
  summarise(n = n()) %>%
  mutate(retweet_freq = n / sum(n))

## Source: local data frame [2 x 3]
## 
##       retweeted    n retweet_freq
## 1 not retweeted 2097    0.7276197
## 2     retweeted  785    0.2723803

Calculate the average number of retweets per user and explore the distribution of number of retweets.

avg_retweets <- d %>%
  group_by(userID) %>%
  summarise(avg_retweets = mean(retweets))

ggplot(avg_retweets, aes(x = avg_retweets)) +
  geom_histogram()

Identify at least two topic areas found in the text of these tweets (for example, many of the tweets discuss sports).

library(tm)
library(wordcloud)
library(RColorBrewer)

clean_tweets <- VCorpus(VectorSource(d$tweet))
clean_tweets <- tm_map(clean_tweets, stripWhitespace)
clean_tweets <- tm_map(clean_tweets, function(x)removeWords(x,stopwords()))

wordcloud(clean_tweets, scale = c(5,0.5), max.words = 100, 
          random.order = FALSE, rot.per = 0.35, colors = brewer.pal(8, "GnBu"))

Leo, WSSU, and Rihanna

What additional questions might you ask based on these data?

Who is tweeting about Rihanna and when?

to_match = c("rihanna", "riri")

riri_tweeters <- d %>%
  filter(grepl(paste(to_match, collapse="|"), tweet)) %>%
  mutate(day = substring(date, 1, 10)) %>%
  group_by(userID, day) %>%
  summarise(n())

kable(riri_tweeters)

userID	day	n()
UID179	Mon May 11	21
UID179	Sat May 09	3
UID179	Sun May 10	20
UID179	Thu May 14	18
UID179	Tue May 12	21
UID179	Wed May 13	21

All tweets mentioning rihanna or riri are by user 179 between May 9 and May 14.

What are the tweets about `today`? Is it a filler word that doesn’t tell us anything about the topic of the tweet?

Let’s take a look at 20 tweets…

today_tweets <- d %>%
  filter(grepl("today", tweet)) %>%
  select(tweet) %>%
  head(n = 20)

kable(today_tweets)

rt i love my ram family and i appreciate all of the birthday wishes on today
concealing your feelings today seems like your smartest course more for scorpio
congratulations to the rams football team and cross country team for a job well done on today
as of 500pm today there are only 12 tickets remaining for the pajama party period
today was the first day all players returned stay tuned and i will let you know what the rams are working with
wear red and be apart of the the go red picture at the clocktower from 315 to 330 pm today
pound 4 pound challenge is going down in the cafe from 12 to 3 today
get your student advance tickets today before prices go up at the door
ght family feud tonight 900 in the ght courtyard sign up for a team by 4pm today see you there
if u see ra bridget today make sure you wish her a happy birthday
get 10 off your spartan race at spartan race today httptcoaashyfuqrh
rt #ramfest is next week at #wssu today from 27 there’s an event for the hip hop heads at thompson center #djbattle #mcbat
rt rt so to lady c #hiphopisrealityfestival from 2pm7pm #wssu #today
greek cookout today be there
stroll off today be there
how was church today
hey how you doing today beautiful
reminder sga amp royal court applications are on orgsync deadline today by 5pm in tc 305 you will need a resume and unofficial transcript show some #rampride today espn 2 #wssu v valdosta state in florence al at 1pm for the ncaa division ii football national championship
come here about proposed increases in your tuition at tuition amp fee committee presentations today rjr 136 12302pm amp 7pm830pm

Looks like we can just remove the word today and make the word cloud again:

clean_tweets <- tm_map(clean_tweets, removeWords, "today")

wordcloud(clean_tweets, scale=c(5,0.5), max.words=100, random.order=FALSE, rot.per=0.35, colors=brewer.pal(8, "GnBu"))

Who is `leo`?

What are tweets about `leo` look like?

leo_tweets <- d %>%
  filter(grepl("leo", tweet)) %>%
  select(tweet) %>%
  head(n = 20)

kable(leo_tweets)

invest in the human soul who knows it might be a diamond in the rough mary mc leod bethune you receive a magical message from the universe today that ult more for leo
you receive a magical message from the universe today that ult more for leo
maintaining a positive perspective can be nearly impossible if more for leo
the fickle winds of change are blowing in like spring breezes more for leo
you could easily convince yourself you are doing the right thi more for leo
you want to take time off to play today but you might have too more for leo
you might think you have too much on your plate and are worrie more for leo
you might wish you could kick back and let everyone continue o more for leo
you could grow annoyed with people being so nice to you now be more for leo
you may be itching to do something totally unusual today and t more for leo
you might not be forthcoming with your emotions today but the more for leo
you are playing the lead role in your own personal movie today more for leo
the winds have changed and you are breathing easier today you more for leo
your actions may produce lasting results today without a lot o more for leo
you think you are ready for nearly anything that might happen more for leo
a coworker or friend may be exaggerating the truth today but more for leo
achieving your goals today is much more likely if you are real more for leo
blind optimism might run away with your day and lead you down more for leo
even if you aren’t the overly jealous type someone’s actions more for leo

Seems like leo often appears as part of more for leo. Still no idea who leo is.

Who is tweeting about `leo`? and when?

leo_tweeters <- d %>%
  filter(grepl("leo", tweet)) %>%
  group_by(userID) %>%
  summarise(count = n()) %>%
  arrange(desc(count))

kable(leo_tweeters)

userID	count
UID99	229
UID120	13
UID89	13
UID127	1
UID73	1

User 99 tweeted the most about leo, let’s take a look at a few of their tweets

user99 <- d %>%
  filter(userID == "UID99") %>%
  select(tweet) %>%
  head(n = 20)

kable(user99)

you receive a magical message from the universe today that ult more for leo maintaining a positive perspective can be nearly impossible if more for leo the fickle winds of change are blowing in like spring breezes more for leo you could easily convince yourself you are doing the right thi more for leo you want to take time off to play today but you might have too more for leo you might think you have too much on your plate and are worrie more for leo you might wish you could kick back and let everyone continue o more for leo you could grow annoyed with people being so nice to you now be more for leo you may be itching to do something totally unusual today and t more for leo you might not be forthcoming with your emotions today but the more for leo
you are playing the lead role in your own personal movie today more for leo the winds have changed and you are breathing easier today you more for leo
your actions may produce lasting results today without a lot o more for leo you think you are ready for nearly anything that might happen more for leo
a coworker or friend may be exaggerating the truth today but more for leo
achieving your goals today is much more likely if you are real more for leo blind optimism might run away with your day and lead you down more for leo even if you aren’t the overly jealous type someone’s actions more for leo
relationship stress might build to a crescendo today especial more for leo
it’s nearly impossible to be sure if you’re growing closer to more for leo

Looks like astrology?

Tweet analysis

Mine Cetinkaya-Rundel

May 28, 2015

Import or read the file into your preferred programming environment.

Identify what message was retweeted the most?

Calculate the proportion of tweets that were retweeted.

Calculate the average number of retweets per user and explore the distribution of number of retweets.

Identify at least two topic areas found in the text of these tweets (for example, many of the tweets discuss sports).

What additional questions might you ask based on these data?

Who is tweeting about Rihanna and when?

What are the tweets about `today`? Is it a filler word that doesn’t tell us anything about the topic of the tweet?

tweet

Who is `leo`?

What are tweets about `leo` look like?

tweet

Who is tweeting about `leo`? and when?

tweet

Tweet analysis

Mine Cetinkaya-Rundel

May 28, 2015

Import or read the file into your preferred programming environment.

Identify what message was retweeted the most?

Calculate the proportion of tweets that were retweeted.

Calculate the average number of retweets per user and explore the distribution of number of retweets.

Identify at least two topic areas found in the text of these tweets (for example, many of the tweets discuss sports).

What additional questions might you ask based on these data?

Who is tweeting about Rihanna and when?

What are the tweets about today? Is it a filler word that doesn’t tell us anything about the topic of the tweet?

tweet

Who is leo?

What are tweets about leo look like?

tweet

Who is tweeting about leo? and when?

tweet

What are the tweets about `today`? Is it a filler word that doesn’t tell us anything about the topic of the tweet?

Who is `leo`?

What are tweets about `leo` look like?

Who is tweeting about `leo`? and when?