Sentiment Analysis of Trump Tweet Data

Chad Allison and Summer Hyland

SOCI290.001.FA21 - Dr. Scott Duxbury - 17 November 2021

Introduction

The general topic that we have decided to study is political polarization. More specifically, we have chosen to study this topic through the lens of the Trump presidency using a collection of Trump’s tweets while he was in office. This topic is relevant as Trump was a controversial figure during his presidency and continues to be controversial even now that he has been out of office for nearly a year and has been banned from Twitter. Trump’s use of social media influenced the political landscape as well, making it much more common for political figures to be active on social media sharing their opinions and using it as a news source. The prevalence of social media has likely contributed to an increase in political polarization as it becomes easier to expose oneself to similarly minded people and in turn, potentially become more hostile toward others who may see things differently. Controversial and polarizing content is easier than ever to share and engage with on social media, which leads to our research question: is there a relationship between the tone of Trump’s tweets and the level of engagement that those tweets receive?

Background

As mentioned above, we are studying political polarization from the perspective of former President Trump’s tweets while in office. In their study published in the Signs and Society journal, Galen Stolee and Steve Caton discuss how Trump communicated with his base via Twitter and study “Trump’s use of Twitter as a speech practice” (Stolee & Caton). Another study, conducted by Bryan Anderson at Elon University, explores how Trump used Twitter to “[express] himself in front of a wide audience” (Anderson) and the tactics Trump employed to persuade his audience. Similar to how we chose to analyze Trump’s tweets, Stephan Lewandowsky et. al. discuss how Trump was reliant on social media and used it to divert attention from undesirable topics onto others, “[presenting] evidence suggesting that President Trump’s use of Twitter diverts crucial media … from topics that are potentially harmful to him” (Lewandowsky et. al.). Trump’s ability to use social media in the manner he did certainly played a role in building his support base and was a key factor in winning the 2016 election.

Another study, conducted by Paromita Pain and Gina Masullo Chen, takes a slightly different approach to studying Trump’s tweets and analyzes those from 2009 until he took office in 2017. They state that Trump portrayed himself as “a political outsider who can alone save America” (Pain et. al.), providing insight into how Trump used social media to portray himself as a savior of some sort; a figure deserving of a following. In a study published in the Digital Scholarship in Humanities journal, Jacques Savoy analyzed the “stylistic and rhetorical aspects of Trump’s tweets” using the tweets posted by Obama and the White House as a comparison. In general, Savoy concluded, “Trump’s rhetoric can be characterized by a definite and negative tone without many references to people. His judgments are definitive and emphasized with grandiose adjectives” (Savoy). This analysis gives us an overview of how Trump communicates on social media. If his tweets are predominantly negative in tone and contain many grandiose sentiments toward himself and his ideas, this indicates that his tweets likely only consist of information that is helpful to him, and therefore potentially polarizing.

In a study conducted by Erendira Morales, Cindy Shultz, and Kristen Landreville, they focus on the impact of Trump’s tweets in the media. Over 3 weeks, researchers analyzed news stories from four large news sources including, Fox News, ABC, NBC, and CNN. They found the largest correlation between Fox News and Trump’s Twitter account. Overall, his tweets were mentioned in the media about 10% of the time and Fox News was referenced in over half of Trump’s tweets during the 3 weeks. Trump himself was mentioned in 40% of the news stories. While this is only a snapshot in time of the prevalence of Trump and his tweets in the media, it indicates an increasing trend toward how much social media, and Twitter specifically, is affecting what journalists report on. It also shows how pervasive Trump’s ideology was permeating through national media and therefore the country (Morales, et al.).

In a study published in the Critical Studies in Communication journal, Brain Ott explores how drastically public discourse has changed in the “Age of Twitter” (Ott) using Trump’s tweets as a case study. Ott argues that “Twitter ultimately trains us to devalue others, thereby, cultivating mean and malicious discourse” (Ott). He illustrates this by highlighting how Twitter’s character limit hinders the ability for users to participate in factual and more sophisticated communication. The ease with which one can use Twitter leads to impulsive tweets with little regard for any consequences. Ott also discusses how the informality of Twitter allows for depersonalized interactions which almost permit users to post demeaning and degrading tweets about any topic or group of people. Trump capitalized on the pitfalls of communicating via Twitter to relay his own highly controversial messages. Ott refers to Trump’s simple lexicon, the negative connotation of many of his tweets, and the repeated use of exclamation points and fully capitalized words as, “stylistic practices [that] reinforce the negative sentiment of his Tweets and heighten their emotional impact, which is, in turn, reflected in the intense emotion of his followers, a phenomenon scholars refer to as ‘emotional contagion’” (Ott). Ott’s analysis of how Trump uses Twitter demonstrates why our research question is important to investigate as this particular mode of social media can be easily used to disseminate politically charged and controversial messaging which could contribute to further polarization. We hypothesize that there is a correlation between negative tone or polarizing tweets and increased likes or retweets. If we find this to be true, similar to why misinformation spreads, it could indicate that people may be drawn to the novelty, scandal, or conflict of polarizing politicians which poses some challenges in terms of finding solutions for narrowing the divide between parties.

Data

The data that we are using to conduct our analyses is a collection of all of Trump’s tweets spanning from February 5, 2017 to May 18, 2018. The key variables included are the time and date of each tweet, the actual text of each tweet, information on if the tweet is a reply to another tweet or if it is a quoted tweet, the number of favorites and retweets, and any hashtags or links included in the tweet. The text of each tweet is the most important component of our analysis, as we will be inspecting the tone of Trump’s tweets and trying to identify relationships between tone and engagement with the tweets, measured by the number of favorites or retweets. We will also explore the times and dates of the tweets and see if we can identify any tendencies for when Trump tweets the most or if there is any time where his tweets received the most engagement. The structure of the data is perfect for the analysis we want to conduct: the ability to analyze the text of Trump’s tweets will allow us to identify trends in his usage of Twitter as a means of communicating and will be insightful for social media’s role on political polarization. Particularly considering that Trump was a particularly polarizing public figure, having tweet data for him specifically is certainly a strength of the data at hand.

We will be conducting sentiment analysis to quantify the tone in Trump’s tweets. Sentiment analysis is a process of detecting different emotions in textual data, whether those emotions be strictly positive or negative, or even more complex emotions like hope and fear. This is an appropriate tool for addressing our research question as it will give us the ability to quantify the tone of a tweet and compare that to other quantifiable figures that already exist, which are favorites and retweets in our case. With a figure like Trump who has a large base who is very passionate about their support for him, emotion certainly plays an immense role in creating and maintaining that support base, and we have the opportunity to use sentiment analysis to identify how Trump is using that emotion in his favor and the tactics he may be deploying to foster support and engagement. We will also visualize tweets with respect to time and try to identify if there are times where Trump is more likely to tweet or if a tweet is more likely to receive more engagement. The level of specificity we have in our data gives us great flexibility with our analysis.

Results

The first thing we sought to explore was ways to see how engagement with Trump’s tweets evolved over the period that we have data for; we wanted to see if engagement increased, decreased, or stayed the same over the roughly fifteen months of tweet data we have access to. To accomplish this, we created a line graph that visualizes engagement over time with color-coding to identify which engagement metric we are using.

From this plot, we observe two things. Firstly, favorite counts are consistently significantly higher than retweet counts. This is likely a result of favorites being somewhat more private than retweets, as users are not explicitly shown when someone they follow likes a tweet, whereas users are explicitly shown when someone they follow retweets a tweet. Retweets may also be more closely aligned with an endorsement of a tweet’s content than favorites - a user liking Trump’s tweet may just mean they find it interesting or entertaining while retweeting a Trump tweet could mean that the user agrees with the content of the tweet. Secondly, we observe that there does seem to be a gradual increase over time in overall engagement with Trump’s tweets. We also observe that engagement appears to increase at a higher rate during a period starting around June 2017 and ending around August 2017; this engagement spike is something we will explore further later in our analysis.

Next, we wanted to see if different times of day were more popular for Trump to tweet or more popular for engagement. To accomplish this, we created a plot that shows the average number of favorites by the hour of the day along with also showing how often Trump is tweeting during that hour.

From this graph, where the y-axis represents the number of favorites a tweet received, we can see there is a general tendency for tweets to be engaged with more between the hours of 11 AM and 2 PM. It is worth noting, though, that the most popular hour with regard to favorites is actually 4 AM; however, we also observe that the size of the points at the top of the bars represents the number of tweets during that hour, and we can see it is rather small for the 4 AM bar where they are larger during that midday window. This smaller sample size means the average will be more easily influenced by outliers, which is likely the case with the high 4 AM bar.

For our text analysis, we chose to use the NRC lexicon to evaluate the tone of tweets. We chose this lexicon over others because of the ability to identify emotions other than just positivity or negativity; we wanted to explore a range of different emotions and see if engagement differs between them. We chose to explore the following tones: trust, fear, positivity, negativity, anger, and joy. Some of these tones are similar to each other, but we chose to do all six of these as we wanted to see if trends existed in any of them that might not exist in the others. The NRC lexicon is the only lexicon at our disposal that allows us to do this; with the Bing and AFINN lexicons, we would be limited to only looking at positivity or negativity without being any more specific than that. And while that could still yield an interesting analysis, the NRC lexicon enables us to explore more complex emotions like fear and anger which will greatly enhance our analysis and allow us to fully explore our research question. Our analysis yielded the following table.

The different tones are listed on the leftmost column of the table, with the two rightmost columns corresponding to the average number of favorites or retweets for each of the tones. The other column indicates if that number of favorites/retweets corresponds to the average when that sentiment was detected or if it was not detected; this column also details the raw increase in favorites/retweets when that tone is detected as opposed to when it is not detected along with the percentage increase. In terms of the largest raw increase in favorites/retweets among the different tones, the negative tone takes the cake. Negatively-toned tweets received over 87,000 favorites and over 21,000 retweets on average (the largest values among all the tones), while tweets where negativity was not detected only received roughly 67,000 favorites and nearly 17,000 retweets on average (the smallest values among all tones). These are also both the largest percentage increases among the different tones. Looking at average favorites specifically, the smallest raw increase and percentage increase belongs to tweets where fear was detected: tweets with fear detected received nearly 81,000 favorites on average, while still receiving nearly 74,000 favorites when fear was not detected. Still focusing on specifically average favorites, tweets where trust was detected received nearly 84,000 favorites on average and only around 69,000 favorites when it was not detected, corresponding to a 21% increase in engagement, the second-largest among all tones. When it comes to the drastic increase in engagement with negatively-toned tweets, it seems as if anger has more to do with it than fear; tweets where anger was detected received nearly 86,000 favorites on average as opposed to roughly 75,000 when it was not detected, a 14% increase (and a 19% increase with retweets). We will continue to investigate this possibility.

Looking further into the sentiments of Trump’s tweets over time, we produced the following graph visualizing the average sentiment value over time where each curve represents one of the different tones in our analysis.

It certainly seems as if all of the tones follow a similar distribution, with all of their average values seemingly increasing slightly around September or October of 2017. Some of them, however, appear to have increased more dramatically, including negativity and anger as we have previously discussed. We also recalled that when we visualized engagement with Trump’s tweets over time, there was a similar trend where engagement increased more sharply for some time around June or July of 2017.

This led us to one of our final pieces of analysis, where we intended to determine if there was a significant increase in the sentiment value of anger-driven tweets that may have had something to do with that increase in engagement. To do this, we utilized Google’s CausalImpact R package to determine if there was a significant increase in anger-driven tweets during the period where engagement saw an abnormal increase. Doing this yielded the following graph.

The top graph contains two lines: the solid black line, which is the actual experienced average sentiment value of anger-driven tweets from February 5, 2017, to August 1, 2017, and the dotted blue line which represents the predicted sentiment value of anger-driven tweets over the same period. The predictions are made using the sentiment value data for other tones (trust, fear, etc.). The dotted vertical line represents June 10, 2017, where we first noticed engagement values beginning to increase more rapidly. The bottom graph represents the cumulative effect of the actual sentiment value of the anger-driven tweets in what we call the post-intervention period (June 10 to August 1) minus the predicted sentiment value, with the shaded blue area being a 95% confidence interval. The lower bound of this confidence interval is greater than zero at the end of this post-intervention period, which corresponds to our small p-value of 0.009. Interpreting this, we can say the positive increase in average sentiment value during this post-intervention period is statistically significant and unlikely to be due to random fluctuations. In context, we can say we have a statistically significant difference in the average sentiment value of Trump’s anger-driven tweets in the period before June 10, 2017, and the following couple of months, where we saw a spike in engagement with his tweets. This gives us good reason to believe that this increase in engagement could be a result of increased anger and hostility in Trump’s tweets. However, we must take all of this information in with a grain of salt; causal inference is a rather difficult subject, especially when working with observational data. The inherent problem is that with causal inference, we want to compare what happened to what might have happened otherwise, which of course we can never know for sure. Despite these limitations, we still find it intriguing that we did find a statistically significant increase in average anger levels in Trump’s tweets, we just have to proceed with caution when drawing conclusions based on our findings.

Conclusion

Looking at our results, we can observe a few things. First, we notice that throughout the time we have tweet data for former president Trump, engagement with his tweets gradually increased for both favorites and retweets. There seems to be a slight bump around June or July of 2017 where engagement increased more rapidly for some time, which would be a good follow-up analysis to try and understand why this might be. Second, we observe that the most popular hours of the day for Trump to tweet were mid-day, usually between 11 AM and 2 PM. These were not the only times he tweeted, but they were consistently the most popular. These were also the hours of the day where tweets tended to receive the most engagement, and we discussed briefly in the results section above how some of the other more popular hours for engagement could be a result of outliers due to the smaller sample size of tweets. Third, when conducting our sentiment analysis on Trump’s tweets, we observe that his negatively-toned tweets appear to have the biggest difference between average engagement when a tweet was classified as negative instead of non-negative. There was also a noticeable difference in engagement for the trust sentiment. These findings are intriguing in the context of political polarization, as it could indicate that Trump’s supporters are engaging more with negative content, some of which could be hostility toward those who disagree with Trump and his supporters. A potential limitation in our study is our trust in the lexicon we are using and how we are classifying tweets as having a sentiment or not; the word-in-a-bag approach of sentiment analysis could potentially lead us in the wrong direction in some cases, and we need to keep that in mind when drawing conclusions or generalizing to a larger population. Another limitation in our study is the use of observational data for our causal inference; the data we have access to was not collected for causal inference, and therefore we have to proceed with caution in conducting that analysis and when drawing conclusions from it. If we relate these findings to governmental policy, it may be enlightening to know that social media is a growing source of news and communication and we may be able to expect future political figures to be more active on social media given how Trump was able to make it work in his favor. This information could potentially shape governmental policy on things such as censorship, as it is extremely easy to share and engage with content on social media that may be either untrue or overly hostile (or both). Concerning ethics, it is important to consider the following principles: respect for persons, beneficence, justice, and respect for the law and public interest. In our analysis, we did not expose any individual’s retweets or use information that could negatively affect them in the future, therefore minimizing any risks to users who favorited or retweeted Trump’s posts. In regards to justice and respect for the law and public interest, our research and policy considerations benefit the populous at large. Engagement with social media and its effect on political polarization is a serious and current issue that affects all Americans; and one that needs careful, educated, and evidence-based public discourse to find a solution. One ethical quandary we must address is the potential for our research to encourage the censorship of free speech. If negatively toned tweets receive more engagement, one may jump to the conclusion that those tweets should be taken out altogether to prevent the increasing political polarization in our country. While informed censorship of misinformation may help, an important question to consider is who gets to censor misinformation? We need to be careful with how we present our findings and continue to encourage the spread of truth and facts versus championing invasive censorship of the people, their ideas, and the constitutional right to free speech.

Works Cited