Which campaign had more bots tweeting for them, T A M A M or DEVAM?

Intro:

Yesterday, Erdogan said ‘If my nation says “enough (tamam in Turkish)” then I will stand aside’. This comment sparked a weirdly pleasing reaction on Turkish Twittersphere; Turkish Twitter users started tweeting ‘T A M A M’ (roughly enough in English), making it the most trending topic worldwide. As expected, pro-Erdogan accounts started a counter campaign tweeting DEVAM (not enough, or continue in English). At the time of writing this (19:30 GMT 08/05/2018), both T A M A M and DEVAM was trending worldwide; while the former campaign attracted 884K tweets, the latter attracted 214K tweets.

Within the first minutes following the launch of the DEVAM campaign, I have observed many bot accounts tweeting for the DEVAM campaign:

Devam Bots-1

Devam Bots-2

Devam Bots-3

This was not a scientific observation but an anecdotal one. Moreover, bots were not only tweeting for the DEVAM campaign, they were also tweeting for the T A M A M campaign. For an objective observer, it was obvious that there were bot accounts tweeting for both campaigns. Equality of arms. Also, it was quite possible to observe the well-known limitations of the social media research: Turkish Twittersphere is an active playfield of the bots. Bot-masters are getting paid to influence the public opinion. Twitter is not representative of the general population. No surprises here.

What is the problem then?

Obviously, Erdogan and AKP was not happy about the amazing public support the T A M A M campaign received. This was expected. In fact, I am amazed that Erdogan did not throttled access to Twitter yet.

Pro-Erdogan media hastily published stories trying to underplay the public support for the T A M A M campaign by conflating the facts. Pro-Erdogan media outlet Yeni Akit argued that it was thousands of bots who were tweeting for the opposition’s T A M A M campaign, whereas it was the ‘people’ who tweeted for the DEVAM campaign. The people! How Akit moved to question the ability of the opposition to govern is beyond me. But again, they are doing their jobs so no further comment needed, really.

Akit’s above claims sounded ridiculous. I wanted to find out what was really going on. After all, ‘You are entitled to your opinion. But you are not entitled to your own facts’, as Moynihan put it.

Yeni Akit:“Bots say ‘enough’, the people say ‘keep going’ to erdogan”

The Hyphotheses

Therefore, I wanted to see which campaign had more bots tweeting for them using traditional data science methods. Rather than trying to test/disprove Yeni Akit’s dogmatic hypothesis, I have formed rather traditional hypotheses which can actually be tested with data.

\(H_{1}\)- Both T A M A M and DEVAM campaigns had bot accounts tweeting for them.

This hypothesis is pretty straightforward. It can be illustrated by using descriptive stats only. Histograms of user bot probability score distributions will also be used to illustrate there were bot accounts tweeting for both campaigns.

\(H_{2}\)- Bots were more active in the DEVAM campaign than the T A M A M campaign.

It is possible to test which campaign had more bots tweeting for it after calculating a bot probability distribution for each user in both campaigns. I’ll test this hypothesis by comparing distributions of the bot scores for both campaigns using the appropriate statistical test.

Methodology and Results

First step was to collect data. I scraped tweets from both campaigns using “DEVAM” and “T A M A M” using Twitter’s standard search API. Since I did data collection in certain point of time, there will be a certain bias associated with the cross-sectional nature of the data collection. However, to minimise the temporal bias between both campaigns, I collected data consequently. First the DEVAM and then the T A M A M tweets were collected. Data collection was started at 2018-05-08 20:35:42 and completed at 2018-05-08 20:57:23. In order to stay inside the Twitter’s API rate limits, I queried most recent 15000 tweets from both campaigns (rather than top tweets which are more likely to be shared by human users). I also did not include retweets as I was not interested in information propagation.

Interestingly, Twitter API returned 13400 DEVAM tweets and 14999 T A M A M tweets. In order to get even distribution sizes for both campaigns, I randomly dropped 1599 tweets from the T A M A M campaign and ended up with two datasets pertaining to both campaigns with n=13400.

Then, I have calculated bot probability scores for each user in both campaigns. You can see if you’re classified as a bot or not yourself here. Interesingly,

Results

Conclusion