Introduction

In this series, I’ll replicate the steps taken in R for my MSc dissertation at Peking University, submitted in June, 2020. Over the coming weeks, I’ll continue to update this page and publish other segments.

This section will detail the process of scraping ads and steps to wrangle the data so it is usable. Other sections will deal with language detection, topic modeling/keyword extraction, correlation & paired sample t-tests, and text scaling.

For other parts to this series, or if you’re interested in this template & how I’ve put it together, check out my guide here.

Abstract of my thesis:

Despite its notoriety, political advertising on social media has steadily gained prominence because of its potential reach and proven efficacy. Its controversies share the common theme of misuse or potential for misuse by malevolent actors who may use the platforms’ capabilities to subvert political processes. Less attention has been paid to the platform itself, and how it may fundamentally change the way an essential social service is distributed: the dissemination of truthful political information and solicitation during elections. This paper investigates the mechanisms of the Facebook advertising platform and how it affected political advertising in the 2019 Canadian federal election. It finds and measures two effects: Planned Distortion (or the deliberate segmentation of the population for targeting) and Algorithmic Distortion (the interjection of automatic biases in the Facebook platform that determine the final reach and efficacy of ads). This paper briefly touches on two social dimensions in which these processes may generate negative externalities: language policy and accuracy in political coverage.

Keywords: public opinion, social media, political advertising, computational politics, spatial politics

Please note: the code used below was used between September 2019 to June 2020. I expect several key aspects of Facebook’s API and the packages I’ve used have changed, improved, or even been removed.

Scraping Data from Facebook with R

First, we set our working directory and load the necessary packages.

# Set working directory
setwd("~/Documents/RPubs/PKUThesis")

# Loading packages; install if missing
if (!require(tidyverse)) install.packages(tidyverse)
if (!require(lubridate)) install.packages(lubridate)
library(tidyverse)
library(lubridate)

Next, we establish a connection with the Facebook API, set our query, and provide our access token.

Defining the link to Facebook’s API:

my_link <- "https://graph.facebook.com"

Next, we define the variables we want to query for. For a full and updated list of possible parameters, please see the Facebook Ad Library API page.

search_fields=c( #We'll search for:
                "ad_creation_time", #The UTC date and time when someone created the ad.
                "ad_delivery_start_time", #Date an time when an advertiser wants Facebook to start delivering an of the ad.
                "ad_delivery_stop_time", #The time when an advertiser wants to stop delivery of their ad. If blank, Facebook runs the ad until the advertiser stops it or they spend their entire campaign budget.
                "ad_creative_body", #The text which displays in the ad. Typically 90 characters.
                "page_id", #ID of the Facebook Page that ran the ad.
                "page_name", #Name of the Facebook Page which ran the ad.
                "currency", #The currency used to pay for the ad.
                "spend", #A string showing amount of money spent running the ad as specified in currency.
                "demographic_distribution", #The demographic distribution of people reached by the ad. Provided as age ranges and gender.
                "funding_entity", #A string containing the name of the person, company, or entity that provided funding for the ad.
                "impressions", #A string containing the number of times the ad created an impression.
                "region_distribution" #Regional distribution of people reached by the ad.
                ) %>% 
  stringr::str_c(., collapse=", ")

Now, we define the token provided by Facebook to gain access to the Ad Library API. I was granted access tokens once I verified my identity and location, created a Facebook Developer account, and added a new app (for more details, see: the Facebook Ad Library API page).

my_token = "[INSERT TOKEN]"  #Copy and paste your access token over [INSERT TOKEN]

The following is one method of scraping the returns to our API query. This is a method I’ve adapted from Roland Schmidt (2019).

# This retrieves the data from the first page of data the API returns
page_one_response <- GET(my_link, path = "v4.0/ads_archive", query = list(access_token = my_token, 
    limit = 100, ad_active_status = "ALL", search_terms = "''", fields = search_fields, 
    ad_reached_countries = "CA"))
page_one_content <- content(page_one_response)
# This converts that data into a tibble data frame
x <- tibble(data = page_one_content$data)
df_imp <- x %>% unnest_wider(data)
# This gets the link referring to the next page
next_link <- page_one_content$paging$`next`
# Iterate over all pages until there is no further pages (i.e., all data returned
# are scraped)
while (length(next_link) > 0) {
    
    next_response <- GET(next_link)
    next_content <- content(next_response)
    
    y <- tibble(data = next_content$data)
    df_next <- y %>% unnest_wider(data)
    
    df_imp <- bind_rows(df_imp, df_next)  #df_imp is the data frame comprised of ALL ads returned for Canada
    
    next_link <- next_content$paging$`next`
    
}

Data Wrangling

# New table with only ads created in Canada from 30 June until 21 October
# (election date = 21 October)
all_ads_in_period <- df_imp %>% mutate(ad_creation_time = lubridate::ymd_hms(ad_creation_time) %>% 
    lubridate::date(.)) %>% filter(ad_creation_time %within% interval(as.Date("2019-06-30"), 
    as.Date("2019-10-21")))

# Organize our new table according to page name/ad creator and number of ads run
df_all_creators <- all_ads_in_period %>% group_by(page_name) %>% summarise(n_ads = n()) %>% 
    arrange(desc(n_ads)) %>% rename(page = page_name, number_of_ads = n_ads)

NROW(all_ads_in_period)

[1] 77328

length(unique(all_ads_in_period$page_name))

[1] 3837

library(kableExtra)
knitr::kable(df_all_creators, caption = "Pages and total number of ads", ) %>% kable_styling() %>% 
    scroll_box(height = "500px")