Notes: The midterm is due at 11:59PM tonight (Wednesday Feb 28). Submit a PDF (html converted to pdf is fine).

If you have any questions, feel free to email me at . I will also start a discussion board where I will post any clarifications arising from students’ questions.

Make sure your analysis results print out as code output (e.g., don’t just create an object ‘DiD_treatment_A’ then tell me in a text response what the value of this object is, make sure this value prints as output as well). Remember to always knit as you go so you don’t run into knitting issues at 11:55PM. And for the love of God stop printing entire datasets!

You are not allowed to collaborate with anyone else on the midterm, or use AI for your code or text responses, but you are allowed to use google or any other online resource to find solutions.

Good luck!

Part 1 - Same-Sex Marriage Canvassing Experiment

In this exercise, we analyze the data from an experiment in which households were canvassed for support on same-sex marriage. Canvassers were given a script leading to conversations that averaged about twenty minutes. A distinctive feature of this study is that gay and straight canvassers were randomly assigned to households and canvassers revealed whether they were straight or gay in the course of the conversation. The experiment aims to test the ‘contact hypothesis,’ which contends that out-group hostility (towards homosexuals in this case) diminishes when people from different groups interact with one another.

This exercise is based on: LaCour, M. J., and D. P. Green. 2014. “When Contact Changes Minds: An Experiment on Transmission of Support for Gay Equality.” Science 346(6215): 1366–69.

The data file is ssm.csv, which is a CSV file. The names and descriptions of variables are:

Name Description
treatment Treatment assignment: No contact, Same-Sex Marriage Script by Gay Canvasser, Same-Sex Marriage Script by Straight Canvasser, Recycling Script by Gay Canvasser, and Recycling Script by Straight Canvasser
ssm_w1 Support for gay marriage in wave 1, prior to treatment (1 to 5). Higher scores indicate more support.
ssm_w2 Support for gay marriage in wave 2 (1 to 5). Higher scores indicate more support.
ssm_w3 Support for gay marriage in wave 3 (1 to 5). Higher scores indicate more support.
ssm_w4 Support for gay marriage in wave 4 (1 to 5). Higher scores indicate more support.
ssm_w5 Support for gay marriage in wave 5 (1 to 5). Higher scores indicate more support.
ssm_w6 Support for gay marriage in wave 6 (1 to 5). Higher scores indicate more support.
ssm_w7 Support for gay marriage in wave 7 (1 to 5). Higher scores indicate more support.

Each observation of this data set is a respondent giving responses to a five-point survey item on same-sex marriage during 7 different time periods (i.e. 7 waves). The first wave consists of the interview before the canvassing treatment occurs, later waves consist of interviews occurring at successively later time points.

Question 1 (5 points)

Using the interview wave before treatment (wave 1), examine whether there are baseline differences between experimental groups in support for same-sex marriage. Give a brief interpretation of your results.

Answer 1

ssm <- read.csv("ssm.csv")
tapply(ssm$ssm_w1, ssm$treatment, mean)
##                                     No Contact 
##                                       2.996397 
##              Recycling Script by Gay Canvasser 
##                                       3.059722 
##         Recycling Script by Straight Canvasser 
##                                       2.931660 
##      Same-Sex Marriage Script by Gay Canvasser 
##                                       2.963568 
## Same-Sex Marriage Script by Straight Canvasser 
##                                       3.105702
tapply(ssm$ssm_w1, ssm$treatment, median)
##                                     No Contact 
##                                              3 
##              Recycling Script by Gay Canvasser 
##                                              3 
##         Recycling Script by Straight Canvasser 
##                                              3 
##      Same-Sex Marriage Script by Gay Canvasser 
##                                              3 
## Same-Sex Marriage Script by Straight Canvasser 
##                                              3

The baselines for this wave are the same. The means are close to 3 and the medians are 3. There are no baseline differences between the groups.

Question 2 (10 points)

Wave 2 of the survey was implemented two months after the canvassing treatments. Estimate the average treatment effects of conversations about gay marriage with gay and straight canvassers on support for same-sex marriage. Estimate the effect of each experimental condition separately (i.e., produce an estimated treatment effect for the ‘Same-Sex Marriage Script by Gay Canvasser’ experimental condition and another estimated treatment effect for the ‘Same-Sex Marriage Script by Straight Canvasser’ experimental condition, with both compared to the ‘No Contact’ control group).

Remember that in a difference in difference design, the treatment effect is the difference between (a) the change in the treated group and (b) the change in the control group (the difference in differences). Also remember that some cases might be missing data on wave 1 or 2, so should not be included if we want to compare changes in attitudes (we don’t know how these peoples’ attitudes changed).

Give a brief interpretation of your results.

Answer 2

suppressWarnings(avg.gay.1 <- mean(ssm$ssm_w1[ssm$treatment == "Same-Sex Marriage Script by Gay Canvasser"] - ssm$ssm_w1[ssm$treatment == "No Contact"], na.rm = TRUE))

 

suppressWarnings(avg.straight.1 <- mean(ssm$ssm_w1[ssm$treatment == "Same-Sex Marriage Script by Straight Canvasser"] - ssm$ssm_w1[ssm$treatment == "No Contact"], na.rm = TRUE))

 



suppressWarnings(avg.gay.2 <- mean(ssm$ssm_w2[ssm$treatment == "Same-Sex Marriage Script by Gay Canvasser"] - ssm$ssm_w2[ssm$treatment == "No Contact"], na.rm = TRUE))

 

suppressWarnings(avg.straight.2 <- mean(ssm$ssm_w2[ssm$treatment == "Same-Sex Marriage Script by Straight Canvasser"] - ssm$ssm_w2[ssm$treatment == "No Contact"], na.rm = TRUE))

 

avg.gay <- (avg.gay.2 - avg.gay.1)

avg.straight <- (avg.straight.2 - avg.straight.1)

 

avg.gay
## [1] 0.1358818
avg.straight
## [1] 0.05411678

The gay canvassers had better treatment effects. Both groups of canvassers showed positive growth, but the gay canvassers had better results. This result makes sense as a gay canvasser would likely lead to a more positive attitude on same-sex marriage.

Question 3 (10 points)

Create a plot displaying the results of your analysis in question 2 (a bar plot or a line graph are two possible options we have covered).

Answer 3

treat.effect <- c(avg.gay, avg.straight)
treat.labels <- c("Gay Canvasser", "Straight Canvasser")

barplot(treat.effect,
        names.arg = treat.labels,
        ylab = "Average Treatment Effect",
        col = "green",
        main = "Average Treatment Effects by Canvasser Type", 
        ylim = c(0, 0.2)
        )

Question 4 (5 points)

The study contained other treatments that involved contact, but did not involve the same-sex marriage script. Specifically, the authors used a script to encourage people to recycle, and had this script delivered by both straight and gay canvassers. What do you think the purpose of these extra treatments is? I.e., what extra information can we learn from analyzing these conditions that we could not learn from analyzing the ‘No contact’, ‘Same-Sex Marriage Script by Gay Canvasser’, and ‘Same-Sex Marriage Script by Straight Canvasser’ conditions alone?

Answer 4

The script allows for different social issues to be discussed. Asking more questions would give the canvasser more informaton to determine if the person being surveyed is biased on the topic in some way. The recycling aspect would provide additional information on the responders bias and if they are responding the testing question in a way to not offend the canvasser.

Question 5 (10 points)

Using waves 1 and 2, compare the effect of the treatment ‘Same-Sex Marriage Script by Gay Canvasser’ compared with the treatment ‘Recycling Script by Gay Canvasser.’ Repeat the same for straight canvassers, comparing the treatment ‘Same-Sex Marriage Script by Straight Canvasser’ to the treatment ‘Recycling Script by Straight Canvasser.’ Again, base your analysis on the ‘difference in difference’ logic and exclude cases missing data on either wave 1 or wave 2. What do these comparisons reveal? Give an interpretation of the results.

Answer 5

suppressWarnings(avg.gay.recycle.1 <- mean(ssm$ssm_w1[ssm$treatment == "Same-Sex Marriage Script by Gay Canvasser"] - ssm$ssm_w1[ssm$treatment == "Recycling Script by Gay Canvasser"], na.rm = TRUE))

suppressWarnings(avg.straight.recycle.1 <- mean(ssm$ssm_w1[ssm$treatment == "Same-Sex Marriage Script by Straight Canvasser"] - ssm$ssm_w1[ssm$treatment == "Recycling Script by Straight Canvasser"], na.rm = TRUE))


suppressWarnings(avg.gay.recycle.2 <- mean(ssm$ssm_w2[ssm$treatment == "Same-Sex Marriage Script by Gay Canvasser"] - ssm$ssm_w2[ssm$treatment == "Recycling Script by Gay Canvasser"], na.rm = TRUE))

suppressWarnings(avg.straight.recycle.2 <- mean(ssm$ssm_w2[ssm$treatment == "Same-Sex Marriage Script by Straight Canvasser"] - ssm$ssm_w2[ssm$treatment == "Recycling Script by Straight Canvasser"], na.rm = TRUE))

avg.gay.recycle <- (avg.gay.recycle.2 - avg.gay.recycle.1)
avg.straight.recycle <- (avg.straight.recycle.2 - avg.straight.recycle.1)

avg.gay.recycle
## [1] 0.1686625
avg.straight.recycle
## [1] 0.1091087

There is a positive increase between wave 1 and 2 for both scripts. The greater increase comes from the gay canvasser. The final values are much closer than in the previous question, showing the recycling affected the straight canvasser more.

Question 6 (20 points)

The authors re-interviewed the respondents 6 different times (in waves 2 to 7) after treatment, at two month intervals. The last interview in wave 7 occurred one year after treatment. Do the treatment effects last? And if so, under what conditions? As in Question 2, focus on the straight and gay canvassers with the same-sex marriage script, and answer these questions by separately computing and examining the treatment effects of each condition, compared to the no contact condition, for each of the subsequent waves (3 through 7). Note that at each wave, treatment effects are still differences in differences. You will therefore need to compute 10 total treatment effects - 2 conditions and five treatment effects per condition). Make sure to print out the results directly from your R code chunk, but also to provide an interpretation in your own words of what the results indicate about the lasting effects of each treatment.

Answer 6

suppressWarnings(avg.gay.3 <- mean(ssm$ssm_w3[ssm$treatment == "Same-Sex Marriage Script by Gay Canvasser"] - ssm$ssm_w3[ssm$treatment == "No Contact"], na.rm = TRUE))

 

suppressWarnings(avg.straight.3 <- mean(ssm$ssm_w3[ssm$treatment == "Same-Sex Marriage Script by Straight Canvasser"] - ssm$ssm_w3[ssm$treatment == "No Contact"], na.rm = TRUE))

 



suppressWarnings(avg.gay.4 <- mean(ssm$ssm_w4[ssm$treatment == "Same-Sex Marriage Script by Gay Canvasser"] - ssm$ssm_w4[ssm$treatment == "No Contact"], na.rm = TRUE))

 

suppressWarnings(avg.straight.4 <- mean(ssm$ssm_w4[ssm$treatment == "Same-Sex Marriage Script by Straight Canvasser"] - ssm$ssm_w4[ssm$treatment == "No Contact"], na.rm = TRUE))



suppressWarnings(avg.gay.5 <- mean(ssm$ssm_w5[ssm$treatment == "Same-Sex Marriage Script by Gay Canvasser"] - ssm$ssm_w5[ssm$treatment == "No Contact"], na.rm = TRUE))

 

suppressWarnings(avg.straight.5 <- mean(ssm$ssm_w5[ssm$treatment == "Same-Sex Marriage Script by Straight Canvasser"] - ssm$ssm_w5[ssm$treatment == "No Contact"], na.rm = TRUE))

 



suppressWarnings(avg.gay.6 <- mean(ssm$ssm_w6[ssm$treatment == "Same-Sex Marriage Script by Gay Canvasser"] - ssm$ssm_w6[ssm$treatment == "No Contact"], na.rm = TRUE))

 

suppressWarnings(avg.straight.6 <- mean(ssm$ssm_w6[ssm$treatment == "Same-Sex Marriage Script by Straight Canvasser"] - ssm$ssm_w6[ssm$treatment == "No Contact"], na.rm = TRUE))

 



suppressWarnings(avg.gay.7 <- mean(ssm$ssm_w7[ssm$treatment == "Same-Sex Marriage Script by Gay Canvasser"] - ssm$ssm_w7[ssm$treatment == "No Contact"], na.rm = TRUE))

 

suppressWarnings(avg.straight.7 <- mean(ssm$ssm_w7[ssm$treatment == "Same-Sex Marriage Script by Straight Canvasser"] - ssm$ssm_w7[ssm$treatment == "No Contact"], na.rm = TRUE))

 

(avg.gay.32 <- avg.gay.3 - avg.gay.2)
## [1] -0.002495056
(avg.gay.43 <- avg.gay.4 - avg.gay.3)
## [1] 0.009024753
(avg.gay.54 <- avg.gay.5 - avg.gay.4)
## [1] 0.05598466
(avg.gay.65 <- avg.gay.6 - avg.gay.5)
## [1] -0.1113925
(avg.gay.76 <- avg.gay.7 - avg.gay.6)
## [1] 0.01955094
(avg.straight.32 <- avg.straight.3 - avg.straight.2)
## [1] -0.04866317
(avg.straight.43 <- avg.straight.4 - avg.straight.3)
## [1] 0.05181527
(avg.straight.54 <- avg.straight.5 - avg.straight.4)
## [1] -0.02497415
(avg.straight.65 <- avg.straight.6 - avg.straight.5)
## [1] -0.04390816
(avg.straight.76 <- avg.straight.7 - avg.straight.6)
## [1] -0.1370371

Question 7 (10 points)

Create a plot (a bar plot or a line graph are options we have covered) displaying the progression of the treatment effects over time. Make the x axis variable the wave, make the y axis variable the treatment effect relative to the non-contact control, and then create different colored bars (or different colored lines) showing the successive estimated treatment effects for the two focal conditions (gay and straight canvasser with same-sex marriage script) for each wave (also include the wave 2 results you computed in Q2). Add a legend specifying which color indicates which condition.

Answer 7

datframe <- data.frame(
  Waves = c("2-1","3-2", "4-3", "5-4", "6-5","7-6"),
  gay.canvasser = c(avg.gay, avg.gay.32, avg.gay.43, avg.gay.54, avg.gay.65, avg.gay.76),
  straight.canvasser = c(avg.straight, avg.straight.32, avg.straight.43, avg.straight.54, avg.straight.65,     avg.straight.76)
)
canvasser <- t(cbind(datframe$gay.canvasser, datframe$straight.canvasser))

barplot(canvasser, 
        beside=T,
        names.arg = datframe$Waves, 
        ylab = "Average Treatment Effect", 
        col = c("green", "red"),
        main = "Average Treatment Effects by Canvasser Type",
        ylim = c(-0.2, 0.2)
        )
legend("topright", 
       legend=c("Gay","Straight"), 
       col=scales::alpha(c("green","red")), 
       pch=c(15), 
       bty="n" 
       )

Part 2 - Measurement: Oil and democracy

Researchers have theorized that natural resources may have an inhibiting effect on the democratization process. Although there are multiple explanations as to why this might be the case, one hypothesis posits that governments in countries with large natural resource endowments (like oil) are able to fund their operations without taxing civilians. Since representation (and other democratic institutions) are a compromise offered by governments in exchange for tax revenue, resource-rich countries do not need to make this trade. In this exercise, we will not investigate causal effects of oil on democracy. Instead, we examine whether the association between oil and democracy is consistent with the aforementioned hypothesis.

This exercise is in part based on Michael L. Ross. (2001). ‘Does Oil Hinder Democracy?’ World Politics,53:3,pp.325-361.

The data set is in the csv file oil.csv. The names and descriptions of variables are:

Name Description
cty_name Country name
logGDPcp Logged GDP per capita
regime A measure of a country’s level of democracy: -10 (authoritarian) to 10 (democratic)
oil Amount of oil exports as a percentage of the country’s GDP
illit Percentage of the population that is illiterate
life Life expectancy in the country
logGDPcp_z logGDPcp standardized (mean = 0, SD = 1)
regime_z regime standardized (mean = 0, SD = 1)
oil_z oil standardized (mean = 0, SD = 1)
illit_z illit standardized (mean = 0, SD = 1)
life_z life standardized (mean = 0, SD = 1)

Note that each of the variables (except country name) has two versions, a raw version and a standardized version (the standardized versions of the variables have the ’_z’ addendum). The standardized vesions are the raw versions converted to z scores, so each have a mean of 0 and a standard deviation of 1. We will use the standardized versions for clustering, this helps to prevent variables with larger scales dominating the clustering process.

Question 8 (5 points)

Are there any countries who are in the top quartile of GDP per capita, but are also above the median in illiteracy? If so, which ones?

Note: when answering this question, be aware that both oil and illit have many missing values, and that the result of indexing a value with an NA is an NA. I.e., the result of c(“dog”,“cat”,“bird”)[c(TRUE,NA,FALSE)] is (“dog”,NA).

Answer 8

oil <- read.csv("oil.csv")


GDPq <- quantile(na.omit(oil$logGDPcp), c(0, 0.25, 0.50, 0.75), na.rm= TRUE)
fourthq.cities <- oil$cty_name[oil$logGDPcp>=GDPq[4]]
illit.median <- median(na.omit(oil$illit))


above.illit <- oil$cty_name[oil$illit > illit.median]

intersect(na.omit(fourthq.cities), na.omit(above.illit))
## [1] "Kuwait"               "Oman"                 "Qatar"               
## [4] "Saudi Arabia"         "United Arab Emirates"

Question 9 (10 points)

Use scatterplots to examine the bivariate relationship between logged GDP per capita and life expectancy as well as between logged GDP per capita and illiteracy. Be sure to add informative axis labels. Also, compute the correlation for both bivariate relationships. Briefly summarize the results. To ignore missing data when applying the cor() function, add the extra command use = "complete.obs".

Answer 9

plot(x = oil$logGDPcp, 
     y = oil$life, 
     pch = 16, 
     col = "red", 
     xlim = c(5, 10), 
     ylim = c(0, 100), 
     xlab = "Logged GDP per Capita", 
     ylab = "Life Expectancy", 
     main = "Life Expectancy based on GDP" 
     ) 

plot(x = oil$logGDPcp, 
     y = oil$illit,
     pch = 16, 
     col = "blue", 
     xlim = c(5, 10), 
     ylim = c(0, 100), 
     xlab = "Logged GDP per Capita", 
     ylab = "Illiteracy Percentage", 
     main = "Illiteracy Percentage based on GDP" 
     ) 

cor(oil$logGDPcp, oil$life, use = "complete.obs")
## [1] 0.8679223
cor(oil$logGDPcp, oil$illit, use = "complete.obs")
## [1] -0.6695359

Based on these graphs, there is a positive correlation between life expentency and GDP. The graphs also show that the higher than the GDP, the lower the illiteracy rate. ### Question 10 (15 points)

We focus on the following subset of the variables: regime, oil, logGDPcp, and illit. First, remove countries that have missing values in any of these variables. Then, using the standardized versions of each of these variables (regime_z, oil_z, logGDPcp_z, illit_z), fit the k-means clustering algorithm to divide countries into two clusters. Before clustering, set a seed using the set.seed() function so your clustering algorithm returns the same results each time. After clustering, compute the means of each cluster on the original unstandardized variables regime, oil, logGDPcp, and illit.

Answer 10

subset.oil <- na.omit(oil[, c("regime_z", "oil_z", "logGDPcp_z", "illit_z")])

set.seed(123)

kmeans_result <- kmeans(subset.oil, centers = 2)
cluster_means <- aggregate(subset.oil, by = list(kmeans_result$cluster), FUN = mean)
colnames(cluster_means) <- c("Cluster", "Regime", "Oil", "Logged GDP per Capita", "Illiteracy")

cluster_means
##   Cluster     Regime        Oil Logged GDP per Capita Illiteracy
## 1       1 -0.5928124 -0.1761290            -0.8642908   1.002291
## 2       2  0.3898304  0.1317338             0.4585391  -0.731718

Question 11 (5 points)

Interpret your results from Question 10. Based on their mean values of regime, oil, logGDPcp, and illit, what differences do you observe between the two clusters of countries? Are these differences consistent with the theory that natural resources may have an inhibiting effect on the democratization process?

Answer 11

The authoritarian regimes in cluster 1 have less oil exported, a lower GDP, and a higher illiteracy rate. The second cluster follows has a democratic system, higher GDP, more literate citizens, and more oil exported. The data suggests that a democratic system leads to a higher GDP and a lower percentage of illiterate citizens.

Question 12 (15 points)

Repeat the previous two questions but this time with three clusters instead of two. How are the results different? Does adding a third cluster seem useful to you, with regard to assessing the theory that natural resources may have an inhibiting effect on the democratization process? Remember to set a seed before clustering.

Answer 12

set.seed(123)

kmeans_result <- kmeans(subset.oil, centers = 3)
cluster_means <- aggregate(subset.oil, by = list(kmeans_result$cluster), FUN = mean)
colnames(cluster_means) <- c("Cluster", "Regime", "Oil", "Logged GDP per Capita", "Illiteracy")

cluster_means
##   Cluster     Regime        Oil Logged GDP per Capita  Illiteracy
## 1       1 -0.5917316 -0.2437875            -0.8608404  1.00723186
## 2       2 -1.3233293  3.1904453             0.7371990  0.06566713
## 3       3  0.6390877 -0.3019062             0.3810843 -0.82354458

This is similar to the previous question, except cluster 2 has less democratic countries with high GDP and oil export with a low illiteracy rate. The third cluster has democratic countries with lower GDP and oil exported with a higher illiteracy rate. This set of data contradicts the previous question. There is no clear interpretation to be made with this data, especially considering the data from the previous question.