In this assignment you will mainly work on data processing and data cleaning.Besides, you need to benefit from what we have covered so far regarding the missing data or potential outlier detection.
About Data set
The data come from OpenSecrets.org, a “website tracking the influence of money on U.S. politics, and how that money affects policy and citizens’ lives”.
This website is hosted by The Center for Responsive Politics, which is a nonpartisan, independent non-profit that “tracks money in U.S. politics and its effect on elections and public policy.”[^1]
Our first goal is to merge the different data sets provided, to create a single dataset. Since that means repeating a task many times, you should write a function to create the data.
Data cleaning
In this section we clean the pac_all data frame to prepare it for analysis and visualization. We have two goals in data cleaning:
Separate the country_parent into two such that country and parent company appear in different columns for country-level analysis.
Convert contribution amounts in total, dems, and repubs from character strings to numeric values.
Check each column whether you have any missing values or not (You can benefit from different packages to visualize missingness if you like)
The following exercises walk you through how to make these fixes to the data.
Looking at the functions separete() str_remove() and str_remove_all() are strongly recommended !!
# Add your codes
pac_2016 <- read_csv("pac_2016.csv")
## New names:
## Rows: 243 Columns: 7
## ── Column specification
## ──────────────────────────────────────────────────────── Delimiter: "," chr
## (5): name, country_parent, total, dems, repubs dbl (2): ...1, year
## ℹ Use `spec()` to retrieve the full column specification for this data. ℹ
## Specify the column types or set `show_col_types = FALSE` to quiet this message.
## • `` -> `...1`
View(pac_2016)
pac_2018 <- read_csv("pac_2018.csv")
## New names:
## Rows: 248 Columns: 7
## ── Column specification
## ──────────────────────────────────────────────────────── Delimiter: "," chr
## (5): name, country_parent, total, dems, repubs dbl (2): ...1, year
## ℹ Use `spec()` to retrieve the full column specification for this data. ℹ
## Specify the column types or set `show_col_types = FALSE` to quiet this message.
## • `` -> `...1`
View(pac_2018)
pac_2020 <- read_csv("pac_2020.csv")
## New names:
## Rows: 230 Columns: 7
## ── Column specification
## ──────────────────────────────────────────────────────── Delimiter: "," chr
## (5): name, country_parent, total, dems, repubs dbl (2): ...1, year
## ℹ Use `spec()` to retrieve the full column specification for this data. ℹ
## Specify the column types or set `show_col_types = FALSE` to quiet this message.
## • `` -> `...1`
View(pac_2020)
pac_all <- list(pac_2020, pac_2018, pac_2016)
pac_all <- rbind(pac_2016, pac_2018, pac_2020)
create_pac_all <- function() {
years <- c(2020, 2018, 2016)
pac_all <- NULL
for (year in years) {
filename <- paste0("pac_", year, ".csv")
dataset <- read.csv(filename)
pac_all <- rbind(pac_all, dataset)
}
return(pac_all)
}
pac_all <- create_pac_all()
pac_all <- read.csv("pac_all.csv")
write.csv(pac_all, "pac_all.csv", row.names = FALSE)
observations <- dim(pac_all)[1]
variables <- dim(pac_all)[2]
cat("The pac_all dataset contains", observations, "observations and", variables, "variables.")
## The pac_all dataset contains 721 observations and 7 variables.
library(tidyr)
pac_all <- separate(pac_all, country_parent, into = c("country", "parent_company"), sep = "/")
## Warning: Expected 2 pieces. Additional pieces discarded in 6 rows [129, 152, 371, 400,
## 612, 640].
## X name country parent_company total dems
## 1 1 7-Eleven Japan Seven & I Holdings $20,000 $1,000
## 2 2 ABB Group (ABB Group) Switzerland Asea Brown Boveri $16,900 $6,800
## 3 3 Accenture (Accenture) Ireland Accenture plc $83,500 $50,500
## 4 4 Air Liquide America France L'Air Liquide SA $37,800 $15,800
## 5 5 Airbus Group Netherlands Airbus Group $182,000 $79,000
## 6 6 Alkermes Inc Ireland Alkermes Plc $94,750 $30,750
## repubs year
## 1 $19,000 2020
## 2 $10,100 2020
## 3 $33,000 2020
## 4 $22,000 2020
## 5 $103,000 2020
## 6 $64,000 2020
pac_all$total <- as.numeric(str_remove_all(pac_all$total, "[^0-9.]"))
pac_all$dems <- as.numeric(str_remove_all(pac_all$dems, "[^0-9.]"))
pac_all$repubs <- as.numeric(str_remove_all(pac_all$repubs, "[^0-9.]"))
missing_values <- colSums(is.na(pac_all))
print(missing_values)
## X name country parent_company total
## 0 0 0 0 0
## dems repubs year
## 0 0 0
Add your comments
Exercise 2
Use the separate() function to separate country_parent into country and parent columns.
Remove the character strings including $ and , signs in the total, dems, and repubs columns and convert these columns to numeric.
End your code chunk by printing out the top 10 rows of your data frame (if you just type the data frame name it should automatically do this for you).
✏️️🧶 ✅ ⬆️ Write your answer in your R Markdown document , knit the document, make sure that your added R code chunk works well and you did not face with any knitting problem
# Add your codes
pac_all$total <- as.numeric(gsub("[$,]", "", pac_all$total))
pac_all$dems <- as.numeric(gsub("[$,]", "", pac_all$dems))
pac_all$repubs <- as.numeric(gsub("[$,]", "", pac_all$repubs))
head(pac_all, 10)
## X name country
## 1 1 7-Eleven Japan
## 2 2 ABB Group (ABB Group) Switzerland
## 3 3 Accenture (Accenture) Ireland
## 4 4 Air Liquide America France
## 5 5 Airbus Group Netherlands
## 6 6 Alkermes Inc Ireland
## 7 7 Allianz of America (Allianz) Germany
## 8 8 AMG Vanadium Netherlands
## 9 9 Anheuser-Busch (Anheuser-Busch InBev) Belgium
## 10 10 AON Corp (AON plc) UK
## parent_company total dems repubs year
## 1 Seven & I Holdings 20000 1000 19000 2020
## 2 Asea Brown Boveri 16900 6800 10100 2020
## 3 Accenture plc 83500 50500 33000 2020
## 4 L'Air Liquide SA 37800 15800 22000 2020
## 5 Airbus Group 182000 79000 103000 2020
## 6 Alkermes Plc 94750 30750 64000 2020
## 7 Allianz AG Holding 71300 36150 35150 2020
## 8 AMG Advanced Metallurgical Group 2000 0 2000 2020
## 9 Anheuser-Busch InBev 336500 174000 162500 2020
## 10 AON PLC 80500 44000 36500 2020
Add your comments
Exercise 3
Consider one of the variables in your data set (say total column),
Create a boxplot for this numerical variable separately for each year.
Comment on the distributional behavior of the variable, are you able to spot any potential outliers (value far away from the others in general)
Apply related tests from outlier package if you have any potential outlier observation in your data. Confirm or not the considered point is a real outlier or not by interpreting your statistical test results (ie. Dixon’s test etc.)
✏️️🧶 ✅ ⬆️ Write your answer in your R Markdown document , knit the document, make sure that your added R code chunk works well and you did not face with any knitting problem
# Add your codes
library(ggplot2)
ggplot(pac_all, aes(x = as.factor(year), y = total)) +
geom_boxplot(fill = "red", color = "yellow", alpha = 0.7) +
geom_jitter(color = "green", alpha = 0.5, width = 0.2) +
labs(x = "Year", y = "Total Contribution Amount", title = "Boxplot of Total Contribution Amount by Year") +
theme_minimal() +
theme(axis.text = element_text(size = 12),
axis.title = element_text(size = 14, face = "bold"),
plot.title = element_text(size = 16, face = "bold"))
library(outliers)
dixon_test_results <- lapply(unique(pac_all$year), function(year) {
subset_data <- pac_all[pac_all$year == year, "total"]
if (length(subset_data) >= 3 && length(subset_data) <= 30) {
dixon.test(subset_data)
} else {
NA
}
})
length(dixon_test_results)
## [1] 3
Add your comments
Exercise 4
Create a line plot of total contributions from all foreign-connected PACs in the Canada and Mexico over the years.
Once you have made the plot, write a brief interpretation of what the graph reveals.
✏️️🧶 ✅ ⬆️ Write your answer in your R Markdown document , knit the document, make sure that your added R code chunk works well and you did not face with any knitting problem
# Add your codes
colnames(pac_all)
## [1] "X" "name" "country" "parent_company"
## [5] "total" "dems" "repubs" "year"
foreign_pacs_can_mex <- subset(pac_all, country %in% c("Canada", "Mexico"))
total_contributions <- aggregate(total ~ year, foreign_pacs_can_mex, sum)
ggplot(total_contributions, aes(x = year, y = total, group = 1)) +
geom_line() +
labs(title = "Total Contributions from Foreign-Connected PACs in Canada and Mexico",
x = "Year",
y = "Total Contributions") +
theme_minimal()
Add your comments
#This trend suggests that there was a growing interest in contributing to political campaigns from foreign-connected PACs in Canada and Mexico during the years leading up to 2018. However, the subsequent decline in contributions after 2018 may indicate a shift in priorities or external factors influencing the contribution patterns.