CMD oppositional terms

CMD can be used to scale documents relative to their engagement with two poles of a given concept. We worked with this earlier when trying to measure engagement with both poles of inequality. The output of for these models is a value indicating a position on a scale between the two polar concepts. For the time being I can only use unigram concepts (like inequality) because computing larger n-grams requires an extra step that I have not yet figured out.

I computed polar CMD values for the following antonym pairs: Inequality - Prosperity; Justice - Order; taxation - charity; justice - meritocracy. Each speech in the corpus can be represented as a scaled position relative to these term pairs. Then, I aggregated the speeches by year and used a density function to visualize the yearly distribution of CMD values. Finally, I stacked the density ridges to illustrate the polar distribution of the corpus relative to the different term pairs.

Update 21 Jan

Taxation - Charity

I think that this is the most interesting plot.

dplot_taxation.charity

## Picking joint bandwidth of 0.284

## Warning: Removed 8 rows containing non-finite values (stat_density_ridges).

Yellow region: The yellow region corresponds to the location of the highest density of speeches. Most of the speeches Following the yellow ridge, it appears that the starting position was skewed towards taxation, and the mid-point oscillated, but pushed further towards taxation until 1983. Then, the mid-point drifts towards charity, crossing 0 on the x-axis during the early 1990s. The mid-point hovers around the 0-marker until 2000, but then lurches sharply towards charity in 2001. The mid-point remains scaled towards charity until 2018, except for in 2009, when it briefly crosses the 0-marker.

Height: The height of these density plots is also important, as it indicates greater homogeneity between the speeches. Flatter curves indicate a wider distribution of values. I still need to work on this type of visualization to figure out how to make this information apparent.

Inequality - Prosperity

dplot_inequality.prosperity

## Picking joint bandwidth of 0.293

## Warning: Removed 10 rows containing non-finite values (stat_density_ridges).

Justice - Meritocracy

dplot_justice.meritocracy

## Picking joint bandwidth of 0.27

## Warning: Removed 20 rows containing non-finite values (stat_density_ridges).

Justice - Order

dplot_justice.order

## Picking joint bandwidth of 0.292

## Warning: Removed 12 rows containing non-finite values (stat_density_ridges).

Additional scaled terms for 22 January

Note: I have to figure out how to do this type of scaling for complex tokens. For the time being, I included only unigram oppositional terms. I should be able to figure out how to do complex tokens for scaling as well, I just need to look into the documentation. For the time being, I added the following term pairs: welfare - market poverty - wealth international - domestic redistribution - growth development - debt tax - aid debt - investment solidarity - responsibility crisis - opportunity

Welfare - Market

dplot_welfare.market

## Picking joint bandwidth of 0.289

## Warning: Removed 30 rows containing non-finite values (stat_density_ridges).

Poverty - Wealth

dplot_poverty.wealth

## Picking joint bandwidth of 0.269

## Warning: Removed 8 rows containing non-finite values (stat_density_ridges).

International - Domestic

dplot_international.domestic

## Picking joint bandwidth of 0.291

## Warning: Removed 25 rows containing non-finite values (stat_density_ridges).

redistribution - growth

dplot_redistribution.growth

## Picking joint bandwidth of 0.269

## Warning: Removed 13 rows containing non-finite values (stat_density_ridges).

development - debt

dplot_development.debt

## Picking joint bandwidth of 0.273

## Warning: Removed 40 rows containing non-finite values (stat_density_ridges).

tax - aid

dplot_tax.aid

## Picking joint bandwidth of 0.276

## Warning: Removed 30 rows containing non-finite values (stat_density_ridges).

debt - investment

dplot_debt.investment

## Picking joint bandwidth of 0.276

## Warning: Removed 36 rows containing non-finite values (stat_density_ridges).

solidarity - responsibility

dplot_solidarity.responsibility

## Picking joint bandwidth of 0.297

## Warning: Removed 19 rows containing non-finite values (stat_density_ridges).

crisis - opportunity

dplot_crisis.opportunity

## Picking joint bandwidth of 0.276

## Warning: Removed 25 rows containing non-finite values (stat_density_ridges).

CMD for 22 January

Order—justice
Inequality—prosperity
Welfare state—market forces
Social justice—meritocracy
Poverty—wealth
NIEO—Washington consensus
International—domestic
Redistribution—growth
Right to development—Debt-relief
Global tax—foreign aid
Self-sufficiency—free trade
Local ownership—foreign investment
Sovereignty over resources—foreign investment
Global solidarity—National responsibility
Structural problems—opportunities

# order
ungdc18$cmd_order <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("order"),wv = my.wv) %>% select(,2)
ungdc18$cmd_order <- unlist(ungdc18$cmd_order)
ungdc18$cmd_order <- as.numeric(ungdc18$cmd_order)

# justice
ungdc18$cmd_justice <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("justice"),wv = my.wv) %>% select(,2)
ungdc18$cmd_justice <- unlist(ungdc18$cmd_justice)
ungdc18$cmd_justice <- as.numeric(ungdc18$cmd_justice)

# inequality
ungdc18$cmd_inequality <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("inequality"),wv = my.wv) %>% select(,2)
ungdc18$cmd_inequality <- unlist(ungdc18$cmd_inequality)
ungdc18$cmd_inequality <- as.numeric(ungdc18$cmd_inequality)

# prosperity
ungdc18$cmd_prosperity <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("prosperity"),wv = my.wv) %>% select(,2)
ungdc18$cmd_prosperity <- unlist(ungdc18$cmd_prosperity)
ungdc18$cmd_prosperity <- as.numeric(ungdc18$cmd_prosperity)

# welfarestate
ungdc18$cmd_welfarestate <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("welfare state"),wv = my.wv) %>% select(,2)
ungdc18$cmd_welfarestate <- unlist(ungdc18$cmd_welfarestate)
ungdc18$cmd_welfarestate <- as.numeric(ungdc18$cmd_welfarestate)

# marketforces
ungdc18$cmd_marketforces <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("market forces"),wv = my.wv) %>% select(,2)
ungdc18$cmd_marketforces <- unlist(ungdc18$cmd_marketforces)
ungdc18$cmd_marketforces <- as.numeric(ungdc18$cmd_marketforces)

# socialjustice
ungdc18$cmd_socialjustice <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("social justice"),wv = my.wv) %>% select(,2)
ungdc18$cmd_socialjustice <- unlist(ungdc18$cmd_socialjustice)
ungdc18$cmd_socialjustice <- as.numeric(ungdc18$cmd_socialjustice)

# meritocracy
ungdc18$cmd_meritocracy <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("meritocracy"),wv = my.wv) %>% select(,2)
ungdc18$cmd_meritocracy <- unlist(ungdc18$cmd_meritocracy)
ungdc18$cmd_meritocracy <- as.numeric(ungdc18$cmd_meritocracy)

# poverty
ungdc18$cmd_poverty <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("poverty"),wv = my.wv) %>% select(,2)
ungdc18$cmd_poverty <- unlist(ungdc18$cmd_poverty)
ungdc18$cmd_poverty <- as.numeric(ungdc18$cmd_poverty)

# wealth
ungdc18$cmd_wealth <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("wealth"),wv = my.wv) %>% select(,2)
ungdc18$cmd_wealth <- unlist(ungdc18$cmd_wealth)
ungdc18$cmd_wealth <- as.numeric(ungdc18$cmd_wealth)

# newinternationaleconomicorder
ungdc18$cmd_newinternationaleconomicorder <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("New International Economic Order"),wv = my.wv) %>% select(,2)
ungdc18$cmd_newinternationaleconomicorder <- unlist(ungdc18$cmd_newinternationaleconomicorder)
ungdc18$cmd_newinternationaleconomicorder <- as.numeric(ungdc18$cmd_newinternationaleconomicorder)

# washingtonconsensus
ungdc18$cmd_washingtonconsensus <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("Washington Consensus"),wv = my.wv) %>% select(,2)
ungdc18$cmd_washingtonconsensus <- unlist(ungdc18$cmd_washingtonconsensus)
ungdc18$cmd_washingtonconsensus <- as.numeric(ungdc18$cmd_washingtonconsensus)

# international
ungdc18$cmd_international <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("international"),wv = my.wv) %>% select(,2)
ungdc18$cmd_international <- unlist(ungdc18$cmd_international)
ungdc18$cmd_international <- as.numeric(ungdc18$cmd_international)

# domestic
ungdc18$cmd_domestic <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("domestic"),wv = my.wv) %>% select(,2)
ungdc18$cmd_domestic <- unlist(ungdc18$cmd_domestic)
ungdc18$cmd_domestic <- as.numeric(ungdc18$cmd_domestic)

# redistribution
ungdc18$cmd_redistribution <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("redistribution"),wv = my.wv) %>% select(,2)
ungdc18$cmd_redistribution <- unlist(ungdc18$cmd_redistribution)
ungdc18$cmd_redistribution <- as.numeric(ungdc18$cmd_redistribution)

# growth
ungdc18$cmd_growth <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("growth"),wv = my.wv) %>% select(,2)
ungdc18$cmd_growth <- unlist(ungdc18$cmd_growth)
ungdc18$cmd_growth <- as.numeric(ungdc18$cmd_growth)

# righttodevelopment
ungdc18$cmd_righttodevelopment <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("right to development"),wv = my.wv) %>% select(,2)
ungdc18$cmd_righttodevelopment <- unlist(ungdc18$cmd_righttodevelopment)
ungdc18$cmd_righttodevelopment <- as.numeric(ungdc18$cmd_righttodevelopment)

# debtrelief
ungdc18$cmd_debtrelief <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("debt relief"),wv = my.wv) %>% select(,2)
ungdc18$cmd_debtrelief <- unlist(ungdc18$cmd_debtrelief)
ungdc18$cmd_debtrelief <- as.numeric(ungdc18$cmd_debtrelief)

# globaltax
ungdc18$cmd_globaltax <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("global tax"),wv = my.wv) %>% select(,2)
ungdc18$cmd_globaltax <- unlist(ungdc18$cmd_globaltax)
ungdc18$cmd_globaltax <- as.numeric(ungdc18$cmd_globaltax)

# foreignaid
ungdc18$cmd_foreignaid <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("foreign aid"),wv = my.wv) %>% select(,2)
ungdc18$cmd_foreignaid <- unlist(ungdc18$cmd_foreignaid)
ungdc18$cmd_foreignaid <- as.numeric(ungdc18$cmd_foreignaid)

# selfsufficiency
ungdc18$cmd_selfsufficiency <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("self-sufficiency"),wv = my.wv) %>% select(,2)
ungdc18$cmd_selfsufficiency <- unlist(ungdc18$cmd_selfsufficiency)
ungdc18$cmd_selfsufficiency <- as.numeric(ungdc18$cmd_selfsufficiency)

# freetrade
ungdc18$cmd_freetrade <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("free trade"),wv = my.wv) %>% select(,2)
ungdc18$cmd_freetrade <- unlist(ungdc18$cmd_freetrade)
ungdc18$cmd_freetrade <- as.numeric(ungdc18$cmd_freetrade)

# localownership
ungdc18$cmd_localownership <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("local ownership"),wv = my.wv) %>% select(,2)
ungdc18$cmd_localownership <- unlist(ungdc18$cmd_localownership)
ungdc18$cmd_localownership <- as.numeric(ungdc18$cmd_localownership)

# foreigninvestment
ungdc18$cmd_foreigninvestment <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("foreign investment"),wv = my.wv) %>% select(,2)
ungdc18$cmd_foreigninvestment <- unlist(ungdc18$cmd_foreigninvestment)
ungdc18$cmd_foreigninvestment <- as.numeric(ungdc18$cmd_foreigninvestment)

# resourcesovereignty
ungdc18$cmd_resourcesovereignty <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("resource sovereignty"),wv = my.wv) %>% select(,2)
ungdc18$cmd_resourcesovereignty <- unlist(ungdc18$cmd_resourcesovereignty)
ungdc18$cmd_resourcesovereignty <- as.numeric(ungdc18$cmd_resourcesovereignty)

# foreigninvestment
ungdc18$cmd_foreigninvestment <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("foreign investment"),wv = my.wv) %>% select(,2)
ungdc18$cmd_foreigninvestment <- unlist(ungdc18$cmd_foreigninvestment)
ungdc18$cmd_foreigninvestment <- as.numeric(ungdc18$cmd_foreigninvestment)

# globalsolidarity
ungdc18$cmd_globalsolidarity <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("global solidarity"),wv = my.wv) %>% select(,2)
ungdc18$cmd_globalsolidarity <- unlist(ungdc18$cmd_globalsolidarity)
ungdc18$cmd_globalsolidarity <- as.numeric(ungdc18$cmd_globalsolidarity)

# national responsibility
ungdc18$cmd_nationalresponsibility <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("national responsibility"),wv = my.wv) %>% select(,2)
ungdc18$cmd_nationalresponsibility <- unlist(ungdc18$cmd_nationalresponsibility)
ungdc18$cmd_nationalresponsibility <- as.numeric(ungdc18$cmd_nationalresponsibility)

# structuralproblem
ungdc18$cmd_structuralproblem <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("structural problem"),wv = my.wv) %>% select(,2)
ungdc18$cmd_structuralproblem <- unlist(ungdc18$cmd_structuralproblem)
ungdc18$cmd_structuralproblem <- as.numeric(ungdc18$cmd_structuralproblem)

# opportunity
ungdc18$cmd_opportunity <- ungdc_unnest %>% 
                    cast_dtm(term = word, 
                                document = doc_id, 
                                value = n, 
                                weighting = tm::weightTf) %>%
                            removeSparseTerms(.999) %>%
                    CMDist(cw =c("opportunity"),wv = my.wv) %>% select(,2)
ungdc18$cmd_opportunity <- unlist(ungdc18$cmd_opportunity)
ungdc18$cmd_opportunity <- as.numeric(ungdc18$cmd_opportunity)

Cosine-scaling Documents

Reagan Speech v. NIEO Declaration

Reagan - NIEO: more positive = more reagan, more negative = more NIEO

NIEO - Reagan

dplot_scale_reagan.nieo

## Picking joint bandwidth of 0.00384

## Warning: Removed 104 rows containing non-finite values (stat_density_ridges).

jan21_1