Tutorial on how to use the universal U.S. Raking Template
Published
Last Updated on 2026-08-28
Introduction
You’ve begun the exciting journey of pivoting from propensity scoring and post-stratification to Iterative Proportional Fitting (IPF), also known as raking! This tutorial will teach you the flow and logic of how to setup the universal raking script for your own project, in addition to some insight in the event that you need to adapt to a more specific population of interest. By the end, you ought to be prepped to conduct your own raking weighting projects, and save enough time to contemplate the virtues of capybaras with the remainder of your time.
Some directory notes and other asides
Before beginning, note that this tutorial is designed for the U.S. at large and most of the states as drawn from the Current Population Survey (CPS) Voting Registration Supplement (VRS) of 2024 and the AP Votecast Exit Poll data sets. In the event that you are conducting a non-political (i.e. no political balancing on presidential vote joint distributions and the like), then you should go with the American Community Survey (ACS) 2024 data, as prepped by Jay Ehlinger.1 This tutorial will therefore be of use insofar as you’d like to skip to some of the techniques when it comes to raking.
However, in the event that raking to CPS-VRS/AP Vote cast data is of interest, then please follow along. Notably, the main file path for the CPS data will be,
“/Volumes/projects/Sampling/CPS/2024/November/”, with the “frames” directory containing the 50,000 row resampled frames2, and the “inprogress” directory containing the pipeline workflow in cleaning and setting up the CPS-VRS and AP Votecast data for use in sampling and weighting. Please under no circumstance should you save or overwrite anything, lest you cause me to have a stroke.3 There are additionally some useful functions within this directory for ease of processing CPS data. These can be copied into local folders for a given project, though for the time being, the source files will be read directly from there.
The same is true for the working directory of this tutorial; there are some helpful functions within the “setup_us_all.R” script that will be used in creating the recodes and lists necessary for raking to work. These can also be copied, though will be likewise read from the directory of “/Volumes/projects/Internal_Projects/Analytics/weighting_document/R_WG_Development/weighting_comparison.” This directory refers to the R Code Managment Working Group directory, which is home to the set of test/pilot functions created by said working group.4 As with the CPS data, copy as needed and adjust directories accordingly, though please do not modify existing files without permission.
With all of that out of the way, let’s begin our raking adventure!5
Setting things up
The preamble for the new raking script will be the same as the past weighting template for the most part, with only two additional objects, source_path and cps_path. These two objects will simply read in the source files found within the R Working Group and CPS directories, respectively. While you might eventually not need the source_path, assuming you copy over the setup script, you’ll want to keep the cps_path for the purpose of reading in the various targets objects derived from the CPS-VRS.
## Modified by: WHO_ELSE_TO_BLAME## Date: Unix format## Please, please, please, remember to put this project under local version control## Also, Canyen needs to stop punching out koalas at Zoological parks rm(list=ls()); gc();
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 572805 30.6 1303514 69.7 666142 35.6
Vcells 1059241 8.1 8388608 64.0 1819013 13.9
## save sessionInfosink("sessionInfo.txt")sessionInfo()
sink()## set random seedset.seed(20130731)switch(Sys.info()[['sysname']],Linux = {startpath <-"/Volumes/projects"},Windows = {startpath <-"/vfs01/projects"},Darwin = {startpath <-"/Volumes/projects"})# path for fxns source_path ="/home/surveys/projects/Internal_Projects/Analytics/weighting_document/R_WG_Development/weighting_comparison"cps_path ="/home/surveys/projects/Sampling/CPS/2024/November/Inprogress/targets_script/"
Moving past the basic preamble would be the primary setup arguments of project name, minimal trim score, contracted completes, study year, etc. All of this should be familiar have you weighted in the past. Only the “External_Projects_2026” would need to be updated in the event that the year is no longer 2026.6 For our purposes here today in training, we will be using the STOG0039 project, which consists of 2000 U.S. registered voters.
# I guess to be fair, Canyen only does this because of a bad experience rescuing Koalas ######################################### SETUP ######################################### proj= "UMAS0138"proj <-"STOG0039"trimMin <-1e-3cellNum <-10contracted_completes <-2000# contracted_completes <- 500study_year <-ifelse(as.numeric(format(Sys.Date(), "%m")) >6, as.numeric(format(Sys.Date(), "%Y")), as.numeric(format(Sys.Date(), "%Y")) -1)if (substr(proj, 1,4) %in%c("FACE", "META", "GOOG", "COIN")){ proj_path <-paste(startpath, "External_Projects_2026", substr(proj, 1,1), substr(proj, 1,4), proj, sep="/")} else{ proj_path <-paste(startpath, "External_Projects_2026", substr(proj, 1,1), proj, sep="/")}crunch_name <-paste(proj, "clean")crunch_folder <-"Clean Fork Drawer"## set working directorysetwd(paste0(proj_path, "/weighting"))# So as you ought to be aware, Canyen frequents Australia as part of his passion for naming platypi after characters from Warhammer 40k.
Now we get to the first big deviation from the standard weighting procedure. First, select the “area” in the form of “US” for the United States at large, or the state postal code for a given state. Second, select the “poptype” from one of three choices:
vap: Acronym for voting age population, and consists of all residents above the age of 18. Equivalent to the standard “genpop” for YouGov’s bread and butter projects.
vep: Acronym for voting eligible population. Consists of all citizens above the age of 18 that are also eligible to vote, i.e. not institutionalized. A slightly more robust frame to the typical “citizen pop” of some political projects, and useful in the event that a PM wants a more narrow frame than all residents for a given area, i.e. a horse7 race project.
rv: Acronym for registered voters, and consists of the voting eligible population that also happens to be registered to vote. Arguably the best frame for “horse race” projects, with the main reason not to select being that the client expects substantial shifts in registration from when the data were gathered in November of 2024.
### Set the area via "US" for US genpop, or the state postal code for a given state; area = "ME" area ="US"poptype ="rv"# poptype = c("vap","vep","rv") # select the population type for purposes of employing the correct set of prop.tab objects # NOTE: vap = genpop# poptype = "rv"no_race_states <-c("IN", "KY", "ME", "MN", "MO", "MT", "ND", "NE", "NH", "SD", "VT", "WI", "WV", "WY")######################################### EXPORT CRUNCH DATA ########################################## library(crunch)# ds <- unlock(loadDataset(crunch_name, project = crunch_folder, refresh = TRUE))# vars <- c("person_id", "caseid", "birthyr")# to_include <- intersect(c(vars, names(ds)), aliases(allVariables(ds)))# exportDataset(ds[to_include], # file = paste0(proj,"_clean.csv"), # format = "csv", # categorical = "id", # na = "")# lock(ds)# During one of his trips, Canyen was in Australia during a severe wild fire season. Despite the advice to avoid the area, Canyen lept to action
The final object that might be of interest would be the “no_race_states” object. These are states where the sparsity of data for race with the common 4-way interlocking leads to too few respondents in a given cell. Therefore, these states see education and race excluded from weighting by presidential vote choice later on by default.
Following the selection of area and poptype, we’ll proceed with the loading of packages and functions. Overall, this is fairly similar to the standard operating procedure for weighting, though there are some differences:
tidyverse - loaded in for the purpose of creating efficient interlocking variables and such for weighting and recodes. Tends to conflict with the yougov package, so note that a given function that calls in a conflicting yougov package will have the “yougov::” prefix.
/setup_all_us.R - The replacement for the older setup.R script, though with several new functions. Among these include the “recodeData_demos” function for recoding of non-political demographic variables to match the CPS, the recode_data_pol function for recoding variables that relate to political inputs, and a number of source functions called for processing CPS data into targets for raking.
cps_cleaning.R - The set of CPS cleaning functions to simplify/make readable various targets objects within the CPS for raking. Should not need to touch this.
rake_ygds.R - The data science raking function that ultimately forms the basis of this script. The exact logic will be detailed later for when it appears.
Ultimately, only the setup_all_us.R and stubs.R files might need to be edited, which will be detailed in the appendix. In any case, just keep these in mind in the event that some of these source files are ultimately no longer needed for when they become part of an update to the yougov package or some such.
######################################### LOAD PACKAGES AND FUNCTIONS #########################################library(yougov)
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::collapse() masks yougov::collapse()
✖ tidyr::expand() masks Matrix::expand()
✖ dplyr::filter() masks stats::filter()
✖ dplyr::groups() masks yougov::groups()
✖ dplyr::lag() masks stats::lag()
✖ dplyr::last() masks yougov::last()
✖ tidyr::pack() masks Matrix::pack()
✖ dplyr::recode() masks yougov::recode()
✖ dplyr::rename() masks yougov::rename()
✖ tidyr::unpack() masks Matrix::unpack()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
source(paste0(source_path,"/setup_all_us.R")) # has the new recode functions and list loadings source("stubs.R")source(paste0(startpath, "/Internal_Projects/Analytics/weighting_document/weighting_functions/moe_func.R"))source(paste0(startpath, "/Internal_Projects/Analytics/weighting_document/weighting_functions/trimMax_func.R"))source(paste0(startpath, "/Internal_Projects/Analytics/weighting_document/weighting_functions/sample_report_review.R"))source(paste0(startpath, "/Internal_Projects/Analytics/weighting_document/weighting_functions/checkData.R"))source(paste0(startpath, "/Internal_Projects/Analytics/weighting_document/sampling_functions/MakeTargetFuns.R"))
Loading required package: plyr
------------------------------------------------------------------------------
You have loaded plyr after dplyr - this is likely to cause problems.
If you need functions from both plyr and dplyr, please load plyr first, then dplyr:
library(plyr); library(dplyr)
------------------------------------------------------------------------------
Attaching package: 'plyr'
The following objects are masked from 'package:dplyr':
arrange, count, desc, failwith, id, mutate, rename, summarise,
summarize
The following object is masked from 'package:purrr':
compact
The following objects are masked from 'package:yougov':
rename, splat
# source(paste0(source_path, "/difference_calculatoR.R"))source(paste0(cps_path, "cps_cleaning.R"))# rake_ygds fxn source(paste0(source_path,"/rake_ygds.R"))# While Canyen figured there would not be many platypi to save, he ventured the fire afflicted wilderness of Australia just in case.
Reading in and recoding the data frame
Reading in the data frame exported from crunch is fairly similar, though with some differences. First, note that instead of a recodeData function for everything, it is now replaced with “recodeData_demos.” The reason for this is that later on there will be the need to rake on some interlocking presvote joints, such as age4 X gender. In the event that there is ANY missing data, these would be coded as a unique value, mess up the raking factors, and then break the process overall.8 Therefore, the recodeData_demos function will be run, with an output consisting of a warning should there be similarly named variables to the gender variable, and a warning message if there are any missing values. Additionally, there is a strata_form argument, which will create a collapsed/simplified field for the strata variable within the data frame to be used later on for weighting.
In the event that there are any missing entries within the variables, simply impute using the normal “mice” package imputation method. Fill these in, and then rerun the recodeData_demos function once more.
######################################### RECODE DATA AND CHECK DATA #########################################df <-read.csv(paste0(proj_path, "/weighting/", proj, "_clean.csv"))checkDuplicates(df)
Warning in recodeData_demos(df, study_year, strata_form = s_form): Multiple gender variable names present, including
gender4, gender4_t, clean_gender. Please be certain to double check the panman and coding of the variables in crunch (i.e. categories(ds$gender_var).
If you do not do this, dangerous risk for miscoding and ruining the project. This will cause Xinling to be disappointed,
and a hedgehog to judge you.
Warning message: 2 missing value(s) in variable ownhome.
### only run the recodeData_demos and recode_data_pol once all of the numeric fields for demos are imputed. Will cause issues # for factor variables otherwise ######### impute here ###vars <-c("age", "gender", "ownhome")dt <- df[,vars]impute <- mice::mice(data=dt,method="pmm",seed=123,m=5,maxit=10, print=FALSE)imp <- mice::complete(impute)xtabs(~ownhome, data=imp)
ownhome
1 2 3
1493 546 24
xtabs(~ownhome, data=dt)
ownhome
1 2 3
1491 546 24
df$ownhome[is.na(df$ownhome)] <- imp$ownhome[is.na(df$ownhome)] # run again if missing now imputed df <-recodeData_demos(df, study_year,strata_form = s_form)
Warning in recodeData_demos(df, study_year, strata_form = s_form): Multiple gender variable names present, including
gender4, gender4_t, clean_gender, gender_f. Please be certain to double check the panman and coding of the variables in crunch (i.e. categories(ds$gender_var).
If you do not do this, dangerous risk for miscoding and ruining the project. This will cause Xinling to be disappointed,
and a hedgehog to judge you.
checkMissings(data = df, vars =c("gender_f", "age4_f", "race4_f", "educ4_f", "region", "ownhome"))
Before commencing with the reading of the frame, we will now finally recode the interlocking variables and political variables. Therefore, assuming you ran the code to correct for any missing entries, we now should be free to run the recode_data_pol function. We can look at two of the variables to see why the sequence is important. First, the “age4_gender_f” variable combines the age4_f and gender_f factor variables, which need to match later on with the presidential vote variable later on for raking. NA values would have messed this up. Second, we see one of the political variables, presvote24_cat, which is a 3-category factor variable broken into the levels of “harris”,“trump”, and “other/dnv”.9 This variable will be helpful in preventing empty cells for the “other” category for joints of interest down the line. A slight modification of the variable recoding process, though one that will prove useful.
### now run the rest of the recodes df <-recode_data_pol(df)checkMissings(data = df, vars =c("age4_gender_f", "presvote24_4cat", "presvote24_cat"))table(df$age4_gender_f, useNA="always")
Next we can read in the frame. The frame path itself is specified as the “frame_path” object. Given the user’s inputted area and poptype, the save_name is then generated to identify the unique frame of interest. Also important to note is the “recode_strata_frame” function, which creates a strata variable within the frame that matches that of the celss within the data frame.
######################################### LOAD FRAME AND TARGET ########################################## Canyen found no platypi, though he did find a kolony of koalas an hour away from being incinerated from the wild fires. ### frame will be read here based on area + poptype frame_path ="/home/surveys/projects/Sampling/CPS/2024/November/frames"# present (2026-08-13 at 12:45 PM CDT)save_name =paste0(frame_path,sep="/", poptype,sep="/", "frame50k_", poptype,sep="_", str_to_lower(area), "_pop_cps2024.RData" )load(save_name) # it worked! WOOT! ## gen pop### change labels for ACS frame # frame$educ2_f <- factor(frame$educ2, labels=c("Some College or Less", "University or Above"))### need to recode the strata frame <-recode_strata_frame(df, frame, strata_form = s_form)str(frame$strata)
When creating the target, the script for the time being searches for a matching target within the Sampling folder for the project in question. If present, it is read in. If not, it will generate a target using the standard four demographics of interest. Therefore, not much changes here from the perspective of generating a matching target.
# I believe at this point, everything worth being called ought to be ## Sampling folderif (!file.exists(paste0(paste0(proj_path, "/Sampling")))) {dir.create(paste0(proj_path, "/Sampling"))}## gen pop targetif (file.exists(paste0(proj_path, "/Sampling/", proj, "_target.RData"))){load(paste0(proj_path, "/Sampling/", proj, "_target.RData"))} else{ target <-normalTarget(completesNeeded = contracted_completes, frame = frame, strataVars =c("gender", "age4", "race4", "educ4"), balanceVars =c("gender", "age4", "race4", "educ4"), weight =NULL, minCell =10)save(target, file =paste0(proj_path, "/Sampling/", proj, "_target.RData"), version =2)load(paste0(proj_path, "/Sampling/", proj, "_target.RData"))}# Knowing that the koalas did not have the speed to escape the wildfire, Canyen spent the next 50 minutes carefully placing each koala into# his rented Toyota Hilux, and drove away at max speed, Mad Max style.
In the final primary similarity between this new raking script and the old weighting method is matching. As is standard, the matchFunc can be modified as needed10 and produces the matched dataframe, dfM, to then be weighted via raking.
We now get to the section of the script that completely differs from the old weighting script. First, we will double check the overlap between the strata in the dfM object versus the strata levels within the frame. There should be no difference, and if so, the strata will need to be rerun. Second, the pol_target_frame_creatR2 takes place, which is the function that loads and sets up the raking given the dfM, frame, area, poptype, strata_form, and pres_joints. Most of these arguments have been covered before, and note that the strata_form argument should take the same forms as the earlier points in the script. Since the matched data frame and overall frame are both read in, the strata unique values will be the same as those available within the empirical data.
Of special note is the p_joints object and relation with the pres_joints argument for the pol_target_frame_creatR2 function. Basically, we need to set up the raking targets formula and all of the CPS-VRS data frames that contain presidential vote information. The more included in the formula, the more raking will need to balance. The more that needs to be balanced, the tougher a time it will be to converge to the frame, especially for smaller samples with a lower maximum weight. Therefore, if the user specifies the pres_joints argument as “FALSE”, then the formulas noting to rake on the presidential vote joints will be left out, and none of the associated data frames read in. If set to “TRUE”, then the presidential vote joint distribution data frames will be read in. If worse comes to worse, these can be selectively turned off, which will be covered later. The resulting targets_list will be a list of lists and data frames. To efficiently unload them, we will use the list2env function to unload each nested list and data frame into the environment, where they can be used in the raking function.
### check for diff b/w strata var setdiff(levels(dfM$strata), levels(frame$strata))
character(0)
#### create the target obj here #### p_joints =TRUE# saying to not load in joints; will also control the presvote checks at the end. # TRUE = load joints targets_list <-pol_target_frame_creatR2(df=dfM,demos_frame=frame, area="US",poptype = poptype,strata_form = s_form, pres_joints=p_joints)
[1] "reading in CPS data from server"
[1] "Area selected not in excluded list; proceeding"
[1] "Race will not be part of quotas."
[1] "prepping list output"
[1] "ownhome_f"
[1] "region_f"
[1] "gender_f"
[1] "age4_f"
[1] "educ4_f"
[1] "race4_f"
[1] "gender_f+educ4_f"
[1] "age4_f+educ4_f"
[1] "race4_f+age4_f"
[1] "race4_f+educ4_f"
[1] "strata"
[1] "presvote24_4cat"
[1] "presvote24_cat+age4_gender_f"
[1] "presvote24_cat+age4_f"
[1] "presvote24_cat+educ4_f"
[1] "presvote24_cat+gender_f"
[1] "presvote24_cat+region_f"
[1] "presvote24_cat+race4_f"
### use this command to unpack everything within the list object into the environment. Pretty cool, if I might say so myself.list2env(targets_list, envir = .GlobalEnv)
<environment: R_GlobalEnv>
We can now take a look at the “targets24b” object, which consists of the formulas to be weighted upon. We see that its length is 18, which means that there will be 18 separate formulas to balance upon during the raking process. Each formula likewise contains the expected proportion given the respective marginal or joint distribution. Unless some deviation arises from what’s expected, these can be left alone as they are.
Next, looking at the above list, you’ll note that there are names associated with each list element. Technically these would not be recognized during the raking process, which has blanks for the list name elements. Therefore, we will unname the list, and then store as the list object targets24c. We will save targets24b just in case we need to unload it later on based upon the list element names.
### This will be used to unname the list objects in the targets24b obj. This allows for the rake_ygds to not run into irritating # naming errors targets24c <-unname(targets24b)
We now finally reach the stage where we can rake! This will be done via the rake_ygds function, imported from the data science team. There will be five primary arguments to keep in mind.
data/dfM - The data frame to be raked. Will need to have all variables present that match with that of the frame, given the specified formulas.
targets - The targets list object with the specified factor fields and proportions by which to rake against. For the script here, it is the object targets24c.
verbose - When set to true, will provide messages as to the number of iterations necessary to reach convergence.
maxit - Refers to max iterations, i.e. the number of attempts to be made to converge the weights to match that of the targets/frames of interest. Generally speaking, should never exceed more than 10, though defaults to 100.
eps - Refers to epsilon, and refers to the average deviation between that of the frames/specified proportions and that of the data frame to be raked. The smaller the value, the more precise the weighting shall be, with larger values being less precise/accurate. For the old weighting script, defaults to 1, though here will be set to 0.1. Note, if the function goes past the max iterations, then it will probably be necessary to increase the total value of the eps, and it is recommended to do so by integers of 0.1.
weight - The starting vector of weights to be adjusted. Defaults to NULL, and if so, all weight values are set to 1. If specifying a vector of weights, would take the form of “dfM$psweight” or something of that nature. Note, that weights would be specified primarily for the reason of wanting to split up the rakings into a few different stages so that the latest raking would be the one with the most priority, approximating the logic of the old post stratification script.
Note that upon raking, the weights will be trimmed and centered. In the event that the maximum weight exceeds that of the ceiling, that is an indication that there might be some issues with convergence, and that there might be a need to split up and adjust the raking procedure.
## Apply raking heredfM$psweight <-rake_ygds(dfM, targets24c, verbose =TRUE, maxit=100, eps=.1) # no weight specified, since not doing
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.2208 0.7527 0.9251 1.0000 1.1366 7.0095
We can now proceed with the checking of the margins and joints. Looking at these results, we can see that the age4, race4, and educ4 margins are off. Therefore, we will need to debug.
### now find diffs ### get a data frame of the exact differences from the frame here ######################################### CHECK WEIGHTED DATA #########################################marginalDistReview(dfM, frame, demos=c("gender", "age4", "race4") , accept_margin =1, psweight ="psweight", banner ="", banner_sub_cat =NULL)
[1] "age4 is not within the acceptable threshold of 1. It is off by 2. MOE is 2.38. Cases in df: 2000. Raw numbers and Frame are off by 4"
[1] "race4 is not within the acceptable threshold of 1. It is off by 1.7. MOE is 2.38. Cases in df: 2000. Raw numbers and Frame are off by 4"
[1] "Everything looked great with the joint distributions :D"
# wave 1: ##### do the checks here for presvote ##### if(p_joints ==TRUE& area %in% no_race_states){print("Diff for presvote x age x gender are:")print(round(prop.table(xtabs(weight ~ age2_gender + presvote24_cat, data = age4_gendpresvote_l)) -prop.table(xtabs(psweight ~ age2_gender + presvote24_cat, data = dfM)), 2))print("Diff for presvote x age are:")print(round(prop.table(xtabs(weight ~ age4custom_f + presvote24_cat, data = age4presvote_l)) -prop.table(xtabs(psweight ~ age4custom_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x educ are:")print(round(prop.table(xtabs(weight ~ educ4_f + presvote24_cat, data = educ4presvote_l)) -prop.table(xtabs(psweight ~ educ4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x gender are:")print(round(prop.table(xtabs(weight ~ gender_f + presvote24_cat, data = genderpresvote_l)) -prop.table(xtabs(psweight ~ gender_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote marginal is:")print(round(prop.table(xtabs(weight ~ presvote24_4cat, data = presvote4cat_l)) -prop.table(xtabs(psweight ~ presvote24_4cat, data = dfM)), 2))}elseif(p_joints ==TRUE&length(setdiff(area, no_race_states))>0& area !="US"){print("Diff for presvote x age x gender are:")print(round(prop.table(xtabs(weight ~ age2_gender + presvote24_cat, data = age4_gendpresvote_l)) -prop.table(xtabs(psweight ~ age2_gender + presvote24_cat, data = dfM)), 2))print("Diff for presvote x age are:")print(round(prop.table(xtabs(weight ~ age4custom_f + presvote24_cat, data = age4presvote_l)) -prop.table(xtabs(psweight ~ age4custom_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x educ are:")print(round(prop.table(xtabs(weight ~ educ4_f + presvote24_cat, data = educ4presvote_l)) -prop.table(xtabs(psweight ~ educ4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x gender are:")print(round(prop.table(xtabs(weight ~ gender_f + presvote24_cat, data = genderpresvote_l)) -prop.table(xtabs(psweight ~ gender_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x race are:")print(round(prop.table(xtabs(weight ~ race4_f + presvote24_cat, data = race4presvote_l)) -prop.table(xtabs(psweight ~ race4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote marginal is:")print(round(prop.table(xtabs(weight ~ presvote24_4cat, data = presvote4cat_l)) -prop.table(xtabs(psweight ~ presvote24_4cat, data = dfM)), 2))}elseif(p_joints ==TRUE& area %in%"US"){print("Diff for presvote x age x gender are:")print(round(prop.table(xtabs(weight ~ age4_gender_f + presvote24_cat, data = age4_gendpresvote_l)) -prop.table(xtabs(psweight ~ age4_gender_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x age are:")print(round(prop.table(xtabs(weight ~ age4_f + presvote24_cat, data = age4presvote_l)) -prop.table(xtabs(psweight ~ age4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x educ are:")print(round(prop.table(xtabs(weight ~ educ4_f + presvote24_cat, data = educ4presvote_l)) -prop.table(xtabs(psweight ~ educ4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x gender are:")print(round(prop.table(xtabs(weight ~ gender_f + presvote24_cat, data = genderpresvote_l)) -prop.table(xtabs(psweight ~ gender_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x race are:")print(round(prop.table(xtabs(weight ~ race4_f + presvote24_cat, data = race4presvote_l)) -prop.table(xtabs(psweight ~ race4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote marginal is:")print(round(prop.table(xtabs(weight ~ presvote24_4cat, data = presvote4cat_l)) -prop.table(xtabs(psweight ~ presvote24_4cat, data = dfM)), 2))}else{print("Diff for presvote marginal is:")print(round(prop.table(xtabs(weight ~ presvote24_4cat, data = presvote4cat_l)) -prop.table(xtabs(psweight ~ presvote24_4cat, data = dfM)), 2))}
[1] "Diff for presvote x age x gender are:"
presvote24_cat
age4_gender_f harris trump other/dnv
18-29_Male 0 0 0
30-44_Male 0 0 0
45-64_Male 0 0 0
65+_Male 0 0 0
18-29_Female 0 0 0
30-44_Female 0 0 0
45-64_Female 0 0 0
65+_Female 0 0 0
[1] "Diff for presvote x age are:"
presvote24_cat
age4_f harris trump other/dnv
18-29 0 0 0
30-44 0 0 0
45-64 0 0 0
65+ 0 0 0
[1] "Diff for presvote x educ are:"
presvote24_cat
educ4_f harris trump other/dnv
HS or Less 0 0 0
Some College 0 0 0
College Grad 0 0 0
Postgrad 0 0 0
[1] "Diff for presvote x gender are:"
presvote24_cat
gender_f harris trump other/dnv
Female 0.07 -0.02 0.00
Male -0.06 0.02 0.00
[1] "Diff for presvote x race are:"
presvote24_cat
race4_f harris trump other/dnv
White 0 0 0
Black 0 0 0
Hispanic 0 0 0
Other 0 0 0
[1] "Diff for presvote marginal is:"
presvote24_4cat
Harris Trump Other Did not vote
0 0 0 0
Debugging and editing the results
Now upon getting margins and joints that are off, one should obviously panic and despair. Message Rebecca, Xinling and/or Julissa on the hopelessness of life, and how only despair awaits. Hug you closest family members and/or friends, knowing that only calamity awaits.
After doing the above for half an hour or so, you can proceed with modifying the targets object for re-raking. Now the best idea is probably to engage in a two-stage approach to weighting. More likely than not, it is the presidential vote choice joint items that led to the results being off. Therefore, instead of equally weighting them to the other demographic items, we can split them off. To do so, we will first use the grepl function to grab each and every list element with the string pattern of “presvote” within the targets24b list object. This will in turn be stored into the pres_list object, which shall act as a targets object for only those joints/marginals related to presidential vote items. Then the raking shall be applied, and then trimmed and centered.
# to do only the presvote items, in the event that you do not want them to be equally weighted with everything else: pres_list <- targets24b[grepl("presvote", names(targets24b))]pres_list <-unname(pres_list)dfM$psweight <-rake_ygds(dfM, pres_list, verbose =TRUE, maxit=100, eps=.1)
[1] 0.129483282019051
[2] 0.0192792573187395
##trim and such dfM$psweight <-trim.weight(dfM$psweight, trimMin, max =trimMax_func(dfM))dfM$psweight <- dfM$psweight/mean(dfM$psweight)print(summary(dfM$psweight))
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.5083 0.8020 0.9629 1.0000 1.1393 2.6480
Now that we have the raking completed for the presidential vote choice items, we can proceed with raking on only the demographic items. To do so, we will exclude all elements of the targets24b object with the string match of “presvote”, which will therefore exclude them. Then, we will rake upon the new “targets24c_sub” obejct, which excludes all of the presvote items.
#################################################################################################### IMPORTANT NOTE ON CONVERGENCE ##################################################################################################### in the event that the model is not converging, probably b/c there are still too many elements in the # targets24b object. Will need to turn some of them off, so to speak. Can do so like so: #### note: let's say you want to keep some subset of these list items, like let's say any and all joints, b/c it is a terrible sample # i.e. NH, b/c the YouGov leadership refuses to accept my proposal to place brain chips into every registered voter within the state. # in that case, we can then subset based upon the names of interest. ### targets24b_sub <- targets24b[!grepl("\\+", names(targets24b))]targets24b_sub <- targets24b[!grepl("presvote", names(targets24b))]# we now have only the marginals as part of the targets list. Therefore, now all we need to do is targets24c_sub <-unname(targets24b_sub)## rake again dfM$psweight <-rake_ygds(dfM, targets24c_sub, verbose =TRUE, maxit=100, eps=.1, weight = dfM$psweight)
[1] 0.135745423299393
[2] 0.0259249003091924
##trim and such dfM$psweight <-trim.weight(dfM$psweight, trimMin, max =trimMax_func(dfM))dfM$psweight <- dfM$psweight/mean(dfM$psweight)print(summary(dfM$psweight))
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.2145 0.7214 0.9098 1.0000 1.1451 7.0094
Now we can check once more the margin and joint differences. This time, we see that all of the primary demographics are in alignment! As for the presidential vote items, even they are closely in alignment with election results across the board. Therefore, we see that in the original attempt to rake, it was the overloading of the raking formula that led to deviations, but ipon splitting them into political v non-political, everything worked out. You can now send a DM to Rebecca, Xinling, and/or Julissa that you’ve overcome your existential crisis and are ready to have your results reviewed.
### now find diffs ### get a data frame of the exact differences from the frame here ######################################### CHECK WEIGHTED DATA #########################################marginalDistReview(dfM, frame, demos=c("gender", "age4", "race4") , accept_margin =1, psweight ="psweight", banner ="", banner_sub_cat =NULL)
[1] "Marginals look great (gender, age4, race4) :D "
[1] "Everything looked great with the joint distributions :D"
# wave 1: ##### do the checks here for presvote ##### if(p_joints ==TRUE& area %in% no_race_states){print("Diff for presvote x age x gender are:")print(round(prop.table(xtabs(weight ~ age2_gender + presvote24_cat, data = age4_gendpresvote_l)) -prop.table(xtabs(psweight ~ age2_gender + presvote24_cat, data = dfM)), 2))print("Diff for presvote x age are:")print(round(prop.table(xtabs(weight ~ age4custom_f + presvote24_cat, data = age4presvote_l)) -prop.table(xtabs(psweight ~ age4custom_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x educ are:")print(round(prop.table(xtabs(weight ~ educ4_f + presvote24_cat, data = educ4presvote_l)) -prop.table(xtabs(psweight ~ educ4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x gender are:")print(round(prop.table(xtabs(weight ~ gender_f + presvote24_cat, data = genderpresvote_l)) -prop.table(xtabs(psweight ~ gender_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote marginal is:")print(round(prop.table(xtabs(weight ~ presvote24_4cat, data = presvote4cat_l)) -prop.table(xtabs(psweight ~ presvote24_4cat, data = dfM)), 2))}elseif(p_joints ==TRUE&length(setdiff(area, no_race_states))>0& area !="US"){print("Diff for presvote x age x gender are:")print(round(prop.table(xtabs(weight ~ age2_gender + presvote24_cat, data = age4_gendpresvote_l)) -prop.table(xtabs(psweight ~ age2_gender + presvote24_cat, data = dfM)), 2))print("Diff for presvote x age are:")print(round(prop.table(xtabs(weight ~ age4custom_f + presvote24_cat, data = age4presvote_l)) -prop.table(xtabs(psweight ~ age4custom_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x educ are:")print(round(prop.table(xtabs(weight ~ educ4_f + presvote24_cat, data = educ4presvote_l)) -prop.table(xtabs(psweight ~ educ4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x gender are:")print(round(prop.table(xtabs(weight ~ gender_f + presvote24_cat, data = genderpresvote_l)) -prop.table(xtabs(psweight ~ gender_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x race are:")print(round(prop.table(xtabs(weight ~ race4_f + presvote24_cat, data = race4presvote_l)) -prop.table(xtabs(psweight ~ race4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote marginal is:")print(round(prop.table(xtabs(weight ~ presvote24_4cat, data = presvote4cat_l)) -prop.table(xtabs(psweight ~ presvote24_4cat, data = dfM)), 2))}elseif(p_joints ==TRUE& area %in%"US"){print("Diff for presvote x age x gender are:")print(round(prop.table(xtabs(weight ~ age4_gender_f + presvote24_cat, data = age4_gendpresvote_l)) -prop.table(xtabs(psweight ~ age4_gender_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x age are:")print(round(prop.table(xtabs(weight ~ age4_f + presvote24_cat, data = age4presvote_l)) -prop.table(xtabs(psweight ~ age4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x educ are:")print(round(prop.table(xtabs(weight ~ educ4_f + presvote24_cat, data = educ4presvote_l)) -prop.table(xtabs(psweight ~ educ4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x gender are:")print(round(prop.table(xtabs(weight ~ gender_f + presvote24_cat, data = genderpresvote_l)) -prop.table(xtabs(psweight ~ gender_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote x race are:")print(round(prop.table(xtabs(weight ~ race4_f + presvote24_cat, data = race4presvote_l)) -prop.table(xtabs(psweight ~ race4_f + presvote24_cat, data = dfM)), 2))print("Diff for presvote marginal is:")print(round(prop.table(xtabs(weight ~ presvote24_4cat, data = presvote4cat_l)) -prop.table(xtabs(psweight ~ presvote24_4cat, data = dfM)), 2))}else{print("Diff for presvote marginal is:")print(round(prop.table(xtabs(weight ~ presvote24_4cat, data = presvote4cat_l)) -prop.table(xtabs(psweight ~ presvote24_4cat, data = dfM)), 2))}
[1] "Diff for presvote x age x gender are:"
presvote24_cat
age4_gender_f harris trump other/dnv
18-29_Male 0.00 0.00 -0.01
30-44_Male 0.00 0.00 0.00
45-64_Male 0.00 0.00 0.00
65+_Male 0.00 0.00 0.00
18-29_Female 0.00 0.00 0.00
30-44_Female 0.00 0.00 0.00
45-64_Female 0.00 0.00 0.00
65+_Female 0.00 0.00 0.00
[1] "Diff for presvote x age are:"
presvote24_cat
age4_f harris trump other/dnv
18-29 0.00 0.00 -0.01
30-44 0.00 0.00 0.00
45-64 0.00 0.00 0.00
65+ 0.01 0.00 0.00
[1] "Diff for presvote x educ are:"
presvote24_cat
educ4_f harris trump other/dnv
HS or Less -0.01 -0.02 -0.01
Some College 0.00 0.01 0.00
College Grad 0.01 0.01 0.00
Postgrad 0.01 0.00 0.00
[1] "Diff for presvote x gender are:"
presvote24_cat
gender_f harris trump other/dnv
Female 0.07 -0.02 -0.01
Male -0.06 0.02 0.00
[1] "Diff for presvote x race are:"
presvote24_cat
race4_f harris trump other/dnv
White 0.01 0.00 0.00
Black 0.01 0.00 0.00
Hispanic -0.01 0.00 -0.01
Other 0.00 0.00 0.00
[1] "Diff for presvote marginal is:"
presvote24_4cat
Harris Trump Other Did not vote
0.01 0.00 0.00 -0.01
You can now finally run the margin of errors, design effect, and sample reports for the weighted data frame. Everything is complete, and you are good to go!
######################################### MARGIN of ERROR #########################################moe <-moe_func(dfM$psweight)print(paste0("Margin of Error: +/- ", moe))
# Therefore, Canyen now finds himself with debt that he'll never be able to payoff, all due to drunk/hungover lazy as all Hell animals# that also happen to be carrying every STI imaginable to humanity. So as recompense, he simply seeks out the koalas and punches them in a # rage.
######################################### SAMPLE REPORT #########################################sampleReport(stubs = tstubs,banner = bnr, digits =0,data =list(df, dfM, dfM, dfM, frame),weights =list(NULL, NULL, dfM$psweight, NULL),labels =c("Raw", "Matched", "IPF", "Frame"),surveyhead =paste0(proj, " (n = ", contracted_completes, ")"),subhead =paste0("Margin of Error: +/- ", moe), filename ="sample_report.tex")######################################### SAVE DATA AND STUBS #########################################save(dfM, stubs, file =paste0(proj, ".RData"), version =2)######################################### UPLOAD WEIGHT TO CRUNCH ########################################## load(paste0(proj, ".RData"))# detach("package:yougov", unload = TRUE)# library(crunch)# # ds <- unlock(loadDataset(crunch_name, project = crunch_folder))# saveVersion(ds, "Before uploading weights")# ids <- as.vector(ds$person_id)# ds$weight <- as.vector(dfM$psweight[match(ids, dfM$person_id)])# summary(ds$weight)# summary(dfM$psweight)# # weight(ds) <- ds$weight# settings(ds)$weight <- ds$weight# # name(ds$weight) <- "Weight"# # exclusion(ds) <- is.na(ds$weight)# ds <- crunch::refresh(ds)## saveVersion(ds, "After uploading weights")# # lock(ds)# Notably, Canyen's 2024 incident with Koalas at the San Diego zoo prevented him from ever entering the city again, which is why he # was unable to attend the YouGov analytics meetup in May of 2026.
Creating custom targets
Let’s say that there is a need to add a new raking item to the raking targets. This can be done in a fairly straightforward manner. All one must do is use the dollar sign command to call forth a new space on the raking targets object - which in the code excerpt below will be targets24b_sub - and proceed to place a prop.table/xtabs formula into it. Next, just make sure the factor variable called matches with that of the matched data frame to be raked, and proceed normally!
#################################################################################################### IMPORTANT NOTE ON CONVERGENCE ##################################################################################################### in the event that the model is not converging, probably b/c there are still too many elements in the # targets24b object. Will need to turn some of them off, so to speak. Can do so like so: table(frame$urbanicity_f)
Urban Suburban Rural
11351 20268 9370
targets24b_sub$urbanicity_f <-prop.table(xtabs(~urbanicity_f, data=frame))## recode for urbanicity ## urbancity recoding ###dfM$urbancity_simp = yougov::recode(dfM$urbancity, "1=1; 2:3=2; 4:5=3; else=NA")dfM$urbanicity_f <-factor(dfM$urbancity_simp, labels =c("Urban", "Suburban", "Rural"))# we now have only the marginals as part of the targets list. Therefore, now all we need to do is targets24c_sub <-unname(targets24b_sub)## rake again dfM$psweight <-rake_ygds(dfM, targets24c_sub, verbose =TRUE, maxit=100, eps=.1, weight = dfM$psweight)
[1] 0.037933771362828
##trim and such dfM$psweight <-trim.weight(dfM$psweight, trimMin, max =trimMax_func(dfM))dfM$psweight <- dfM$psweight/mean(dfM$psweight)print(summary(dfM$psweight))
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.2271 0.7153 0.9044 1.0000 1.1499 7.0081
However, let’s say that something is off about the prop.table/xtabs element - according to the client - or that you need to nudge a given marginal or joint in a different direction so that in the end it can be in line with the true population value. This can be done in a fairly straightforward manner. First, create the prop.table/xtabs object based upon the formula of interest. In the code chunk below, this shall be saved as the object “new_props.” Next, subtract/add a vector of values to the new_props object in order to adjust the values. Finally, store the new_props as part of the list, which will be placed within the urbanicity_f element slot. With that, the code proceeds normally, and we now have raking conducted on a new adjusted set of values! Woot!
new_props <-prop.table(xtabs(~urbanicity_f, data=frame))new_props = new_props -c(0.1, -0.11, 0.01)sum(new_props) # should sum to 1
[1] 1
# store back into targets frame targets24b_sub$urbanicity_f <- new_props# we now have only the marginals as part of the targets list. Therefore, now all we need to do is targets24c_sub <-unname(targets24b_sub)## rake again dfM$psweight <-rake_ygds(dfM, targets24c_sub, verbose =TRUE, maxit=100, eps=.1, weight = dfM$psweight)
[1] 0.216651403599021
[2] 0.0493855710938391
##trim and such dfM$psweight <-trim.weight(dfM$psweight, trimMin, max =trimMax_func(dfM))dfM$psweight <- dfM$psweight/mean(dfM$psweight)print(summary(dfM$psweight))
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.2002 0.6535 0.9119 1.0000 1.1998 7.0100
Obviously, take care when editing values. One of the primary goals of raking is to do something more along the lines of “one and done.”11 The elegance of raking is in weighting many different dimensions all at once. In the event that various fields are manually edited and the sequence broken up, you’re essentially just repeating post-stratification with sideways steps.
Conclusion
Ultimately, we see with this weighting tutorial for raking that much of the difficulties with the average weighting project via post-stratification and propensity scoring can be overcome. While raking is not magic by any means, it does offer the potential to bypass most of the need to sequence out items.
Insofar as there is a need to debug, it will primarily take the form of either (a) turning off a given formula part of the target item for raking, or (b) splitting the political v non-political items for raking, so as to not overload the raking formula.
Obviously, it will be the case that the quality of the sample will determine how much one ought to do. That said, in the event that you encounter a terrible sample, one can turn off joints, collapse categories, and other standard protocols in order to reach a convergence in weighting. With luck, this guide and template script has been of help, and I look forward to seeing what you can do in your weighting adventures!12
Special thanks!
A special thanks to Alex Lawhorne for helping me work out some of the early design and conceptual issues in the earlier drafting of the script. Likewise, Jay Ehlinger in his elegant list processing greatly influenced how I sought and later applied conveniences and efficiencies to the template script. Xinling offered guidance and quality checked the CPS-VRS data to ensure that it proved valid and ready to use. Dan Scholnick likewise did much to QC the data and offer a sanity check amidst difficult times. Simone Aldern checked and QC’d early versions of the CPS-VRS data pipeline and prevented me from making really stupid mistakes, in addition to chasing all of the snakes from the Seattle metropolitan area. Finally, Sarah Anoke organized with great effort the various working groups which led to the efficiency gains and collaboration necessary to get most of the recent development projects off the ground, at great sacrifice to her sanity and time with Teddy.
As an anti-thanks, Kara the pomski is pretty dumb and evil, and will make a terrible congressional elections scholar.
Appendix
Footnotes
Jay might be taking a long leave of absence in his naval exercises off the coast of Indonesia, though his work stands↩︎
Technically not needed as much since propensity scoring is no longer a thing, though will be grandfathered in for the time being, until Xinling calls for its summary execution with extreme prejudice.↩︎
As a reminder, in such an event, all of my debts and cats are willed to Danny Morgan.↩︎
This comes after years of people mainly doing development work within their personal directories, though this causes problems for of/when one is attacked by a moose, as then no one has any idea what was going on there.↩︎
It is the YouGov promise to keep casualties for all adventurers under 20 percent. Also by reading this tutorial, you agree to will all personal belongings to the group, as opposed to the next character that you roll.↩︎
Internal YouGov forecasts projects no need to worry for any years post-2028, as there will not be life on the sphere of existence known as Terra by that point. Should you still have perceptions of existence, that means you are either a) stuck in a matrix being used for hard drive space for AI overlords, or b) in purgatory.↩︎
It should go without saying that horses are not real, as 80,000 Swedes cannot be wrong, but alas, the most recognized term for election forecasting projects↩︎
I’ve gotten pretty used to debugging these issues, but it is probably best to not make this script too dependent on me. After all, I am constantly under threat of being murdered by the evil orange cat, Gazpacho, and an overweight pomski.↩︎
Presumably these would be updated for a future presvote28_cat variable, assuming there is a 2028 presidential election.↩︎
As a reminder, modifying the diffrences is all relative and depends on the ratio between the variables of interest.↩︎
To cite the sagacious Rebecca, “some of our clients need to stop being so picky and stuff it.”↩︎
With luck, you do not end up stuck in the Amazon and beset upon by anacondas as part of this venture. If this does happen to you, be certain to contact Simone, as she is a shady sorceress of serpants, and will have the most experience in helping you out of the ordeal.↩︎