This example will cover the use of R functions for analyzing complex survey data. Most social and health surveys are not simple random samples of the population, but instead consist of respondents from a complex survey design. These designs often stratify the population based on one or more characteristics, including geography, race, age, etc. In addition the designs can be multi-stage, meaning that initial strata are created, then respondents are sampled from smaller units within those strata. An example would be if a school district was chosen as a sample strata, and then schools were then chosen as the primary sampling units (PSUs) within the district. From this 2 stage design, we could further sample classrooms within the school (3 stage design) or simply sample students (or whatever our unit of interest is).

A second feature of survey data we often want to account for is differential respondent weighting. This means that each respondent is given a weight to represent how common that particular respondent is within the population. This reflects the differenital probability of sampling based on respondent characteristics. As demographers, we are also often interested in making inference for the population, not just the sample, so our results must be generalizable to the population at large. Sample weights are used in the process as well.

When such data are analyzed, we must take into account this nesting structure (sample design) as well as the respondent sample weight in order to make valid estimates of ANY statistical parameter. If we do not account for design, the parameter standard errors will be incorrect, and if we do not account for weighting, the parameters themselves will be incorrect and biased.

In general there are typically three things we need to find in our survey data codebooks: The sample strata identifier, the sample primary sampling unit identifier (often called a cluster identifier) and the respondent survey weight. These will typically have one of these names and should be easily identifiable in the codebook.

Statistical software will have special routines for analyzing these types of data and you must be aware that the diversity of statistical routines that generally exists will be lower for analyzing complex survey data, and some forms of analysis may not be available!

Below I illustrate the use of survey characteristics when conducting descriptive analysis of a survey data set and a linear regression model estimated from that data. For this example I am using 2011 CDC Behavioral Risk Factor Surveillance System (BRFSS) SMART county data. Link

#load brfss
library(car)
library(stargazer)
## 
## Please cite as: 
## 
##  Hlavac, Marek (2014). stargazer: LaTeX code and ASCII text for well-formatted regression and summary statistics tables.
##  R package version 5.1. http://CRAN.R-project.org/package=stargazer
load("~/Google Drive/dem7283/data/brfss_11.Rdata")

#The names in the data are very ugly, so I make them less ugly
nams<-names(brfss_11)
head(nams, n=40)
##  [1] "_STATE"   "precall"  "fmonth"   "intvid"   "seqno"    "_PSU"    
##  [7] "ctelenum" "cellfon"  "pvtresid" "genhlth"  "physhlth" "menthlth"
## [13] "poorhlth" "HLTHPLN1" "PERSDOC2" "medcost"  "CHECKUP1" "BPHIGH4" 
## [19] "bpmeds"   "bloodcho" "cholchk"  "TOLDHI2"  "CVDINFR4" "CVDCRHD4"
## [25] "CVDSTRK3" "ASTHMA3"  "asthnow"  "chcscncr" "chcocncr" "chccopd" 
## [31] "HAVARTH3" "ADDEPEV2" "chckidny" "chcvison" "DIABETE3" "SMOKE100"
## [37] "SMOKDAY2" "STOPSMK2" "LASTSMK2" "USENOW3"
#we see some names are lower case, some are upper and some have a little _ in the first position. This is a nightmare.
newnames<-gsub(pattern = "_",replacement =  "",x =  nams)
names(brfss_11)<-tolower(newnames)

#Poor or fair self rated health
brfss_11$badhealth<-ifelse(brfss_11$genhlth %in% c(4,5),1,0)

#race/ethnicity
brfss_11$black<-recode(brfss_11$racegr2, recodes="2=1; 9=NA; else=0")
brfss_11$white<-recode(brfss_11$racegr2, recodes="1=1; 9=NA; else=0")
brfss_11$other<-recode(brfss_11$racegr2, recodes="3:4=1; 9=NA; else=0")
brfss_11$hispanic<-recode(brfss_11$racegr2, recodes="5=1; 9=NA; else=0")

#insurance
brfss_11$ins<-ifelse(brfss_11$hlthpln1==1,1,0)

#income grouping
brfss_11$inc<-ifelse(brfss_11$incomg==9, NA, brfss_11$incomg)

#education level
brfss_11$educ<-recode(brfss_11$educa, recodes="1:2='0Prim'; 3='1somehs'; 4='2hsgrad'; 5='3somecol'; 6='4colgrad';9=NA", as.factor.result=T)
brfss_11$educ<-relevel(brfss_11$educ, ref='2hsgrad')

#employment
brfss_11$employ<-recode(brfss_11$employ, recodes="1:2='Employed'; 2:6='nilf'; 7='retired'; 8='unable'; else=NA", as.factor.result=T)
brfss_11$employ<-relevel(brfss_11$employ, ref='Employed')

#marital status
brfss_11$marst<-recode(brfss_11$marital, recodes="1='married'; 2='divorced'; 3='widowed'; 4='separated'; 5='nm';6='cohab'; else=NA", as.factor.result=T)
brfss_11$marst<-relevel(brfss_11$marst, ref='married')

#Age cut into intervals
brfss_11$agec<-cut(brfss_11$age, breaks=c(0,24,39,59,79,99))

#BMI, in the brfss_11a the bmi variable has 2 implied decimal places, so we must divide by 100 to get real bmi's
brfss_11$bmi<-brfss_11$bmi5/100

Analysis

First, we will do some descriptive analysis, such as means and cross tabulations.

#First we will do some tables
#Raw frequencies
table(brfss_11$badhealt, brfss_11$educ)
##    
##     2hsgrad 0Prim 1somehs 3somecol 4colgrad
##   0   48414  2932    7473    53446    88637
##   1   14546  2920    4525    11108     8343
#column percentages
prop.table(table(brfss_11$badhealt, brfss_11$educ), margin=2)
##    
##        2hsgrad      0Prim    1somehs   3somecol   4colgrad
##   0 0.76896442 0.50102529 0.62285381 0.82792701 0.91397195
##   1 0.23103558 0.49897471 0.37714619 0.17207299 0.08602805
#basic chi square test of independence
chisq.test(table(brfss_11$badhealt, brfss_11$educ))
## 
##  Pearson's Chi-squared test
## 
## data:  table(brfss_11$badhealt, brfss_11$educ)
## X-squared = 14577.74, df = 4, p-value < 2.2e-16

So basically all of these numbers are incorrect, since they all assume random sampling. Now, we must tell R what the survey design is and what the weight variable is, then we can re-do these so they are correct.

#Create a survey design object
#we must use functions in a new library, called survey
library(survey)
## Loading required package: grid
## 
## Attaching package: 'survey'
## 
## The following object is masked from 'package:graphics':
## 
##     dotchart

Now we identify the survey design. ids = PSU identifers, strata=strata identifiers, weights=case weights, data= the data frame where these variables are located. Lastly, I only include respondents with NON-MISSING case weights.

options(survey.lonely.psu = "adjust")
des<-svydesign(ids=~1, strata=~ststr, weights=~cntywt, data = brfss_11[is.na(brfss_11$cntywt)==F,] )

#re-do the analysis from above using only weights
library(questionr)
wtd.table(brfss_11$badhealt, brfss_11$educ, weights = brfss_11$cntywt)
##    2hsgrad    0Prim  1somehs 3somecol 4colgrad
## 0 20995409  2765052  6789606 26263911 28808091
## 1  5099712  2282053  3009468  4263418  2223490
prop.table(wtd.table(brfss_11$badhealth, brfss_11$educ, weights = brfss_11$cntywt), margin=2)
##      2hsgrad      0Prim    1somehs   3somecol   4colgrad
## 0 0.80457219 0.54784913 0.69288237 0.86034095 0.92834751
## 1 0.19542781 0.45215087 0.30711763 0.13965905 0.07165249
#compare that with the original
prop.table(table(brfss_11$badhealth, brfss_11$educ), margin=2)
##    
##        2hsgrad      0Prim    1somehs   3somecol   4colgrad
##   0 0.76896442 0.50102529 0.62285381 0.82792701 0.91397195
##   1 0.23103558 0.49897471 0.37714619 0.17207299 0.08602805

There are differences, notably that the prevalence of poor SRH is higher in the sample than the population. This is important!

Let’s say we also want the standard errors of these percentages. This can be found for a proportion by: \(s.e. (p)={\sqrt {p(1-p)} \over {n}}\)

So we need to get n and p, that’s easy:

n<-table(is.na(brfss_11$badhealth)==F)
n
## 
##   TRUE 
## 243403
p<-prop.table(wtd.table(brfss_11$badhealth, brfss_11$educ, weights = brfss_11$cntywt), margin=2)[2,]
se<-sqrt((p*(1-p))/n)

data.frame(proportion=p, se=se)
##          proportion           se
## 2hsgrad  0.19542781 0.0008037347
## 0Prim    0.45215087 0.0010088096
## 1somehs  0.30711763 0.0009350163
## 3somecol 0.13965905 0.0007025980
## 4colgrad 0.07165249 0.0005227668

Which shows us the errors in the estimates based on the weighted proportions. That’s nice, but since we basically inflated the n to be the population of the US, these se’s are too small. This is another example of using survey statistical methods, to get the right se for a statistic.

#Now consider the full sample design + weights
svytable(~badhealth+educ, design = des)
##          educ
## badhealth  2hsgrad    0Prim  1somehs 3somecol 4colgrad
##         0 20995409  2765052  6789606 26263911 28808091
##         1  5099712  2282053  3009468  4263418  2223490
prop.table(svytable(~badhealth+educ, design = des), margin = 2)
##          educ
## badhealth    2hsgrad      0Prim    1somehs   3somecol   4colgrad
##         0 0.80457219 0.54784913 0.69288237 0.86034095 0.92834751
##         1 0.19542781 0.45215087 0.30711763 0.13965905 0.07165249

Which gives the same %’s as the weighted table above, but we also want the correct standard errors for our bad health prevalences.

The svyby() function will calculate statistics by groups, in this case we want the % in bad health by each level of education. The %’s can be gotten using the svymean() function, which finds means of variables using survey design:

svyby(formula = ~badhealth, by = ~educ, design = des, FUN = svymean)
##              educ  badhealth          se
## 2hsgrad   2hsgrad 0.19542781 0.003368281
## 0Prim       0Prim 0.45215087 0.013413084
## 1somehs   1somehs 0.30711763 0.008232550
## 3somecol 3somecol 0.13965905 0.002601595
## 4colgrad 4colgrad 0.07165249 0.001831015

And we see the same point estimates of our prevalences as in the simple weighted table, but the standard errors have now been adjusted for survey design as well, so they are also correct.

Regression example

Next we apply this logic to a regression case. First we fit the OLS model for our BMI outcome using education and age as predictors:

fit1<-lm(bmi~educ+agec, data=brfss_11)
#summary(fit1)

#Next we incorporate case weights
fit2<-lm(bmi~educ+agec, data=brfss_11, weights = cntywt)
#summary(fit2)

We see the low education effect reduce and the age effects increase. Now we will incorporate design effects as well:

fit3<-svyglm(bmi~educ+agec,des, family=gaussian)
#summary(fit3)

stargazer(fit1, fit2, fit3, type="html", style="demography", column.labels = c("OLS", "Weights", "Survey"), covariate.labels=c("PrimarySchool", "SomeHS", "SomeColl", "CollGrad", "Age 24-39","Age 39-59" ,"Age 59-79", "Age 80+"), keep.stat="n", model.names=F, align=T)
bmi
OLS Weights Survey
Model 1 Model 2 Model 3
PrimarySchool 1.001*** 0.880*** 0.880***
(0.085) (0.062) (0.218)
SomeHS 0.617*** 0.395*** 0.395**
(0.060) (0.046) (0.124)
SomeColl -0.215*** -0.232*** -0.232**
(0.034) (0.033) (0.073)
CollGrad -1.614*** -1.747*** -1.747***
(0.031) (0.033) (0.065)
Age 24-39 2.319*** 2.530*** 2.530***
(0.061) (0.041) (0.107)
Age 39-59 3.094*** 3.325*** 3.325***
(0.056) (0.040) (0.099)
Age 59-79 2.799*** 3.005*** 3.005***
(0.057) (0.044) (0.098)
Age 80+ 0.271*** 0.621*** 0.621***
(0.067) (0.068) (0.112)
Constant 25.553*** 25.253*** 25.253***
(0.056) (0.038) (0.099)
N 229,224 229,224 229,224
p < .05; p < .01; p < .001

Which shows the same \(\beta\)’s as the weighted model but the standard errors are larger, so the test statistics are more conservative (smaller t statistics). While in this simple model, our overall interpretation of the effects do not change (positive effects of education, negative effects of age), it is entirely possible that they could once we include our survey design effects.

It may be informative to plot the results of the models to see how different the coefficients are from one another:

plot(coef(fit1)[-1], ylab="Beta parameters",ylim=c(-2, 4), xlab=NULL,axes=T,xaxt="n",main=expression(paste(beta , " Parameters from Survey Regression and non survey regression models")))
axis(side=1, at=1:8, labels=F)
text(x=1:8, y=-2.5,  srt = 45, pos = 1, xpd = TRUE,labels = c( "PrimarySch", "SomeHS", "somecol", "colgrad", "25_40", "40_60", "60_80", "80+" ))
#add the coefficients for the normal logit model
points(coef(fit3)[-1], col=2, pch=4, cex=1.5)
legend("topleft", legend=c("Non-survey model", "Survey Model"), col=c(1,2), pch=c(1,4))

Which shows us that the betas are similar but have some differences between the two models