2.1 Summary of findings

• Likelihood-based Poisson estimator yielded the most stable SE estimates

• Robust methods had the largest variability, making them the least stable

• Trade-off of loosening parametric assumptions included lower stability and greater bias under DGMs explored here

2.2 Outline

• Chapter 1: Statistical background; models & derivations
• Chapter 2: Simulations
• Chapter 3: Application to FAMS
• Chapter 4: Conclusion

3.1 Generalized Linear Models

Generalized linear models (GLMs) “generalize” linear regression for exponential family distributions. They increase model flexibility by:

• Transforming the mean using “link” function - allows function of the response to vary linearly with the predictors
• Defining a mean-variance relationship

4.1 Estimator validity

Table 1. Validity of variance estimators under three data generating mechanisms

Figure 2. Probability densities under Poisson \((\lambda = 2)\), quasi-Poisson \((\lambda = 2, \phi = 2)\), and negative binomial \((\mu = 2, \phi = 2)\) frameworks. These discrete distributions are presented continuously to visualize differences in density and dispersion.

4.2 Derivations

First specified a GLM and defined the mean model and mean-variance relationship. We did this using a Poisson distribution with the mean model defined through a log link.

The Poisson density is re-written in canonical form, where the natural parameter is \(\theta = log(\lambda)\) and scale parameter is \(\phi = 1\). Our mean model is defined as \(g(E[Y]) = g(\mu) = g(b'(\theta)) = log(\theta)\):

\[ f(Y|x;\theta,\phi) = exp\Bigg[\frac{y\theta(x^T\beta) - b(\theta(x^T\beta))}{\phi} + c(y, \phi)\Bigg] \] \[f(y|\lambda) = \frac{\lambda^ye^{-\lambda}}{y!}\] \[=exp\bigg[ylog(\lambda) - \lambda - log(y!)\bigg]\]

Score equations used to estimate parameters via maximum likelihood estimation

\[D^TV^{-1}(y-\mu) = X^T(y-exp(X\beta)) = 0\]

Asymptotically, it can be shown that the distribution of \(\beta\) is:

\[\widehat{\beta_N} \sim N \Bigg( \beta_0, \frac{1}{N} \big[A_N(\beta_0) \big]^{-1}B(\beta_0) \big[A_N(\beta_0) \big]^{-1}\Bigg)\]

• \(A_N = A_N^{obs}(\widehat{\beta}) = X^T diag(exp(x_i^T \eta) (1-exp(x_i^T \beta)))X = X^T V(\beta)X\)

• \(B_N^{obs} (\widehat{\beta})= \sum_{i=1}^N G(\beta;x_i,Y_i )\times G(\beta;,x_i,Y_i )^T =D^T (\beta) V^{-1} (\beta)diag(Y_i- \mu_i (\beta))^2 V^{-1} (\beta)D(\beta)\)

Finally, \(\phi\) estimated to use in quasi-Poisson variance estimation:

\[\widehat{\phi} = \frac{1}{N-K}\sum_{i=1}^N \frac{(y_i-\widehat{\mu}_i)^2}{V(\widehat{\mu_i})}\]

\[ = \frac{1}{N-K}\sum_{i=1}^N \frac{(y_i-exp(x_i^T\widehat{\beta}))^2}{exp(x_i^T\widehat{\beta})}\]

The six variance estimators are constructed as follows:

1. Poisson

   \[\widehat{Cov}[\widehat{\beta}] = A_N^{-1}(\beta)\]

2. Quasi-Poisson

   \[\widehat{Cov}[\widehat{\beta}] = \widehat{\phi} \times A_N^{-1}(\beta)\]

3. Sandwich

   \[\widehat{Cov}[\widehat{\beta}] = A_N^{-1}(\beta)\bigg( B_N^{obs}(\widehat{\beta})\bigg)A_N^{-1}(\beta)\]

4. Unconditional bootstrap

   • With-replacement sampling with 500 bootstrap replicates conducted over the entire dataset to produce point estimates and SEs.

5. Conditional bootstrap

   • With-replacement sampling with 500 bootstrap replicates conducted from within subsets of the data for each X indicator to produce point estimates and SEs.

6. Negative Binomial

• Assumed mean-variance relationship: \[Var[Y_i|x_i] = \mu_i + \frac{\mu_i^2}{\phi}\]

• Estimating equations: \[D^T (\beta) V^{-1} (\beta,\phi)(y-\mu(\beta))=0\]

5.1 Estimator bias

The average bias of each estimator was computed as a percentage of the ESE. The majority of estimators displayed “negative bias”, resulting in an underestimate of the true variance and overstated certainty of the true parameter value.

Figure 4. Average bias of SE estimators as a proportion of ESE in the Poisson framework.

• Lower bias as sample sizes increase

• The two bootstrap estimators demonstrate the greatest bias across sample sizes, except in N=1,280

Figure 5. Average bias of SE estimators as a proportion of the ESE in the quasi-Poisson framework.

• The Poisson estimator is highly biased compared to all other estimators.
• Bias decreases overall in non-Poisson estimators as sample sizes increase, but trend is inconsistent

Figure 6. Average bias of SE estimators as a proportion of the ESE in the negative binomial framework.

• The Poisson estimator is highly biased compared to all other estimators.
• Bootstrap and sandwich estimators exhibit more bias than other non-Poisson estimators at smaller sample sizes.
• Most estimators are more biased at smaller samples compared to quasi-Poisson scenario
• Consistent downward trajectory with increasing sample sizes

5.2 Estimator stability

The stability of each estimator was illustrated using the SD of the estimated SE.

Figure 7. SD of the estimated SE over finite sample sizes

• Likelihood-based Poisson estimator was the most stable (smallest SD of estimated SE) under Poisson, quasi-Poisson and negative binomial frameworks
• Quasi-Poisson and negative binomial estimators had comparable performance
• Robust estimators had lowest stability with similar patterns

Figure 8. Ratio of individual model SD of the estimated SE to Poisson model SE of the estimated SE

• Quasi-Poisson and negative binomial performed similarly with constant ratios at N>160
• Sandwich reached constant at higher ratio in scenarios A and B
• Bootstrap estimators had largest SD of SE ratios and failed to level off

Figure 9. Ratio of individual model SDvof the estimated SE to the ”correct” model SE of the estimated SE for each respective framework

• Poisson was the denominator in scenario A
• Quasi-Poisson denominator in scenario B
• Negative binomial denominator in scenario C

Negative binomial variance estimator was most stable in scenario B. Meanwhile, the sandwich had lower stability, and bootstrap estimators performed similarly as in the prior example.

6.1 Eligibility

• Diagnosis of Type II diabetes mellitus (T2DM)
• At least one daily diabetes medication
• Aged 18-75 years
• English-speaking
• Community-dwelling
• Owned mobile phone

6.2 Trial procedures

• Randomization was 1:1 to intervention or control
• Intervention included monthly coaching sessions for the person with T2DM, text message support, and optional text messages to their designated support person.
• All participants received printed material on diabetes management and access to on-study HbA1c

6.3 Study cohort

Performance of our six variance estimators was studied in a subset of the FAMS dataset. Participants were included if they had complete data for:

• Text message responses (participant “engagement”)
• Number of texts received (offset)
• Age
• Race/ethnicity
• Gender
• Education
• Income
• Insurance
• Baseline physical activity *

Table 2. FAMS physical activity groups

6.4 Variance estimation

\(\beta\) and SE estimates calculated treating physical activity categorically and continuously

Table 3. Estimates of \(\beta_1\) and SE for physical activity group, treated categorically

Table 4. Estimates of \(\beta_1\) and SE for physical activity group, treated continuously