##EMA Compliance for NIMH Family Study

EMA assessments were given out to n=563 participants from ages 7-80.


The palm pilot/android phone prompted EMA assessments 4 times a day over 2 weeks for all participants. For questions asked at every assessment (excluding branched questions or daily questions) the max amount of responses is 56.


The assessments took an average of 3 minutes to complete and included 60+ questions (not sure exact number). All the variables (questions) in the EMA have very similar compliance levels, since the participant usually answers all the questions when he/she starts the first one. However, there is a very slight decrease in compliance as questions get closer to the end of the assessment, which can a result the rare instances where the participant has to stop in the middle of an assessment. (Pic 1)


Since across all the variables are practically the same, compliance from any one variable can be generalized to the entire EMA assessment. From this point on I will be showing data from the now.sadness variable, which is close to the middle of the 3 min EMA assessment and is a question which is given at all 56 assessments.


Summary statistic for % compliance for now.sadness
x
median 83.93
mean 77.90
SE.mean 0.81
CI.mean.0.95 1.58
std.dev 19.31


Age distribution in NIMH EMA sample


Compliance for now.sadness variable across age


Plot showing participant completetion rate for now.sadness variable. For example, if x=10%, y would be the percent of all participants (n=563) who completed at least 10% of the 56 assessments for the now.sadness variable. We can see that 1.9% of the participants had a perfect completion for this variable.


Fatigue effect

NIMH_Compliance_day <- 
      NIMHMerged %>% 
      group_by_("ID", "day") %>%
      summarise_all(countna) %>%
      #remove the first day, it is incomplete by design.
      filter(day > 1)

summary(lm(now.sadness ~ day, data = NIMH_Compliance_day))
## 
## Call:
## lm(formula = now.sadness ~ day, data = NIMH_Compliance_day)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.4463 -0.3630  0.5954  0.8037  1.0538 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  3.529659   0.031113  113.45   <2e-16 ***
## day         -0.041674   0.003523  -11.83   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.138 on 7447 degrees of freedom
## Multiple R-squared:  0.01845,    Adjusted R-squared:  0.01831 
## F-statistic: 139.9 on 1 and 7447 DF,  p-value: < 2.2e-16

For this analysis, day 1 was removed because for a large portion of the sample, day 1 was missing the first 3 assessments of the day by design.

There is a fatigue effect, beta = -0.041. The number of assessments completed per day (out of 4) decreases by an average of 0.041 for each assessment day (p<0.001).

This effect is statistically significant because of the large sample size but practically speaking, it’s pretty small. We’re averaging 3.42 completed assessments on day 2 and 2.92 on day 14.