creating the “test-retest reliability measure” - since the DEEP estimates are just single numbers the most intuitive measure is just the difference between the new and old estimates

then testing whether this measure of preference certainty interacts with default condition to predict choice of SS

r$betadiff<-abs(r$beta.x - r$beta.y)
summary(glm(SS~cond1SS2LL3NO*betadiff, data=r, family=binomial))
## 
## Call:
## glm(formula = SS ~ cond1SS2LL3NO * betadiff, family = binomial, 
##     data = r)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.9460  -1.0438  -0.9639   1.2561   1.4148  
## 
## Coefficients:
##                        Estimate Std. Error z value Pr(>|z|)  
## (Intercept)             -0.4015     0.2812  -1.428   0.1533  
## cond1SS2LL3NO           -0.0471     0.1339  -0.352   0.7250  
## betadiff                 5.1525     2.3315   2.210   0.0271 *
## cond1SS2LL3NO:betadiff  -0.6720     1.0688  -0.629   0.5295  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 887.01  on 642  degrees of freedom
## Residual deviance: 864.12  on 639  degrees of freedom
## AIC: 872.12
## 
## Number of Fisher Scoring iterations: 4
r$deltadiff<-abs(r$delta.x - r$delta.y)
summary(glm(SS~cond1SS2LL3NO*deltadiff, data=r, family=binomial))
## 
## Call:
## glm(formula = SS ~ cond1SS2LL3NO * deltadiff, family = binomial, 
##     data = r)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.7231  -1.0340  -0.9399   1.2501   1.4457  
## 
## Coefficients:
##                         Estimate Std. Error z value Pr(>|z|)
## (Intercept)              -0.2548     0.2791  -0.913    0.361
## cond1SS2LL3NO            -0.1190     0.1337  -0.890    0.374
## deltadiff               223.3119   145.0674   1.539    0.124
## cond1SS2LL3NO:deltadiff   9.0705    67.2151   0.135    0.893
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 887.01  on 642  degrees of freedom
## Residual deviance: 864.61  on 639  degrees of freedom
## AIC: 872.61
## 
## Number of Fisher Scoring iterations: 4

neither reliability measure (beta nor delta) interacts with condition the original and latest estimates of beta and delta are moderately correlated, but the original estimate is a stronger predictor of choice (the choice occured 1 week after the original measure vs. several months later) the original estimate of beta also interact with condition - the latest one does not

cor.test(r$beta.x, r$beta.y)
## 
##  Pearson's product-moment correlation
## 
## data:  r$beta.x and r$beta.y
## t = 19.484, df = 641, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.5589122 0.6562501
## sample estimates:
##       cor 
## 0.6098762
cor.test(r$delta.x, r$delta.y)
## 
##  Pearson's product-moment correlation
## 
## data:  r$delta.x and r$delta.y
## t = 21.046, df = 641, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.5911448 0.6828170
## sample estimates:
##       cor 
## 0.6392464
#original measure of beta
summary(glm(SS~cond1SS2LL3NO*beta.y, data=r, family=binomial)) 
## 
## Call:
## glm(formula = SS ~ cond1SS2LL3NO * beta.y, family = binomial, 
##     data = r)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -2.5718  -0.9970  -0.7367   1.1728   2.0241  
## 
## Coefficients:
##                      Estimate Std. Error z value Pr(>|z|)    
## (Intercept)            12.097      2.421   4.996 5.84e-07 ***
## cond1SS2LL3NO          -3.042      1.019  -2.984  0.00284 ** 
## beta.y                -12.098      2.384  -5.075 3.87e-07 ***
## cond1SS2LL3NO:beta.y    2.952      1.002   2.947  0.00321 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 887.01  on 642  degrees of freedom
## Residual deviance: 798.69  on 639  degrees of freedom
## AIC: 806.69
## 
## Number of Fisher Scoring iterations: 4
#newer measure of beta
summary(glm(SS~cond1SS2LL3NO*beta.x, data=r, family=binomial))
## 
## Call:
## glm(formula = SS ~ cond1SS2LL3NO * beta.x, family = binomial, 
##     data = r)
## 
## Deviance Residuals: 
##    Min      1Q  Median      3Q     Max  
## -2.419  -1.000  -0.661   1.150   2.147  
## 
## Coefficients:
##                      Estimate Std. Error z value Pr(>|z|)    
## (Intercept)            6.6263     2.0220   3.277  0.00105 ** 
## cond1SS2LL3NO         -0.3621     0.9589  -0.378  0.70570    
## beta.x                -6.5756     1.9950  -3.296  0.00098 ***
## cond1SS2LL3NO:beta.x   0.2536     0.9480   0.267  0.78909    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 887.01  on 642  degrees of freedom
## Residual deviance: 796.19  on 639  degrees of freedom
## AIC: 804.19
## 
## Number of Fisher Scoring iterations: 4

the new measure of preference certainty (alpha=.91) is very skewed - people are very certain about their DEEP preferences

hist(r$certain)

summary(r$certain)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   2.600   6.000   6.600   6.292   7.000   7.000       9

nevertheless, there’s an almost-marginally significant interaction between preference certainty and default condition

summary(glm(SS~cond1SS2LL3NO*certain, data=r, family=binomial))
## 
## Call:
## glm(formula = SS ~ cond1SS2LL3NO * certain, family = binomial, 
##     data = r)
## 
## Deviance Residuals: 
##    Min      1Q  Median      3Q     Max  
## -1.886  -1.078  -1.023   1.269   1.340  
## 
## Coefficients:
##                       Estimate Std. Error z value Pr(>|z|)  
## (Intercept)             4.0832     1.7634   2.315   0.0206 *
## cond1SS2LL3NO          -1.3256     0.8057  -1.645   0.0999 .
## certain                -0.6416     0.2780  -2.308   0.0210 *
## cond1SS2LL3NO:certain   0.1942     0.1275   1.523   0.1277  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 874.96  on 633  degrees of freedom
## Residual deviance: 865.25  on 630  degrees of freedom
##   (9 observations deleted due to missingness)
## AIC: 873.25
## 
## Number of Fisher Scoring iterations: 4

a spotlight analysis (in SPSS) shows that the effect of condition on choice of SS is strongest when preference certainty is lowest, and becomes marginally significant when certain is less than 5.3 on the 7 point scale (N=65, so only about 20 per default condition)

certain<-subset(r, certain>5.3)
un<-subset(r, certain<=5.3)
summary(glm(SS~cond1SS2LL3NO, data=un, family=binomial))
## 
## Call:
## glm(formula = SS ~ cond1SS2LL3NO, family = binomial, data = un)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.6770  -1.1854   0.7499   0.9461   1.1694  
## 
## Coefficients:
##               Estimate Std. Error z value Pr(>|z|)  
## (Intercept)     1.6781     0.7312   2.295   0.0217 *
## cond1SS2LL3NO  -0.5531     0.3223  -1.716   0.0862 .
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 85.611  on 64  degrees of freedom
## Residual deviance: 82.540  on 63  degrees of freedom
## AIC: 86.54
## 
## Number of Fisher Scoring iterations: 4
summary(glm(SS~cond1SS2LL3NO, data=certain, family=binomial))
## 
## Call:
## glm(formula = SS ~ cond1SS2LL3NO, family = binomial, data = certain)
## 
## Deviance Residuals: 
##    Min      1Q  Median      3Q     Max  
## -1.103  -1.077  -1.050   1.282   1.310  
## 
## Coefficients:
##               Estimate Std. Error z value Pr(>|z|)
## (Intercept)   -0.11332    0.22016  -0.515    0.607
## cond1SS2LL3NO -0.06432    0.10621  -0.606    0.545
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 780.89  on 568  degrees of freedom
## Residual deviance: 780.53  on 567  degrees of freedom
## AIC: 784.53
## 
## Number of Fisher Scoring iterations: 3