1-1 Progress Report
2023-07-25
Demographics Stats
| knower_level_cp_subset | age_years | n |
|---|---|---|
| CP | 3 | 14 |
| CP | 4 | 18 |
| CP | 5 | 16 |
| subset | 2 | 11 |
| subset | 3 | 19 |
| subset | 4 | 10 |
| gender | n |
|---|---|
| F | 43 |
| M | 45 |
| knower_level_cp_subset | n |
|---|---|
| CP | 48 |
| subset | 40 |
Results
Both CP knowers and subset knowers are successful! The CP-knowers perform better than subset-knowers, in both large vs small sets. Subset knowers only succeed with large sets, not small sets.
Subset knowers are only above chance for 5 vs 6??
## `summarise()` has grouped output by 'id', 'knower_level'. You can override
## using the `.groups` argument.
## `summarise()` has grouped output by 'id', 'quantity', 'knower_level_cp_subset'.
## You can override using the `.groups` argument.
## `summarise()` has grouped output by 'id', 'knower_level_cp_subset'. You can
## override using the `.groups` argument.
Only 64.6% of CP-knowers (31/48) were above chance, and 35% of subset knowers (14/40) (following binomial p < .05 –> chose correctly at least 7 out of 8 trials).
## `summarise()` has grouped output by 'id'. You can override using the `.groups`
## argument.## # A tibble: 2 × 4
## knower_level_cp_subset n_total n_succeed prop_succeed
## <chr> <int> <int> <dbl>
## 1 CP 48 31 0.646
## 2 subset 40 14 0.35Age effect:
## `geom_smooth()` using formula = 'y ~ x'
## `geom_smooth()` using formula = 'y ~ x'
T-tests
Both subset-knowers and CP knowers perform better than chance.
## Effect sizes were labelled following Cohen's (1988) recommendations.
##
## The One Sample t-test testing the difference between df.trial %>%
## filter(knower_level_cp_subset == "subset") %>% pull(correct_set_chosen) (mean =
## 0.60) and mu = 0.5 suggests that the effect is positive, statistically
## significant, and very small (difference = 0.10, 95% CI [0.54, 0.65], t(319) =
## 3.53, p < .001; Cohen's d = 0.20, 95% CI [0.09, 0.31])## Effect sizes were labelled following Cohen's (1988) recommendations.
##
## The One Sample t-test testing the difference between df.trial %>%
## filter(knower_level_cp_subset == "CP") %>% pull(correct_set_chosen) (mean =
## 0.81) and mu = 0.5 suggests that the effect is positive, statistically
## significant, and medium (difference = 0.31, 95% CI [0.77, 0.85], t(381) =
## 15.34, p < .001; Cohen's d = 0.78, 95% CI [0.67, 0.90])Regressions
Base model
correct_set_chosen ~ magnitude + age_zscored + (1|id) + (1|quantity) Effect of age, not magnitude. Makes sense – they only need to attend to the last pair.
Registered random effect structure results in overfitting. So I
removed quantity as a random effect, and will note any
deviance from the registered models (none so far).
## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula: correct_set_chosen ~ magnitude + age_zscored + (1 | id)
## Data: df.trial
##
## AIC BIC logLik deviance df.resid
## 685.7 703.9 -338.9 677.7 698
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.1887 -0.5112 0.2280 0.4162 1.9981
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 3.389 1.841
## Number of obs: 702, groups: id, 88
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 1.7449 0.2885 6.047 1.47e-09 ***
## magnitudesmall -0.2636 0.2115 -1.246 0.212627
## age_zscored 0.9300 0.2516 3.697 0.000218 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) mgntds
## magnitdsmll -0.402
## age_zscored 0.177 -0.022## Analysis of Deviance Table (Type III Wald chisquare tests)
##
## Response: correct_set_chosen
## Chisq Df Pr(>Chisq)
## (Intercept) 36.5716 1 1.472e-09 ***
## magnitude 1.5535 1 0.2126267
## age_zscored 13.6656 1 0.0002184 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Age actually isn’t significant for CP knowers or subset knowers (marginally significant for subset knowers).
## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula: correct_set_chosen ~ magnitude + age_zscored + (1 | id)
## Data: df.trial %>% filter(knower_level_cp_subset == "CP")
##
## AIC BIC logLik deviance df.resid
## 311.3 327.0 -151.6 303.3 378
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.0131 0.1308 0.1685 0.3507 1.8630
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 4.888 2.211
## Number of obs: 382, groups: id, 48
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 2.35418 0.57178 4.117 3.83e-05 ***
## magnitudesmall -0.04735 0.32398 -0.146 0.884
## age_zscored 0.59559 0.48787 1.221 0.222
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) mgntds
## magnitdsmll -0.293
## age_zscored -0.350 -0.001## Analysis of Deviance Table (Type III Wald chisquare tests)
##
## Response: correct_set_chosen
## Chisq Df Pr(>Chisq)
## (Intercept) 16.9520 1 3.834e-05 ***
## magnitude 0.0214 1 0.8838
## age_zscored 1.4904 1 0.2222
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula: correct_set_chosen ~ magnitude + age_zscored + (1 | id)
## Data: df.trial %>% filter(knower_level_cp_subset == "subset")
##
## AIC BIC logLik deviance df.resid
## 377.2 392.3 -184.6 369.2 316
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.6748 -0.7127 0.2957 0.5405 1.8487
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 2.364 1.537
## Number of obs: 320, groups: id, 40
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 1.3117 0.4173 3.144 0.00167 **
## magnitudesmall -0.4221 0.2725 -1.549 0.12147
## age_zscored 0.7152 0.3902 1.833 0.06682 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) mgntds
## magnitdsmll -0.356
## age_zscored 0.639 -0.019## Analysis of Deviance Table (Type III Wald chisquare tests)
##
## Response: correct_set_chosen
## Chisq Df Pr(>Chisq)
## (Intercept) 9.8828 1 0.001668 **
## magnitude 2.3983 1 0.121469
## age_zscored 3.3595 1 0.066818 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1What if we use specific set size instead of small vs large? (e.g., 1 vs 2 –> quantity = 2). Also not significant.
## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula: correct_set_chosen ~ quantity + age_zscored + (1 | id)
## Data: df.trial
##
## AIC BIC logLik deviance df.resid
## 686.7 704.9 -339.4 678.7 698
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.0074 -0.4996 0.2293 0.4063 2.1037
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 3.37 1.836
## Number of obs: 702, groups: id, 88
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 1.47652 0.31485 4.690 2.74e-06 ***
## quantity 0.02523 0.03399 0.742 0.458045
## age_zscored 0.92779 0.25094 3.697 0.000218 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) quntty
## quantity -0.547
## age_zscored 0.147 0.013## Analysis of Deviance Table (Type III Wald chisquare tests)
##
## Response: correct_set_chosen
## Chisq Df Pr(>Chisq)
## (Intercept) 21.9920 1 2.738e-06 ***
## quantity 0.5507 1 0.458045
## age_zscored 13.6695 1 0.000218 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1CP knowledge
correct_set_chosen ~ cp_knowledge + magnitude + age_zscored + (1|id) + (1|quantity). Effect of age, not magnitude or CP knowledge. Including CP knowledge doesn’t improve the model.
## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula:
## correct_set_chosen ~ knower_level_cp_subset + magnitude + age_zscored +
## (1 | id)
## Data: df.trial
##
## AIC BIC logLik deviance df.resid
## 685.0 707.8 -337.5 675.0 697
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.2198 -0.5100 0.2107 0.4214 1.9894
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 3.317 1.821
## Number of obs: 702, groups: id, 88
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 2.1879 0.4153 5.269 1.37e-07 ***
## knower_level_cp_subsetsubset -0.9559 0.5929 -1.612 0.1069
## magnitudesmall -0.2644 0.2116 -1.249 0.2116
## age_zscored 0.6293 0.3049 2.064 0.0391 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) knw___ mgntds
## knwr_lvl_c_ -0.724
## magnitdsmll -0.287 0.012
## age_zscored -0.319 0.577 -0.011## Analysis of Deviance Table (Type III Wald chisquare tests)
##
## Response: correct_set_chosen
## Chisq Df Pr(>Chisq)
## (Intercept) 27.7590 1 1.374e-07 ***
## knower_level_cp_subset 2.5991 1 0.10693
## magnitude 1.5606 1 0.21157
## age_zscored 4.2583 1 0.03906 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## Data: df.trial
## Models:
## fit.base: correct_set_chosen ~ magnitude + age_zscored + (1 | id)
## fit.cp: correct_set_chosen ~ knower_level_cp_subset + magnitude + age_zscored + (1 | id)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## fit.base 4 685.71 703.92 -338.85 677.71
## fit.cp 5 685.05 707.82 -337.52 675.05 2.6598 1 0.1029CP knowledge only
(Not registered)
CP knowledge is significant if we take away age.
## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula: correct_set_chosen ~ knower_level_cp_subset + magnitude + (1 |
## id)
## Data: df.trial
##
## AIC BIC logLik deviance df.resid
## 687.3 705.5 -339.6 679.3 698
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.0092 -0.5285 0.1964 0.4051 1.8920
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 3.633 1.906
## Number of obs: 702, groups: id, 88
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 2.5628 0.4083 6.276 3.46e-10 ***
## knower_level_cp_subsetsubset -1.7327 0.5030 -3.445 0.000572 ***
## magnitudesmall -0.2650 0.2117 -1.252 0.210665
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) knw___
## knwr_lvl_c_ -0.700
## magnitdsmll -0.297 0.022## Analysis of Deviance Table (Type III Wald chisquare tests)
##
## Response: correct_set_chosen
## Chisq Df Pr(>Chisq)
## (Intercept) 39.3939 1 3.464e-10 ***
## knower_level_cp_subset 11.8650 1 0.000572 ***
## magnitude 1.5669 1 0.210665
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Interaction
Still, only effect of age. Does not improve the model.
## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula:
## correct_set_chosen ~ knower_level_cp_subset * magnitude + age_zscored +
## (1 | id)
## Data: df.trial
##
## AIC BIC logLik deviance df.resid
## 686.2 713.5 -337.1 674.2 696
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.0383 -0.5308 0.2162 0.4030 1.9115
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 3.328 1.824
## Number of obs: 702, groups: id, 88
##
## Fixed effects:
## Estimate Std. Error z value
## (Intercept) 2.07444 0.43004 4.824
## knower_level_cp_subsetsubset -0.75002 0.63230 -1.186
## magnitudesmall -0.04389 0.31642 -0.139
## age_zscored 0.63086 0.30542 2.066
## knower_level_cp_subsetsubset:magnitudesmall -0.39782 0.42583 -0.934
## Pr(>|z|)
## (Intercept) 1.41e-06 ***
## knower_level_cp_subsetsubset 0.2355
## magnitudesmall 0.8897
## age_zscored 0.0389 *
## knower_level_cp_subsetsubset:magnitudesmall 0.3502
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) knw___ mgntds ag_zsc
## knwr_lvl_c_ -0.747
## magnitdsmll -0.377 0.255
## age_zscored -0.311 0.546 0.000
## knwr_lvl__: 0.269 -0.344 -0.743 -0.010## Analysis of Deviance Table (Type III Wald chisquare tests)
##
## Response: correct_set_chosen
## Chisq Df Pr(>Chisq)
## (Intercept) 23.2695 1 1.408e-06 ***
## knower_level_cp_subset 1.4070 1 0.23555
## magnitude 0.0192 1 0.88968
## age_zscored 4.2664 1 0.03887 *
## knower_level_cp_subset:magnitude 0.8728 1 0.35019
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## Data: df.trial
## Models:
## fit.base: correct_set_chosen ~ magnitude + age_zscored + (1 | id)
## fit.cp_int: correct_set_chosen ~ knower_level_cp_subset * magnitude + age_zscored + (1 | id)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## fit.base 4 685.71 703.92 -338.85 677.71
## fit.cp_int 6 686.17 713.50 -337.09 674.17 3.5336 2 0.1709## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula:
## correct_set_chosen ~ knower_level_cp_subset * quantity + age_zscored +
## (1 | id)
## Data: df.trial
##
## AIC BIC logLik deviance df.resid
## 687.6 714.9 -337.8 675.6 696
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.0037 -0.5258 0.2161 0.4088 2.0846
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 3.304 1.818
## Number of obs: 702, groups: id, 88
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 2.0520033 0.4775811 4.297 1.73e-05
## knower_level_cp_subsetsubset -1.1949034 0.6908465 -1.730 0.0837
## quantity -0.0005816 0.0507684 -0.011 0.9909
## age_zscored 0.6288288 0.3044496 2.065 0.0389
## knower_level_cp_subsetsubset:quantity 0.0466700 0.0683095 0.683 0.4945
##
## (Intercept) ***
## knower_level_cp_subsetsubset .
## quantity
## age_zscored *
## knower_level_cp_subsetsubset:quantity
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) knw___ quntty ag_zsc
## knwr_lvl_c_ -0.752
## quantity -0.555 0.384
## age_zscored -0.279 0.491 -0.001
## knwr_lvl__: 0.419 -0.516 -0.743 0.007## Analysis of Deviance Table (Type III Wald chisquare tests)
##
## Response: correct_set_chosen
## Chisq Df Pr(>Chisq)
## (Intercept) 18.4613 1 1.734e-05 ***
## knower_level_cp_subset 2.9916 1 0.08370 .
## quantity 0.0001 1 0.99086
## age_zscored 4.2661 1 0.03888 *
## knower_level_cp_subset:quantity 0.4668 1 0.49447
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Highest Count
No effect of highest count.
## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula: correct_set_chosen ~ highest_count + magnitude + age_zscored +
## (1 | id)
## Data: df.trial %>% filter(!is.na(highest_count))
##
## AIC BIC logLik deviance df.resid
## 599.7 621.6 -294.8 589.7 593
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.1490 -0.5804 0.2415 0.4181 1.9491
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 3.338 1.827
## Number of obs: 598, groups: id, 75
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 1.36781 0.45081 3.034 0.00241 **
## highest_count 0.01423 0.01993 0.714 0.47527
## magnitudesmall -0.17490 0.22596 -0.774 0.43893
## age_zscored 0.59130 0.33876 1.746 0.08089 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) hghst_ mgntds
## highest_cnt -0.741
## magnitdsmll -0.262 -0.005
## age_zscored 0.451 -0.559 -0.006## Analysis of Deviance Table (Type III Wald chisquare tests)
##
## Response: correct_set_chosen
## Chisq Df Pr(>Chisq)
## (Intercept) 9.2056 1 0.002413 **
## highest_count 0.5097 1 0.475269
## magnitude 0.5991 1 0.438928
## age_zscored 3.0468 1 0.080895 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula: correct_set_chosen ~ magnitude + age_zscored + (1 | id)
## Data: df.trial %>% filter(!is.na(highest_count))
##
## AIC BIC logLik deviance df.resid
## 598.2 615.8 -295.1 590.2 594
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.9693 -0.5729 0.2390 0.4131 1.9846
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 3.346 1.829
## Number of obs: 598, groups: id, 75
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 1.6153 0.3029 5.332 9.7e-08 ***
## magnitudesmall -0.1745 0.2259 -0.772 0.43995
## age_zscored 0.7308 0.2805 2.605 0.00918 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) mgntds
## magnitdsmll -0.397
## age_zscored 0.065 -0.011## Data: df.trial %>% filter(!is.na(highest_count))
## Models:
## fit.hc_compare: correct_set_chosen ~ magnitude + age_zscored + (1 | id)
## fit.hc: correct_set_chosen ~ highest_count + magnitude + age_zscored + (1 | id)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## fit.hc_compare 4 598.20 615.77 -295.10 590.20
## fit.hc 5 599.67 621.64 -294.84 589.67 0.5207 1 0.4705## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula: correct_set_chosen ~ highest_count + magnitude + age_zscored +
## (1 | id)
## Data: df.trial %>% filter(knower_level_cp_subset == "CP")
##
## AIC BIC logLik deviance df.resid
## 308.2 327.4 -149.1 298.2 345
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.0351 0.1352 0.1901 0.3669 1.8009
##
## Random effects:
## Groups Name Variance Std.Dev.
## id (Intercept) 4.178 2.044
## Number of obs: 350, groups: id, 44
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 1.85748 0.69630 2.668 0.00764 **
## highest_count 0.01728 0.02660 0.649 0.51606
## magnitudesmall -0.04725 0.32336 -0.146 0.88383
## age_zscored 0.13618 0.59351 0.229 0.81851
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) hghst_ mgntds
## highest_cnt -0.649
## magnitdsmll -0.237 -0.004
## age_zscored 0.169 -0.610 0.003## Analysis of Deviance Table (Type III Wald chisquare tests)
##
## Response: correct_set_chosen
## Chisq Df Pr(>Chisq)
## (Intercept) 7.1164 1 0.007638 **
## highest_count 0.4218 1 0.516061
## magnitude 0.0214 1 0.883827
## age_zscored 0.0526 1 0.818515
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Exploratory
Just for funsies, I check whether kids perform better in mismatch or match trials. There is no difference.
## `summarise()` has grouped output by 'id', 'is_mismatch'. You can override using
## the `.groups` argument.