Marios Frog ABC with fixed priors
1 Can we differentiate between models even with small sample sizes?
In this section I’ll do only the error classification with the random forest algorithm for the different samples sizes of the call populations. Doing this with the priors of the simulations fixed to 3mya Year & 100k Ne and also with those with simulations fixed to 1mya & 100k. Under each tab are also PCA’s fo only the simulations for visual inspection.
1.1 1mya Years & 100k Ne
call GH
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 912 0 9 0.009771987
## exp 0 903 18 0.019543974
## neu 2 6 913 0.008686211call WUG1
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 953 0 8 0.008324662
## exp 0 938 23 0.023933403
## neu 4 3 954 0.007284079call WUG2
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 916 0 11 0.011866235
## exp 0 905 22 0.023732470
## neu 0 4 923 0.004314995call WUG3
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 926 0 12 0.01279318
## exp 0 915 23 0.02452026
## neu 1 9 928 0.01066098call WUG4
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 906 0 11 0.01199564
## exp 0 896 21 0.02290076
## neu 3 5 909 0.00872410call WUG5
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 914 0 15 0.016146394
## exp 0 904 25 0.026910657
## neu 0 5 924 0.0053821311.1.1 Summary Table
Shows for each population the sample size and classification errors of each demographic scenario (contraction, expansion, neutral).
| Population | Sample_Size | con | exp | neu | |
|---|---|---|---|---|---|
| call_GH_ce | call_GH | 5 | 0.00977198697068404 | 0.0195439739413681 | 0.00868621064060804 |
| call_WUG1_ce | call_WUG1 | 8 | 0.00832466181061394 | 0.0239334027055151 | 0.0072840790842872 |
| call_WUG2_ce | call_WUG2 | 11 | 0.011866235167206 | 0.0237324703344121 | 0.00431499460625674 |
| call_WUG3_ce | call_WUG3 | 6 | 0.0127931769722814 | 0.0245202558635394 | 0.0106609808102345 |
| call_WUG4_ce | call_WUG4 | 11 | 0.0119956379498364 | 0.0229007633587786 | 0.00872410032715376 |
| call_WUG5_ce | call_WUG5 | 14 | 0.0161463939720129 | 0.0269106566200215 | 0.00538213132400431 |
1.2 3mya Years & 100k Ne
call GH
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 397 249 250 0.5569196
## exp 235 328 333 0.6339286
## neu 238 306 352 0.6071429call WUG1
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 547 173 225 0.4211640
## exp 159 521 265 0.4486772
## neu 223 265 457 0.5164021call WUG2
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 522 188 203 0.4282585
## exp 152 488 273 0.4654984
## neu 161 296 456 0.5005476call WUG3
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 569 162 189 0.3815217
## exp 223 442 255 0.5195652
## neu 196 252 472 0.4869565call WUG4
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 539 135 216 0.3943820
## exp 182 419 289 0.5292135
## neu 252 297 341 0.6168539call WUG5
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 458 234 200 0.4865471
## exp 201 373 318 0.5818386
## neu 232 342 318 0.6434978pall GH
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 368 254 256 0.5808656
## exp 174 399 305 0.5455581
## neu 231 339 308 0.6492027pall WUG1
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 439 220 217 0.4988584
## exp 193 344 339 0.6073059
## neu 194 351 331 0.6221461pall WUG2
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 509 157 197 0.4101970
## exp 181 389 293 0.5492468
## neu 198 327 338 0.6083430plib GH
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 357 172 170 0.4892704
## exp 146 312 241 0.5536481
## neu 166 289 244 0.6509299plib WUG1
## con exp neu class.error
## con 499 205 194 0.4443207
## exp 178 388 332 0.5679287
## neu 205 346 347 0.6135857ppli GH
## con exp neu class.error
## con 502 189 173 0.4189815
## exp 164 353 347 0.5914352
## neu 172 341 351 0.5937500ppli WUG1
## con exp neu class.error
## con 499 178 199 0.4303653
## exp 174 352 350 0.5981735
## neu 201 390 285 0.67465751.2.1 Summary Table
Shows for each population the sample size and classification errors of each demographic scenario (contraction, expansion, neutral).
| Population | Sample_Size | con | exp | neu | |
|---|---|---|---|---|---|
| call_GH_ce | call_GH | 5 | 0.556919642857143 | 0.633928571428571 | 0.607142857142857 |
| call_WUG1_ce | call_WUG1 | 8 | 0.421164021164021 | 0.448677248677249 | 0.516402116402116 |
| call_WUG2_ce | call_WUG2 | 11 | 0.428258488499452 | 0.465498357064622 | 0.500547645125958 |
| call_WUG3_ce | call_WUG3 | 6 | 0.381521739130435 | 0.519565217391304 | 0.48695652173913 |
| call_WUG4_ce | call_WUG4 | 11 | 0.39438202247191 | 0.529213483146067 | 0.61685393258427 |
| call_WUG5_ce | call_WUG5 | 14 | 0.486547085201794 | 0.581838565022421 | 0.643497757847534 |
| pall_GH_ce | pall_GH | 3 | 0.580865603644647 | 0.545558086560364 | 0.649202733485194 |
| pall_WUG1_ce | pall_WUG1 | 6 | 0.498858447488584 | 0.607305936073059 | 0.622146118721461 |
| pall_WUG2_ce | pall_WUG2 | 9 | 0.410196987253766 | 0.549246813441483 | 0.608342989571263 |
| plib_GH_ce | plib_GH | 4 | 0.489270386266094 | 0.553648068669528 | 0.650929899856938 |
| plib_WUG1_ce | plib_WUG1 | 10 | 0.444320712694877 | 0.567928730512249 | 0.61358574610245 |
| ppli_GH_ce | ppli_GH | 4 | 0.418981481481481 | 0.591435185185185 | 0.59375 |
| ppli_WUG1_ce | ppli_WUG1 | 7 | 0.430365296803653 | 0.598173515981735 | 0.674657534246575 |
1.3 1mya Years & 50k Ne
call WUG5
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 718 50 129 0.1995541
## exp 15 500 382 0.4425864
## neu 48 354 495 0.4481605plib GH
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 780 25 97 0.1352550
## exp 8 543 351 0.3980044
## neu 24 384 494 0.45232821.3.1 Summary Table
Shows for each population the sample size and classification errors of each demographic scenario (contraction, expansion, neutral).
| Population | Sample_Size | con | exp | neu | |
|---|---|---|---|---|---|
| call_WUG5_ce | call_WUG5 | 14 | 0.199554069119286 | 0.442586399108138 | 0.448160535117057 |
| plib_GH_ce | plib_GH | 4 | 0.135254988913525 | 0.3980044345898 | 0.452328159645233 |
1.4 3mya Years & 50k Ne
call WUG5
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 320 263 311 0.6420582
## exp 268 326 300 0.6353468
## neu 293 278 323 0.6387025plib GH
## Warning in lda.default(x, grouping, ...): variables are collinear## con exp neu class.error
## con 331 283 287 0.6326304
## exp 277 324 300 0.6403996
## neu 293 315 293 0.67480581.4.1 Summary Table
Shows for each population the sample size and classification errors of each demographic scenario (contraction, expansion, neutral).
| Population | Sample_Size | con | exp | neu | |
|---|---|---|---|---|---|
| call_WUG5_ce | call_WUG5 | 14 | 0.642058165548098 | 0.635346756152125 | 0.638702460850112 |
| plib_GH_ce | plib_GH | 4 | 0.632630410654828 | 0.640399556048835 | 0.67480577136515 |
1.5 Conclusion
The classification errors show that the algorithm ability to differentiate between the demographic scenarios does not rly depent on the sample size of the different populations. However, I found vast differences in the error classification dependent on the combination of priors. Using the simulations ran on 3 million years, the error classification does not show a pattern at all with errors around 0.5. The situation is a bit better for the 1 million years and 50k NE, and pretty ideal with 1 million years and 100k NE, the latter with errors around 0.01.
2 How does the target fit to the simulations?
The next step is to see if a scenario can be selected plausibly for the different targets. For this it is not only important that the scenarios can generally be differentiated by an algorithm, but also if the simulations at least somewhat represent the target data, which can be seen in the PCA’s under each tab. Because the experiment was overall quite unsuccessful, I also tried using the classical ABC methods instead of the random forest.
2.1 Random Forest
2.1.1 Call WUG5
1 million years & 100k Ne
## con exp neu class.error
## con 913 0 16 0.017222820
## exp 0 906 23 0.024757804
## neu 1 4 924 0.005382131## selected model votes model1 votes model2 votes model3 post.proba
## 1 neu 0 1 999 0.995253 million years & 100k Ne
## con exp neu class.error
## con 432 211 249 0.5156951
## exp 192 371 329 0.5840807
## neu 241 348 303 0.6603139## selected model votes model1 votes model2 votes model3 post.proba
## 1 neu 107 158 735 0.18866671 million years & 50k Ne
## con exp neu class.error
## con 719 45 133 0.1984392
## exp 19 525 353 0.4147157
## neu 58 366 473 0.4726867## selected model votes model1 votes model2 votes model3 post.proba
## 1 neu 67 105 828 0.76531673 million years & 50k Ne
## con exp neu class.error
## con 267 306 321 0.7013423
## exp 278 294 322 0.6711409
## neu 290 286 318 0.6442953## selected model votes model1 votes model2 votes model3 post.proba
## 1 neu 135 158 707 0.15111672.1.1.1 Summary Table
| Priors | Selected Model | Votes Contraction | Votes Expansion | Votes Neutral | Posterior Probability | |
|---|---|---|---|---|---|---|
| ms_1mya_100k | 1mya_100k | neu | 0 | 1 | 999 | 0.99525 |
| ms_3mya_100k | 3mya_100k | neu | 107 | 158 | 735 | 0.188666666666667 |
| ms_1mya_50k | 1mya_50k | neu | 67 | 105 | 828 | 0.765316666666667 |
| ms_3mya_50k | 3mya_50k | neu | 135 | 158 | 707 | 0.151116666666667 |
2.1.2 Plib GH
3 million years & 100k Ne
## con exp neu class.error
## con 331 178 190 0.5264664
## exp 157 303 239 0.5665236
## neu 177 277 245 0.6494993## selected model votes model1 votes model2 votes model3 post.proba
## 1 neu 75 162 763 0.14951 million years & 50k Ne
## con exp neu class.error
## con 733 45 124 0.1873614
## exp 19 508 375 0.4368071
## neu 44 362 496 0.4501109## Warning in ranger(formula, data, num.trees = ntree, sample.fraction =
## sampsize/nrow(data), : Unused arguments: obs, training, paral.predict## selected model votes model1 votes model2 votes model3 post.proba
## 1 neu 75 162 763 0.14953 million years & 50k Ne
## con exp neu class.error
## con 331 278 292 0.6326304
## exp 299 316 286 0.6492786
## neu 314 305 282 0.6870144## Warning in ranger(formula, data, num.trees = ntree, sample.fraction =
## sampsize/nrow(data), : Unused arguments: obs, training, paral.predict## selected model votes model1 votes model2 votes model3 post.proba
## 1 neu 75 162 763 0.14952.1.2.1 Summary Table
| Priors | Selected Model | Votes Contraction | Votes Expansion | Votes Neutral | Posterior Probability | |
|---|---|---|---|---|---|---|
| ms_3mya_100k | 3mya_100k | neu | 75 | 162 | 763 | 0.1495 |
| ms_1mya_50k | 1mya_50k | neu | 75 | 162 | 763 | 0.1495 |
| ms_3mya_50k | 3mya_50k | neu | 75 | 162 | 763 | 0.1495 |
2.2 Classical ABC
2.2.1 Call WUG5
1 million years & 100k Ne
## Confusion matrix based on 100 samples for each model.
##
## $tol0.01
## con exp neu
## con 94 1 5
## exp 0 92 8
## neu 2 6 92
##
## $tol0.05
## con exp neu
## con 95 1 4
## exp 0 98 2
## neu 9 7 84
##
## $tol0.1
## con exp neu
## con 99 0 1
## exp 1 94 5
## neu 3 5 92
##
##
## Mean model posterior probabilities (mnlogistic)
##
## $tol0.01
## con exp neu
## con 0.9375 0.0107 0.0518
## exp 0.0004 0.8005 0.1991
## neu 0.0372 0.1524 0.8104
##
## $tol0.05
## con exp neu
## con 0.9391 0.0142 0.0467
## exp 0.0000 0.8743 0.1256
## neu 0.0931 0.0508 0.8561
##
## $tol0.1
## con exp neu
## con 0.9659 0.0017 0.0324
## exp 0.0100 0.8858 0.1042
## neu 0.0401 0.0487 0.9112## Call:
## postpr(target = tar, index = model, sumstat = sumstat, tol = 0.05,
## method = "mnlogistic")
## Data:
## postpr.out$values (140 posterior samples)
## Models a priori:
## con, exp, neu
## Models a posteriori:
## con, exp, neu
##
## Proportion of accepted simulations (rejection):
## con exp neu
## 0.0071 0.3286 0.6643
##
## Bayes factors:
## con exp neu
## con 1.0000 0.0217 0.0108
## exp 46.0000 1.0000 0.4946
## neu 93.0000 2.0217 1.0000
##
##
## Posterior model probabilities (mnlogistic):
## con exp neu
## 0 0 1
##
## Bayes factors:
## con exp neu
## con 1.000000e+00 9.748584e+84 0.000000e+00
## exp 0.000000e+00 1.000000e+00 0.000000e+00
## neu 3.403383e+87 3.317816e+172 1.000000e+003 million years & 100k Ne
## Confusion matrix based on 100 samples for each model.
##
## $tol0.01
## con exp neu
## con 29 30 41
## exp 39 27 34
## neu 29 32 39
##
## $tol0.05
## con exp neu
## con 30 29 41
## exp 29 28 43
## neu 30 38 32
##
## $tol0.1
## con exp neu
## con 35 26 39
## exp 31 32 37
## neu 26 38 36
##
##
## Mean model posterior probabilities (mnlogistic)
##
## $tol0.01
## con exp neu
## con 0.2887 0.2968 0.4145
## exp 0.3947 0.2719 0.3334
## neu 0.3009 0.3169 0.3822
##
## $tol0.05
## con exp neu
## con 0.2965 0.3164 0.3872
## exp 0.3193 0.2944 0.3862
## neu 0.2909 0.3503 0.3588
##
## $tol0.1
## con exp neu
## con 0.3267 0.3160 0.3573
## exp 0.3337 0.3102 0.3561
## neu 0.3133 0.3432 0.3435## Call:
## postpr(target = tar, index = model, sumstat = sumstat, tol = 0.05,
## method = "mnlogistic")
## Data:
## postpr.out$values (134 posterior samples)
## Models a priori:
## con, exp, neu
## Models a posteriori:
## con, exp, neu
##
## Proportion of accepted simulations (rejection):
## con exp neu
## 0.2687 0.3731 0.3582
##
## Bayes factors:
## con exp neu
## con 1.0000 0.7200 0.7500
## exp 1.3889 1.0000 1.0417
## neu 1.3333 0.9600 1.0000
##
##
## Posterior model probabilities (mnlogistic):
## con exp neu
## 0.5043 0.3671 0.1286
##
## Bayes factors:
## con exp neu
## con 1.0000 1.3738 3.9222
## exp 0.7279 1.0000 2.8551
## neu 0.2550 0.3502 1.00001 million years & 50k Ne
## Confusion matrix based on 100 samples for each model.
##
## $tol0.01
## con exp neu
## con 62 18 20
## exp 10 55 35
## neu 23 30 47
##
## $tol0.05
## con exp neu
## con 67 6 27
## exp 8 46 46
## neu 15 34 51
##
## $tol0.1
## con exp neu
## con 69 3 28
## exp 9 30 61
## neu 14 24 62
##
##
## Mean model posterior probabilities (mnlogistic)
##
## $tol0.01
## con exp neu
## con 0.6181 0.1814 0.2005
## exp 0.0993 0.5451 0.3557
## neu 0.2201 0.3049 0.4750
##
## $tol0.05
## con exp neu
## con 0.6580 0.0906 0.2515
## exp 0.1220 0.4076 0.4704
## neu 0.1943 0.3338 0.4720
##
## $tol0.1
## con exp neu
## con 0.6731 0.0998 0.2272
## exp 0.1346 0.3744 0.4909
## neu 0.1948 0.3178 0.4875## Call:
## postpr(target = tar, index = model, sumstat = sumstat, tol = 0.05,
## method = "mnlogistic")
## Data:
## postpr.out$values (135 posterior samples)
## Models a priori:
## con, exp, neu
## Models a posteriori:
## con, exp, neu
##
## Proportion of accepted simulations (rejection):
## con exp neu
## 0.2593 0.4444 0.2963
##
## Bayes factors:
## con exp neu
## con 1.0000 0.5833 0.8750
## exp 1.7143 1.0000 1.5000
## neu 1.1429 0.6667 1.0000
##
##
## Posterior model probabilities (mnlogistic):
## con exp neu
## 0.0982 0.2643 0.6375
##
## Bayes factors:
## con exp neu
## con 1.0000 0.3714 0.1540
## exp 2.6923 1.0000 0.4146
## neu 6.4937 2.4119 1.00003 million years & 50k Ne
## Confusion matrix based on 100 samples for each model.
##
## $tol0.01
## con exp neu
## con 34 29 37
## exp 31 35 34
## neu 32 31 37
##
## $tol0.05
## con exp neu
## con 26 42 32
## exp 31 44 25
## neu 30 37 33
##
## $tol0.1
## con exp neu
## con 30 45 25
## exp 38 40 22
## neu 38 37 25
##
##
## Mean model posterior probabilities (mnlogistic)
##
## $tol0.01
## con exp neu
## con 0.3366 0.2887 0.3746
## exp 0.3060 0.3500 0.3440
## neu 0.3224 0.3078 0.3698
##
## $tol0.05
## con exp neu
## con 0.3080 0.3901 0.3019
## exp 0.3611 0.3681 0.2708
## neu 0.3430 0.3467 0.3104
##
## $tol0.1
## con exp neu
## con 0.3386 0.3892 0.2721
## exp 0.3599 0.3773 0.2628
## neu 0.3449 0.3573 0.2977## Call:
## postpr(target = tar, index = model, sumstat = sumstat, tol = 0.05,
## method = "mnlogistic")
## Data:
## postpr.out$values (135 posterior samples)
## Models a priori:
## con, exp, neu
## Models a posteriori:
## con, exp, neu
##
## Proportion of accepted simulations (rejection):
## con exp neu
## 0.3556 0.3481 0.2963
##
## Bayes factors:
## con exp neu
## con 1.0000 1.0213 1.2000
## exp 0.9792 1.0000 1.1750
## neu 0.8333 0.8511 1.0000
##
##
## Posterior model probabilities (mnlogistic):
## con exp neu
## 0.4404 0.4413 0.1183
##
## Bayes factors:
## con exp neu
## con 1.0000 0.9980 3.7222
## exp 1.0020 1.0000 3.7295
## neu 0.2687 0.2681 1.00002.2.1.1 Summary Table
| Priors | PostProb Contraction | PostProb Expansion | PostProb Neutral | |
|---|---|---|---|---|
| ms_1mya_100k | 1mya_100k | 2.93825333261164e-88 | 3.01403098296649e-173 | 1 |
| ms_3mya_100k | 3mya_100k | 0.504313748532904 | 0.367107054529299 | 0.128579196937797 |
| ms_1mya_50k | 1mya_50k | 0.098174098527296 | 0.264317511300487 | 0.637508390172217 |
| ms_3mya_50k | 3mya_50k | 0.440407787772727 | 0.441271626805481 | 0.118320585421791 |
2.2.2 Plib GH
3 million years & 100k Ne
## Confusion matrix based on 100 samples for each model.
##
## $tol0.01
## con exp neu
## con 42 18 40
## exp 31 31 38
## neu 24 40 36
##
## $tol0.05
## con exp neu
## con 50 23 27
## exp 27 42 31
## neu 30 36 34
##
## $tol0.1
## con exp neu
## con 54 19 27
## exp 21 39 40
## neu 31 43 26
##
##
## Mean model posterior probabilities (mnlogistic)
##
## $tol0.01
## con exp neu
## con 0.4192 0.2140 0.3668
## exp 0.3003 0.3272 0.3725
## neu 0.2542 0.3856 0.3602
##
## $tol0.05
## con exp neu
## con 0.4388 0.2743 0.2869
## exp 0.2960 0.3520 0.3520
## neu 0.3115 0.3547 0.3337
##
## $tol0.1
## con exp neu
## con 0.4380 0.2721 0.2899
## exp 0.2861 0.3657 0.3482
## neu 0.3115 0.3662 0.3223## Call:
## postpr(target = tar, index = model, sumstat = sumstat, tol = 0.05,
## method = "mnlogistic")
## Data:
## postpr.out$values (105 posterior samples)
## Models a priori:
## con, exp, neu
## Models a posteriori:
## con, exp, neu
##
## Proportion of accepted simulations (rejection):
## con exp neu
## 0.3238 0.2667 0.4095
##
## Bayes factors:
## con exp neu
## con 1.0000 1.2143 0.7907
## exp 0.8235 1.0000 0.6512
## neu 1.2647 1.5357 1.0000
##
##
## Posterior model probabilities (mnlogistic):
## con exp neu
## 0.2205 0.3867 0.3928
##
## Bayes factors:
## con exp neu
## con 1.0000 0.5703 0.5615
## exp 1.7534 1.0000 0.9846
## neu 1.7809 1.0157 1.00001 million years & 50k Ne
## Confusion matrix based on 100 samples for each model.
##
## $tol0.01
## con exp neu
## con 82 10 8
## exp 6 53 41
## neu 7 46 47
##
## $tol0.05
## con exp neu
## con 77 15 8
## exp 5 55 40
## neu 7 40 53
##
## $tol0.1
## con exp neu
## con 79 10 11
## exp 5 58 37
## neu 3 45 52
##
##
## Mean model posterior probabilities (mnlogistic)
##
## $tol0.01
## con exp neu
## con 0.8051 0.0955 0.0995
## exp 0.0731 0.5066 0.4204
## neu 0.0732 0.4766 0.4503
##
## $tol0.05
## con exp neu
## con 0.7878 0.1079 0.1043
## exp 0.0918 0.4887 0.4195
## neu 0.0934 0.4296 0.4770
##
## $tol0.1
## con exp neu
## con 0.7875 0.1043 0.1082
## exp 0.0950 0.4827 0.4223
## neu 0.0984 0.4463 0.4554## Call:
## postpr(target = tar, index = model, sumstat = sumstat, tol = 0.05,
## method = "mnlogistic")
## Data:
## postpr.out$values (136 posterior samples)
## Models a priori:
## con, exp, neu
## Models a posteriori:
## con, exp, neu
##
## Proportion of accepted simulations (rejection):
## con exp neu
## 0.1838 0.3456 0.4706
##
## Bayes factors:
## con exp neu
## con 1.0000 0.5319 0.3906
## exp 1.8800 1.0000 0.7344
## neu 2.5600 1.3617 1.0000
##
##
## Posterior model probabilities (mnlogistic):
## con exp neu
## 0.0738 0.3618 0.5645
##
## Bayes factors:
## con exp neu
## con 1.0000 0.2039 0.1307
## exp 4.9050 1.0000 0.6409
## neu 7.6532 1.5603 1.00003 million years & 50k Ne
## Confusion matrix based on 100 samples for each model.
##
## $tol0.01
## con exp neu
## con 28 37 35
## exp 33 30 37
## neu 36 28 36
##
## $tol0.05
## con exp neu
## con 32 26 42
## exp 38 34 28
## neu 34 30 36
##
## $tol0.1
## con exp neu
## con 30 24 46
## exp 30 35 35
## neu 40 33 27
##
##
## Mean model posterior probabilities (mnlogistic)
##
## $tol0.01
## con exp neu
## con 0.3278 0.3589 0.3133
## exp 0.3270 0.3366 0.3365
## neu 0.3474 0.3155 0.3372
##
## $tol0.05
## con exp neu
## con 0.3337 0.3159 0.3504
## exp 0.3347 0.3254 0.3399
## neu 0.3349 0.3237 0.3414
##
## $tol0.1
## con exp neu
## con 0.3335 0.3076 0.3589
## exp 0.3208 0.3312 0.3480
## neu 0.3383 0.3213 0.3404## Call:
## postpr(target = tar, index = model, sumstat = sumstat, tol = 0.05,
## method = "mnlogistic")
## Data:
## postpr.out$values (136 posterior samples)
## Models a priori:
## con, exp, neu
## Models a posteriori:
## con, exp, neu
##
## Proportion of accepted simulations (rejection):
## con exp neu
## 0.3088 0.3824 0.3088
##
## Bayes factors:
## con exp neu
## con 1.0000 0.8077 1.0000
## exp 1.2381 1.0000 1.2381
## neu 1.0000 0.8077 1.0000
##
##
## Posterior model probabilities (mnlogistic):
## con exp neu
## 0.2301 0.4194 0.3505
##
## Bayes factors:
## con exp neu
## con 1.0000 0.5486 0.6565
## exp 1.8228 1.0000 1.1967
## neu 1.5232 0.8356 1.00002.2.2.1 Summary Table
| Priors | PostProb Contraction | PostProb Expansion | PostProb Neutral | |
|---|---|---|---|---|
| ms_3mya_100k | 3mya_100k | 0.220541214686921 | 0.386698777755728 | 0.392760007557351 |
| ms_1mya_50k | 1mya_50k | 0.0737560632067556 | 0.361777086146564 | 0.564466850646681 |
| ms_3mya_50k | 3mya_50k | 0.230093545702135 | 0.41941982878794 | 0.350486625509925 |
3 Overall Conclusion
It seems we don’t yet have a combination of priors that fit the target data very well. I also see differences between call WUG5 and plib GH, which indicates that maybe different priors for different populations might be necessary to run this experiment reliably. Another problem I see is that the error classification, so how well the scenarios can be differentiated by the algorithm in general, depend on the choice of priors. This leads me to the conclusion, that the experiment as we’re trying it here, might be a lost cause, because it will be very hard to find the best fitting combination of priors for each population, but even if we manage to do this, eventually many of them will have a bad error classification. I guess we have to either broaden the priors again to find a middle ground between the first attempts with the very broad priors and the here fixed priors, or you just go the dadi route for those populations that you can. To end on a more positive note: It is a very cool finding, that ABC here is basically not all influenced by the sample size.