Comparison of length based idicators and methods
L Kell
25 May, 2022
Figure 1 Length data
Figure 2. Example of length based indicators and estimators for Sprat.
\(F/F_{MSY}\) true positive rate (sensitivity) and true negative rate (specificity)
Calculate the True Skill Statistic TSS = TPR + TNR - 1 = TPR - FPR and the area under the ROC curve (AUC)
A Perfect forecasts receive a TSS of 1, while a random forecast receive a score of 0 and forecasts inferior to random forecasts receive a negative score.
TSS depends on the chosen reference level, in contrast AUC indicates whether a indicator can classifiy i.e. rank status, i.e. AUC=1 if there is a choice of a reference value where TRP=1 & FPR=0. If AUC=0.5 then the estimator is no better than a coin toss.
TSS allows the best estimator for species and its robustness to be evaluated, the AUC identifies whether the indicator can be improved by tuning.
LIME and Haupt estimators and \(L_{mean}\) are biased (as shown by TSS) but can be improved by tuning (as shown by AUC). Tuning of \(L_{mean}\) results in performance equivalent to LBSPR. There are some scenarios, however, where performance is poor, as seen by the outliers, these are for the high CV, and so \(L_{mean}\) can be improved by improved sampling, or combinimng data from multiple years.
Figure 3. True skill score by species
Figure 4. True skill score by Estimator scenario
Figure 5. True skill score by sampling error
Area under the ROC curve
Figure 6. Area under the ROC by species
Figure 7. Area under the ROC by scenario
Figure 8. Area under the ROC by sampling error.
Figure 9. Estimates of slope.
Figure 10. P values
MSE
Figure 11. Comparison of Operating Model run for F with HCR run with perfect CPUE and with \(L_{Mean}\)
References
Acknowledgements
Software Versions
- R version 4.0.3 (2020-10-10)
- FLCore: 2.6.18.9020
- FLBRP: 2.5.8
- FLasher: 0.6.8.9003
- FLife: 3.4.0
- mydas: 1.2.2
- Compiled: Wed May 25 08:43:57 2022
